2024-05-09T07:50:00.969 INFO:root:teuthology version: 0.0.1.dev271+g1ae7ad8 2024-05-09T07:50:00.969 DEBUG:teuthology.run:Teuthology command: teuthology --description fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a5s-mds-1c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-ec-root overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/no prefetch_entire_dirfrags/no races session_timeout thrashosds-health} ranks/1 tasks/{1-thrash/mon 2-workunit/suites/iozone}} --verbose --name teuthology-2024-05-08_21:24:04-fs-squid-distro-default-smithi --archive /home/teuthworker/archive/teuthology-2024-05-08_21:24:04-fs-squid-distro-default-smithi/7698643 --owner scheduled_teuthology@teuthology -- /home/teuthworker/archive/teuthology-2024-05-08_21:24:04-fs-squid-distro-default-smithi/7698643/orig.config.yaml 2024-05-09T07:50:01.021 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-09T07:50:01.117 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-05-08_21:24:04-fs-squid-distro-default-smithi/7698643 branch: squid description: fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a5s-mds-1c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-ec-root overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/no prefetch_entire_dirfrags/no races session_timeout thrashosds-health} ranks/1 tasks/{1-thrash/mon 2-workunit/suites/iozone}} email: ceph-qa@ceph.com first_in_suite: false job_id: '7698643' kernel: client: branch: testing kdb: 1 sha1: distro ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: teuthology-2024-05-08_21:24:04-fs-squid-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: squid ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd session_timeout: 300 conf: client: client mount timeout: 600 client_shutdown_timeout: 120 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds oft prefetch dirfrags: false mds op complaint time: 180 mds verify scatter: true mds_dir_prefetch: false mds_sleep_rank_change: 5000000.0 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd blocked scrub grace period: 3600 osd failsafe full ratio: 0.95 osd max markdown count: 1000 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(OSD_ - \(PG_ - \(POOL_ - \(CACHE_POOL_ - \(SMALLER_PGP_NUM\) - \(OBJECT_ - \(SLOW_OPS\) - \(REQUEST_SLOW\) - \(TOO_FEW_PGS\) - slow request - timeout on replica - late reservation from - MON_DOWN - OSDMAP_FLAGS - OSD_DOWN - PG_DEGRADED - PG_AVAILABILITY - POOL_APP_NOT_ENABLED - overall HEALTH_ - \(MON_DOWN\) sha1: 96805dcf909541c4ca9a1e90258de8ae5b6fa1bc ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 96805dcf909541c4ca9a1e90258de8ae5b6fa1bc kclient: syntax: v1 selinux: allowlist: - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: be85c202969d25af6e130678e7d73a20e9c4ca11 owner: scheduled_teuthology@teuthology priority: 100 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mgr.y - mds.a - mds.c - mds.e - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - - mon.b - mon.c - mgr.x - mds.b - mds.d - mds.f - osd.4 - osd.5 - osd.6 - osd.7 seed: 2334 sha1: 96805dcf909541c4ca9a1e90258de8ae5b6fa1bc sleep_before_teardown: 0 subset: 359/512 suite: fs suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_be85c202969d25af6e130678e7d73a20e9c4ca11/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: be85c202969d25af6e130678e7d73a20e9c4ca11 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi001.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOuQ3EjDSvcgqGy6BS5Ig9vVHLkMq6JjkSO277ytzuWqwDKQ91Dn6OR3QzTzTVxopO6YAyuYfxePNnGMHVaZS2M= smithi007.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBANhHY74xUlJgJLHFh1AMaT07n0TE4qfCorQtgll2/4p7Jtmzv31b/1Z4GDoHSfqANJQPCCv2wQY8lvidNaPGfw= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - sequential: - mgrmodules - kclient: null - mon_thrash: check_mds_failover: true revive_delay: 20 thrash_delay: 10 - workunit: clients: all: - suites/iozone.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 1ae7ad82388e92a475afff437d49054826c019a1 timestamp: 2024-05-08_21:24:04 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-05-09T07:50:01.117 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_be85c202969d25af6e130678e7d73a20e9c4ca11/qa; will attempt to use it 2024-05-09T07:50:01.118 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_be85c202969d25af6e130678e7d73a20e9c4ca11/qa/tasks 2024-05-09T07:50:01.118 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-05-09T07:50:01.120 INFO:teuthology.task.internal:Checking packages... 2024-05-09T07:50:01.145 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '96805dcf909541c4ca9a1e90258de8ae5b6fa1bc' 2024-05-09T07:50:01.146 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-05-09T07:50:01.146 INFO:teuthology.packaging:ref: None 2024-05-09T07:50:01.146 INFO:teuthology.packaging:tag: None 2024-05-09T07:50:01.146 INFO:teuthology.packaging:branch: squid 2024-05-09T07:50:01.146 INFO:teuthology.packaging:sha1: 96805dcf909541c4ca9a1e90258de8ae5b6fa1bc 2024-05-09T07:50:01.146 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=squid 2024-05-09T07:50:01.361 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-2529.gbf77c77e 2024-05-09T07:50:01.363 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-05-09T07:50:01.370 INFO:teuthology.task.internal:no buildpackages task found 2024-05-09T07:50:01.370 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-05-09T07:50:01.378 INFO:teuthology.task.internal:Saving configuration 2024-05-09T07:50:01.398 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-05-09T07:50:01.404 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-09T07:50:01.429 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi001.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-05-08_21:24:04-fs-squid-distro-default-smithi/7698643', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-09 07:44:24.800744', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOuQ3EjDSvcgqGy6BS5Ig9vVHLkMq6JjkSO277ytzuWqwDKQ91Dn6OR3QzTzTVxopO6YAyuYfxePNnGMHVaZS2M='} 2024-05-09T07:50:01.451 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi007.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-05-08_21:24:04-fs-squid-distro-default-smithi/7698643', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-09 07:44:24.802160', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBANhHY74xUlJgJLHFh1AMaT07n0TE4qfCorQtgll2/4p7Jtmzv31b/1Z4GDoHSfqANJQPCCv2wQY8lvidNaPGfw='} 2024-05-09T07:50:01.452 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-05-09T07:50:01.458 INFO:teuthology.task.internal:roles: ubuntu@smithi001.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'mds.e', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'] 2024-05-09T07:50:01.458 INFO:teuthology.task.internal:roles: ubuntu@smithi007.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'mds.f', 'osd.4', 'osd.5', 'osd.6', 'osd.7'] 2024-05-09T07:50:01.458 INFO:teuthology.run_tasks:Running task console_log... 2024-05-09T07:50:01.524 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f8d1bd6a0d0>, signals=[15]) 2024-05-09T07:50:01.524 INFO:teuthology.run_tasks:Running task internal.connect... 2024-05-09T07:50:01.530 INFO:teuthology.task.internal:Opening connections... 2024-05-09T07:50:01.530 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi001.front.sepia.ceph.com 2024-05-09T07:50:01.532 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi001.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T07:50:01.608 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi007.front.sepia.ceph.com 2024-05-09T07:50:01.609 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi007.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T07:50:01.684 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-05-09T07:50:01.691 DEBUG:teuthology.orchestra.run.smithi001:> uname -m 2024-05-09T07:50:01.710 INFO:teuthology.orchestra.run.smithi001.stdout:x86_64 2024-05-09T07:50:01.710 DEBUG:teuthology.orchestra.run.smithi001:> cat /etc/os-release 2024-05-09T07:50:01.768 INFO:teuthology.orchestra.run.smithi001.stdout:NAME="CentOS Stream" 2024-05-09T07:50:01.768 INFO:teuthology.orchestra.run.smithi001.stdout:VERSION="9" 2024-05-09T07:50:01.768 INFO:teuthology.orchestra.run.smithi001.stdout:ID="centos" 2024-05-09T07:50:01.768 INFO:teuthology.orchestra.run.smithi001.stdout:ID_LIKE="rhel fedora" 2024-05-09T07:50:01.768 INFO:teuthology.orchestra.run.smithi001.stdout:VERSION_ID="9" 2024-05-09T07:50:01.768 INFO:teuthology.orchestra.run.smithi001.stdout:PLATFORM_ID="platform:el9" 2024-05-09T07:50:01.768 INFO:teuthology.orchestra.run.smithi001.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-09T07:50:01.769 INFO:teuthology.orchestra.run.smithi001.stdout:ANSI_COLOR="0;31" 2024-05-09T07:50:01.769 INFO:teuthology.orchestra.run.smithi001.stdout:LOGO="fedora-logo-icon" 2024-05-09T07:50:01.769 INFO:teuthology.orchestra.run.smithi001.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-09T07:50:01.769 INFO:teuthology.orchestra.run.smithi001.stdout:HOME_URL="https://centos.org/" 2024-05-09T07:50:01.769 INFO:teuthology.orchestra.run.smithi001.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-09T07:50:01.769 INFO:teuthology.orchestra.run.smithi001.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-09T07:50:01.769 INFO:teuthology.orchestra.run.smithi001.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-09T07:50:01.770 INFO:teuthology.lock.ops:Updating smithi001.front.sepia.ceph.com on lock server 2024-05-09T07:50:01.796 DEBUG:teuthology.orchestra.run.smithi007:> uname -m 2024-05-09T07:50:01.815 INFO:teuthology.orchestra.run.smithi007.stdout:x86_64 2024-05-09T07:50:01.815 DEBUG:teuthology.orchestra.run.smithi007:> cat /etc/os-release 2024-05-09T07:50:01.872 INFO:teuthology.orchestra.run.smithi007.stdout:NAME="CentOS Stream" 2024-05-09T07:50:01.873 INFO:teuthology.orchestra.run.smithi007.stdout:VERSION="9" 2024-05-09T07:50:01.873 INFO:teuthology.orchestra.run.smithi007.stdout:ID="centos" 2024-05-09T07:50:01.873 INFO:teuthology.orchestra.run.smithi007.stdout:ID_LIKE="rhel fedora" 2024-05-09T07:50:01.873 INFO:teuthology.orchestra.run.smithi007.stdout:VERSION_ID="9" 2024-05-09T07:50:01.873 INFO:teuthology.orchestra.run.smithi007.stdout:PLATFORM_ID="platform:el9" 2024-05-09T07:50:01.873 INFO:teuthology.orchestra.run.smithi007.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-09T07:50:01.873 INFO:teuthology.orchestra.run.smithi007.stdout:ANSI_COLOR="0;31" 2024-05-09T07:50:01.873 INFO:teuthology.orchestra.run.smithi007.stdout:LOGO="fedora-logo-icon" 2024-05-09T07:50:01.873 INFO:teuthology.orchestra.run.smithi007.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-09T07:50:01.873 INFO:teuthology.orchestra.run.smithi007.stdout:HOME_URL="https://centos.org/" 2024-05-09T07:50:01.873 INFO:teuthology.orchestra.run.smithi007.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-09T07:50:01.873 INFO:teuthology.orchestra.run.smithi007.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-09T07:50:01.874 INFO:teuthology.orchestra.run.smithi007.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-09T07:50:01.874 INFO:teuthology.lock.ops:Updating smithi007.front.sepia.ceph.com on lock server 2024-05-09T07:50:01.896 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-05-09T07:50:01.908 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-05-09T07:50:01.914 INFO:teuthology.task.internal:Checking for old test directory... 2024-05-09T07:50:01.914 DEBUG:teuthology.orchestra.run.smithi001:> test '!' -e /home/ubuntu/cephtest 2024-05-09T07:50:01.917 DEBUG:teuthology.orchestra.run.smithi007:> test '!' -e /home/ubuntu/cephtest 2024-05-09T07:50:01.933 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-05-09T07:50:01.940 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-05-09T07:50:01.940 DEBUG:teuthology.orchestra.run.smithi001:> test -z $(ls -A /var/lib/ceph) 2024-05-09T07:50:01.975 DEBUG:teuthology.orchestra.run.smithi007:> test -z $(ls -A /var/lib/ceph) 2024-05-09T07:50:02.002 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-05-09T07:50:02.058 INFO:teuthology.run_tasks:Running task kernel... 2024-05-09T07:50:02.073 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'kdb': 1, 'sha1': 'distro'} 2024-05-09T07:50:02.073 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}} 2024-05-09T07:50:02.073 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}}, timeout 300 2024-05-09T07:50:02.073 DEBUG:teuthology.orchestra.run.smithi001:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-09T07:50:02.091 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T07:50:02.092 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2024-05-09T07:50:02.348 DEBUG:teuthology.task.kernel:sha1 for client.0 is 806ec36b21129f113ce356c007dd59694d340b37 2024-05-09T07:50:02.348 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "806ec36b21129f113ce356c007dd59694d340b37"... 2024-05-09T07:50:02.348 DEBUG:teuthology.orchestra.run.smithi001:> uname -r 2024-05-09T07:50:02.367 INFO:teuthology.orchestra.run.smithi001.stdout:5.14.0-437.el9.x86_64 2024-05-09T07:50:02.367 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-437.el9.x86_64 vs 806ec36b21129f113ce356c007dd59694d340b37 2024-05-09T07:50:02.368 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-05-09T07:50:02.368 DEBUG:teuthology.orchestra.run.smithi001:> sudo yum upgrade -y linux-firmware 2024-05-09T07:50:08.173 INFO:teuthology.orchestra.run.smithi001.stdout:CentOS Stream 9 - BaseOS 2.0 MB/s | 8.1 MB 00:04 2024-05-09T07:50:10.482 INFO:teuthology.orchestra.run.smithi001.stdout:CentOS Stream 9 - AppStream 16 MB/s | 19 MB 00:01 2024-05-09T07:50:14.891 INFO:teuthology.orchestra.run.smithi001.stdout:CentOS Stream 9 - CRB 9.6 MB/s | 6.2 MB 00:00 2024-05-09T07:50:16.400 INFO:teuthology.orchestra.run.smithi001.stdout:CentOS Stream 9 - Extras packages 54 kB/s | 16 kB 00:00 2024-05-09T07:50:17.829 INFO:teuthology.orchestra.run.smithi001.stdout:Extra Packages for Enterprise Linux 16 MB/s | 21 MB 00:01 2024-05-09T07:50:23.218 INFO:teuthology.orchestra.run.smithi001.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-05-09T07:50:25.432 INFO:teuthology.orchestra.run.smithi001.stdout:Dependencies resolved. 2024-05-09T07:50:25.433 INFO:teuthology.orchestra.run.smithi001.stdout:Nothing to do. 2024-05-09T07:50:25.433 INFO:teuthology.orchestra.run.smithi001.stdout:Complete! 2024-05-09T07:50:25.557 INFO:teuthology.task.kernel:Downloading kernel 806ec36b21129f113ce356c007dd59694d340b37 on client.0... 2024-05-09T07:50:25.557 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=806ec36b21129f113ce356c007dd59694d340b37 2024-05-09T07:50:25.791 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ 2024-05-09T07:50:25.791 DEBUG:teuthology.orchestra.run.smithi001:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ --input-file=- 2024-05-09T07:50:25.794 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi001.front.sepia.ceph.com to complete... 2024-05-09T07:50:37.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09 07:50:37 URL:https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm [357367792/357367792] -> "/tmp/kernel.x86_64.rpm" [1] 2024-05-09T07:50:37.914 INFO:teuthology.orchestra.run.smithi001.stderr:FINISHED --2024-05-09 07:50:37-- 2024-05-09T07:50:37.914 INFO:teuthology.orchestra.run.smithi001.stderr:Total wall clock time: 12s 2024-05-09T07:50:37.914 INFO:teuthology.orchestra.run.smithi001.stderr:Downloaded: 1 files, 341M in 12s (28.9 MB/s) 2024-05-09T07:50:37.917 INFO:teuthology.task.kernel:Installing kernel 806ec36b21129f113ce356c007dd59694d340b37 on client.0... 2024-05-09T07:50:37.917 DEBUG:teuthology.orchestra.run.smithi001:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-05-09T07:50:40.100 INFO:teuthology.orchestra.run.smithi001.stdout:Verifying... ######################################## 2024-05-09T07:50:40.212 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing... ######################################## 2024-05-09T07:50:40.326 INFO:teuthology.orchestra.run.smithi001.stdout:Updating / installing... 2024-05-09T07:50:48.592 INFO:teuthology.orchestra.run.smithi001.stdout:kernel-6.9.0_rc3_g806ec36b2112-1 ######################################## 2024-05-09T07:51:28.916 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi001.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-05-09T07:51:28.916 DEBUG:teuthology.orchestra.run.smithi001:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-05-09T07:51:28.998 INFO:teuthology.orchestra.run.smithi001.stdout:/boot/System.map-6.9.0-rc3-g806ec36b2112 2024-05-09T07:51:28.998 INFO:teuthology.orchestra.run.smithi001.stdout:/boot/config-6.9.0-rc3-g806ec36b2112 2024-05-09T07:51:28.998 INFO:teuthology.orchestra.run.smithi001.stdout:/boot/vmlinuz-6.9.0-rc3-g806ec36b2112 2024-05-09T07:51:28.998 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/System.map 2024-05-09T07:51:28.999 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/config 2024-05-09T07:51:28.999 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel 2024-05-09T07:51:28.999 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch 2024-05-09T07:51:28.999 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86 2024-05-09T07:51:28.999 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto 2024-05-09T07:51:28.999 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-05-09T07:51:28.999 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-05-09T07:51:28.999 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-05-09T07:51:28.999 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-05-09T07:51:28.999 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-05-09T07:51:28.999 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-05-09T07:51:28.999 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-05-09T07:51:28.999 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32c-intel.ko 2024-05-09T07:51:28.999 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-05-09T07:51:29.000 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-05-09T07:51:29.000 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-05-09T07:51:29.000 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-05-09T07:51:29.000 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx2.ko 2024-05-09T07:51:29.000 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-05-09T07:51:29.000 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-05-09T07:51:29.000 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-05-09T07:51:29.000 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-05-09T07:51:29.000 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-05-09T07:51:29.000 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-05-09T07:51:29.000 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-05-09T07:51:29.001 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel 2024-05-09T07:51:29.001 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu 2024-05-09T07:51:29.001 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce 2024-05-09T07:51:29.001 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-05-09T07:51:29.001 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm 2024-05-09T07:51:29.001 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-amd.ko 2024-05-09T07:51:29.001 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-intel.ko 2024-05-09T07:51:29.001 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm.ko 2024-05-09T07:51:29.001 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto 2024-05-09T07:51:29.001 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ansi_cprng.ko 2024-05-09T07:51:29.001 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/anubis.ko 2024-05-09T07:51:29.001 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/arc4.ko 2024-05-09T07:51:29.001 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx 2024-05-09T07:51:29.002 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_memcpy.ko 2024-05-09T07:51:29.002 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_pq.ko 2024-05-09T07:51:29.002 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_raid6_recov.ko 2024-05-09T07:51:29.002 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_tx.ko 2024-05-09T07:51:29.002 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_xor.ko 2024-05-09T07:51:29.002 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/raid6test.ko 2024-05-09T07:51:29.002 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authenc.ko 2024-05-09T07:51:29.002 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authencesn.ko 2024-05-09T07:51:29.002 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blake2b_generic.ko 2024-05-09T07:51:29.002 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_common.ko 2024-05-09T07:51:29.002 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_generic.ko 2024-05-09T07:51:29.002 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/camellia_generic.ko 2024-05-09T07:51:29.003 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast5_generic.ko 2024-05-09T07:51:29.003 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast6_generic.ko 2024-05-09T07:51:29.003 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast_common.ko 2024-05-09T07:51:29.003 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ccm.ko 2024-05-09T07:51:29.003 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cmac.ko 2024-05-09T07:51:29.003 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crc32_generic.ko 2024-05-09T07:51:29.003 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_engine.ko 2024-05-09T07:51:29.003 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_user.ko 2024-05-09T07:51:29.003 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cts.ko 2024-05-09T07:51:29.003 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/des_generic.ko 2024-05-09T07:51:29.003 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/dh_generic.ko 2024-05-09T07:51:29.003 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/echainiv.ko 2024-05-09T07:51:29.004 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/essiv.ko 2024-05-09T07:51:29.004 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/fcrypt.ko 2024-05-09T07:51:29.004 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/khazad.ko 2024-05-09T07:51:29.004 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4.ko 2024-05-09T07:51:29.004 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4hc.ko 2024-05-09T07:51:29.004 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/md4.ko 2024-05-09T07:51:29.004 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/michael_mic.ko 2024-05-09T07:51:29.004 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcbc.ko 2024-05-09T07:51:29.004 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcrypt.ko 2024-05-09T07:51:29.004 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/rmd160.ko 2024-05-09T07:51:29.004 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/seed.ko 2024-05-09T07:51:29.004 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/serpent_generic.ko 2024-05-09T07:51:29.004 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tcrypt.ko 2024-05-09T07:51:29.005 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tea.ko 2024-05-09T07:51:29.005 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_common.ko 2024-05-09T07:51:29.005 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_generic.ko 2024-05-09T07:51:29.005 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/vmac.ko 2024-05-09T07:51:29.005 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/wp512.ko 2024-05-09T07:51:29.005 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xcbc.ko 2024-05-09T07:51:29.005 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xor.ko 2024-05-09T07:51:29.005 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xxhash_generic.ko 2024-05-09T07:51:29.005 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers 2024-05-09T07:51:29.005 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi 2024-05-09T07:51:29.005 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_ipmi.ko 2024-05-09T07:51:29.005 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_pad.ko 2024-05-09T07:51:29.005 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/ec_sys.ko 2024-05-09T07:51:29.006 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/platform_profile.ko 2024-05-09T07:51:29.006 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbs.ko 2024-05-09T07:51:29.006 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbshc.ko 2024-05-09T07:51:29.006 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/video.ko 2024-05-09T07:51:29.006 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata 2024-05-09T07:51:29.006 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/acard-ahci.ko 2024-05-09T07:51:29.006 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ahci_platform.ko 2024-05-09T07:51:29.006 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ata_generic.ko 2024-05-09T07:51:29.006 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/libahci_platform.ko 2024-05-09T07:51:29.006 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_acpi.ko 2024-05-09T07:51:29.006 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ali.ko 2024-05-09T07:51:29.006 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_amd.ko 2024-05-09T07:51:29.006 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_artop.ko 2024-05-09T07:51:29.007 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atiixp.ko 2024-05-09T07:51:29.007 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atp867x.ko 2024-05-09T07:51:29.007 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd640.ko 2024-05-09T07:51:29.007 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd64x.ko 2024-05-09T07:51:29.007 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cypress.ko 2024-05-09T07:51:29.007 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_efar.ko 2024-05-09T07:51:29.007 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt366.ko 2024-05-09T07:51:29.007 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt37x.ko 2024-05-09T07:51:29.007 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x2n.ko 2024-05-09T07:51:29.007 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x3.ko 2024-05-09T07:51:29.007 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it8213.ko 2024-05-09T07:51:29.007 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it821x.ko 2024-05-09T07:51:29.007 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_jmicron.ko 2024-05-09T07:51:29.008 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_marvell.ko 2024-05-09T07:51:29.008 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_mpiix.ko 2024-05-09T07:51:29.008 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_netcell.ko 2024-05-09T07:51:29.008 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ninja32.ko 2024-05-09T07:51:29.008 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87410.ko 2024-05-09T07:51:29.008 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87415.ko 2024-05-09T07:51:29.008 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_oldpiix.ko 2024-05-09T07:51:29.008 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_opti.ko 2024-05-09T07:51:29.008 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_optidma.ko 2024-05-09T07:51:29.008 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pcmcia.ko 2024-05-09T07:51:29.008 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc2027x.ko 2024-05-09T07:51:29.008 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-05-09T07:51:29.008 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_piccolo.ko 2024-05-09T07:51:29.009 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_rdc.ko 2024-05-09T07:51:29.009 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sch.ko 2024-05-09T07:51:29.009 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_serverworks.ko 2024-05-09T07:51:29.009 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sil680.ko 2024-05-09T07:51:29.009 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sis.ko 2024-05-09T07:51:29.009 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sl82c105.ko 2024-05-09T07:51:29.009 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_triflex.ko 2024-05-09T07:51:29.009 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_via.ko 2024-05-09T07:51:29.009 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pdc_adma.ko 2024-05-09T07:51:29.009 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_inic162x.ko 2024-05-09T07:51:29.009 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_mv.ko 2024-05-09T07:51:29.009 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_nv.ko 2024-05-09T07:51:29.010 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_promise.ko 2024-05-09T07:51:29.010 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_qstor.ko 2024-05-09T07:51:29.010 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil.ko 2024-05-09T07:51:29.010 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil24.ko 2024-05-09T07:51:29.010 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sis.ko 2024-05-09T07:51:29.010 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_svw.ko 2024-05-09T07:51:29.010 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sx4.ko 2024-05-09T07:51:29.010 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_uli.ko 2024-05-09T07:51:29.010 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_via.ko 2024-05-09T07:51:29.010 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_vsc.ko 2024-05-09T07:51:29.010 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay 2024-05-09T07:51:29.010 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay/ks0108.ko 2024-05-09T07:51:29.011 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base 2024-05-09T07:51:29.011 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap 2024-05-09T07:51:29.011 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-i2c.ko 2024-05-09T07:51:29.011 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-mmio.ko 2024-05-09T07:51:29.011 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma 2024-05-09T07:51:29.011 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma/bcma.ko 2024-05-09T07:51:29.011 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block 2024-05-09T07:51:29.011 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe 2024-05-09T07:51:29.011 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe/aoe.ko 2024-05-09T07:51:29.011 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/brd.ko 2024-05-09T07:51:29.011 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd 2024-05-09T07:51:29.011 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd/drbd.ko 2024-05-09T07:51:29.011 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/floppy.ko 2024-05-09T07:51:29.011 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/loop.ko 2024-05-09T07:51:29.011 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx 2024-05-09T07:51:29.011 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-05-09T07:51:29.012 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/nbd.ko 2024-05-09T07:51:29.012 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk 2024-05-09T07:51:29.012 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk/null_blk.ko 2024-05-09T07:51:29.012 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/pktcdvd.ko 2024-05-09T07:51:29.012 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/rbd.ko 2024-05-09T07:51:29.012 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/virtio_blk.ko 2024-05-09T07:51:29.012 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback 2024-05-09T07:51:29.012 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-05-09T07:51:29.012 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkfront.ko 2024-05-09T07:51:29.012 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram 2024-05-09T07:51:29.012 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram/zram.ko 2024-05-09T07:51:29.012 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char 2024-05-09T07:51:29.012 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hangcheck-timer.ko 2024-05-09T07:51:29.012 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random 2024-05-09T07:51:29.012 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/amd-rng.ko 2024-05-09T07:51:29.013 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/intel-rng.ko 2024-05-09T07:51:29.013 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-05-09T07:51:29.013 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/via-rng.ko 2024-05-09T07:51:29.013 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/virtio-rng.ko 2024-05-09T07:51:29.013 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi 2024-05-09T07:51:29.013 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-05-09T07:51:29.013 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-05-09T07:51:29.013 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-05-09T07:51:29.013 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_si.ko 2024-05-09T07:51:29.013 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-05-09T07:51:29.013 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-05-09T07:51:29.013 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/lp.ko 2024-05-09T07:51:29.013 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave 2024-05-09T07:51:29.013 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave/mwave.ko 2024-05-09T07:51:29.014 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ppdev.ko 2024-05-09T07:51:29.014 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tlclk.ko 2024-05-09T07:51:29.014 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm 2024-05-09T07:51:29.014 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm.ko 2024-05-09T07:51:29.014 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_atmel.ko 2024-05-09T07:51:29.014 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_infineon.ko 2024-05-09T07:51:29.014 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_nsc.ko 2024-05-09T07:51:29.014 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis.ko 2024-05-09T07:51:29.014 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-05-09T07:51:29.014 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/uv_mmtimer.ko 2024-05-09T07:51:29.014 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/virtio_console.ko 2024-05-09T07:51:29.014 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq 2024-05-09T07:51:29.014 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-05-09T07:51:29.015 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-05-09T07:51:29.015 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/p4-clockmod.ko 2024-05-09T07:51:29.015 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-05-09T07:51:29.015 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/powernow-k8.ko 2024-05-09T07:51:29.015 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/speedstep-lib.ko 2024-05-09T07:51:29.015 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto 2024-05-09T07:51:29.015 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp 2024-05-09T07:51:29.015 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-05-09T07:51:29.015 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp.ko 2024-05-09T07:51:29.015 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel 2024-05-09T07:51:29.015 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat 2024-05-09T07:51:29.015 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common 2024-05-09T07:51:29.015 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-05-09T07:51:29.015 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-05-09T07:51:29.015 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-05-09T07:51:29.016 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-aes.ko 2024-05-09T07:51:29.016 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-sha.ko 2024-05-09T07:51:29.016 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio 2024-05-09T07:51:29.016 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-05-09T07:51:29.016 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca 2024-05-09T07:51:29.016 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca/dca.ko 2024-05-09T07:51:29.016 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq 2024-05-09T07:51:29.016 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-05-09T07:51:29.016 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma 2024-05-09T07:51:29.016 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw 2024-05-09T07:51:29.016 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw/dw_dmac.ko 2024-05-09T07:51:29.016 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat 2024-05-09T07:51:29.016 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat/ioatdma.ko 2024-05-09T07:51:29.017 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac 2024-05-09T07:51:29.017 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/amd64_edac.ko 2024-05-09T07:51:29.017 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/e752x_edac.ko 2024-05-09T07:51:29.017 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/edac_mce_amd.ko 2024-05-09T07:51:29.017 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3000_edac.ko 2024-05-09T07:51:29.017 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3200_edac.ko 2024-05-09T07:51:29.017 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5100_edac.ko 2024-05-09T07:51:29.017 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5400_edac.ko 2024-05-09T07:51:29.017 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7300_edac.ko 2024-05-09T07:51:29.017 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7core_edac.ko 2024-05-09T07:51:29.017 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i82975x_edac.ko 2024-05-09T07:51:29.017 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/ie31200_edac.ko 2024-05-09T07:51:29.017 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/sb_edac.ko 2024-05-09T07:51:29.017 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/x38_edac.ko 2024-05-09T07:51:29.018 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware 2024-05-09T07:51:29.018 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/edd.ko 2024-05-09T07:51:29.018 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/iscsi_ibft.ko 2024-05-09T07:51:29.018 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio 2024-05-09T07:51:29.018 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-ich.ko 2024-05-09T07:51:29.018 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-viperboard.ko 2024-05-09T07:51:29.018 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid 2024-05-09T07:51:29.018 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-appleir.ko 2024-05-09T07:51:29.018 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-aureal.ko 2024-05-09T07:51:29.018 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-axff.ko 2024-05-09T07:51:29.018 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-dr.ko 2024-05-09T07:51:29.018 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elecom.ko 2024-05-09T07:51:29.018 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elo.ko 2024-05-09T07:51:29.019 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-emsff.ko 2024-05-09T07:51:29.019 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gaff.ko 2024-05-09T07:51:29.019 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gt683r.ko 2024-05-09T07:51:29.019 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gyration.ko 2024-05-09T07:51:29.019 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-kbd.ko 2024-05-09T07:51:29.019 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-mouse.ko 2024-05-09T07:51:29.019 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtekff.ko 2024-05-09T07:51:29.019 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-hyperv.ko 2024-05-09T07:51:29.019 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-icade.ko 2024-05-09T07:51:29.019 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-keytouch.ko 2024-05-09T07:51:29.019 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-kye.ko 2024-05-09T07:51:29.019 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lcpower.ko 2024-05-09T07:51:29.019 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-led.ko 2024-05-09T07:51:29.019 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lenovo.ko 2024-05-09T07:51:29.020 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-dj.ko 2024-05-09T07:51:29.020 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-05-09T07:51:29.020 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-multitouch.ko 2024-05-09T07:51:29.020 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-ortek.ko 2024-05-09T07:51:29.020 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-penmount.ko 2024-05-09T07:51:29.020 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-petalynx.ko 2024-05-09T07:51:29.020 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-picolcd.ko 2024-05-09T07:51:29.020 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-pl.ko 2024-05-09T07:51:29.020 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-plantronics.ko 2024-05-09T07:51:29.020 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-primax.ko 2024-05-09T07:51:29.020 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-rmi.ko 2024-05-09T07:51:29.020 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-arvo.ko 2024-05-09T07:51:29.020 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-common.ko 2024-05-09T07:51:29.020 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-isku.ko 2024-05-09T07:51:29.021 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kone.ko 2024-05-09T07:51:29.021 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-05-09T07:51:29.021 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-konepure.ko 2024-05-09T07:51:29.021 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-05-09T07:51:29.021 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-lua.ko 2024-05-09T07:51:29.021 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-pyra.ko 2024-05-09T07:51:29.021 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-ryos.ko 2024-05-09T07:51:29.021 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-savu.ko 2024-05-09T07:51:29.021 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat.ko 2024-05-09T07:51:29.021 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-saitek.ko 2024-05-09T07:51:29.021 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-samsung.ko 2024-05-09T07:51:29.021 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sensor-hub.ko 2024-05-09T07:51:29.021 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sjoy.ko 2024-05-09T07:51:29.021 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sony.ko 2024-05-09T07:51:29.022 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-speedlink.ko 2024-05-09T07:51:29.022 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-steelseries.ko 2024-05-09T07:51:29.022 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sunplus.ko 2024-05-09T07:51:29.022 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-thrustmaster.ko 2024-05-09T07:51:29.022 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tivo.ko 2024-05-09T07:51:29.022 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tmff.ko 2024-05-09T07:51:29.022 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-topseed.ko 2024-05-09T07:51:29.022 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-twinhan.ko 2024-05-09T07:51:29.022 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-uclogic.ko 2024-05-09T07:51:29.022 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-waltop.ko 2024-05-09T07:51:29.022 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-wiimote.ko 2024-05-09T07:51:29.022 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-xinmo.ko 2024-05-09T07:51:29.022 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zpff.ko 2024-05-09T07:51:29.023 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zydacron.ko 2024-05-09T07:51:29.023 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/uhid.ko 2024-05-09T07:51:29.023 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/wacom.ko 2024-05-09T07:51:29.023 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv 2024-05-09T07:51:29.023 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_balloon.ko 2024-05-09T07:51:29.023 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_utils.ko 2024-05-09T07:51:29.023 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_vmbus.ko 2024-05-09T07:51:29.023 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon 2024-05-09T07:51:29.023 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru.ko 2024-05-09T07:51:29.023 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru3.ko 2024-05-09T07:51:29.023 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/acpi_power_meter.ko 2024-05-09T07:51:29.023 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7414.ko 2024-05-09T07:51:29.023 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7418.ko 2024-05-09T07:51:29.023 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adc128d818.ko 2024-05-09T07:51:29.024 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1025.ko 2024-05-09T07:51:29.024 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1026.ko 2024-05-09T07:51:29.024 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1029.ko 2024-05-09T07:51:29.024 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1031.ko 2024-05-09T07:51:29.024 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm9240.ko 2024-05-09T07:51:29.024 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ads7828.ko 2024-05-09T07:51:29.024 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7410.ko 2024-05-09T07:51:29.024 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7411.ko 2024-05-09T07:51:29.024 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7462.ko 2024-05-09T07:51:29.024 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7470.ko 2024-05-09T07:51:29.024 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7475.ko 2024-05-09T07:51:29.024 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7x10.ko 2024-05-09T07:51:29.024 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/amc6821.ko 2024-05-09T07:51:29.025 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/applesmc.ko 2024-05-09T07:51:29.025 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asb100.ko 2024-05-09T07:51:29.025 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asc7621.ko 2024-05-09T07:51:29.025 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asus_atk0110.ko 2024-05-09T07:51:29.025 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/atxp1.ko 2024-05-09T07:51:29.025 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/coretemp.ko 2024-05-09T07:51:29.025 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-05-09T07:51:29.025 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dme1737.ko 2024-05-09T07:51:29.025 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds1621.ko 2024-05-09T07:51:29.025 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds620.ko 2024-05-09T07:51:29.025 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc1403.ko 2024-05-09T07:51:29.025 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc6w201.ko 2024-05-09T07:51:29.025 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71805f.ko 2024-05-09T07:51:29.026 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71882fg.ko 2024-05-09T07:51:29.026 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f75375s.ko 2024-05-09T07:51:29.026 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fam15h_power.ko 2024-05-09T07:51:29.026 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fschmd.ko 2024-05-09T07:51:29.026 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g760a.ko 2024-05-09T07:51:29.026 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g762.ko 2024-05-09T07:51:29.026 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl518sm.ko 2024-05-09T07:51:29.026 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl520sm.ko 2024-05-09T07:51:29.026 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/hwmon-vid.ko 2024-05-09T07:51:29.026 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5500_temp.ko 2024-05-09T07:51:29.026 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5k_amb.ko 2024-05-09T07:51:29.026 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmaem.ko 2024-05-09T07:51:29.026 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmpex.ko 2024-05-09T07:51:29.027 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina209.ko 2024-05-09T07:51:29.027 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina2xx.ko 2024-05-09T07:51:29.027 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/it87.ko 2024-05-09T07:51:29.027 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k10temp.ko 2024-05-09T07:51:29.027 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k8temp.ko 2024-05-09T07:51:29.027 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lineage-pem.ko 2024-05-09T07:51:29.027 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm63.ko 2024-05-09T07:51:29.027 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm73.ko 2024-05-09T07:51:29.027 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm75.ko 2024-05-09T07:51:29.027 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm77.ko 2024-05-09T07:51:29.027 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm78.ko 2024-05-09T07:51:29.027 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm80.ko 2024-05-09T07:51:29.027 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm83.ko 2024-05-09T07:51:29.027 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm85.ko 2024-05-09T07:51:29.028 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm87.ko 2024-05-09T07:51:29.028 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm90.ko 2024-05-09T07:51:29.028 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm92.ko 2024-05-09T07:51:29.028 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm93.ko 2024-05-09T07:51:29.028 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95234.ko 2024-05-09T07:51:29.028 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95241.ko 2024-05-09T07:51:29.028 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95245.ko 2024-05-09T07:51:29.028 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc2945.ko 2024-05-09T07:51:29.028 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4151.ko 2024-05-09T07:51:29.028 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4215.ko 2024-05-09T07:51:29.028 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4222.ko 2024-05-09T07:51:29.028 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4245.ko 2024-05-09T07:51:29.028 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4260.ko 2024-05-09T07:51:29.029 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4261.ko 2024-05-09T07:51:29.029 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max16065.ko 2024-05-09T07:51:29.029 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1619.ko 2024-05-09T07:51:29.029 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1668.ko 2024-05-09T07:51:29.029 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max197.ko 2024-05-09T07:51:29.029 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6639.ko 2024-05-09T07:51:29.029 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6650.ko 2024-05-09T07:51:29.029 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6697.ko 2024-05-09T07:51:29.029 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/mcp3021.ko 2024-05-09T07:51:29.029 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6683.ko 2024-05-09T07:51:29.029 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775-core.ko 2024-05-09T07:51:29.029 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775.ko 2024-05-09T07:51:29.029 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct7802.ko 2024-05-09T07:51:29.030 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87360.ko 2024-05-09T07:51:29.030 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87427.ko 2024-05-09T07:51:29.030 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pcf8591.ko 2024-05-09T07:51:29.030 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus 2024-05-09T07:51:29.030 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-05-09T07:51:29.030 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-05-09T07:51:29.030 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-05-09T07:51:29.030 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max16064.ko 2024-05-09T07:51:29.030 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max34440.ko 2024-05-09T07:51:29.030 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max8688.ko 2024-05-09T07:51:29.030 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-05-09T07:51:29.030 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-05-09T07:51:29.030 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-05-09T07:51:29.030 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-05-09T07:51:29.031 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-05-09T07:51:29.031 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-05-09T07:51:29.031 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/powr1220.ko 2024-05-09T07:51:29.031 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5627.ko 2024-05-09T07:51:29.031 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5636.ko 2024-05-09T07:51:29.031 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch56xx-common.ko 2024-05-09T07:51:29.031 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht15.ko 2024-05-09T07:51:29.031 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht21.ko 2024-05-09T07:51:29.031 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/shtc1.ko 2024-05-09T07:51:29.031 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sis5595.ko 2024-05-09T07:51:29.031 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47b397.ko 2024-05-09T07:51:29.031 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m1.ko 2024-05-09T07:51:29.031 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m192.ko 2024-05-09T07:51:29.031 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/thmc50.ko 2024-05-09T07:51:29.032 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp102.ko 2024-05-09T07:51:29.032 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp103.ko 2024-05-09T07:51:29.032 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp401.ko 2024-05-09T07:51:29.032 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp421.ko 2024-05-09T07:51:29.032 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via-cputemp.ko 2024-05-09T07:51:29.032 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via686a.ko 2024-05-09T07:51:29.032 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt1211.ko 2024-05-09T07:51:29.032 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt8231.ko 2024-05-09T07:51:29.032 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627ehf.ko 2024-05-09T07:51:29.032 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627hf.ko 2024-05-09T07:51:29.032 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83781d.ko 2024-05-09T07:51:29.032 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83791d.ko 2024-05-09T07:51:29.032 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83792d.ko 2024-05-09T07:51:29.032 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83793.ko 2024-05-09T07:51:29.033 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83795.ko 2024-05-09T07:51:29.033 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l785ts.ko 2024-05-09T07:51:29.033 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l786ng.ko 2024-05-09T07:51:29.033 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c 2024-05-09T07:51:29.033 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos 2024-05-09T07:51:29.033 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-05-09T07:51:29.033 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-05-09T07:51:29.033 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses 2024-05-09T07:51:29.033 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-05-09T07:51:29.033 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-05-09T07:51:29.033 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-05-09T07:51:29.033 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-05-09T07:51:29.033 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-05-09T07:51:29.034 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-05-09T07:51:29.034 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-05-09T07:51:29.034 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-05-09T07:51:29.034 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-i801.ko 2024-05-09T07:51:29.034 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-isch.ko 2024-05-09T07:51:29.034 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-05-09T07:51:29.034 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-05-09T07:51:29.034 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-05-09T07:51:29.034 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-parport.ko 2024-05-09T07:51:29.034 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-05-09T07:51:29.034 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-05-09T07:51:29.034 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-05-09T07:51:29.035 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-05-09T07:51:29.035 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-05-09T07:51:29.035 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-05-09T07:51:29.035 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-via.ko 2024-05-09T07:51:29.035 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-05-09T07:51:29.035 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-05-09T07:51:29.035 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-dev.ko 2024-05-09T07:51:29.035 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-mux.ko 2024-05-09T07:51:29.035 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-05-09T07:51:29.035 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-smbus.ko 2024-05-09T07:51:29.035 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-stub.ko 2024-05-09T07:51:29.035 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband 2024-05-09T07:51:29.035 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core 2024-05-09T07:51:29.035 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_cm.ko 2024-05-09T07:51:29.036 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_core.ko 2024-05-09T07:51:29.036 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_umad.ko 2024-05-09T07:51:29.036 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-05-09T07:51:29.036 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/iw_cm.ko 2024-05-09T07:51:29.036 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_cm.ko 2024-05-09T07:51:29.036 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-05-09T07:51:29.036 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw 2024-05-09T07:51:29.036 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4 2024-05-09T07:51:29.036 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-05-09T07:51:29.036 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4 2024-05-09T07:51:29.036 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-05-09T07:51:29.036 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5 2024-05-09T07:51:29.036 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-05-09T07:51:29.037 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca 2024-05-09T07:51:29.037 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-05-09T07:51:29.037 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp 2024-05-09T07:51:29.037 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib 2024-05-09T07:51:29.037 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-05-09T07:51:29.037 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser 2024-05-09T07:51:29.037 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-05-09T07:51:29.037 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert 2024-05-09T07:51:29.037 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-05-09T07:51:29.037 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp 2024-05-09T07:51:29.037 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-05-09T07:51:29.037 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt 2024-05-09T07:51:29.037 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-05-09T07:51:29.037 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input 2024-05-09T07:51:29.037 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport 2024-05-09T07:51:29.038 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-05-09T07:51:29.038 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/fm801-gp.ko 2024-05-09T07:51:29.038 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/gameport.ko 2024-05-09T07:51:29.038 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/joydev.ko 2024-05-09T07:51:29.038 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard 2024-05-09T07:51:29.038 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/gpio_keys.ko 2024-05-09T07:51:29.038 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse 2024-05-09T07:51:29.038 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/appletouch.ko 2024-05-09T07:51:29.038 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/bcm5974.ko 2024-05-09T07:51:29.038 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/cyapatp.ko 2024-05-09T07:51:29.038 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/elan_i2c.ko 2024-05-09T07:51:29.038 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/sermouse.ko 2024-05-09T07:51:29.038 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-05-09T07:51:29.039 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_usb.ko 2024-05-09T07:51:29.039 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/vsxxxaa.ko 2024-05-09T07:51:29.039 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4 2024-05-09T07:51:29.039 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4/rmi_core.ko 2024-05-09T07:51:29.039 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio 2024-05-09T07:51:29.039 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/altera_ps2.ko 2024-05-09T07:51:29.039 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/arc_ps2.ko 2024-05-09T07:51:29.039 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-05-09T07:51:29.039 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/serio_raw.ko 2024-05-09T07:51:29.039 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/sparse-keymap.ko 2024-05-09T07:51:29.039 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds 2024-05-09T07:51:29.039 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-blinkm.ko 2024-05-09T07:51:29.039 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lm3530.ko 2024-05-09T07:51:29.039 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lp3944.ko 2024-05-09T07:51:29.039 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-ss4200.ko 2024-05-09T07:51:29.040 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger 2024-05-09T07:51:29.040 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-05-09T07:51:29.040 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-05-09T07:51:29.040 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-05-09T07:51:29.040 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-05-09T07:51:29.040 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-05-09T07:51:29.040 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-05-09T07:51:29.040 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-05-09T07:51:29.040 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-05-09T07:51:29.040 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-05-09T07:51:29.040 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md 2024-05-09T07:51:29.040 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache 2024-05-09T07:51:29.040 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache/bcache.ko 2024-05-09T07:51:29.040 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-bio-prison.ko 2024-05-09T07:51:29.040 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache-smq.ko 2024-05-09T07:51:29.041 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache.ko 2024-05-09T07:51:29.041 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-crypt.ko 2024-05-09T07:51:29.041 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-delay.ko 2024-05-09T07:51:29.041 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-flakey.ko 2024-05-09T07:51:29.041 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-log-userspace.ko 2024-05-09T07:51:29.041 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-multipath.ko 2024-05-09T07:51:29.041 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-queue-length.ko 2024-05-09T07:51:29.041 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-raid.ko 2024-05-09T07:51:29.041 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-round-robin.ko 2024-05-09T07:51:29.041 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-service-time.ko 2024-05-09T07:51:29.041 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-switch.ko 2024-05-09T07:51:29.041 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-thin-pool.ko 2024-05-09T07:51:29.041 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-verity.ko 2024-05-09T07:51:29.041 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data 2024-05-09T07:51:29.042 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-05-09T07:51:29.042 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid0.ko 2024-05-09T07:51:29.042 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid1.ko 2024-05-09T07:51:29.042 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid10.ko 2024-05-09T07:51:29.042 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid456.ko 2024-05-09T07:51:29.042 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message 2024-05-09T07:51:29.042 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion 2024-05-09T07:51:29.042 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptbase.ko 2024-05-09T07:51:29.042 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptctl.ko 2024-05-09T07:51:29.042 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptfc.ko 2024-05-09T07:51:29.042 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptlan.ko 2024-05-09T07:51:29.042 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptsas.ko 2024-05-09T07:51:29.042 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptscsih.ko 2024-05-09T07:51:29.042 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptspi.ko 2024-05-09T07:51:29.043 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd 2024-05-09T07:51:29.043 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_ich.ko 2024-05-09T07:51:29.043 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_sch.ko 2024-05-09T07:51:29.043 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/mfd-core.ko 2024-05-09T07:51:29.043 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/sm501.ko 2024-05-09T07:51:29.043 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/viperboard.ko 2024-05-09T07:51:29.043 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/vx855.ko 2024-05-09T07:51:29.043 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/wl1273-core.ko 2024-05-09T07:51:29.043 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc 2024-05-09T07:51:29.043 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl 2024-05-09T07:51:29.043 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-05-09T07:51:29.043 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds9802als.ko 2024-05-09T07:51:29.043 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds990x.ko 2024-05-09T07:51:29.043 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/bh1770glc.ko 2024-05-09T07:51:29.044 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710 2024-05-09T07:51:29.044 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710/cb710.ko 2024-05-09T07:51:29.044 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom 2024-05-09T07:51:29.044 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/at24.ko 2024-05-09T07:51:29.044 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-05-09T07:51:29.044 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/max6875.ko 2024-05-09T07:51:29.044 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/enclosure.ko 2024-05-09T07:51:29.044 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/hpilo.ko 2024-05-09T07:51:29.044 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29003.ko 2024-05-09T07:51:29.044 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29020.ko 2024-05-09T07:51:29.044 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d 2024-05-09T07:51:29.044 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-05-09T07:51:29.044 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-05-09T07:51:29.045 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei 2024-05-09T07:51:29.045 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-me.ko 2024-05-09T07:51:29.045 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-txe.ko 2024-05-09T07:51:29.045 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei.ko 2024-05-09T07:51:29.045 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru 2024-05-09T07:51:29.045 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru/gru.ko 2024-05-09T07:51:29.045 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp 2024-05-09T07:51:29.045 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xp.ko 2024-05-09T07:51:29.045 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpc.ko 2024-05-09T07:51:29.045 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-05-09T07:51:29.045 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_7xx1.ko 2024-05-09T07:51:29.045 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_core.ko 2024-05-09T07:51:29.045 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tsl2550.ko 2024-05-09T07:51:29.045 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_balloon.ko 2024-05-09T07:51:29.046 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci 2024-05-09T07:51:29.046 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-05-09T07:51:29.046 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net 2024-05-09T07:51:29.046 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding 2024-05-09T07:51:29.046 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding/bonding.ko 2024-05-09T07:51:29.046 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa 2024-05-09T07:51:29.046 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53 2024-05-09T07:51:29.046 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53/b53_common.ko 2024-05-09T07:51:29.046 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/bcm-sf2.ko 2024-05-09T07:51:29.046 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6060.ko 2024-05-09T07:51:29.046 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx 2024-05-09T07:51:29.046 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-05-09T07:51:29.046 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dummy.ko 2024-05-09T07:51:29.047 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/eql.ko 2024-05-09T07:51:29.047 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet 2024-05-09T07:51:29.047 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com 2024-05-09T07:51:29.047 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-05-09T07:51:29.047 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-05-09T07:51:29.047 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-05-09T07:51:29.047 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-05-09T07:51:29.047 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390 2024-05-09T07:51:29.047 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/8390.ko 2024-05-09T07:51:29.047 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-05-09T07:51:29.047 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-05-09T07:51:29.047 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-05-09T07:51:29.047 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec 2024-05-09T07:51:29.048 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-05-09T07:51:29.048 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere 2024-05-09T07:51:29.048 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere/et131x.ko 2024-05-09T07:51:29.048 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon 2024-05-09T07:51:29.048 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-05-09T07:51:29.048 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera 2024-05-09T07:51:29.048 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-05-09T07:51:29.048 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd 2024-05-09T07:51:29.048 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-05-09T07:51:29.048 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-05-09T07:51:29.048 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-05-09T07:51:29.048 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros 2024-05-09T07:51:29.048 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx 2024-05-09T07:51:29.048 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-05-09T07:51:29.048 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c 2024-05-09T07:51:29.049 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-05-09T07:51:29.049 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e 2024-05-09T07:51:29.049 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-05-09T07:51:29.049 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx 2024-05-09T07:51:29.049 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-05-09T07:51:29.049 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-05-09T07:51:29.049 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom 2024-05-09T07:51:29.049 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-05-09T07:51:29.049 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-05-09T07:51:29.049 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-05-09T07:51:29.049 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-05-09T07:51:29.049 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-05-09T07:51:29.049 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet 2024-05-09T07:51:29.049 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-05-09T07:51:29.050 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-05-09T07:51:29.050 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade 2024-05-09T07:51:29.050 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna 2024-05-09T07:51:29.050 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-05-09T07:51:29.050 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio 2024-05-09T07:51:29.050 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb 2024-05-09T07:51:29.050 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-05-09T07:51:29.050 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-05-09T07:51:29.050 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-05-09T07:51:29.050 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-05-09T07:51:29.050 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-05-09T07:51:29.050 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-05-09T07:51:29.051 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-05-09T07:51:29.051 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-05-09T07:51:29.051 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-05-09T07:51:29.051 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco 2024-05-09T07:51:29.051 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic 2024-05-09T07:51:29.051 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-05-09T07:51:29.051 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec 2024-05-09T07:51:29.051 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip 2024-05-09T07:51:29.051 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-05-09T07:51:29.051 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-05-09T07:51:29.051 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-05-09T07:51:29.051 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-05-09T07:51:29.051 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-05-09T07:51:29.052 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-05-09T07:51:29.052 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink 2024-05-09T07:51:29.052 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-05-09T07:51:29.052 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-05-09T07:51:29.052 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dnet.ko 2024-05-09T07:51:29.052 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex 2024-05-09T07:51:29.052 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet 2024-05-09T07:51:29.052 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-05-09T07:51:29.052 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ethoc.ko 2024-05-09T07:51:29.052 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/fealnx.ko 2024-05-09T07:51:29.052 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel 2024-05-09T07:51:29.052 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e100.ko 2024-05-09T07:51:29.052 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000 2024-05-09T07:51:29.052 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-05-09T07:51:29.053 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e 2024-05-09T07:51:29.053 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-05-09T07:51:29.053 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k 2024-05-09T07:51:29.053 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-05-09T07:51:29.053 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e 2024-05-09T07:51:29.053 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-05-09T07:51:29.053 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf 2024-05-09T07:51:29.053 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-05-09T07:51:29.053 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb 2024-05-09T07:51:29.053 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-05-09T07:51:29.053 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf 2024-05-09T07:51:29.053 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-05-09T07:51:29.053 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe 2024-05-09T07:51:29.053 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-05-09T07:51:29.054 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf 2024-05-09T07:51:29.054 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-05-09T07:51:29.054 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/jme.ko 2024-05-09T07:51:29.054 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell 2024-05-09T07:51:29.054 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-05-09T07:51:29.054 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/skge.ko 2024-05-09T07:51:29.054 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-05-09T07:51:29.054 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox 2024-05-09T07:51:29.054 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4 2024-05-09T07:51:29.054 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-05-09T07:51:29.054 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-05-09T07:51:29.054 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5 2024-05-09T07:51:29.054 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-05-09T07:51:29.055 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-05-09T07:51:29.055 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel 2024-05-09T07:51:29.055 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-05-09T07:51:29.055 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom 2024-05-09T07:51:29.055 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge 2024-05-09T07:51:29.055 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-05-09T07:51:29.055 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi 2024-05-09T07:51:29.055 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-05-09T07:51:29.055 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-05-09T07:51:29.055 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion 2024-05-09T07:51:29.055 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-05-09T07:51:29.055 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia 2024-05-09T07:51:29.055 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-05-09T07:51:29.055 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines 2024-05-09T07:51:29.056 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-05-09T07:51:29.056 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-05-09T07:51:29.056 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic 2024-05-09T07:51:29.056 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen 2024-05-09T07:51:29.056 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-05-09T07:51:29.056 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-05-09T07:51:29.056 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-05-09T07:51:29.056 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-05-09T07:51:29.056 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc 2024-05-09T07:51:29.056 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-05-09T07:51:29.056 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek 2024-05-09T07:51:29.056 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-05-09T07:51:29.056 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-05-09T07:51:29.056 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/atp.ko 2024-05-09T07:51:29.057 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-05-09T07:51:29.057 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker 2024-05-09T07:51:29.057 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-05-09T07:51:29.057 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc 2024-05-09T07:51:29.057 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-05-09T07:51:29.057 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan 2024-05-09T07:51:29.057 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-05-09T07:51:29.057 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis 2024-05-09T07:51:29.057 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis190.ko 2024-05-09T07:51:29.057 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis900.ko 2024-05-09T07:51:29.057 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc 2024-05-09T07:51:29.057 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-05-09T07:51:29.057 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-05-09T07:51:29.057 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-05-09T07:51:29.058 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-05-09T07:51:29.058 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro 2024-05-09T07:51:29.058 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac 2024-05-09T07:51:29.058 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-05-09T07:51:29.058 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-05-09T07:51:29.058 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun 2024-05-09T07:51:29.058 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/cassini.ko 2024-05-09T07:51:29.058 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/niu.ko 2024-05-09T07:51:29.058 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sungem.ko 2024-05-09T07:51:29.058 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-05-09T07:51:29.058 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti 2024-05-09T07:51:29.058 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-05-09T07:51:29.058 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti 2024-05-09T07:51:29.058 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti/tlan.ko 2024-05-09T07:51:29.059 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via 2024-05-09T07:51:29.059 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-05-09T07:51:29.059 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-05-09T07:51:29.059 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet 2024-05-09T07:51:29.059 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-05-09T07:51:29.059 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-05-09T07:51:29.059 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom 2024-05-09T07:51:29.059 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-05-09T07:51:29.059 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/geneve.ko 2024-05-09T07:51:29.059 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv 2024-05-09T07:51:29.059 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-05-09T07:51:29.059 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154 2024-05-09T07:51:29.059 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154/fakelb.ko 2024-05-09T07:51:29.059 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ifb.ko 2024-05-09T07:51:29.060 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan 2024-05-09T07:51:29.060 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan/ipvlan.ko 2024-05-09T07:51:29.060 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvlan.ko 2024-05-09T07:51:29.060 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvtap.ko 2024-05-09T07:51:29.060 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio 2024-05-09T07:51:29.060 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio.ko 2024-05-09T07:51:29.060 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-05-09T07:51:29.060 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-05-09T07:51:29.060 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-regmap.ko 2024-05-09T07:51:29.060 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mii.ko 2024-05-09T07:51:29.060 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/net_failover.ko 2024-05-09T07:51:29.060 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/netconsole.ko 2024-05-09T07:51:29.060 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/nlmon.ko 2024-05-09T07:51:29.060 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs 2024-05-09T07:51:29.061 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs-lynx.ko 2024-05-09T07:51:29.061 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-05-09T07:51:29.061 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy 2024-05-09T07:51:29.061 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/amd.ko 2024-05-09T07:51:29.061 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-05-09T07:51:29.061 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-05-09T07:51:29.061 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm7xxx.ko 2024-05-09T07:51:29.061 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm87xx.ko 2024-05-09T07:51:29.061 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/broadcom.ko 2024-05-09T07:51:29.061 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/cicada.ko 2024-05-09T07:51:29.061 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/davicom.ko 2024-05-09T07:51:29.061 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/dp83640.ko 2024-05-09T07:51:29.061 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/et1011c.ko 2024-05-09T07:51:29.061 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/icplus.ko 2024-05-09T07:51:29.062 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/lxt.ko 2024-05-09T07:51:29.062 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/marvell.ko 2024-05-09T07:51:29.062 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/micrel.ko 2024-05-09T07:51:29.062 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/national.ko 2024-05-09T07:51:29.062 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/phylink.ko 2024-05-09T07:51:29.062 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/qsemi.ko 2024-05-09T07:51:29.062 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/realtek.ko 2024-05-09T07:51:29.062 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/smsc.ko 2024-05-09T07:51:29.062 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/ste10Xp.ko 2024-05-09T07:51:29.062 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/vitesse.ko 2024-05-09T07:51:29.062 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp 2024-05-09T07:51:29.062 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/bsd_comp.ko 2024-05-09T07:51:29.062 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_async.ko 2024-05-09T07:51:29.062 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_deflate.ko 2024-05-09T07:51:29.063 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_generic.ko 2024-05-09T07:51:29.063 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_mppe.ko 2024-05-09T07:51:29.063 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_synctty.ko 2024-05-09T07:51:29.063 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppoe.ko 2024-05-09T07:51:29.063 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppox.ko 2024-05-09T07:51:29.063 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pptp.ko 2024-05-09T07:51:29.063 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip 2024-05-09T07:51:29.063 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slhc.ko 2024-05-09T07:51:29.063 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slip.ko 2024-05-09T07:51:29.063 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/sungem_phy.ko 2024-05-09T07:51:29.063 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tap.ko 2024-05-09T07:51:29.063 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team 2024-05-09T07:51:29.063 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team.ko 2024-05-09T07:51:29.064 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_activebackup.ko 2024-05-09T07:51:29.064 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_broadcast.ko 2024-05-09T07:51:29.064 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-05-09T07:51:29.064 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_random.ko 2024-05-09T07:51:29.064 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-05-09T07:51:29.064 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tun.ko 2024-05-09T07:51:29.064 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/veth.ko 2024-05-09T07:51:29.064 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/virtio_net.ko 2024-05-09T07:51:29.064 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3 2024-05-09T07:51:29.064 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-05-09T07:51:29.064 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan 2024-05-09T07:51:29.064 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan/vxlan.ko 2024-05-09T07:51:29.064 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback 2024-05-09T07:51:29.064 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback/xen-netback.ko 2024-05-09T07:51:29.065 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netfront.ko 2024-05-09T07:51:29.065 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb 2024-05-09T07:51:29.065 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb/ntb.ko 2024-05-09T07:51:29.065 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme 2024-05-09T07:51:29.065 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host 2024-05-09T07:51:29.065 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme-core.ko 2024-05-09T07:51:29.065 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme.ko 2024-05-09T07:51:29.065 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport 2024-05-09T07:51:29.065 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport.ko 2024-05-09T07:51:29.065 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_cs.ko 2024-05-09T07:51:29.065 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_pc.ko 2024-05-09T07:51:29.065 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_serial.ko 2024-05-09T07:51:29.065 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci 2024-05-09T07:51:29.065 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug 2024-05-09T07:51:29.066 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-05-09T07:51:29.066 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie 2024-05-09T07:51:29.066 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie/aer_inject.ko 2024-05-09T07:51:29.066 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/xen-pcifront.ko 2024-05-09T07:51:29.066 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia 2024-05-09T07:51:29.066 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/i82092.ko 2024-05-09T07:51:29.066 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/pd6729.ko 2024-05-09T07:51:29.066 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/yenta_socket.ko 2024-05-09T07:51:29.066 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl 2024-05-09T07:51:29.066 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel 2024-05-09T07:51:29.066 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-05-09T07:51:29.066 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform 2024-05-09T07:51:29.066 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86 2024-05-09T07:51:29.067 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acer-wmi.ko 2024-05-09T07:51:29.067 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acerhdf.ko 2024-05-09T07:51:29.067 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/apple-gmux.ko 2024-05-09T07:51:29.067 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-laptop.ko 2024-05-09T07:51:29.067 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-05-09T07:51:29.067 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-wmi.ko 2024-05-09T07:51:29.067 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/classmate-laptop.ko 2024-05-09T07:51:29.067 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-05-09T07:51:29.067 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-05-09T07:51:29.067 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-05-09T07:51:29.067 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-05-09T07:51:29.067 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/hdaps.ko 2024-05-09T07:51:29.067 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel 2024-05-09T07:51:29.067 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-05-09T07:51:29.068 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel_ips.ko 2024-05-09T07:51:29.068 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/msi-wmi.ko 2024-05-09T07:51:29.068 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/mxm-wmi.ko 2024-05-09T07:51:29.068 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-05-09T07:51:29.068 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-laptop.ko 2024-05-09T07:51:29.068 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-q10.ko 2024-05-09T07:51:29.068 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/topstar-laptop.ko 2024-05-09T07:51:29.068 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-05-09T07:51:29.068 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_haps.ko 2024-05-09T07:51:29.068 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi-bmof.ko 2024-05-09T07:51:29.068 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi.ko 2024-05-09T07:51:29.068 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap 2024-05-09T07:51:29.068 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_common.ko 2024-05-09T07:51:29.069 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_msr.ko 2024-05-09T07:51:29.069 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps 2024-05-09T07:51:29.069 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients 2024-05-09T07:51:29.069 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-gpio.ko 2024-05-09T07:51:29.069 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-ldisc.ko 2024-05-09T07:51:29.069 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps_parport.ko 2024-05-09T07:51:29.069 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/pps_core.ko 2024-05-09T07:51:29.069 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp 2024-05-09T07:51:29.069 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp.ko 2024-05-09T07:51:29.069 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp_kvm.ko 2024-05-09T07:51:29.069 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras 2024-05-09T07:51:29.069 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd 2024-05-09T07:51:29.069 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl 2024-05-09T07:51:29.069 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-05-09T07:51:29.070 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/fmpm.ko 2024-05-09T07:51:29.070 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc 2024-05-09T07:51:29.070 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-bq32k.ko 2024-05-09T07:51:29.070 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1286.ko 2024-05-09T07:51:29.070 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1307.ko 2024-05-09T07:51:29.070 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1374.ko 2024-05-09T07:51:29.070 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1511.ko 2024-05-09T07:51:29.070 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1553.ko 2024-05-09T07:51:29.070 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1672.ko 2024-05-09T07:51:29.070 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1742.ko 2024-05-09T07:51:29.070 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds2404.ko 2024-05-09T07:51:29.070 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds3232.ko 2024-05-09T07:51:29.070 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-em3027.ko 2024-05-09T07:51:29.070 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-fm3130.ko 2024-05-09T07:51:29.071 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl12022.ko 2024-05-09T07:51:29.071 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl1208.ko 2024-05-09T07:51:29.071 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m41t80.ko 2024-05-09T07:51:29.071 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t35.ko 2024-05-09T07:51:29.071 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t59.ko 2024-05-09T07:51:29.071 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-max6900.ko 2024-05-09T07:51:29.071 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-msm6242.ko 2024-05-09T07:51:29.071 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf2127.ko 2024-05-09T07:51:29.071 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf85063.ko 2024-05-09T07:51:29.071 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8523.ko 2024-05-09T07:51:29.071 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8563.ko 2024-05-09T07:51:29.071 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8583.ko 2024-05-09T07:51:29.072 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rp5c01.ko 2024-05-09T07:51:29.072 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rs5c372.ko 2024-05-09T07:51:29.072 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-05-09T07:51:29.072 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8025.ko 2024-05-09T07:51:29.072 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8581.ko 2024-05-09T07:51:29.072 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-05-09T07:51:29.072 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-x1205.ko 2024-05-09T07:51:29.072 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi 2024-05-09T07:51:29.072 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-9xxx.ko 2024-05-09T07:51:29.072 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-sas.ko 2024-05-09T07:51:29.072 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-xxxx.ko 2024-05-09T07:51:29.072 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/BusLogic.ko 2024-05-09T07:51:29.072 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/a100u2w.ko 2024-05-09T07:51:29.073 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid 2024-05-09T07:51:29.073 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid/aacraid.ko 2024-05-09T07:51:29.073 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/advansys.ko 2024-05-09T07:51:29.073 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx 2024-05-09T07:51:29.073 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-05-09T07:51:29.073 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-05-09T07:51:29.073 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/am53c974.ko 2024-05-09T07:51:29.073 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr 2024-05-09T07:51:29.073 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-05-09T07:51:29.073 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/atp870u.ko 2024-05-09T07:51:29.073 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi 2024-05-09T07:51:29.073 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-05-09T07:51:29.073 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa 2024-05-09T07:51:29.073 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa/bfa.ko 2024-05-09T07:51:29.074 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc 2024-05-09T07:51:29.074 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-05-09T07:51:29.074 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i 2024-05-09T07:51:29.074 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-05-09T07:51:29.074 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ch.ko 2024-05-09T07:51:29.074 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor 2024-05-09T07:51:29.074 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor/csiostor.ko 2024-05-09T07:51:29.074 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi 2024-05-09T07:51:29.074 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i 2024-05-09T07:51:29.074 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-05-09T07:51:29.074 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i 2024-05-09T07:51:29.074 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-05-09T07:51:29.074 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-05-09T07:51:29.075 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dc395x.ko 2024-05-09T07:51:29.075 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler 2024-05-09T07:51:29.075 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-05-09T07:51:29.075 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-05-09T07:51:29.075 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-05-09T07:51:29.075 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-05-09T07:51:29.075 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dmx3191d.ko 2024-05-09T07:51:29.075 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r 2024-05-09T07:51:29.075 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r/esas2r.ko 2024-05-09T07:51:29.075 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esp_scsi.ko 2024-05-09T07:51:29.075 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe 2024-05-09T07:51:29.075 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/fcoe.ko 2024-05-09T07:51:29.075 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-05-09T07:51:29.075 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic 2024-05-09T07:51:29.075 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic/fnic.ko 2024-05-09T07:51:29.076 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hpsa.ko 2024-05-09T07:51:29.076 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hptiop.ko 2024-05-09T07:51:29.076 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hv_storvsc.ko 2024-05-09T07:51:29.076 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/initio.ko 2024-05-09T07:51:29.076 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ipr.ko 2024-05-09T07:51:29.076 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ips.ko 2024-05-09T07:51:29.076 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci 2024-05-09T07:51:29.076 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci/isci.ko 2024-05-09T07:51:29.076 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-05-09T07:51:29.076 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_tcp.ko 2024-05-09T07:51:29.076 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc 2024-05-09T07:51:29.076 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc/libfc.ko 2024-05-09T07:51:29.076 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi.ko 2024-05-09T07:51:29.076 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi_tcp.ko 2024-05-09T07:51:29.076 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas 2024-05-09T07:51:29.076 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas/libsas.ko 2024-05-09T07:51:29.077 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc 2024-05-09T07:51:29.077 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc/lpfc.ko 2024-05-09T07:51:29.077 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid 2024-05-09T07:51:29.077 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid.ko 2024-05-09T07:51:29.077 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-05-09T07:51:29.077 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-05-09T07:51:29.077 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-05-09T07:51:29.077 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas 2024-05-09T07:51:29.077 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-05-09T07:51:29.077 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas 2024-05-09T07:51:29.077 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas/mvsas.ko 2024-05-09T07:51:29.077 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvumi.ko 2024-05-09T07:51:29.078 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001 2024-05-09T07:51:29.078 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-05-09T07:51:29.078 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pmcraid.ko 2024-05-09T07:51:29.078 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla1280.ko 2024-05-09T07:51:29.078 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx 2024-05-09T07:51:29.078 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-05-09T07:51:29.078 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-05-09T07:51:29.078 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx 2024-05-09T07:51:29.078 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-05-09T07:51:29.078 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/raid_class.ko 2024-05-09T07:51:29.078 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_debug.ko 2024-05-09T07:51:29.078 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_fc.ko 2024-05-09T07:51:29.078 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-05-09T07:51:29.078 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_sas.ko 2024-05-09T07:51:29.079 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_spi.ko 2024-05-09T07:51:29.079 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_srp.ko 2024-05-09T07:51:29.079 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ses.ko 2024-05-09T07:51:29.079 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/st.ko 2024-05-09T07:51:29.079 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/stex.ko 2024-05-09T07:51:29.079 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2 2024-05-09T07:51:29.079 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-05-09T07:51:29.079 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/virtio_scsi.ko 2024-05-09T07:51:29.079 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/vmw_pvscsi.ko 2024-05-09T07:51:29.079 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/wd719x.ko 2024-05-09T07:51:29.079 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb 2024-05-09T07:51:29.079 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb/ssb.ko 2024-05-09T07:51:29.079 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target 2024-05-09T07:51:29.079 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi 2024-05-09T07:51:29.079 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-05-09T07:51:29.080 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback 2024-05-09T07:51:29.080 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback/tcm_loop.ko 2024-05-09T07:51:29.080 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_file.ko 2024-05-09T07:51:29.080 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_iblock.ko 2024-05-09T07:51:29.080 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_mod.ko 2024-05-09T07:51:29.080 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_pscsi.ko 2024-05-09T07:51:29.080 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_user.ko 2024-05-09T07:51:29.080 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc 2024-05-09T07:51:29.080 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-05-09T07:51:29.080 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal 2024-05-09T07:51:29.080 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel 2024-05-09T07:51:29.080 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal 2024-05-09T07:51:29.080 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-05-09T07:51:29.080 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-05-09T07:51:29.080 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-05-09T07:51:29.081 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-05-09T07:51:29.081 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-05-09T07:51:29.081 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-05-09T07:51:29.081 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-05-09T07:51:29.081 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-05-09T07:51:29.081 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-05-09T07:51:29.081 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-05-09T07:51:29.081 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-05-09T07:51:29.081 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-05-09T07:51:29.081 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-05-09T07:51:29.081 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-05-09T07:51:29.081 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-05-09T07:51:29.081 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-05-09T07:51:29.082 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-05-09T07:51:29.082 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-05-09T07:51:29.082 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty 2024-05-09T07:51:29.082 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless 2024-05-09T07:51:29.082 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-05-09T07:51:29.082 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_gsm.ko 2024-05-09T07:51:29.082 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_hdlc.ko 2024-05-09T07:51:29.082 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/nozomi.ko 2024-05-09T07:51:29.082 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial 2024-05-09T07:51:29.082 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250 2024-05-09T07:51:29.082 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-05-09T07:51:29.082 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/arc_uart.ko 2024-05-09T07:51:29.082 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm 2024-05-09T07:51:29.082 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm/jsm.ko 2024-05-09T07:51:29.083 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs 2024-05-09T07:51:29.083 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core 2024-05-09T07:51:29.083 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core/ufshcd-core.ko 2024-05-09T07:51:29.083 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host 2024-05-09T07:51:29.083 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-05-09T07:51:29.083 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio 2024-05-09T07:51:29.083 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio.ko 2024-05-09T07:51:29.083 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_aec.ko 2024-05-09T07:51:29.083 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_cif.ko 2024-05-09T07:51:29.083 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_pci_generic.ko 2024-05-09T07:51:29.083 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_sercos3.ko 2024-05-09T07:51:29.083 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb 2024-05-09T07:51:29.083 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class 2024-05-09T07:51:29.083 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-acm.ko 2024-05-09T07:51:29.084 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-wdm.ko 2024-05-09T07:51:29.084 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usblp.ko 2024-05-09T07:51:29.084 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usbtmc.ko 2024-05-09T07:51:29.084 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host 2024-05-09T07:51:29.084 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/sl811-hcd.ko 2024-05-09T07:51:29.084 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image 2024-05-09T07:51:29.084 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/mdc800.ko 2024-05-09T07:51:29.084 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/microtek.ko 2024-05-09T07:51:29.084 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc 2024-05-09T07:51:29.084 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/adutux.ko 2024-05-09T07:51:29.084 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/appledisplay.ko 2024-05-09T07:51:29.084 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi26.ko 2024-05-09T07:51:29.085 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi62.ko 2024-05-09T07:51:29.085 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ezusb.ko 2024-05-09T07:51:29.085 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/idmouse.ko 2024-05-09T07:51:29.085 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/iowarrior.ko 2024-05-09T07:51:29.085 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/isight_firmware.ko 2024-05-09T07:51:29.085 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ldusb.ko 2024-05-09T07:51:29.085 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/legousbtower.ko 2024-05-09T07:51:29.085 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga 2024-05-09T07:51:29.085 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-05-09T07:51:29.085 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/trancevibrator.ko 2024-05-09T07:51:29.085 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usb3503.ko 2024-05-09T07:51:29.085 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usblcd.ko 2024-05-09T07:51:29.085 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usbsevseg.ko 2024-05-09T07:51:29.086 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/uss720.ko 2024-05-09T07:51:29.086 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/yurex.ko 2024-05-09T07:51:29.086 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy 2024-05-09T07:51:29.086 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy/phy-generic.ko 2024-05-09T07:51:29.086 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial 2024-05-09T07:51:29.086 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/aircable.ko 2024-05-09T07:51:29.086 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ark3116.ko 2024-05-09T07:51:29.086 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/belkin_sa.ko 2024-05-09T07:51:29.086 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ch341.ko 2024-05-09T07:51:29.086 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cp210x.ko 2024-05-09T07:51:29.086 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cyberjack.ko 2024-05-09T07:51:29.087 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cypress_m8.ko 2024-05-09T07:51:29.087 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/digi_acceleport.ko 2024-05-09T07:51:29.087 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/empeg.ko 2024-05-09T07:51:29.087 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ftdi_sio.ko 2024-05-09T07:51:29.087 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/garmin_gps.ko 2024-05-09T07:51:29.087 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_edgeport.ko 2024-05-09T07:51:29.087 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_ti.ko 2024-05-09T07:51:29.087 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipaq.ko 2024-05-09T07:51:29.087 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipw.ko 2024-05-09T07:51:29.087 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ir-usb.ko 2024-05-09T07:51:29.087 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-05-09T07:51:29.087 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan.ko 2024-05-09T07:51:29.088 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan_pda.ko 2024-05-09T07:51:29.088 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kl5kusb105.ko 2024-05-09T07:51:29.088 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kobil_sct.ko 2024-05-09T07:51:29.088 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mct_u232.ko 2024-05-09T07:51:29.088 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7720.ko 2024-05-09T07:51:29.088 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7840.ko 2024-05-09T07:51:29.088 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/navman.ko 2024-05-09T07:51:29.088 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/omninet.ko 2024-05-09T07:51:29.088 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/opticon.ko 2024-05-09T07:51:29.088 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/option.ko 2024-05-09T07:51:29.088 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/oti6858.ko 2024-05-09T07:51:29.088 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/pl2303.ko 2024-05-09T07:51:29.088 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcaux.ko 2024-05-09T07:51:29.089 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcserial.ko 2024-05-09T07:51:29.089 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/quatech2.ko 2024-05-09T07:51:29.089 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/safe_serial.ko 2024-05-09T07:51:29.089 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/sierra.ko 2024-05-09T07:51:29.089 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/spcp8x5.ko 2024-05-09T07:51:29.089 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ssu100.ko 2024-05-09T07:51:29.089 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/symbolserial.ko 2024-05-09T07:51:29.089 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-05-09T07:51:29.089 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-05-09T07:51:29.089 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_debug.ko 2024-05-09T07:51:29.089 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_wwan.ko 2024-05-09T07:51:29.089 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/visor.ko 2024-05-09T07:51:29.090 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/whiteheat.ko 2024-05-09T07:51:29.090 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/xsens_mt.ko 2024-05-09T07:51:29.090 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage 2024-05-09T07:51:29.090 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/uas.ko 2024-05-09T07:51:29.090 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-alauda.ko 2024-05-09T07:51:29.090 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-cypress.ko 2024-05-09T07:51:29.090 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-datafab.ko 2024-05-09T07:51:29.090 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-05-09T07:51:29.090 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-freecom.ko 2024-05-09T07:51:29.090 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-isd200.ko 2024-05-09T07:51:29.090 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-05-09T07:51:29.090 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-karma.ko 2024-05-09T07:51:29.090 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-onetouch.ko 2024-05-09T07:51:29.091 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-realtek.ko 2024-05-09T07:51:29.091 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr09.ko 2024-05-09T07:51:29.091 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr55.ko 2024-05-09T07:51:29.091 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-usbat.ko 2024-05-09T07:51:29.091 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/usb-storage.ko 2024-05-09T07:51:29.091 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip 2024-05-09T07:51:29.091 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-core.ko 2024-05-09T07:51:29.091 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-host.ko 2024-05-09T07:51:29.091 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-05-09T07:51:29.091 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio 2024-05-09T07:51:29.091 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci 2024-05-09T07:51:29.091 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-05-09T07:51:29.091 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci.ko 2024-05-09T07:51:29.091 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio.ko 2024-05-09T07:51:29.093 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-05-09T07:51:29.093 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost 2024-05-09T07:51:29.094 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost.ko 2024-05-09T07:51:29.094 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_iotlb.ko 2024-05-09T07:51:29.094 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_net.ko 2024-05-09T07:51:29.094 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_scsi.ko 2024-05-09T07:51:29.094 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video 2024-05-09T07:51:29.094 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight 2024-05-09T07:51:29.094 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/apple_bl.ko 2024-05-09T07:51:29.094 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/lcd.ko 2024-05-09T07:51:29.094 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/platform_lcd.ko 2024-05-09T07:51:29.094 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio 2024-05-09T07:51:29.094 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio.ko 2024-05-09T07:51:29.095 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_balloon.ko 2024-05-09T07:51:29.095 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_mmio.ko 2024-05-09T07:51:29.095 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci.ko 2024-05-09T07:51:29.095 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-05-09T07:51:29.095 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-05-09T07:51:29.095 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_ring.ko 2024-05-09T07:51:29.095 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1 2024-05-09T07:51:29.095 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters 2024-05-09T07:51:29.095 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2482.ko 2024-05-09T07:51:29.095 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2490.ko 2024-05-09T07:51:29.095 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves 2024-05-09T07:51:29.095 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-05-09T07:51:29.095 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-05-09T07:51:29.095 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-05-09T07:51:29.096 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-05-09T07:51:29.096 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-05-09T07:51:29.096 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-05-09T07:51:29.096 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-05-09T07:51:29.096 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-05-09T07:51:29.096 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-05-09T07:51:29.096 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_smem.ko 2024-05-09T07:51:29.096 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_therm.ko 2024-05-09T07:51:29.096 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/wire.ko 2024-05-09T07:51:29.096 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog 2024-05-09T07:51:29.096 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim1535_wdt.ko 2024-05-09T07:51:29.096 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim7101_wdt.ko 2024-05-09T07:51:29.096 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/f71808e_wdt.ko 2024-05-09T07:51:29.097 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/hpwdt.ko 2024-05-09T07:51:29.097 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/i6300esb.ko 2024-05-09T07:51:29.097 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-05-09T07:51:29.097 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_wdt.ko 2024-05-09T07:51:29.097 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ib700wdt.ko 2024-05-09T07:51:29.097 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ibmasr.ko 2024-05-09T07:51:29.097 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-05-09T07:51:29.097 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it8712f_wdt.ko 2024-05-09T07:51:29.097 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it87_wdt.ko 2024-05-09T07:51:29.097 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/machzwd.ko 2024-05-09T07:51:29.097 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/nv_tco.ko 2024-05-09T07:51:29.097 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_pci.ko 2024-05-09T07:51:29.097 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_usb.ko 2024-05-09T07:51:29.097 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-05-09T07:51:29.098 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sch311x_wdt.ko 2024-05-09T07:51:29.098 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/softdog.ko 2024-05-09T07:51:29.098 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sp5100_tco.ko 2024-05-09T07:51:29.098 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/via_wdt.ko 2024-05-09T07:51:29.098 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-05-09T07:51:29.098 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83877f_wdt.ko 2024-05-09T07:51:29.098 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83977f_wdt.ko 2024-05-09T07:51:29.098 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/wdt_pci.ko 2024-05-09T07:51:29.098 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/xen_wdt.ko 2024-05-09T07:51:29.098 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen 2024-05-09T07:51:29.098 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-acpi-processor.ko 2024-05-09T07:51:29.098 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-evtchn.ko 2024-05-09T07:51:29.098 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntalloc.ko 2024-05-09T07:51:29.098 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntdev.ko 2024-05-09T07:51:29.099 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback 2024-05-09T07:51:29.099 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-05-09T07:51:29.099 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-privcmd.ko 2024-05-09T07:51:29.099 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs 2024-05-09T07:51:29.099 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs/xenfs.ko 2024-05-09T07:51:29.099 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs 2024-05-09T07:51:29.099 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p 2024-05-09T07:51:29.099 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p/9p.ko 2024-05-09T07:51:29.099 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs 2024-05-09T07:51:29.099 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs/affs.ko 2024-05-09T07:51:29.099 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs 2024-05-09T07:51:29.099 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs/befs.ko 2024-05-09T07:51:29.099 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/binfmt_misc.ko 2024-05-09T07:51:29.099 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs 2024-05-09T07:51:29.100 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs/btrfs.ko 2024-05-09T07:51:29.100 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles 2024-05-09T07:51:29.100 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles/cachefiles.ko 2024-05-09T07:51:29.100 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph 2024-05-09T07:51:29.100 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph/ceph.ko 2024-05-09T07:51:29.100 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda 2024-05-09T07:51:29.100 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda/coda.ko 2024-05-09T07:51:29.100 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs 2024-05-09T07:51:29.100 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs/cramfs.ko 2024-05-09T07:51:29.100 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm 2024-05-09T07:51:29.100 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm/dlm.ko 2024-05-09T07:51:29.100 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs 2024-05-09T07:51:29.100 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs/ecryptfs.ko 2024-05-09T07:51:29.101 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs 2024-05-09T07:51:29.101 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs/f2fs.ko 2024-05-09T07:51:29.101 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat 2024-05-09T07:51:29.101 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/fat.ko 2024-05-09T07:51:29.101 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/msdos.ko 2024-05-09T07:51:29.101 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/vfat.ko 2024-05-09T07:51:29.101 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse 2024-05-09T07:51:29.101 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/cuse.ko 2024-05-09T07:51:29.101 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/fuse.ko 2024-05-09T07:51:29.101 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2 2024-05-09T07:51:29.101 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2/gfs2.ko 2024-05-09T07:51:29.101 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs 2024-05-09T07:51:29.101 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs/hfs.ko 2024-05-09T07:51:29.101 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus 2024-05-09T07:51:29.102 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus/hfsplus.ko 2024-05-09T07:51:29.102 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs 2024-05-09T07:51:29.102 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs/isofs.ko 2024-05-09T07:51:29.102 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs 2024-05-09T07:51:29.102 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs/jfs.ko 2024-05-09T07:51:29.102 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd 2024-05-09T07:51:29.102 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd/lockd.ko 2024-05-09T07:51:29.102 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix 2024-05-09T07:51:29.102 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix/minix.ko 2024-05-09T07:51:29.102 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs 2024-05-09T07:51:29.102 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs/netfs.ko 2024-05-09T07:51:29.102 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs 2024-05-09T07:51:29.102 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout 2024-05-09T07:51:29.103 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-05-09T07:51:29.103 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout 2024-05-09T07:51:29.103 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-05-09T07:51:29.103 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout 2024-05-09T07:51:29.103 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-05-09T07:51:29.103 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfs.ko 2024-05-09T07:51:29.103 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv3.ko 2024-05-09T07:51:29.103 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv4.ko 2024-05-09T07:51:29.103 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common 2024-05-09T07:51:29.103 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/grace.ko 2024-05-09T07:51:29.103 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/nfs_acl.ko 2024-05-09T07:51:29.103 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd 2024-05-09T07:51:29.103 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd/nfsd.ko 2024-05-09T07:51:29.103 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2 2024-05-09T07:51:29.104 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2/nilfs2.ko 2024-05-09T07:51:29.104 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls 2024-05-09T07:51:29.104 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-celtic.ko 2024-05-09T07:51:29.104 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-centeuro.ko 2024-05-09T07:51:29.104 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-croatian.ko 2024-05-09T07:51:29.104 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-cyrillic.ko 2024-05-09T07:51:29.104 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-gaelic.ko 2024-05-09T07:51:29.104 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-greek.ko 2024-05-09T07:51:29.104 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-iceland.ko 2024-05-09T07:51:29.104 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-inuit.ko 2024-05-09T07:51:29.104 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-roman.ko 2024-05-09T07:51:29.104 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-romanian.ko 2024-05-09T07:51:29.104 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-turkish.ko 2024-05-09T07:51:29.104 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1250.ko 2024-05-09T07:51:29.105 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1251.ko 2024-05-09T07:51:29.105 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1255.ko 2024-05-09T07:51:29.105 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp737.ko 2024-05-09T07:51:29.105 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp775.ko 2024-05-09T07:51:29.105 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp850.ko 2024-05-09T07:51:29.105 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp852.ko 2024-05-09T07:51:29.105 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp855.ko 2024-05-09T07:51:29.105 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp857.ko 2024-05-09T07:51:29.105 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp860.ko 2024-05-09T07:51:29.105 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp861.ko 2024-05-09T07:51:29.105 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp862.ko 2024-05-09T07:51:29.105 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp863.ko 2024-05-09T07:51:29.105 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp864.ko 2024-05-09T07:51:29.105 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp865.ko 2024-05-09T07:51:29.106 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp866.ko 2024-05-09T07:51:29.106 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp869.ko 2024-05-09T07:51:29.106 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp874.ko 2024-05-09T07:51:29.106 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp932.ko 2024-05-09T07:51:29.106 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp936.ko 2024-05-09T07:51:29.106 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp949.ko 2024-05-09T07:51:29.106 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp950.ko 2024-05-09T07:51:29.106 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_euc-jp.ko 2024-05-09T07:51:29.106 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-1.ko 2024-05-09T07:51:29.106 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-13.ko 2024-05-09T07:51:29.106 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-14.ko 2024-05-09T07:51:29.106 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-15.ko 2024-05-09T07:51:29.106 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-2.ko 2024-05-09T07:51:29.106 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-3.ko 2024-05-09T07:51:29.107 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-4.ko 2024-05-09T07:51:29.107 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-5.ko 2024-05-09T07:51:29.107 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-6.ko 2024-05-09T07:51:29.107 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-7.ko 2024-05-09T07:51:29.107 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-9.ko 2024-05-09T07:51:29.107 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-r.ko 2024-05-09T07:51:29.107 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-ru.ko 2024-05-09T07:51:29.107 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-u.ko 2024-05-09T07:51:29.107 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_ucs2_utils.ko 2024-05-09T07:51:29.107 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_utf8.ko 2024-05-09T07:51:29.107 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2 2024-05-09T07:51:29.107 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster 2024-05-09T07:51:29.107 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-05-09T07:51:29.108 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm 2024-05-09T07:51:29.108 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-05-09T07:51:29.108 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs 2024-05-09T07:51:29.108 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-05-09T07:51:29.108 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2.ko 2024-05-09T07:51:29.108 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-05-09T07:51:29.108 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-05-09T07:51:29.108 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-05-09T07:51:29.108 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs 2024-05-09T07:51:29.108 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs/overlay.ko 2024-05-09T07:51:29.108 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore 2024-05-09T07:51:29.108 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore/ramoops.ko 2024-05-09T07:51:29.108 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs 2024-05-09T07:51:29.108 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs/reiserfs.ko 2024-05-09T07:51:29.109 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs 2024-05-09T07:51:29.109 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs/romfs.ko 2024-05-09T07:51:29.109 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb 2024-05-09T07:51:29.109 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client 2024-05-09T07:51:29.109 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client/cifs.ko 2024-05-09T07:51:29.109 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common 2024-05-09T07:51:29.109 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_arc4.ko 2024-05-09T07:51:29.109 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_md4.ko 2024-05-09T07:51:29.109 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs 2024-05-09T07:51:29.109 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs/squashfs.ko 2024-05-09T07:51:29.109 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv 2024-05-09T07:51:29.109 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv/sysv.ko 2024-05-09T07:51:29.109 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf 2024-05-09T07:51:29.110 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf/udf.ko 2024-05-09T07:51:29.110 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs 2024-05-09T07:51:29.110 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs/ufs.ko 2024-05-09T07:51:29.110 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs 2024-05-09T07:51:29.110 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs/xfs.ko 2024-05-09T07:51:29.110 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel 2024-05-09T07:51:29.110 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu 2024-05-09T07:51:29.110 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu/rcutorture.ko 2024-05-09T07:51:29.110 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/torture.ko 2024-05-09T07:51:29.110 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace 2024-05-09T07:51:29.110 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-05-09T07:51:29.110 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib 2024-05-09T07:51:29.110 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/asn1_encoder.ko 2024-05-09T07:51:29.110 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc-itu-t.ko 2024-05-09T07:51:29.111 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc8.ko 2024-05-09T07:51:29.111 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto 2024-05-09T07:51:29.111 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libarc4.ko 2024-05-09T07:51:29.111 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libdes.ko 2024-05-09T07:51:29.111 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/libcrc32c.ko 2024-05-09T07:51:29.111 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lru_cache.ko 2024-05-09T07:51:29.111 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4 2024-05-09T07:51:29.111 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4_compress.ko 2024-05-09T07:51:29.111 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4hc_compress.ko 2024-05-09T07:51:29.111 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math 2024-05-09T07:51:29.111 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math/cordic.ko 2024-05-09T07:51:29.111 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6 2024-05-09T07:51:29.111 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6/raid6_pq.ko 2024-05-09T07:51:29.111 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon 2024-05-09T07:51:29.112 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon/reed_solomon.ko 2024-05-09T07:51:29.112 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_bm.ko 2024-05-09T07:51:29.112 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_fsm.ko 2024-05-09T07:51:29.112 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_kmp.ko 2024-05-09T07:51:29.112 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd 2024-05-09T07:51:29.112 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd/zstd_compress.ko 2024-05-09T07:51:29.112 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm 2024-05-09T07:51:29.112 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm/hwpoison-inject.ko 2024-05-09T07:51:29.112 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net 2024-05-09T07:51:29.112 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan 2024-05-09T07:51:29.112 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/6lowpan.ko 2024-05-09T07:51:29.112 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_dest.ko 2024-05-09T07:51:29.112 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_fragment.ko 2024-05-09T07:51:29.112 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_hop.ko 2024-05-09T07:51:29.113 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_ipv6.ko 2024-05-09T07:51:29.113 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_mobility.ko 2024-05-09T07:51:29.113 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_routing.ko 2024-05-09T07:51:29.113 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_udp.ko 2024-05-09T07:51:29.113 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802 2024-05-09T07:51:29.113 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/garp.ko 2024-05-09T07:51:29.113 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/mrp.ko 2024-05-09T07:51:29.113 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/p8022.ko 2024-05-09T07:51:29.113 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/psnap.ko 2024-05-09T07:51:29.113 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/stp.ko 2024-05-09T07:51:29.113 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q 2024-05-09T07:51:29.113 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q/8021q.ko 2024-05-09T07:51:29.113 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p 2024-05-09T07:51:29.113 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet.ko 2024-05-09T07:51:29.114 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_fd.ko 2024-05-09T07:51:29.114 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_rdma.ko 2024-05-09T07:51:29.114 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_virtio.ko 2024-05-09T07:51:29.114 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv 2024-05-09T07:51:29.114 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv/batman-adv.ko 2024-05-09T07:51:29.114 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge 2024-05-09T07:51:29.114 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/br_netfilter.ko 2024-05-09T07:51:29.114 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/bridge.ko 2024-05-09T07:51:29.114 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter 2024-05-09T07:51:29.114 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-05-09T07:51:29.114 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_among.ko 2024-05-09T07:51:29.114 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arp.ko 2024-05-09T07:51:29.114 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-05-09T07:51:29.114 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-05-09T07:51:29.115 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip.ko 2024-05-09T07:51:29.115 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-05-09T07:51:29.115 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_limit.ko 2024-05-09T07:51:29.115 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_log.ko 2024-05-09T07:51:29.115 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark.ko 2024-05-09T07:51:29.115 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-05-09T07:51:29.115 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-05-09T07:51:29.115 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-05-09T07:51:29.115 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-05-09T07:51:29.115 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_snat.ko 2024-05-09T07:51:29.115 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_stp.ko 2024-05-09T07:51:29.115 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-05-09T07:51:29.115 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-05-09T07:51:29.115 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-05-09T07:51:29.116 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-05-09T07:51:29.116 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtables.ko 2024-05-09T07:51:29.116 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-05-09T07:51:29.116 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-05-09T07:51:29.116 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-05-09T07:51:29.116 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph 2024-05-09T07:51:29.116 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph/libceph.ko 2024-05-09T07:51:29.116 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core 2024-05-09T07:51:29.116 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/failover.ko 2024-05-09T07:51:29.116 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/pktgen.ko 2024-05-09T07:51:29.116 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp 2024-05-09T07:51:29.116 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp.ko 2024-05-09T07:51:29.116 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_diag.ko 2024-05-09T07:51:29.116 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv4.ko 2024-05-09T07:51:29.116 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv6.ko 2024-05-09T07:51:29.117 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver 2024-05-09T07:51:29.117 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver/dns_resolver.ko 2024-05-09T07:51:29.117 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa 2024-05-09T07:51:29.117 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/dsa_core.ko 2024-05-09T07:51:29.117 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_brcm.ko 2024-05-09T07:51:29.117 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_dsa.ko 2024-05-09T07:51:29.117 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_none.ko 2024-05-09T07:51:29.117 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_trailer.ko 2024-05-09T07:51:29.117 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154 2024-05-09T07:51:29.117 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan 2024-05-09T07:51:29.117 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-05-09T07:51:29.117 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154.ko 2024-05-09T07:51:29.117 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154_socket.ko 2024-05-09T07:51:29.118 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4 2024-05-09T07:51:29.118 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ah4.ko 2024-05-09T07:51:29.118 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/esp4.ko 2024-05-09T07:51:29.118 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/fou.ko 2024-05-09T07:51:29.118 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/gre.ko 2024-05-09T07:51:29.118 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/inet_diag.ko 2024-05-09T07:51:29.118 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_gre.ko 2024-05-09T07:51:29.118 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_tunnel.ko 2024-05-09T07:51:29.118 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_vti.ko 2024-05-09T07:51:29.118 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipcomp.ko 2024-05-09T07:51:29.118 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipip.ko 2024-05-09T07:51:29.118 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter 2024-05-09T07:51:29.118 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arp_tables.ko 2024-05-09T07:51:29.119 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-05-09T07:51:29.119 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-05-09T07:51:29.119 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-05-09T07:51:29.119 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-05-09T07:51:29.119 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-05-09T07:51:29.119 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-05-09T07:51:29.119 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-05-09T07:51:29.119 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-05-09T07:51:29.119 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-05-09T07:51:29.119 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_security.ko 2024-05-09T07:51:29.119 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-05-09T07:51:29.119 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-05-09T07:51:29.119 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-05-09T07:51:29.120 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-05-09T07:51:29.120 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-05-09T07:51:29.120 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-05-09T07:51:29.120 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-05-09T07:51:29.120 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-05-09T07:51:29.120 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-05-09T07:51:29.120 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-05-09T07:51:29.120 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_bic.ko 2024-05-09T07:51:29.120 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_dctcp.ko 2024-05-09T07:51:29.120 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_diag.ko 2024-05-09T07:51:29.120 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_highspeed.ko 2024-05-09T07:51:29.120 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_htcp.ko 2024-05-09T07:51:29.120 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_hybla.ko 2024-05-09T07:51:29.121 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_illinois.ko 2024-05-09T07:51:29.121 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_lp.ko 2024-05-09T07:51:29.121 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_scalable.ko 2024-05-09T07:51:29.121 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_vegas.ko 2024-05-09T07:51:29.121 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_veno.ko 2024-05-09T07:51:29.121 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_westwood.ko 2024-05-09T07:51:29.121 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_yeah.ko 2024-05-09T07:51:29.121 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tunnel4.ko 2024-05-09T07:51:29.121 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_diag.ko 2024-05-09T07:51:29.121 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_tunnel.ko 2024-05-09T07:51:29.121 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/xfrm4_tunnel.ko 2024-05-09T07:51:29.121 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6 2024-05-09T07:51:29.121 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ah6.ko 2024-05-09T07:51:29.121 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/esp6.ko 2024-05-09T07:51:29.122 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/fou6.ko 2024-05-09T07:51:29.122 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_tunnel.ko 2024-05-09T07:51:29.122 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-05-09T07:51:29.122 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_vti.ko 2024-05-09T07:51:29.122 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ipcomp6.ko 2024-05-09T07:51:29.122 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter 2024-05-09T07:51:29.122 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-05-09T07:51:29.122 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-05-09T07:51:29.122 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-05-09T07:51:29.122 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-05-09T07:51:29.122 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-05-09T07:51:29.122 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-05-09T07:51:29.122 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-05-09T07:51:29.123 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-05-09T07:51:29.123 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-05-09T07:51:29.123 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-05-09T07:51:29.123 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-05-09T07:51:29.123 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-05-09T07:51:29.123 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-05-09T07:51:29.123 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-05-09T07:51:29.123 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-05-09T07:51:29.123 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-05-09T07:51:29.123 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-05-09T07:51:29.123 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-05-09T07:51:29.123 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-05-09T07:51:29.123 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-05-09T07:51:29.124 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-05-09T07:51:29.124 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-05-09T07:51:29.124 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-05-09T07:51:29.124 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-05-09T07:51:29.124 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/sit.ko 2024-05-09T07:51:29.124 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/tunnel6.ko 2024-05-09T07:51:29.124 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/xfrm6_tunnel.ko 2024-05-09T07:51:29.124 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key 2024-05-09T07:51:29.124 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key/af_key.ko 2024-05-09T07:51:29.124 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp 2024-05-09T07:51:29.124 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_core.ko 2024-05-09T07:51:29.124 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_debugfs.ko 2024-05-09T07:51:29.124 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_eth.ko 2024-05-09T07:51:29.125 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip.ko 2024-05-09T07:51:29.125 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip6.ko 2024-05-09T07:51:29.125 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_netlink.ko 2024-05-09T07:51:29.125 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ppp.ko 2024-05-09T07:51:29.125 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc 2024-05-09T07:51:29.125 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc/llc.ko 2024-05-09T07:51:29.125 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154 2024-05-09T07:51:29.125 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154/mac802154.ko 2024-05-09T07:51:29.125 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls 2024-05-09T07:51:29.125 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls/mpls_gso.ko 2024-05-09T07:51:29.125 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter 2024-05-09T07:51:29.125 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset 2024-05-09T07:51:29.125 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set.ko 2024-05-09T07:51:29.126 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-05-09T07:51:29.126 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-05-09T07:51:29.126 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-05-09T07:51:29.126 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-05-09T07:51:29.126 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-05-09T07:51:29.126 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-05-09T07:51:29.126 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-05-09T07:51:29.126 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-05-09T07:51:29.126 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-05-09T07:51:29.126 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-05-09T07:51:29.126 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-05-09T07:51:29.126 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-05-09T07:51:29.126 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-05-09T07:51:29.127 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-05-09T07:51:29.127 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-05-09T07:51:29.127 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs 2024-05-09T07:51:29.127 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs.ko 2024-05-09T07:51:29.127 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-05-09T07:51:29.127 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-05-09T07:51:29.127 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-05-09T07:51:29.127 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-05-09T07:51:29.127 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-05-09T07:51:29.127 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-05-09T07:51:29.127 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-05-09T07:51:29.127 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-05-09T07:51:29.127 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-05-09T07:51:29.127 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-05-09T07:51:29.128 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-05-09T07:51:29.128 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-05-09T07:51:29.128 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-05-09T07:51:29.128 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conncount.ko 2024-05-09T07:51:29.128 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack.ko 2024-05-09T07:51:29.128 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-05-09T07:51:29.128 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-05-09T07:51:29.128 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-05-09T07:51:29.128 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_h323.ko 2024-05-09T07:51:29.128 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_irc.ko 2024-05-09T07:51:29.128 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-05-09T07:51:29.128 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-05-09T07:51:29.128 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-05-09T07:51:29.128 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sane.ko 2024-05-09T07:51:29.129 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sip.ko 2024-05-09T07:51:29.129 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-05-09T07:51:29.129 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-05-09T07:51:29.129 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_dup_netdev.ko 2024-05-09T07:51:29.129 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table.ko 2024-05-09T07:51:29.129 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table_inet.ko 2024-05-09T07:51:29.129 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_log_syslog.ko 2024-05-09T07:51:29.129 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat.ko 2024-05-09T07:51:29.129 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_amanda.ko 2024-05-09T07:51:29.129 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_ftp.ko 2024-05-09T07:51:29.129 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_irc.ko 2024-05-09T07:51:29.129 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_sip.ko 2024-05-09T07:51:29.129 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_tftp.ko 2024-05-09T07:51:29.130 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_synproxy_core.ko 2024-05-09T07:51:29.130 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_tables.ko 2024-05-09T07:51:29.130 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink.ko 2024-05-09T07:51:29.130 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_acct.ko 2024-05-09T07:51:29.130 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_log.ko 2024-05-09T07:51:29.130 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_osf.ko 2024-05-09T07:51:29.130 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_queue.ko 2024-05-09T07:51:29.130 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_chain_nat.ko 2024-05-09T07:51:29.130 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_compat.ko 2024-05-09T07:51:29.130 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_ct.ko 2024-05-09T07:51:29.130 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_dup_netdev.ko 2024-05-09T07:51:29.130 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fib.ko 2024-05-09T07:51:29.130 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fwd_netdev.ko 2024-05-09T07:51:29.130 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_hash.ko 2024-05-09T07:51:29.131 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_limit.ko 2024-05-09T07:51:29.131 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_log.ko 2024-05-09T07:51:29.131 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_masq.ko 2024-05-09T07:51:29.131 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_nat.ko 2024-05-09T07:51:29.131 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_numgen.ko 2024-05-09T07:51:29.131 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_queue.ko 2024-05-09T07:51:29.131 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_quota.ko 2024-05-09T07:51:29.131 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_redir.ko 2024-05-09T07:51:29.131 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject.ko 2024-05-09T07:51:29.131 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject_inet.ko 2024-05-09T07:51:29.131 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_socket.ko 2024-05-09T07:51:29.131 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_tunnel.ko 2024-05-09T07:51:29.131 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_AUDIT.ko 2024-05-09T07:51:29.131 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CHECKSUM.ko 2024-05-09T07:51:29.132 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CLASSIFY.ko 2024-05-09T07:51:29.132 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-05-09T07:51:29.132 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CT.ko 2024-05-09T07:51:29.132 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_DSCP.ko 2024-05-09T07:51:29.132 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HL.ko 2024-05-09T07:51:29.132 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HMARK.ko 2024-05-09T07:51:29.132 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_IDLETIMER.ko 2024-05-09T07:51:29.132 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LED.ko 2024-05-09T07:51:29.132 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LOG.ko 2024-05-09T07:51:29.132 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_MASQUERADE.ko 2024-05-09T07:51:29.132 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NETMAP.ko 2024-05-09T07:51:29.132 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFLOG.ko 2024-05-09T07:51:29.132 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFQUEUE.ko 2024-05-09T07:51:29.133 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_RATEEST.ko 2024-05-09T07:51:29.133 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_REDIRECT.ko 2024-05-09T07:51:29.133 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_SECMARK.ko 2024-05-09T07:51:29.133 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPMSS.ko 2024-05-09T07:51:29.133 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-05-09T07:51:29.133 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TEE.ko 2024-05-09T07:51:29.133 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TPROXY.ko 2024-05-09T07:51:29.133 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TRACE.ko 2024-05-09T07:51:29.133 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_addrtype.ko 2024-05-09T07:51:29.133 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_bpf.ko 2024-05-09T07:51:29.133 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cgroup.ko 2024-05-09T07:51:29.133 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cluster.ko 2024-05-09T07:51:29.133 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_comment.ko 2024-05-09T07:51:29.133 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connbytes.ko 2024-05-09T07:51:29.134 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlabel.ko 2024-05-09T07:51:29.134 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlimit.ko 2024-05-09T07:51:29.134 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connmark.ko 2024-05-09T07:51:29.134 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_conntrack.ko 2024-05-09T07:51:29.134 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cpu.ko 2024-05-09T07:51:29.134 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dccp.ko 2024-05-09T07:51:29.134 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_devgroup.ko 2024-05-09T07:51:29.134 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dscp.ko 2024-05-09T07:51:29.134 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ecn.ko 2024-05-09T07:51:29.134 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_esp.ko 2024-05-09T07:51:29.134 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hashlimit.ko 2024-05-09T07:51:29.134 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_helper.ko 2024-05-09T07:51:29.134 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hl.ko 2024-05-09T07:51:29.134 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipcomp.ko 2024-05-09T07:51:29.135 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_iprange.ko 2024-05-09T07:51:29.135 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipvs.ko 2024-05-09T07:51:29.135 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_l2tp.ko 2024-05-09T07:51:29.135 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_length.ko 2024-05-09T07:51:29.135 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_limit.ko 2024-05-09T07:51:29.135 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mac.ko 2024-05-09T07:51:29.135 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mark.ko 2024-05-09T07:51:29.135 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_multiport.ko 2024-05-09T07:51:29.135 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nat.ko 2024-05-09T07:51:29.135 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nfacct.ko 2024-05-09T07:51:29.135 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_osf.ko 2024-05-09T07:51:29.135 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_owner.ko 2024-05-09T07:51:29.135 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_physdev.ko 2024-05-09T07:51:29.135 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_pkttype.ko 2024-05-09T07:51:29.136 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_policy.ko 2024-05-09T07:51:29.136 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_quota.ko 2024-05-09T07:51:29.136 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_rateest.ko 2024-05-09T07:51:29.136 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_realm.ko 2024-05-09T07:51:29.136 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_recent.ko 2024-05-09T07:51:29.136 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_sctp.ko 2024-05-09T07:51:29.136 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_set.ko 2024-05-09T07:51:29.136 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_socket.ko 2024-05-09T07:51:29.136 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_state.ko 2024-05-09T07:51:29.136 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_statistic.ko 2024-05-09T07:51:29.136 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_string.ko 2024-05-09T07:51:29.136 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_tcpmss.ko 2024-05-09T07:51:29.136 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_time.ko 2024-05-09T07:51:29.137 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_u32.ko 2024-05-09T07:51:29.137 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink 2024-05-09T07:51:29.137 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink/netlink_diag.ko 2024-05-09T07:51:29.137 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh 2024-05-09T07:51:29.137 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh/nsh.ko 2024-05-09T07:51:29.137 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch 2024-05-09T07:51:29.137 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/openvswitch.ko 2024-05-09T07:51:29.137 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-geneve.ko 2024-05-09T07:51:29.137 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-gre.ko 2024-05-09T07:51:29.137 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-vxlan.ko 2024-05-09T07:51:29.137 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet 2024-05-09T07:51:29.137 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet/af_packet_diag.ko 2024-05-09T07:51:29.137 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds 2024-05-09T07:51:29.137 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds.ko 2024-05-09T07:51:29.138 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_rdma.ko 2024-05-09T07:51:29.138 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_tcp.ko 2024-05-09T07:51:29.138 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched 2024-05-09T07:51:29.138 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_csum.ko 2024-05-09T07:51:29.138 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_gact.ko 2024-05-09T07:51:29.138 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_mirred.ko 2024-05-09T07:51:29.138 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_nat.ko 2024-05-09T07:51:29.138 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_pedit.ko 2024-05-09T07:51:29.138 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_police.ko 2024-05-09T07:51:29.138 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_simple.ko 2024-05-09T07:51:29.138 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_skbedit.ko 2024-05-09T07:51:29.138 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_vlan.ko 2024-05-09T07:51:29.138 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_basic.ko 2024-05-09T07:51:29.139 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_bpf.ko 2024-05-09T07:51:29.139 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_flow.ko 2024-05-09T07:51:29.139 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_fw.ko 2024-05-09T07:51:29.139 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_route.ko 2024-05-09T07:51:29.139 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_u32.ko 2024-05-09T07:51:29.139 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_cmp.ko 2024-05-09T07:51:29.139 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_ipset.ko 2024-05-09T07:51:29.139 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_meta.ko 2024-05-09T07:51:29.139 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_nbyte.ko 2024-05-09T07:51:29.139 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_text.ko 2024-05-09T07:51:29.139 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_u32.ko 2024-05-09T07:51:29.139 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_choke.ko 2024-05-09T07:51:29.139 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_codel.ko 2024-05-09T07:51:29.139 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_drr.ko 2024-05-09T07:51:29.140 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_fq.ko 2024-05-09T07:51:29.140 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_gred.ko 2024-05-09T07:51:29.140 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hfsc.ko 2024-05-09T07:51:29.140 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hhf.ko 2024-05-09T07:51:29.140 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_htb.ko 2024-05-09T07:51:29.140 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_ingress.ko 2024-05-09T07:51:29.140 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio.ko 2024-05-09T07:51:29.140 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio_lib.ko 2024-05-09T07:51:29.140 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_multiq.ko 2024-05-09T07:51:29.140 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_netem.ko 2024-05-09T07:51:29.140 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_pie.ko 2024-05-09T07:51:29.140 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_plug.ko 2024-05-09T07:51:29.140 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_prio.ko 2024-05-09T07:51:29.140 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_qfq.ko 2024-05-09T07:51:29.141 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_red.ko 2024-05-09T07:51:29.141 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfb.ko 2024-05-09T07:51:29.141 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfq.ko 2024-05-09T07:51:29.141 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_tbf.ko 2024-05-09T07:51:29.141 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_teql.ko 2024-05-09T07:51:29.141 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp 2024-05-09T07:51:29.141 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp.ko 2024-05-09T07:51:29.141 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp_diag.ko 2024-05-09T07:51:29.141 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc 2024-05-09T07:51:29.141 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss 2024-05-09T07:51:29.141 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-05-09T07:51:29.141 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-05-09T07:51:29.141 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/sunrpc.ko 2024-05-09T07:51:29.141 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma 2024-05-09T07:51:29.141 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-05-09T07:51:29.142 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc 2024-05-09T07:51:29.142 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc.ko 2024-05-09T07:51:29.142 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc_diag.ko 2024-05-09T07:51:29.142 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix 2024-05-09T07:51:29.142 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix/unix_diag.ko 2024-05-09T07:51:29.142 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock 2024-05-09T07:51:29.142 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-05-09T07:51:29.142 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-05-09T07:51:29.142 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock.ko 2024-05-09T07:51:29.142 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_diag.ko 2024-05-09T07:51:29.142 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_loopback.ko 2024-05-09T07:51:29.142 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm 2024-05-09T07:51:29.142 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm/xfrm_ipcomp.ko 2024-05-09T07:51:29.143 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security 2024-05-09T07:51:29.143 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys 2024-05-09T07:51:29.143 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys 2024-05-09T07:51:29.143 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-05-09T07:51:29.143 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys 2024-05-09T07:51:29.143 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys/trusted.ko 2024-05-09T07:51:29.143 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias 2024-05-09T07:51:29.143 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias.bin 2024-05-09T07:51:29.143 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin 2024-05-09T07:51:29.143 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.alias.bin 2024-05-09T07:51:29.143 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.bin 2024-05-09T07:51:29.143 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.modinfo 2024-05-09T07:51:29.143 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep 2024-05-09T07:51:29.143 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep.bin 2024-05-09T07:51:29.144 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.devname 2024-05-09T07:51:29.144 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.order 2024-05-09T07:51:29.144 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.softdep 2024-05-09T07:51:29.144 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols 2024-05-09T07:51:29.144 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols.bin 2024-05-09T07:51:29.144 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-05-09T07:51:29.148 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-rc3-g806ec36b2112 2024-05-09T07:51:29.148 DEBUG:teuthology.orchestra.run.smithi001:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-05-09T07:51:29.174 INFO:teuthology.orchestra.run.smithi001.stdout:postinstall scriptlet (using /bin/sh): 2024-05-09T07:51:29.174 INFO:teuthology.orchestra.run.smithi001.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-05-09T07:51:29.175 INFO:teuthology.orchestra.run.smithi001.stdout: /usr/bin/kernel-install add 6.9.0-rc3-g806ec36b2112 /lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-05-09T07:51:29.175 INFO:teuthology.orchestra.run.smithi001.stdout:fi 2024-05-09T07:51:29.175 INFO:teuthology.orchestra.run.smithi001.stdout:for file in vmlinuz System.map config; do 2024-05-09T07:51:29.175 INFO:teuthology.orchestra.run.smithi001.stdout: if ! cmp --silent "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112"; then 2024-05-09T07:51:29.175 INFO:teuthology.orchestra.run.smithi001.stdout: cp "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112" 2024-05-09T07:51:29.175 INFO:teuthology.orchestra.run.smithi001.stdout: fi 2024-05-09T07:51:29.175 INFO:teuthology.orchestra.run.smithi001.stdout:done 2024-05-09T07:51:29.175 INFO:teuthology.orchestra.run.smithi001.stdout:if [ -d "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" ] && \ 2024-05-09T07:51:29.175 INFO:teuthology.orchestra.run.smithi001.stdout: ! diff -rq "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" >/dev/null 2>&1; then 2024-05-09T07:51:29.175 INFO:teuthology.orchestra.run.smithi001.stdout: rm -rf "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-05-09T07:51:29.175 INFO:teuthology.orchestra.run.smithi001.stdout: cp -r "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-05-09T07:51:29.175 INFO:teuthology.orchestra.run.smithi001.stdout:fi 2024-05-09T07:51:29.175 INFO:teuthology.orchestra.run.smithi001.stdout:if [ ! -e "/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep" ]; then 2024-05-09T07:51:29.176 INFO:teuthology.orchestra.run.smithi001.stdout: /usr/sbin/depmod 6.9.0-rc3-g806ec36b2112 2024-05-09T07:51:29.176 INFO:teuthology.orchestra.run.smithi001.stdout:fi 2024-05-09T07:51:29.176 INFO:teuthology.orchestra.run.smithi001.stdout:preuninstall scriptlet (using /bin/sh): 2024-05-09T07:51:29.176 INFO:teuthology.orchestra.run.smithi001.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-05-09T07:51:29.176 INFO:teuthology.orchestra.run.smithi001.stdout:kernel-install remove 6.9.0-rc3-g806ec36b2112 2024-05-09T07:51:29.176 INFO:teuthology.orchestra.run.smithi001.stdout:fi 2024-05-09T07:51:29.177 DEBUG:teuthology.orchestra.run.smithi001:> sudo rpm -qi grub2-tools 2024-05-09T07:51:29.254 INFO:teuthology.orchestra.run.smithi001.stdout:Name : grub2-tools 2024-05-09T07:51:29.254 INFO:teuthology.orchestra.run.smithi001.stdout:Epoch : 1 2024-05-09T07:51:29.254 INFO:teuthology.orchestra.run.smithi001.stdout:Version : 2.06 2024-05-09T07:51:29.254 INFO:teuthology.orchestra.run.smithi001.stdout:Release : 68.el9 2024-05-09T07:51:29.254 INFO:teuthology.orchestra.run.smithi001.stdout:Architecture: x86_64 2024-05-09T07:51:29.254 INFO:teuthology.orchestra.run.smithi001.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-05-09T07:51:29.254 INFO:teuthology.orchestra.run.smithi001.stdout:Group : Unspecified 2024-05-09T07:51:29.254 INFO:teuthology.orchestra.run.smithi001.stdout:Size : 8264254 2024-05-09T07:51:29.254 INFO:teuthology.orchestra.run.smithi001.stdout:License : GPLv3+ 2024-05-09T07:51:29.255 INFO:teuthology.orchestra.run.smithi001.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-05-09T07:51:29.255 INFO:teuthology.orchestra.run.smithi001.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-05-09T07:51:29.255 INFO:teuthology.orchestra.run.smithi001.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-05-09T07:51:29.255 INFO:teuthology.orchestra.run.smithi001.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-05-09T07:51:29.255 INFO:teuthology.orchestra.run.smithi001.stdout:Packager : builder@centos.org 2024-05-09T07:51:29.255 INFO:teuthology.orchestra.run.smithi001.stdout:Vendor : CentOS 2024-05-09T07:51:29.255 INFO:teuthology.orchestra.run.smithi001.stdout:URL : http://www.gnu.org/software/grub/ 2024-05-09T07:51:29.255 INFO:teuthology.orchestra.run.smithi001.stdout:Summary : Support tools for GRUB. 2024-05-09T07:51:29.255 INFO:teuthology.orchestra.run.smithi001.stdout:Description : 2024-05-09T07:51:29.255 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T07:51:29.255 INFO:teuthology.orchestra.run.smithi001.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-05-09T07:51:29.255 INFO:teuthology.orchestra.run.smithi001.stdout:customizable bootloader with modular architecture. It supports a rich 2024-05-09T07:51:29.255 INFO:teuthology.orchestra.run.smithi001.stdout:variety of kernel formats, file systems, computer architectures and 2024-05-09T07:51:29.255 INFO:teuthology.orchestra.run.smithi001.stdout:hardware devices. 2024-05-09T07:51:29.255 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T07:51:29.256 INFO:teuthology.orchestra.run.smithi001.stdout:This subpackage provides tools for support of all platforms. 2024-05-09T07:51:29.258 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-05-09T07:51:29.258 INFO:teuthology.task.kernel:Updating grub on smithi001 to boot 6.9.0-rc3-g806ec36b2112 2024-05-09T07:51:29.258 DEBUG:teuthology.orchestra.run.smithi001:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-05-09T07:51:29.880 INFO:teuthology.orchestra.run.smithi001.stderr:Generating grub configuration file ... 2024-05-09T07:51:31.174 INFO:teuthology.orchestra.run.smithi001.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-05-09T07:51:31.199 INFO:teuthology.orchestra.run.smithi001.stderr:done 2024-05-09T07:51:31.201 DEBUG:teuthology.orchestra.run.smithi001:> mktemp 2024-05-09T07:51:31.222 INFO:teuthology.orchestra.run.smithi001.stdout:/tmp/tmp.rbKs6Bv4AT 2024-05-09T07:51:31.222 DEBUG:teuthology.orchestra.run.smithi001:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.rbKs6Bv4AT 2024-05-09T07:51:31.288 DEBUG:teuthology.orchestra.run.smithi001:> sudo chmod 0666 /tmp/tmp.rbKs6Bv4AT 2024-05-09T07:51:31.423 DEBUG:teuthology.orchestra.remote:smithi001:/tmp/tmp.rbKs6Bv4AT is 6KB 2024-05-09T07:51:31.444 DEBUG:teuthology.orchestra.run.smithi001:> rm -fr /tmp/tmp.rbKs6Bv4AT 2024-05-09T07:51:31.458 DEBUG:teuthology.orchestra.run.smithi001:> sudo /bin/ls /boot/loader/entries || true 2024-05-09T07:51:31.526 INFO:teuthology.orchestra.run.smithi001.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-05-09T07:51:31.527 INFO:teuthology.orchestra.run.smithi001.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-05-09T07:51:31.527 INFO:teuthology.orchestra.run.smithi001.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-05-09T07:51:31.527 INFO:teuthology.orchestra.run.smithi001.stdout:7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112.conf 2024-05-09T07:51:31.528 DEBUG:teuthology.orchestra.run.smithi001:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112 2024-05-09T07:51:31.681 DEBUG:teuthology.orchestra.run.smithi001:> sudo shutdown -r now 2024-05-09T07:52:01.684 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-05-09T07:52:01.685 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi001.front.sepia.ceph.com' 2024-05-09T07:52:01.686 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi001.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T07:52:20.254 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.1 2024-05-09T07:52:29.256 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi001.front.sepia.ceph.com' 2024-05-09T07:52:29.257 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi001.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T07:52:32.322 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.1 2024-05-09T07:52:44.323 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi001.front.sepia.ceph.com' 2024-05-09T07:52:44.324 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi001.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T07:53:16.200 DEBUG:teuthology.orchestra.run.smithi001:> true 2024-05-09T07:53:17.089 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi001.front.sepia.ceph.com' 2024-05-09T07:53:17.089 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.9.0-rc3-g806ec36b2112"... 2024-05-09T07:53:17.089 DEBUG:teuthology.orchestra.run.smithi001:> uname -r 2024-05-09T07:53:17.139 INFO:teuthology.orchestra.run.smithi001.stdout:6.9.0-rc3-g806ec36b2112 2024-05-09T07:53:17.139 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-rc3-g806ec36b2112 vs 6.9.0-rc3-g806ec36b2112 2024-05-09T07:53:17.140 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-05-09T07:53:17.140 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-05-09T07:53:18.143 DEBUG:teuthology.parallel:result is None 2024-05-09T07:53:18.144 INFO:teuthology.run_tasks:Running task internal.base... 2024-05-09T07:53:18.155 INFO:teuthology.task.internal:Creating test directory... 2024-05-09T07:53:18.155 DEBUG:teuthology.orchestra.run.smithi001:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-09T07:53:18.160 DEBUG:teuthology.orchestra.run.smithi007:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-09T07:53:18.204 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-05-09T07:53:18.213 INFO:teuthology.run_tasks:Running task internal.archive... 2024-05-09T07:53:18.229 INFO:teuthology.task.internal:Creating archive directory... 2024-05-09T07:53:18.229 DEBUG:teuthology.orchestra.run.smithi001:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-09T07:53:18.256 DEBUG:teuthology.orchestra.run.smithi007:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-09T07:53:18.314 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-05-09T07:53:18.327 INFO:teuthology.task.internal:Enabling coredump saving... 2024-05-09T07:53:18.327 DEBUG:teuthology.orchestra.run.smithi007:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-09T07:53:18.344 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T07:53:18.344 DEBUG:teuthology.orchestra.run.smithi001:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-09T07:53:18.350 DEBUG:teuthology.orchestra.run.smithi007:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-09T07:53:18.418 INFO:teuthology.orchestra.run.smithi007.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-09T07:53:18.431 INFO:teuthology.orchestra.run.smithi007.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-09T07:53:18.492 INFO:teuthology.orchestra.run.smithi001.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-09T07:53:18.598 INFO:teuthology.orchestra.run.smithi001.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-09T07:53:18.604 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-05-09T07:53:18.615 INFO:teuthology.task.internal:Configuring sudo... 2024-05-09T07:53:18.615 DEBUG:teuthology.orchestra.run.smithi001:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-09T07:53:18.620 DEBUG:teuthology.orchestra.run.smithi007:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-09T07:53:18.693 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-05-09T07:53:18.703 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-05-09T07:53:18.703 DEBUG:teuthology.orchestra.run.smithi001:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-09T07:53:18.707 DEBUG:teuthology.orchestra.run.smithi007:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-09T07:53:18.742 DEBUG:teuthology.orchestra.run.smithi001:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-09T07:53:18.859 DEBUG:teuthology.orchestra.run.smithi001:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-09T07:53:18.962 DEBUG:teuthology.orchestra.run.smithi001:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-09T07:53:19.032 DEBUG:teuthology.orchestra.run.smithi001:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-09T07:53:19.121 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-05-09T07:53:19.121 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-09T07:53:19.187 DEBUG:teuthology.orchestra.run.smithi007:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-09T07:53:19.233 DEBUG:teuthology.orchestra.run.smithi007:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-09T07:53:19.289 DEBUG:teuthology.orchestra.run.smithi007:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-09T07:53:19.375 DEBUG:teuthology.orchestra.run.smithi007:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-09T07:53:19.417 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-05-09T07:53:19.417 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-09T07:53:19.484 DEBUG:teuthology.orchestra.run.smithi001:> sudo service rsyslog restart 2024-05-09T07:53:19.490 DEBUG:teuthology.orchestra.run.smithi007:> sudo service rsyslog restart 2024-05-09T07:53:19.559 INFO:teuthology.orchestra.run.smithi007.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-09T07:53:19.587 INFO:teuthology.orchestra.run.smithi001.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-09T07:53:19.922 INFO:teuthology.run_tasks:Running task internal.timer... 2024-05-09T07:53:19.932 INFO:teuthology.task.internal:Starting timer... 2024-05-09T07:53:19.932 INFO:teuthology.run_tasks:Running task pcp... 2024-05-09T07:53:19.952 INFO:teuthology.run_tasks:Running task selinux... 2024-05-09T07:53:19.969 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:getty_t:s0']} 2024-05-09T07:53:19.970 DEBUG:teuthology.orchestra.run.smithi001:> sudo service auditd rotate 2024-05-09T07:53:20.071 INFO:teuthology.orchestra.run.smithi001.stdout:Rotating logs: 2024-05-09T07:53:20.075 DEBUG:teuthology.orchestra.run.smithi007:> sudo service auditd rotate 2024-05-09T07:53:20.164 INFO:teuthology.orchestra.run.smithi007.stdout:Rotating logs: 2024-05-09T07:53:20.166 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-05-09T07:53:20.166 DEBUG:teuthology.orchestra.run.smithi001:> /usr/sbin/getenforce 2024-05-09T07:53:20.254 INFO:teuthology.orchestra.run.smithi001.stdout:Permissive 2024-05-09T07:53:20.255 DEBUG:teuthology.orchestra.run.smithi007:> /usr/sbin/getenforce 2024-05-09T07:53:20.287 INFO:teuthology.orchestra.run.smithi007.stdout:Permissive 2024-05-09T07:53:20.288 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi001.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi007.front.sepia.ceph.com': 'permissive'} 2024-05-09T07:53:20.288 DEBUG:teuthology.orchestra.run.smithi001:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-09T07:53:20.356 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T07:53:20.357 DEBUG:teuthology.orchestra.run.smithi007:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-09T07:53:20.386 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T07:53:20.387 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-05-09T07:53:20.387 DEBUG:teuthology.orchestra.run.smithi001:> sudo /usr/sbin/setenforce permissive 2024-05-09T07:53:20.459 DEBUG:teuthology.orchestra.run.smithi007:> sudo /usr/sbin/setenforce permissive 2024-05-09T07:53:20.491 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-05-09T07:53:20.511 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-05-09T07:53:20.532 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-05-09T07:53:20.533 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi001.front.sepia.ceph.com,smithi007.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-05-09T07:59:10.221 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi001.front.sepia.ceph.com'), Remote(name='ubuntu@smithi007.front.sepia.ceph.com')] 2024-05-09T07:59:10.222 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi001.front.sepia.ceph.com' 2024-05-09T07:59:10.223 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi001.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T07:59:10.340 DEBUG:teuthology.orchestra.run.smithi001:> true 2024-05-09T07:59:10.498 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi001.front.sepia.ceph.com' 2024-05-09T07:59:10.499 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi007.front.sepia.ceph.com' 2024-05-09T07:59:10.499 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi007.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T07:59:10.574 DEBUG:teuthology.orchestra.run.smithi007:> true 2024-05-09T07:59:10.653 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi007.front.sepia.ceph.com' 2024-05-09T07:59:10.653 INFO:teuthology.run_tasks:Running task clock... 2024-05-09T07:59:10.666 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-05-09T07:59:10.666 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-09T07:59:10.666 DEBUG:teuthology.orchestra.run.smithi001:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-09T07:59:10.674 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-09T07:59:10.674 DEBUG:teuthology.orchestra.run.smithi007:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-09T07:59:10.721 INFO:teuthology.orchestra.run.smithi007.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-09T07:59:10.735 INFO:teuthology.orchestra.run.smithi007.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-09T07:59:10.758 INFO:teuthology.orchestra.run.smithi007.stderr:sudo: ntpd: command not found 2024-05-09T07:59:10.768 INFO:teuthology.orchestra.run.smithi001.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-09T07:59:10.769 INFO:teuthology.orchestra.run.smithi007.stdout:506 Cannot talk to daemon 2024-05-09T07:59:10.783 INFO:teuthology.orchestra.run.smithi007.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-09T07:59:10.796 INFO:teuthology.orchestra.run.smithi007.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-09T07:59:10.814 INFO:teuthology.orchestra.run.smithi001.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-09T07:59:10.851 INFO:teuthology.orchestra.run.smithi007.stderr:bash: line 1: ntpq: command not found 2024-05-09T07:59:10.855 INFO:teuthology.orchestra.run.smithi007.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-09T07:59:10.855 INFO:teuthology.orchestra.run.smithi007.stdout:=============================================================================== 2024-05-09T07:59:10.855 INFO:teuthology.orchestra.run.smithi007.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T07:59:10.855 INFO:teuthology.orchestra.run.smithi007.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T07:59:10.855 INFO:teuthology.orchestra.run.smithi007.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T07:59:10.855 INFO:teuthology.orchestra.run.smithi007.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T07:59:10.896 INFO:teuthology.orchestra.run.smithi001.stderr:sudo: ntpd: command not found 2024-05-09T07:59:10.932 INFO:teuthology.orchestra.run.smithi001.stdout:506 Cannot talk to daemon 2024-05-09T07:59:10.974 INFO:teuthology.orchestra.run.smithi001.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-09T07:59:11.017 INFO:teuthology.orchestra.run.smithi001.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-09T07:59:11.162 INFO:teuthology.orchestra.run.smithi001.stderr:bash: line 1: ntpq: command not found 2024-05-09T07:59:11.170 INFO:teuthology.orchestra.run.smithi001.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-09T07:59:11.170 INFO:teuthology.orchestra.run.smithi001.stdout:=============================================================================== 2024-05-09T07:59:11.170 INFO:teuthology.orchestra.run.smithi001.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T07:59:11.170 INFO:teuthology.orchestra.run.smithi001.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T07:59:11.170 INFO:teuthology.orchestra.run.smithi001.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T07:59:11.170 INFO:teuthology.orchestra.run.smithi001.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T07:59:11.171 INFO:teuthology.run_tasks:Running task install... 2024-05-09T07:59:11.196 DEBUG:teuthology.task.install:project ceph 2024-05-09T07:59:11.196 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '96805dcf909541c4ca9a1e90258de8ae5b6fa1bc'}} 2024-05-09T07:59:11.196 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '96805dcf909541c4ca9a1e90258de8ae5b6fa1bc'} 2024-05-09T07:59:11.196 INFO:teuthology.task.install:Using flavor: default 2024-05-09T07:59:11.202 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-05-09T07:59:11.202 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-05-09T07:59:11.202 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '96805dcf909541c4ca9a1e90258de8ae5b6fa1bc', 'tag': None, 'wait_for_package': False} 2024-05-09T07:59:11.203 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=96805dcf909541c4ca9a1e90258de8ae5b6fa1bc 2024-05-09T07:59:11.204 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '96805dcf909541c4ca9a1e90258de8ae5b6fa1bc', 'tag': None, 'wait_for_package': False} 2024-05-09T07:59:11.205 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=96805dcf909541c4ca9a1e90258de8ae5b6fa1bc 2024-05-09T07:59:11.349 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/squid/96805dcf909541c4ca9a1e90258de8ae5b6fa1bc/centos/9/flavors/default/ 2024-05-09T07:59:11.349 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2515.g96805dcf 2024-05-09T07:59:11.362 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/squid/96805dcf909541c4ca9a1e90258de8ae5b6fa1bc/centos/9/flavors/default/ 2024-05-09T07:59:11.362 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2515.g96805dcf 2024-05-09T07:59:11.467 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/squid/96805dcf909541c4ca9a1e90258de8ae5b6fa1bc/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/96805dcf909541c4ca9a1e90258de8ae5b6fa1bc/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/96805dcf909541c4ca9a1e90258de8ae5b6fa1bc/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-09T07:59:11.468 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-05-09T07:59:11.468 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-09T07:59:11.484 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/squid/96805dcf909541c4ca9a1e90258de8ae5b6fa1bc/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/96805dcf909541c4ca9a1e90258de8ae5b6fa1bc/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/96805dcf909541c4ca9a1e90258de8ae5b6fa1bc/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-09T07:59:11.484 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-05-09T07:59:11.484 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-09T07:59:11.512 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-09T07:59:11.513 DEBUG:teuthology.orchestra.run.smithi007:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/96805dcf909541c4ca9a1e90258de8ae5b6fa1bc/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-09T07:59:11.556 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-09T07:59:11.556 DEBUG:teuthology.orchestra.run.smithi001:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/96805dcf909541c4ca9a1e90258de8ae5b6fa1bc/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-09T07:59:11.582 DEBUG:teuthology.orchestra.run.smithi007:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-09T07:59:11.659 DEBUG:teuthology.orchestra.run.smithi007:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-09T07:59:11.694 DEBUG:teuthology.orchestra.run.smithi001:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-09T07:59:11.724 INFO:teuthology.orchestra.run.smithi007.stdout:check_obsoletes = 1 2024-05-09T07:59:11.726 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum clean all 2024-05-09T07:59:11.804 DEBUG:teuthology.orchestra.run.smithi001:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-09T07:59:11.886 INFO:teuthology.orchestra.run.smithi001.stdout:check_obsoletes = 1 2024-05-09T07:59:11.891 DEBUG:teuthology.orchestra.run.smithi001:> sudo yum clean all 2024-05-09T07:59:11.965 INFO:teuthology.orchestra.run.smithi007.stdout:54 files removed 2024-05-09T07:59:11.992 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-09T07:59:12.237 INFO:teuthology.orchestra.run.smithi001.stdout:54 files removed 2024-05-09T07:59:12.278 DEBUG:teuthology.orchestra.run.smithi001:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-09T07:59:12.722 INFO:teuthology.orchestra.run.smithi007.stdout:ceph packages for x86_64 183 kB/s | 83 kB 00:00 2024-05-09T07:59:13.197 INFO:teuthology.orchestra.run.smithi007.stdout:ceph noarch packages 27 kB/s | 11 kB 00:00 2024-05-09T07:59:13.232 INFO:teuthology.orchestra.run.smithi001.stdout:ceph packages for x86_64 166 kB/s | 83 kB 00:00 2024-05-09T07:59:13.581 INFO:teuthology.orchestra.run.smithi007.stdout:ceph source packages 5.4 kB/s | 1.9 kB 00:00 2024-05-09T07:59:13.774 INFO:teuthology.orchestra.run.smithi001.stdout:ceph noarch packages 26 kB/s | 11 kB 00:00 2024-05-09T07:59:14.098 INFO:teuthology.orchestra.run.smithi007.stdout:CentOS Stream 9 - BaseOS 17 MB/s | 8.1 MB 00:00 2024-05-09T07:59:14.308 INFO:teuthology.orchestra.run.smithi001.stdout:ceph source packages 4.2 kB/s | 1.9 kB 00:00 2024-05-09T07:59:14.990 INFO:teuthology.orchestra.run.smithi001.stdout:CentOS Stream 9 - BaseOS 14 MB/s | 8.1 MB 00:00 2024-05-09T07:59:18.158 INFO:teuthology.orchestra.run.smithi001.stdout:CentOS Stream 9 - AppStream 10 MB/s | 19 MB 00:01 2024-05-09T07:59:18.391 INFO:teuthology.orchestra.run.smithi007.stdout:CentOS Stream 9 - AppStream 6.1 MB/s | 19 MB 00:03 2024-05-09T07:59:22.817 INFO:teuthology.orchestra.run.smithi001.stdout:CentOS Stream 9 - CRB 9.6 MB/s | 6.2 MB 00:00 2024-05-09T07:59:24.642 INFO:teuthology.orchestra.run.smithi001.stdout:CentOS Stream 9 - Extras packages 36 kB/s | 16 kB 00:00 2024-05-09T07:59:25.566 INFO:teuthology.orchestra.run.smithi007.stdout:CentOS Stream 9 - CRB 1.8 MB/s | 6.2 MB 00:03 2024-05-09T07:59:27.283 INFO:teuthology.orchestra.run.smithi007.stdout:CentOS Stream 9 - Extras packages 31 kB/s | 16 kB 00:00 2024-05-09T07:59:31.035 INFO:teuthology.orchestra.run.smithi001.stdout:Extra Packages for Enterprise Linux 3.4 MB/s | 21 MB 00:06 2024-05-09T07:59:31.860 INFO:teuthology.orchestra.run.smithi007.stdout:Extra Packages for Enterprise Linux 4.8 MB/s | 21 MB 00:04 2024-05-09T07:59:37.260 INFO:teuthology.orchestra.run.smithi007.stdout:lab-extras 37 kB/s | 1.7 kB 00:00 2024-05-09T07:59:37.502 INFO:teuthology.orchestra.run.smithi001.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-05-09T07:59:39.638 INFO:teuthology.orchestra.run.smithi007.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-09T07:59:39.638 INFO:teuthology.orchestra.run.smithi007.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-09T07:59:39.650 INFO:teuthology.orchestra.run.smithi007.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-09T07:59:39.651 INFO:teuthology.orchestra.run.smithi007.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-09T07:59:39.651 INFO:teuthology.orchestra.run.smithi007.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-09T07:59:39.652 INFO:teuthology.orchestra.run.smithi007.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-09T07:59:39.654 INFO:teuthology.orchestra.run.smithi007.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-09T07:59:39.656 INFO:teuthology.orchestra.run.smithi007.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-09T07:59:39.657 INFO:teuthology.orchestra.run.smithi007.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-09T07:59:39.724 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-05-09T07:59:39.738 INFO:teuthology.orchestra.run.smithi007.stdout:======================================================================================= 2024-05-09T07:59:39.739 INFO:teuthology.orchestra.run.smithi007.stdout: Package Arch Version Repository Size 2024-05-09T07:59:39.739 INFO:teuthology.orchestra.run.smithi007.stdout:======================================================================================= 2024-05-09T07:59:39.739 INFO:teuthology.orchestra.run.smithi007.stdout:Installing: 2024-05-09T07:59:39.739 INFO:teuthology.orchestra.run.smithi007.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-09T07:59:39.739 INFO:teuthology.orchestra.run.smithi007.stdout: ceph x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 6.6 k 2024-05-09T07:59:39.739 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-base x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 5.4 M 2024-05-09T07:59:39.739 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-fuse x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 889 k 2024-05-09T07:59:39.739 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 145 k 2024-05-09T07:59:39.739 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 1.5 M 2024-05-09T07:59:39.739 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 142 k 2024-05-09T07:59:39.740 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 3.5 M 2024-05-09T07:59:39.740 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 7.4 M 2024-05-09T07:59:39.740 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-rook noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 49 k 2024-05-09T07:59:39.740 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-radosgw x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 10 M 2024-05-09T07:59:39.740 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-test x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 48 M 2024-05-09T07:59:39.740 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-volume noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 264 k 2024-05-09T07:59:39.740 INFO:teuthology.orchestra.run.smithi007.stdout: cephadm noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 763 k 2024-05-09T07:59:39.740 INFO:teuthology.orchestra.run.smithi007.stdout: cephfs-mirror x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 222 k 2024-05-09T07:59:39.740 INFO:teuthology.orchestra.run.smithi007.stdout: cephfs-shell x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 27 k 2024-05-09T07:59:39.741 INFO:teuthology.orchestra.run.smithi007.stdout: cephfs-top noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 18 k 2024-05-09T07:59:39.741 INFO:teuthology.orchestra.run.smithi007.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-05-09T07:59:39.741 INFO:teuthology.orchestra.run.smithi007.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-09T07:59:39.741 INFO:teuthology.orchestra.run.smithi007.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-09T07:59:39.741 INFO:teuthology.orchestra.run.smithi007.stdout: libcephfs-devel x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 32 k 2024-05-09T07:59:39.741 INFO:teuthology.orchestra.run.smithi007.stdout: libcephfs2 x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 746 k 2024-05-09T07:59:39.741 INFO:teuthology.orchestra.run.smithi007.stdout: librados-devel x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 127 k 2024-05-09T07:59:39.741 INFO:teuthology.orchestra.run.smithi007.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-09T07:59:39.741 INFO:teuthology.orchestra.run.smithi007.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-09T07:59:39.741 INFO:teuthology.orchestra.run.smithi007.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-09T07:59:39.742 INFO:teuthology.orchestra.run.smithi007.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-09T07:59:39.742 INFO:teuthology.orchestra.run.smithi007.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-09T07:59:39.742 INFO:teuthology.orchestra.run.smithi007.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-09T07:59:39.742 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cephfs x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 162 k 2024-05-09T07:59:39.742 INFO:teuthology.orchestra.run.smithi007.stdout: python3-rados x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 321 k 2024-05-09T07:59:39.742 INFO:teuthology.orchestra.run.smithi007.stdout: python3-rbd x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 298 k 2024-05-09T07:59:39.742 INFO:teuthology.orchestra.run.smithi007.stdout: python3-rgw x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 99 k 2024-05-09T07:59:39.742 INFO:teuthology.orchestra.run.smithi007.stdout: rbd-fuse x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 85 k 2024-05-09T07:59:39.742 INFO:teuthology.orchestra.run.smithi007.stdout: rbd-mirror x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 3.1 M 2024-05-09T07:59:39.742 INFO:teuthology.orchestra.run.smithi007.stdout: rbd-nbd x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 171 k 2024-05-09T07:59:39.743 INFO:teuthology.orchestra.run.smithi007.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-09T07:59:39.743 INFO:teuthology.orchestra.run.smithi007.stdout:Upgrading: 2024-05-09T07:59:39.743 INFO:teuthology.orchestra.run.smithi007.stdout: NetworkManager x86_64 1:1.47.5-1.el9 baseos 2.3 M 2024-05-09T07:59:39.743 INFO:teuthology.orchestra.run.smithi007.stdout: NetworkManager-libnm x86_64 1:1.47.5-1.el9 baseos 1.8 M 2024-05-09T07:59:39.743 INFO:teuthology.orchestra.run.smithi007.stdout: NetworkManager-team x86_64 1:1.47.5-1.el9 baseos 42 k 2024-05-09T07:59:39.743 INFO:teuthology.orchestra.run.smithi007.stdout: NetworkManager-tui x86_64 1:1.47.5-1.el9 baseos 248 k 2024-05-09T07:59:39.743 INFO:teuthology.orchestra.run.smithi007.stdout: librados2 x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 3.4 M 2024-05-09T07:59:39.743 INFO:teuthology.orchestra.run.smithi007.stdout: librbd1 x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 3.1 M 2024-05-09T07:59:39.743 INFO:teuthology.orchestra.run.smithi007.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-09T07:59:39.743 INFO:teuthology.orchestra.run.smithi007.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-09T07:59:39.743 INFO:teuthology.orchestra.run.smithi007.stdout:Installing dependencies: 2024-05-09T07:59:39.743 INFO:teuthology.orchestra.run.smithi007.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-09T07:59:39.743 INFO:teuthology.orchestra.run.smithi007.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-09T07:59:39.744 INFO:teuthology.orchestra.run.smithi007.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-09T07:59:39.744 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-common x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 21 M 2024-05-09T07:59:39.744 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 32 k 2024-05-09T07:59:39.744 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mds x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 2.4 M 2024-05-09T07:59:39.744 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 249 k 2024-05-09T07:59:39.744 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mon x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 4.7 M 2024-05-09T07:59:39.744 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-osd x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 17 M 2024-05-09T07:59:39.744 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 16 k 2024-05-09T07:59:39.744 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-selinux x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 25 k 2024-05-09T07:59:39.744 INFO:teuthology.orchestra.run.smithi007.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-09T07:59:39.744 INFO:teuthology.orchestra.run.smithi007.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-09T07:59:39.744 INFO:teuthology.orchestra.run.smithi007.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-09T07:59:39.744 INFO:teuthology.orchestra.run.smithi007.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-09T07:59:39.745 INFO:teuthology.orchestra.run.smithi007.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-09T07:59:39.745 INFO:teuthology.orchestra.run.smithi007.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-09T07:59:39.745 INFO:teuthology.orchestra.run.smithi007.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-09T07:59:39.745 INFO:teuthology.orchestra.run.smithi007.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-09T07:59:39.745 INFO:teuthology.orchestra.run.smithi007.stdout: libcephsqlite x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 163 k 2024-05-09T07:59:39.745 INFO:teuthology.orchestra.run.smithi007.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-09T07:59:39.745 INFO:teuthology.orchestra.run.smithi007.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-09T07:59:39.745 INFO:teuthology.orchestra.run.smithi007.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-09T07:59:39.745 INFO:teuthology.orchestra.run.smithi007.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-09T07:59:39.745 INFO:teuthology.orchestra.run.smithi007.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-09T07:59:39.745 INFO:teuthology.orchestra.run.smithi007.stdout: libradosstriper1 x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 500 k 2024-05-09T07:59:39.745 INFO:teuthology.orchestra.run.smithi007.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-09T07:59:39.745 INFO:teuthology.orchestra.run.smithi007.stdout: librgw2 x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 5.3 M 2024-05-09T07:59:39.746 INFO:teuthology.orchestra.run.smithi007.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-09T07:59:39.746 INFO:teuthology.orchestra.run.smithi007.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-09T07:59:39.746 INFO:teuthology.orchestra.run.smithi007.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-09T07:59:39.746 INFO:teuthology.orchestra.run.smithi007.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-09T07:59:39.746 INFO:teuthology.orchestra.run.smithi007.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-09T07:59:39.746 INFO:teuthology.orchestra.run.smithi007.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-05-09T07:59:39.746 INFO:teuthology.orchestra.run.smithi007.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-09T07:59:39.747 INFO:teuthology.orchestra.run.smithi007.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-09T07:59:39.747 INFO:teuthology.orchestra.run.smithi007.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-09T07:59:39.747 INFO:teuthology.orchestra.run.smithi007.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-09T07:59:39.747 INFO:teuthology.orchestra.run.smithi007.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-09T07:59:39.747 INFO:teuthology.orchestra.run.smithi007.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-09T07:59:39.747 INFO:teuthology.orchestra.run.smithi007.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-09T07:59:39.747 INFO:teuthology.orchestra.run.smithi007.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-09T07:59:39.747 INFO:teuthology.orchestra.run.smithi007.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-09T07:59:39.747 INFO:teuthology.orchestra.run.smithi007.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-09T07:59:39.748 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-09T07:59:39.748 INFO:teuthology.orchestra.run.smithi007.stdout: python3-ceph-argparse x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 45 k 2024-05-09T07:59:39.748 INFO:teuthology.orchestra.run.smithi007.stdout: python3-ceph-common x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 135 k 2024-05-09T07:59:39.748 INFO:teuthology.orchestra.run.smithi007.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-09T07:59:39.748 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-09T07:59:39.748 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-09T07:59:39.748 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-09T07:59:39.748 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-09T07:59:39.748 INFO:teuthology.orchestra.run.smithi007.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-05-09T07:59:39.748 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-09T07:59:39.748 INFO:teuthology.orchestra.run.smithi007.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-09T07:59:39.748 INFO:teuthology.orchestra.run.smithi007.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-09T07:59:39.748 INFO:teuthology.orchestra.run.smithi007.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-09T07:59:39.748 INFO:teuthology.orchestra.run.smithi007.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-09T07:59:39.749 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-09T07:59:39.749 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-09T07:59:39.749 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-09T07:59:39.749 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-09T07:59:39.749 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-09T07:59:39.749 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-05-09T07:59:39.749 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-09T07:59:39.749 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-09T07:59:39.749 INFO:teuthology.orchestra.run.smithi007.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-09T07:59:39.749 INFO:teuthology.orchestra.run.smithi007.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-09T07:59:39.749 INFO:teuthology.orchestra.run.smithi007.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-09T07:59:39.749 INFO:teuthology.orchestra.run.smithi007.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-09T07:59:39.750 INFO:teuthology.orchestra.run.smithi007.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-09T07:59:39.750 INFO:teuthology.orchestra.run.smithi007.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-09T07:59:39.750 INFO:teuthology.orchestra.run.smithi007.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-09T07:59:39.750 INFO:teuthology.orchestra.run.smithi007.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-09T07:59:39.750 INFO:teuthology.orchestra.run.smithi007.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-09T07:59:39.750 INFO:teuthology.orchestra.run.smithi007.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-09T07:59:39.750 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-09T07:59:39.750 INFO:teuthology.orchestra.run.smithi007.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-09T07:59:39.750 INFO:teuthology.orchestra.run.smithi007.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-09T07:59:39.750 INFO:teuthology.orchestra.run.smithi007.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-09T07:59:39.750 INFO:teuthology.orchestra.run.smithi007.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-09T07:59:39.750 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-09T07:59:39.751 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-09T07:59:39.751 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-09T07:59:39.751 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-09T07:59:39.751 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-09T07:59:39.751 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-09T07:59:39.751 INFO:teuthology.orchestra.run.smithi007.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-09T07:59:39.751 INFO:teuthology.orchestra.run.smithi007.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-09T07:59:39.751 INFO:teuthology.orchestra.run.smithi007.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-09T07:59:39.751 INFO:teuthology.orchestra.run.smithi007.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-09T07:59:39.751 INFO:teuthology.orchestra.run.smithi007.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-09T07:59:39.751 INFO:teuthology.orchestra.run.smithi007.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-09T07:59:39.751 INFO:teuthology.orchestra.run.smithi007.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-09T07:59:39.751 INFO:teuthology.orchestra.run.smithi007.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-09T07:59:39.752 INFO:teuthology.orchestra.run.smithi007.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-09T07:59:39.752 INFO:teuthology.orchestra.run.smithi007.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-09T07:59:39.752 INFO:teuthology.orchestra.run.smithi007.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-09T07:59:39.752 INFO:teuthology.orchestra.run.smithi007.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-09T07:59:39.752 INFO:teuthology.orchestra.run.smithi007.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-09T07:59:39.752 INFO:teuthology.orchestra.run.smithi007.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-09T07:59:39.752 INFO:teuthology.orchestra.run.smithi007.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-09T07:59:39.752 INFO:teuthology.orchestra.run.smithi007.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-09T07:59:39.752 INFO:teuthology.orchestra.run.smithi007.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-09T07:59:39.752 INFO:teuthology.orchestra.run.smithi007.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-09T07:59:39.752 INFO:teuthology.orchestra.run.smithi007.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-09T07:59:39.752 INFO:teuthology.orchestra.run.smithi007.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-09T07:59:39.752 INFO:teuthology.orchestra.run.smithi007.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-09T07:59:39.752 INFO:teuthology.orchestra.run.smithi007.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-09T07:59:39.753 INFO:teuthology.orchestra.run.smithi007.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-09T07:59:39.753 INFO:teuthology.orchestra.run.smithi007.stdout:Installing weak dependencies: 2024-05-09T07:59:39.753 INFO:teuthology.orchestra.run.smithi007.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-09T07:59:39.753 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-05-09T07:59:39.753 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction Summary 2024-05-09T07:59:39.753 INFO:teuthology.orchestra.run.smithi007.stdout:======================================================================================= 2024-05-09T07:59:39.753 INFO:teuthology.orchestra.run.smithi007.stdout:Install 148 Packages 2024-05-09T07:59:39.753 INFO:teuthology.orchestra.run.smithi007.stdout:Upgrade 8 Packages 2024-05-09T07:59:39.753 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-05-09T07:59:39.754 INFO:teuthology.orchestra.run.smithi007.stdout:Total download size: 225 M 2024-05-09T07:59:39.754 INFO:teuthology.orchestra.run.smithi007.stdout:Downloading Packages: 2024-05-09T07:59:40.197 INFO:teuthology.orchestra.run.smithi001.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-09T07:59:40.199 INFO:teuthology.orchestra.run.smithi001.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-09T07:59:40.220 INFO:teuthology.orchestra.run.smithi001.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-09T07:59:40.221 INFO:teuthology.orchestra.run.smithi001.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-09T07:59:40.223 INFO:teuthology.orchestra.run.smithi001.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-09T07:59:40.224 INFO:teuthology.orchestra.run.smithi001.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-09T07:59:40.227 INFO:teuthology.orchestra.run.smithi001.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-09T07:59:40.230 INFO:teuthology.orchestra.run.smithi001.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-09T07:59:40.231 INFO:teuthology.orchestra.run.smithi001.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-09T07:59:40.307 INFO:teuthology.orchestra.run.smithi001.stdout:Dependencies resolved. 2024-05-09T07:59:40.334 INFO:teuthology.orchestra.run.smithi001.stdout:======================================================================================= 2024-05-09T07:59:40.334 INFO:teuthology.orchestra.run.smithi001.stdout: Package Arch Version Repository Size 2024-05-09T07:59:40.335 INFO:teuthology.orchestra.run.smithi001.stdout:======================================================================================= 2024-05-09T07:59:40.335 INFO:teuthology.orchestra.run.smithi001.stdout:Installing: 2024-05-09T07:59:40.335 INFO:teuthology.orchestra.run.smithi001.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-09T07:59:40.335 INFO:teuthology.orchestra.run.smithi001.stdout: ceph x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 6.6 k 2024-05-09T07:59:40.335 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-base x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 5.4 M 2024-05-09T07:59:40.335 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-fuse x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 889 k 2024-05-09T07:59:40.335 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 145 k 2024-05-09T07:59:40.335 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-mgr x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 1.5 M 2024-05-09T07:59:40.335 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 142 k 2024-05-09T07:59:40.335 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 3.5 M 2024-05-09T07:59:40.335 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 7.4 M 2024-05-09T07:59:40.335 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-mgr-rook noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 49 k 2024-05-09T07:59:40.335 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-radosgw x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 10 M 2024-05-09T07:59:40.335 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-test x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 48 M 2024-05-09T07:59:40.336 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-volume noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 264 k 2024-05-09T07:59:40.336 INFO:teuthology.orchestra.run.smithi001.stdout: cephadm noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 763 k 2024-05-09T07:59:40.336 INFO:teuthology.orchestra.run.smithi001.stdout: cephfs-mirror x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 222 k 2024-05-09T07:59:40.336 INFO:teuthology.orchestra.run.smithi001.stdout: cephfs-shell x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 27 k 2024-05-09T07:59:40.336 INFO:teuthology.orchestra.run.smithi001.stdout: cephfs-top noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 18 k 2024-05-09T07:59:40.336 INFO:teuthology.orchestra.run.smithi001.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-05-09T07:59:40.336 INFO:teuthology.orchestra.run.smithi001.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-09T07:59:40.336 INFO:teuthology.orchestra.run.smithi001.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-09T07:59:40.336 INFO:teuthology.orchestra.run.smithi001.stdout: libcephfs-devel x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 32 k 2024-05-09T07:59:40.336 INFO:teuthology.orchestra.run.smithi001.stdout: libcephfs2 x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 746 k 2024-05-09T07:59:40.336 INFO:teuthology.orchestra.run.smithi001.stdout: librados-devel x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 127 k 2024-05-09T07:59:40.336 INFO:teuthology.orchestra.run.smithi001.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-09T07:59:40.336 INFO:teuthology.orchestra.run.smithi001.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-09T07:59:40.336 INFO:teuthology.orchestra.run.smithi001.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-09T07:59:40.337 INFO:teuthology.orchestra.run.smithi001.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-09T07:59:40.337 INFO:teuthology.orchestra.run.smithi001.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-09T07:59:40.337 INFO:teuthology.orchestra.run.smithi001.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-09T07:59:40.337 INFO:teuthology.orchestra.run.smithi001.stdout: python3-cephfs x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 162 k 2024-05-09T07:59:40.337 INFO:teuthology.orchestra.run.smithi001.stdout: python3-rados x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 321 k 2024-05-09T07:59:40.337 INFO:teuthology.orchestra.run.smithi001.stdout: python3-rbd x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 298 k 2024-05-09T07:59:40.337 INFO:teuthology.orchestra.run.smithi001.stdout: python3-rgw x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 99 k 2024-05-09T07:59:40.338 INFO:teuthology.orchestra.run.smithi001.stdout: rbd-fuse x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 85 k 2024-05-09T07:59:40.338 INFO:teuthology.orchestra.run.smithi001.stdout: rbd-mirror x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 3.1 M 2024-05-09T07:59:40.338 INFO:teuthology.orchestra.run.smithi001.stdout: rbd-nbd x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 171 k 2024-05-09T07:59:40.338 INFO:teuthology.orchestra.run.smithi001.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-09T07:59:40.338 INFO:teuthology.orchestra.run.smithi001.stdout:Upgrading: 2024-05-09T07:59:40.338 INFO:teuthology.orchestra.run.smithi001.stdout: NetworkManager x86_64 1:1.47.5-1.el9 baseos 2.3 M 2024-05-09T07:59:40.338 INFO:teuthology.orchestra.run.smithi001.stdout: NetworkManager-libnm x86_64 1:1.47.5-1.el9 baseos 1.8 M 2024-05-09T07:59:40.338 INFO:teuthology.orchestra.run.smithi001.stdout: NetworkManager-team x86_64 1:1.47.5-1.el9 baseos 42 k 2024-05-09T07:59:40.338 INFO:teuthology.orchestra.run.smithi001.stdout: NetworkManager-tui x86_64 1:1.47.5-1.el9 baseos 248 k 2024-05-09T07:59:40.339 INFO:teuthology.orchestra.run.smithi001.stdout: librados2 x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 3.4 M 2024-05-09T07:59:40.339 INFO:teuthology.orchestra.run.smithi001.stdout: librbd1 x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 3.1 M 2024-05-09T07:59:40.339 INFO:teuthology.orchestra.run.smithi001.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-09T07:59:40.339 INFO:teuthology.orchestra.run.smithi001.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-09T07:59:40.339 INFO:teuthology.orchestra.run.smithi001.stdout:Installing dependencies: 2024-05-09T07:59:40.339 INFO:teuthology.orchestra.run.smithi001.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-09T07:59:40.339 INFO:teuthology.orchestra.run.smithi001.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-09T07:59:40.339 INFO:teuthology.orchestra.run.smithi001.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-09T07:59:40.339 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-common x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 21 M 2024-05-09T07:59:40.339 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 32 k 2024-05-09T07:59:40.339 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-mds x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 2.4 M 2024-05-09T07:59:40.340 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 249 k 2024-05-09T07:59:40.340 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-mon x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 4.7 M 2024-05-09T07:59:40.340 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-osd x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 17 M 2024-05-09T07:59:40.340 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 16 k 2024-05-09T07:59:40.340 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-selinux x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 25 k 2024-05-09T07:59:40.340 INFO:teuthology.orchestra.run.smithi001.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-09T07:59:40.340 INFO:teuthology.orchestra.run.smithi001.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-09T07:59:40.340 INFO:teuthology.orchestra.run.smithi001.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-09T07:59:40.340 INFO:teuthology.orchestra.run.smithi001.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-09T07:59:40.341 INFO:teuthology.orchestra.run.smithi001.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-09T07:59:40.341 INFO:teuthology.orchestra.run.smithi001.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-09T07:59:40.341 INFO:teuthology.orchestra.run.smithi001.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-09T07:59:40.341 INFO:teuthology.orchestra.run.smithi001.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-09T07:59:40.341 INFO:teuthology.orchestra.run.smithi001.stdout: libcephsqlite x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 163 k 2024-05-09T07:59:40.341 INFO:teuthology.orchestra.run.smithi001.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-09T07:59:40.341 INFO:teuthology.orchestra.run.smithi001.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-09T07:59:40.341 INFO:teuthology.orchestra.run.smithi001.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-09T07:59:40.341 INFO:teuthology.orchestra.run.smithi001.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-09T07:59:40.341 INFO:teuthology.orchestra.run.smithi001.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-09T07:59:40.341 INFO:teuthology.orchestra.run.smithi001.stdout: libradosstriper1 x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 500 k 2024-05-09T07:59:40.341 INFO:teuthology.orchestra.run.smithi001.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-09T07:59:40.341 INFO:teuthology.orchestra.run.smithi001.stdout: librgw2 x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 5.3 M 2024-05-09T07:59:40.341 INFO:teuthology.orchestra.run.smithi001.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-09T07:59:40.342 INFO:teuthology.orchestra.run.smithi001.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-09T07:59:40.342 INFO:teuthology.orchestra.run.smithi001.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-09T07:59:40.342 INFO:teuthology.orchestra.run.smithi001.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-09T07:59:40.342 INFO:teuthology.orchestra.run.smithi001.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-09T07:59:40.342 INFO:teuthology.orchestra.run.smithi001.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-05-09T07:59:40.342 INFO:teuthology.orchestra.run.smithi001.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-09T07:59:40.342 INFO:teuthology.orchestra.run.smithi001.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-09T07:59:40.342 INFO:teuthology.orchestra.run.smithi001.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-09T07:59:40.342 INFO:teuthology.orchestra.run.smithi001.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-09T07:59:40.342 INFO:teuthology.orchestra.run.smithi001.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-09T07:59:40.342 INFO:teuthology.orchestra.run.smithi001.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-09T07:59:40.342 INFO:teuthology.orchestra.run.smithi001.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-09T07:59:40.342 INFO:teuthology.orchestra.run.smithi001.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-09T07:59:40.343 INFO:teuthology.orchestra.run.smithi001.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-09T07:59:40.343 INFO:teuthology.orchestra.run.smithi001.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-09T07:59:40.343 INFO:teuthology.orchestra.run.smithi001.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-09T07:59:40.343 INFO:teuthology.orchestra.run.smithi001.stdout: python3-ceph-argparse x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 45 k 2024-05-09T07:59:40.343 INFO:teuthology.orchestra.run.smithi001.stdout: python3-ceph-common x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 135 k 2024-05-09T07:59:40.343 INFO:teuthology.orchestra.run.smithi001.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-09T07:59:40.343 INFO:teuthology.orchestra.run.smithi001.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-09T07:59:40.343 INFO:teuthology.orchestra.run.smithi001.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-09T07:59:40.343 INFO:teuthology.orchestra.run.smithi001.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-09T07:59:40.343 INFO:teuthology.orchestra.run.smithi001.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-09T07:59:40.343 INFO:teuthology.orchestra.run.smithi001.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-05-09T07:59:40.344 INFO:teuthology.orchestra.run.smithi001.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-09T07:59:40.344 INFO:teuthology.orchestra.run.smithi001.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-09T07:59:40.344 INFO:teuthology.orchestra.run.smithi001.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-09T07:59:40.344 INFO:teuthology.orchestra.run.smithi001.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-09T07:59:40.344 INFO:teuthology.orchestra.run.smithi001.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-09T07:59:40.344 INFO:teuthology.orchestra.run.smithi001.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-09T07:59:40.344 INFO:teuthology.orchestra.run.smithi001.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-09T07:59:40.344 INFO:teuthology.orchestra.run.smithi001.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-09T07:59:40.344 INFO:teuthology.orchestra.run.smithi001.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-09T07:59:40.344 INFO:teuthology.orchestra.run.smithi001.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-09T07:59:40.344 INFO:teuthology.orchestra.run.smithi001.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-05-09T07:59:40.345 INFO:teuthology.orchestra.run.smithi001.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-09T07:59:40.345 INFO:teuthology.orchestra.run.smithi001.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-09T07:59:40.345 INFO:teuthology.orchestra.run.smithi001.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-09T07:59:40.345 INFO:teuthology.orchestra.run.smithi001.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-09T07:59:40.345 INFO:teuthology.orchestra.run.smithi001.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-09T07:59:40.345 INFO:teuthology.orchestra.run.smithi001.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-09T07:59:40.345 INFO:teuthology.orchestra.run.smithi001.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-09T07:59:40.345 INFO:teuthology.orchestra.run.smithi001.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-09T07:59:40.345 INFO:teuthology.orchestra.run.smithi001.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-09T07:59:40.345 INFO:teuthology.orchestra.run.smithi001.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-09T07:59:40.345 INFO:teuthology.orchestra.run.smithi001.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-09T07:59:40.345 INFO:teuthology.orchestra.run.smithi001.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-09T07:59:40.345 INFO:teuthology.orchestra.run.smithi001.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-09T07:59:40.345 INFO:teuthology.orchestra.run.smithi001.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-09T07:59:40.346 INFO:teuthology.orchestra.run.smithi001.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-09T07:59:40.346 INFO:teuthology.orchestra.run.smithi001.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-09T07:59:40.346 INFO:teuthology.orchestra.run.smithi001.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-09T07:59:40.346 INFO:teuthology.orchestra.run.smithi001.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-09T07:59:40.346 INFO:teuthology.orchestra.run.smithi001.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-09T07:59:40.346 INFO:teuthology.orchestra.run.smithi001.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-09T07:59:40.346 INFO:teuthology.orchestra.run.smithi001.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-09T07:59:40.346 INFO:teuthology.orchestra.run.smithi001.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-09T07:59:40.346 INFO:teuthology.orchestra.run.smithi001.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-09T07:59:40.346 INFO:teuthology.orchestra.run.smithi001.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-09T07:59:40.346 INFO:teuthology.orchestra.run.smithi001.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-09T07:59:40.346 INFO:teuthology.orchestra.run.smithi001.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-09T07:59:40.346 INFO:teuthology.orchestra.run.smithi001.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-09T07:59:40.347 INFO:teuthology.orchestra.run.smithi001.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-09T07:59:40.347 INFO:teuthology.orchestra.run.smithi001.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-09T07:59:40.347 INFO:teuthology.orchestra.run.smithi001.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-09T07:59:40.347 INFO:teuthology.orchestra.run.smithi001.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-09T07:59:40.347 INFO:teuthology.orchestra.run.smithi001.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-09T07:59:40.347 INFO:teuthology.orchestra.run.smithi001.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-09T07:59:40.347 INFO:teuthology.orchestra.run.smithi001.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-09T07:59:40.347 INFO:teuthology.orchestra.run.smithi001.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-09T07:59:40.347 INFO:teuthology.orchestra.run.smithi001.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-09T07:59:40.347 INFO:teuthology.orchestra.run.smithi001.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-09T07:59:40.347 INFO:teuthology.orchestra.run.smithi001.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-09T07:59:40.347 INFO:teuthology.orchestra.run.smithi001.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-09T07:59:40.347 INFO:teuthology.orchestra.run.smithi001.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-09T07:59:40.347 INFO:teuthology.orchestra.run.smithi001.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-09T07:59:40.348 INFO:teuthology.orchestra.run.smithi001.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-09T07:59:40.348 INFO:teuthology.orchestra.run.smithi001.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-09T07:59:40.348 INFO:teuthology.orchestra.run.smithi001.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-09T07:59:40.348 INFO:teuthology.orchestra.run.smithi001.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-09T07:59:40.348 INFO:teuthology.orchestra.run.smithi001.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-09T07:59:40.348 INFO:teuthology.orchestra.run.smithi001.stdout:Installing weak dependencies: 2024-05-09T07:59:40.348 INFO:teuthology.orchestra.run.smithi001.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-09T07:59:40.348 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T07:59:40.348 INFO:teuthology.orchestra.run.smithi001.stdout:Transaction Summary 2024-05-09T07:59:40.348 INFO:teuthology.orchestra.run.smithi001.stdout:======================================================================================= 2024-05-09T07:59:40.349 INFO:teuthology.orchestra.run.smithi001.stdout:Install 148 Packages 2024-05-09T07:59:40.349 INFO:teuthology.orchestra.run.smithi001.stdout:Upgrade 8 Packages 2024-05-09T07:59:40.349 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T07:59:40.350 INFO:teuthology.orchestra.run.smithi001.stdout:Total download size: 225 M 2024-05-09T07:59:40.350 INFO:teuthology.orchestra.run.smithi001.stdout:Downloading Packages: 2024-05-09T07:59:40.745 INFO:teuthology.orchestra.run.smithi007.stdout:(1/156): ceph-19.0.0-2515.g96805dcf.el9.x86_64. 38 kB/s | 6.6 kB 00:00 2024-05-09T07:59:40.962 INFO:teuthology.orchestra.run.smithi007.stdout:(2/156): ceph-fuse-19.0.0-2515.g96805dcf.el9.x8 4.0 MB/s | 889 kB 00:00 2024-05-09T07:59:41.020 INFO:teuthology.orchestra.run.smithi007.stdout:(3/156): ceph-immutable-object-cache-19.0.0-251 2.4 MB/s | 145 kB 00:00 2024-05-09T07:59:41.379 INFO:teuthology.orchestra.run.smithi007.stdout:(4/156): ceph-base-19.0.0-2515.g96805dcf.el9.x8 6.6 MB/s | 5.4 MB 00:00 2024-05-09T07:59:41.396 INFO:teuthology.orchestra.run.smithi001.stdout:(1/156): ceph-19.0.0-2515.g96805dcf.el9.x86_64. 22 kB/s | 6.6 kB 00:00 2024-05-09T07:59:41.437 INFO:teuthology.orchestra.run.smithi007.stdout:(5/156): ceph-mds-19.0.0-2515.g96805dcf.el9.x86 5.7 MB/s | 2.4 MB 00:00 2024-05-09T07:59:41.604 INFO:teuthology.orchestra.run.smithi007.stdout:(6/156): ceph-mgr-19.0.0-2515.g96805dcf.el9.x86 6.6 MB/s | 1.5 MB 00:00 2024-05-09T07:59:42.038 INFO:teuthology.orchestra.run.smithi001.stdout:(2/156): ceph-fuse-19.0.0-2515.g96805dcf.el9.x8 1.4 MB/s | 889 kB 00:00 2024-05-09T07:59:42.196 INFO:teuthology.orchestra.run.smithi001.stdout:(3/156): ceph-immutable-object-cache-19.0.0-251 920 kB/s | 145 kB 00:00 2024-05-09T07:59:42.221 INFO:teuthology.orchestra.run.smithi007.stdout:(7/156): ceph-mon-19.0.0-2515.g96805dcf.el9.x86 6.0 MB/s | 4.7 MB 00:00 2024-05-09T07:59:42.996 INFO:teuthology.orchestra.run.smithi001.stdout:(4/156): ceph-base-19.0.0-2515.g96805dcf.el9.x8 2.8 MB/s | 5.4 MB 00:01 2024-05-09T07:59:43.297 INFO:teuthology.orchestra.run.smithi001.stdout:(5/156): ceph-mds-19.0.0-2515.g96805dcf.el9.x86 2.1 MB/s | 2.4 MB 00:01 2024-05-09T07:59:43.480 INFO:teuthology.orchestra.run.smithi001.stdout:(6/156): ceph-mgr-19.0.0-2515.g96805dcf.el9.x86 3.1 MB/s | 1.5 MB 00:00 2024-05-09T07:59:43.513 INFO:teuthology.orchestra.run.smithi007.stdout:(8/156): ceph-common-19.0.0-2515.g96805dcf.el9. 7.1 MB/s | 21 MB 00:02 2024-05-09T07:59:43.639 INFO:teuthology.orchestra.run.smithi007.stdout:(9/156): ceph-radosgw-19.0.0-2515.g96805dcf.el9 7.2 MB/s | 10 MB 00:01 2024-05-09T07:59:43.664 INFO:teuthology.orchestra.run.smithi007.stdout:(10/156): ceph-selinux-19.0.0-2515.g96805dcf.el 167 kB/s | 25 kB 00:00 2024-05-09T07:59:43.789 INFO:teuthology.orchestra.run.smithi007.stdout:(11/156): cephfs-mirror-19.0.0-2515.g96805dcf.e 1.7 MB/s | 222 kB 00:00 2024-05-09T07:59:43.864 INFO:teuthology.orchestra.run.smithi007.stdout:(12/156): cephfs-shell-19.0.0-2515.g96805dcf.el 361 kB/s | 27 kB 00:00 2024-05-09T07:59:43.939 INFO:teuthology.orchestra.run.smithi007.stdout:(13/156): libcephfs-devel-19.0.0-2515.g96805dcf 423 kB/s | 32 kB 00:00 2024-05-09T07:59:44.098 INFO:teuthology.orchestra.run.smithi007.stdout:(14/156): libcephfs2-19.0.0-2515.g96805dcf.el9. 4.6 MB/s | 746 kB 00:00 2024-05-09T07:59:44.182 INFO:teuthology.orchestra.run.smithi007.stdout:(15/156): libcephsqlite-19.0.0-2515.g96805dcf.e 1.9 MB/s | 163 kB 00:00 2024-05-09T07:59:44.374 INFO:teuthology.orchestra.run.smithi007.stdout:(16/156): ceph-osd-19.0.0-2515.g96805dcf.el9.x8 6.1 MB/s | 17 MB 00:02 2024-05-09T07:59:44.399 INFO:teuthology.orchestra.run.smithi007.stdout:(17/156): librados-devel-19.0.0-2515.g96805dcf. 584 kB/s | 127 kB 00:00 2024-05-09T07:59:44.516 INFO:teuthology.orchestra.run.smithi007.stdout:(18/156): libradosstriper1-19.0.0-2515.g96805dc 3.5 MB/s | 500 kB 00:00 2024-05-09T07:59:44.547 INFO:teuthology.orchestra.run.smithi001.stdout:(7/156): ceph-mon-19.0.0-2515.g96805dcf.el9.x86 3.8 MB/s | 4.7 MB 00:01 2024-05-09T07:59:44.584 INFO:teuthology.orchestra.run.smithi007.stdout:(19/156): python3-ceph-argparse-19.0.0-2515.g96 676 kB/s | 45 kB 00:00 2024-05-09T07:59:44.659 INFO:teuthology.orchestra.run.smithi007.stdout:(20/156): python3-ceph-common-19.0.0-2515.g9680 1.8 MB/s | 135 kB 00:00 2024-05-09T07:59:44.784 INFO:teuthology.orchestra.run.smithi007.stdout:(21/156): python3-cephfs-19.0.0-2515.g96805dcf. 1.3 MB/s | 162 kB 00:00 2024-05-09T07:59:44.901 INFO:teuthology.orchestra.run.smithi007.stdout:(22/156): python3-rados-19.0.0-2515.g96805dcf.e 2.7 MB/s | 321 kB 00:00 2024-05-09T07:59:44.993 INFO:teuthology.orchestra.run.smithi007.stdout:(23/156): python3-rbd-19.0.0-2515.g96805dcf.el9 3.2 MB/s | 298 kB 00:00 2024-05-09T07:59:45.068 INFO:teuthology.orchestra.run.smithi007.stdout:(24/156): python3-rgw-19.0.0-2515.g96805dcf.el9 1.3 MB/s | 99 kB 00:00 2024-05-09T07:59:45.152 INFO:teuthology.orchestra.run.smithi007.stdout:(25/156): rbd-fuse-19.0.0-2515.g96805dcf.el9.x8 1.0 MB/s | 85 kB 00:00 2024-05-09T07:59:45.285 INFO:teuthology.orchestra.run.smithi007.stdout:(26/156): librgw2-19.0.0-2515.g96805dcf.el9.x86 6.0 MB/s | 5.3 MB 00:00 2024-05-09T07:59:45.369 INFO:teuthology.orchestra.run.smithi007.stdout:(27/156): rbd-nbd-19.0.0-2515.g96805dcf.el9.x86 2.0 MB/s | 171 kB 00:00 2024-05-09T07:59:45.453 INFO:teuthology.orchestra.run.smithi007.stdout:(28/156): ceph-grafana-dashboards-19.0.0-2515.g 381 kB/s | 32 kB 00:00 2024-05-09T07:59:45.536 INFO:teuthology.orchestra.run.smithi007.stdout:(29/156): ceph-mgr-cephadm-19.0.0-2515.g96805dc 1.7 MB/s | 142 kB 00:00 2024-05-09T07:59:45.787 INFO:teuthology.orchestra.run.smithi007.stdout:(30/156): rbd-mirror-19.0.0-2515.g96805dcf.el9. 4.9 MB/s | 3.1 MB 00:00 2024-05-09T07:59:46.187 INFO:teuthology.orchestra.run.smithi007.stdout:(31/156): ceph-mgr-dashboard-19.0.0-2515.g96805 5.4 MB/s | 3.5 MB 00:00 2024-05-09T07:59:46.287 INFO:teuthology.orchestra.run.smithi007.stdout:(32/156): ceph-mgr-modules-core-19.0.0-2515.g96 2.4 MB/s | 249 kB 00:00 2024-05-09T07:59:46.379 INFO:teuthology.orchestra.run.smithi007.stdout:(33/156): ceph-mgr-rook-19.0.0-2515.g96805dcf.e 539 kB/s | 49 kB 00:00 2024-05-09T07:59:46.463 INFO:teuthology.orchestra.run.smithi007.stdout:(34/156): ceph-prometheus-alerts-19.0.0-2515.g9 197 kB/s | 16 kB 00:00 2024-05-09T07:59:46.563 INFO:teuthology.orchestra.run.smithi007.stdout:(35/156): ceph-volume-19.0.0-2515.g96805dcf.el9 2.6 MB/s | 264 kB 00:00 2024-05-09T07:59:46.739 INFO:teuthology.orchestra.run.smithi007.stdout:(36/156): cephadm-19.0.0-2515.g96805dcf.el9.noa 4.3 MB/s | 763 kB 00:00 2024-05-09T07:59:46.830 INFO:teuthology.orchestra.run.smithi007.stdout:(37/156): cephfs-top-19.0.0-2515.g96805dcf.el9. 199 kB/s | 18 kB 00:00 2024-05-09T07:59:47.131 INFO:teuthology.orchestra.run.smithi007.stdout:(38/156): fuse-2.9.9-15.el9.x86_64.rpm 268 kB/s | 80 kB 00:00 2024-05-09T07:59:47.214 INFO:teuthology.orchestra.run.smithi007.stdout:(39/156): ceph-mgr-diskprediction-local-19.0.0- 5.2 MB/s | 7.4 MB 00:01 2024-05-09T07:59:47.424 INFO:teuthology.orchestra.run.smithi007.stdout:(40/156): libgfortran-11.4.1-3.el9.x86_64.rpm 2.7 MB/s | 803 kB 00:00 2024-05-09T07:59:47.549 INFO:teuthology.orchestra.run.smithi007.stdout:(41/156): libquadmath-11.4.1-3.el9.x86_64.rpm 585 kB/s | 195 kB 00:00 2024-05-09T07:59:47.772 INFO:teuthology.orchestra.run.smithi001.stdout:(8/156): ceph-radosgw-19.0.0-2515.g96805dcf.el9 3.2 MB/s | 10 MB 00:03 2024-05-09T07:59:48.031 INFO:teuthology.orchestra.run.smithi001.stdout:(9/156): ceph-common-19.0.0-2515.g96805dcf.el9. 3.0 MB/s | 21 MB 00:06 2024-05-09T07:59:48.056 INFO:teuthology.orchestra.run.smithi001.stdout:(10/156): ceph-selinux-19.0.0-2515.g96805dcf.el 89 kB/s | 25 kB 00:00 2024-05-09T07:59:48.183 INFO:teuthology.orchestra.run.smithi007.stdout:(42/156): ceph-test-19.0.0-2515.g96805dcf.el9.x 11 MB/s | 48 MB 00:04 2024-05-09T07:59:48.207 INFO:teuthology.orchestra.run.smithi007.stdout:(43/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 323 kB/s | 253 kB 00:00 2024-05-09T07:59:48.250 INFO:teuthology.orchestra.run.smithi007.stdout:(44/156): python3-cryptography-36.0.1-4.el9.x86 1.7 MB/s | 1.2 MB 00:00 2024-05-09T07:59:48.290 INFO:teuthology.orchestra.run.smithi001.stdout:(11/156): ceph-osd-19.0.0-2515.g96805dcf.el9.x8 3.5 MB/s | 17 MB 00:04 2024-05-09T07:59:48.292 INFO:teuthology.orchestra.run.smithi007.stdout:(45/156): python3-ply-3.11-14.el9.noarch.rpm 981 kB/s | 106 kB 00:00 2024-05-09T07:59:48.323 INFO:teuthology.orchestra.run.smithi001.stdout:(12/156): cephfs-mirror-19.0.0-2515.g96805dcf.e 835 kB/s | 222 kB 00:00 2024-05-09T07:59:48.333 INFO:teuthology.orchestra.run.smithi007.stdout:(46/156): python3-pycparser-2.20-6.el9.noarch.r 1.1 MB/s | 135 kB 00:00 2024-05-09T07:59:48.349 INFO:teuthology.orchestra.run.smithi001.stdout:(13/156): cephfs-shell-19.0.0-2515.g96805dcf.el 470 kB/s | 27 kB 00:00 2024-05-09T07:59:48.359 INFO:teuthology.orchestra.run.smithi007.stdout:(47/156): python3-requests-2.25.1-8.el9.noarch. 1.1 MB/s | 125 kB 00:00 2024-05-09T07:59:48.374 INFO:teuthology.orchestra.run.smithi001.stdout:(14/156): libcephfs-devel-19.0.0-2515.g96805dcf 639 kB/s | 32 kB 00:00 2024-05-09T07:59:48.426 INFO:teuthology.orchestra.run.smithi007.stdout:(48/156): python3-urllib3-1.26.5-5.el9.noarch.r 1.6 MB/s | 215 kB 00:00 2024-05-09T07:59:48.449 INFO:teuthology.orchestra.run.smithi001.stdout:(15/156): libcephsqlite-19.0.0-2515.g96805dcf.e 2.2 MB/s | 163 kB 00:00 2024-05-09T07:59:48.508 INFO:teuthology.orchestra.run.smithi001.stdout:(16/156): libcephfs2-19.0.0-2515.g96805dcf.el9. 4.6 MB/s | 746 kB 00:00 2024-05-09T07:59:48.533 INFO:teuthology.orchestra.run.smithi001.stdout:(17/156): librados-devel-19.0.0-2515.g96805dcf. 1.5 MB/s | 127 kB 00:00 2024-05-09T07:59:48.534 INFO:teuthology.orchestra.run.smithi007.stdout:(49/156): automake-1.16.2-8.el9.noarch.rpm 3.4 MB/s | 689 kB 00:00 2024-05-09T07:59:48.560 INFO:teuthology.orchestra.run.smithi007.stdout:(50/156): boost-program-options-1.75.0-8.el9.x8 800 kB/s | 107 kB 00:00 2024-05-09T07:59:48.585 INFO:teuthology.orchestra.run.smithi007.stdout:(51/156): elfutils-libelf-devel-0.190-2.el9.x86 427 kB/s | 21 kB 00:00 2024-05-09T07:59:48.618 INFO:teuthology.orchestra.run.smithi007.stdout:(52/156): bison-3.7.4-5.el9.x86_64.rpm 3.5 MB/s | 943 kB 00:00 2024-05-09T07:59:48.629 INFO:teuthology.orchestra.run.smithi001.stdout:(18/156): libradosstriper1-19.0.0-2515.g96805dc 4.1 MB/s | 500 kB 00:00 2024-05-09T07:59:48.644 INFO:teuthology.orchestra.run.smithi007.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 559 kB/s | 33 kB 00:00 2024-05-09T07:59:48.669 INFO:teuthology.orchestra.run.smithi007.stdout:(54/156): flex-2.6.4-9.el9.x86_64.rpm 2.8 MB/s | 313 kB 00:00 2024-05-09T07:59:48.684 INFO:teuthology.orchestra.run.smithi001.stdout:(19/156): python3-ceph-argparse-19.0.0-2515.g96 844 kB/s | 45 kB 00:00 2024-05-09T07:59:48.694 INFO:teuthology.orchestra.run.smithi007.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 362 kB/s | 18 kB 00:00 2024-05-09T07:59:48.720 INFO:teuthology.orchestra.run.smithi007.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 221 kB/s | 11 kB 00:00 2024-05-09T07:59:48.742 INFO:teuthology.orchestra.run.smithi001.stdout:(20/156): python3-ceph-common-19.0.0-2515.g9680 2.3 MB/s | 135 kB 00:00 2024-05-09T07:59:48.745 INFO:teuthology.orchestra.run.smithi007.stdout:(57/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 160 kB 00:00 2024-05-09T07:59:48.770 INFO:teuthology.orchestra.run.smithi007.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 904 kB/s | 45 kB 00:00 2024-05-09T07:59:48.801 INFO:teuthology.orchestra.run.smithi001.stdout:(21/156): python3-cephfs-19.0.0-2515.g96805dcf. 2.7 MB/s | 162 kB 00:00 2024-05-09T07:59:48.804 INFO:teuthology.orchestra.run.smithi007.stdout:(59/156): librdkafka-1.6.1-102.el9.x86_64.rpm 11 MB/s | 662 kB 00:00 2024-05-09T07:59:48.863 INFO:teuthology.orchestra.run.smithi007.stdout:(60/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 12 MB/s | 3.0 MB 00:00 2024-05-09T07:59:48.885 INFO:teuthology.orchestra.run.smithi001.stdout:(22/156): python3-rados-19.0.0-2515.g96805dcf.e 3.8 MB/s | 321 kB 00:00 2024-05-09T07:59:48.888 INFO:teuthology.orchestra.run.smithi007.stdout:(61/156): libuuid-devel-2.37.4-18.el9.x86_64.rp 304 kB/s | 25 kB 00:00 2024-05-09T07:59:48.913 INFO:teuthology.orchestra.run.smithi007.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 943 kB/s | 47 kB 00:00 2024-05-09T07:59:48.947 INFO:teuthology.orchestra.run.smithi007.stdout:(63/156): libtool-2.4.6-45.el9.x86_64.rpm 3.2 MB/s | 582 kB 00:00 2024-05-09T07:59:48.968 INFO:teuthology.orchestra.run.smithi001.stdout:(23/156): python3-rbd-19.0.0-2515.g96805dcf.el9 3.5 MB/s | 298 kB 00:00 2024-05-09T07:59:48.972 INFO:teuthology.orchestra.run.smithi007.stdout:(64/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.4 MB/s | 292 kB 00:00 2024-05-09T07:59:48.997 INFO:teuthology.orchestra.run.smithi007.stdout:(65/156): openblas-0.3.21-2.el9.x86_64.rpm 700 kB/s | 35 kB 00:00 2024-05-09T07:59:49.023 INFO:teuthology.orchestra.run.smithi007.stdout:(66/156): lua-5.4.4-4.el9.x86_64.rpm 1.7 MB/s | 188 kB 00:00 2024-05-09T07:59:49.027 INFO:teuthology.orchestra.run.smithi001.stdout:(24/156): python3-rgw-19.0.0-2515.g96805dcf.el9 1.7 MB/s | 99 kB 00:00 2024-05-09T07:59:49.086 INFO:teuthology.orchestra.run.smithi001.stdout:(25/156): rbd-fuse-19.0.0-2515.g96805dcf.el9.x8 1.4 MB/s | 85 kB 00:00 2024-05-09T07:59:49.099 INFO:teuthology.orchestra.run.smithi007.stdout:(67/156): postgresql-13.14-1.el9.x86_64.rpm 21 MB/s | 1.6 MB 00:00 2024-05-09T07:59:49.165 INFO:teuthology.orchestra.run.smithi007.stdout:(68/156): openblas-openmp-0.3.21-2.el9.x86_64.r 25 MB/s | 4.7 MB 00:00 2024-05-09T07:59:49.199 INFO:teuthology.orchestra.run.smithi007.stdout:(69/156): postgresql-contrib-13.14-1.el9.x86_64 8.9 MB/s | 911 kB 00:00 2024-05-09T07:59:49.232 INFO:teuthology.orchestra.run.smithi007.stdout:(70/156): postgresql-private-libs-13.14-1.el9.x 2.0 MB/s | 133 kB 00:00 2024-05-09T07:59:49.300 INFO:teuthology.orchestra.run.smithi007.stdout:(71/156): openssl-devel-3.2.1-1.el9.x86_64.rpm 15 MB/s | 4.4 MB 00:00 2024-05-09T07:59:49.341 INFO:teuthology.orchestra.run.smithi007.stdout:(72/156): protobuf-3.14.0-13.el9.x86_64.rpm 9.3 MB/s | 1.0 MB 00:00 2024-05-09T07:59:49.367 INFO:teuthology.orchestra.run.smithi007.stdout:(73/156): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 87 kB 00:00 2024-05-09T07:59:49.378 INFO:teuthology.orchestra.run.smithi001.stdout:(26/156): librgw2-19.0.0-2515.g96805dcf.el9.x86 6.3 MB/s | 5.3 MB 00:00 2024-05-09T07:59:49.417 INFO:teuthology.orchestra.run.smithi007.stdout:(74/156): python3-devel-3.9.18-3.el9.x86_64.rpm 4.8 MB/s | 244 kB 00:00 2024-05-09T07:59:49.436 INFO:teuthology.orchestra.run.smithi001.stdout:(27/156): rbd-nbd-19.0.0-2515.g96805dcf.el9.x86 2.9 MB/s | 171 kB 00:00 2024-05-09T07:59:49.487 INFO:teuthology.orchestra.run.smithi001.stdout:(28/156): ceph-grafana-dashboards-19.0.0-2515.g 647 kB/s | 32 kB 00:00 2024-05-09T07:59:49.492 INFO:teuthology.orchestra.run.smithi007.stdout:(75/156): postgresql-server-13.14-1.el9.x86_64. 20 MB/s | 5.8 MB 00:00 2024-05-09T07:59:49.518 INFO:teuthology.orchestra.run.smithi007.stdout:(76/156): python3-jinja2-2.11.3-5.el9.noarch.rp 2.4 MB/s | 248 kB 00:00 2024-05-09T07:59:49.543 INFO:teuthology.orchestra.run.smithi007.stdout:(77/156): python3-jsonpatch-1.21-16.el9.noarch. 511 kB/s | 26 kB 00:00 2024-05-09T07:59:49.568 INFO:teuthology.orchestra.run.smithi007.stdout:(78/156): python3-jsonpointer-2.0-4.el9.noarch. 381 kB/s | 19 kB 00:00 2024-05-09T07:59:49.594 INFO:teuthology.orchestra.run.smithi007.stdout:(79/156): python3-mako-1.1.4-6.el9.noarch.rpm 3.4 MB/s | 172 kB 00:00 2024-05-09T07:59:49.595 INFO:teuthology.orchestra.run.smithi001.stdout:(29/156): rbd-mirror-19.0.0-2515.g96805dcf.el9. 6.1 MB/s | 3.1 MB 00:00 2024-05-09T07:59:49.619 INFO:teuthology.orchestra.run.smithi007.stdout:(80/156): python3-markupsafe-1.1.1-12.el9.x86_6 691 kB/s | 35 kB 00:00 2024-05-09T07:59:49.629 INFO:teuthology.orchestra.run.smithi001.stdout:(30/156): ceph-mgr-cephadm-19.0.0-2515.g96805dc 1.0 MB/s | 142 kB 00:00 2024-05-09T07:59:49.703 INFO:teuthology.orchestra.run.smithi007.stdout:(81/156): python3-babel-2.9.1-2.el9.noarch.rpm 17 MB/s | 6.0 MB 00:00 2024-05-09T07:59:49.728 INFO:teuthology.orchestra.run.smithi007.stdout:(82/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.3 MB/s | 250 kB 00:00 2024-05-09T07:59:49.753 INFO:teuthology.orchestra.run.smithi007.stdout:(83/156): python3-oauthlib-3.1.1-5.el9.noarch.r 4.3 MB/s | 222 kB 00:00 2024-05-09T07:59:49.779 INFO:teuthology.orchestra.run.smithi007.stdout:(84/156): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-05-09T07:59:49.804 INFO:teuthology.orchestra.run.smithi007.stdout:(85/156): python3-prettytable-0.7.2-27.el9.noar 845 kB/s | 42 kB 00:00 2024-05-09T07:59:49.829 INFO:teuthology.orchestra.run.smithi007.stdout:(86/156): python3-protobuf-3.14.0-13.el9.noarch 5.3 MB/s | 269 kB 00:00 2024-05-09T07:59:49.854 INFO:teuthology.orchestra.run.smithi007.stdout:(87/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-05-09T07:59:49.880 INFO:teuthology.orchestra.run.smithi007.stdout:(88/156): python3-pyasn1-modules-0.4.8-6.el9.no 5.4 MB/s | 279 kB 00:00 2024-05-09T07:59:49.905 INFO:teuthology.orchestra.run.smithi007.stdout:(89/156): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-05-09T07:59:49.989 INFO:teuthology.orchestra.run.smithi007.stdout:(90/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 13 MB/s | 5.1 MB 00:00 2024-05-09T07:59:50.014 INFO:teuthology.orchestra.run.smithi007.stdout:(91/156): python3-requests-oauthlib-1.3.0-12.el 401 kB/s | 54 kB 00:00 2024-05-09T07:59:50.029 INFO:teuthology.orchestra.run.smithi001.stdout:(31/156): ceph-mgr-dashboard-19.0.0-2515.g96805 8.1 MB/s | 3.5 MB 00:00 2024-05-09T07:59:50.039 INFO:teuthology.orchestra.run.smithi007.stdout:(92/156): python3-toml-0.10.2-6.el9.noarch.rpm 829 kB/s | 42 kB 00:00 2024-05-09T07:59:50.081 INFO:teuthology.orchestra.run.smithi007.stdout:(93/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 661 kB/s | 44 kB 00:00 2024-05-09T07:59:50.088 INFO:teuthology.orchestra.run.smithi001.stdout:(32/156): ceph-mgr-modules-core-19.0.0-2515.g96 4.2 MB/s | 249 kB 00:00 2024-05-09T07:59:50.115 INFO:teuthology.orchestra.run.smithi007.stdout:(94/156): qatlib-24.02.0-1.el9.x86_64.rpm 2.9 MB/s | 221 kB 00:00 2024-05-09T07:59:50.138 INFO:teuthology.orchestra.run.smithi001.stdout:(33/156): ceph-mgr-rook-19.0.0-2515.g96805dcf.e 1.0 MB/s | 49 kB 00:00 2024-05-09T07:59:50.140 INFO:teuthology.orchestra.run.smithi007.stdout:(95/156): qatlib-service-24.02.0-1.el9.x86_64.r 610 kB/s | 36 kB 00:00 2024-05-09T07:59:50.166 INFO:teuthology.orchestra.run.smithi007.stdout:(96/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 942 kB/s | 47 kB 00:00 2024-05-09T07:59:50.188 INFO:teuthology.orchestra.run.smithi001.stdout:(34/156): ceph-prometheus-alerts-19.0.0-2515.g9 335 kB/s | 16 kB 00:00 2024-05-09T07:59:50.212 INFO:teuthology.orchestra.run.smithi007.stdout:(97/156): socat-1.7.4.1-5.el9.x86_64.rpm 4.2 MB/s | 305 kB 00:00 2024-05-09T07:59:50.241 INFO:teuthology.orchestra.run.smithi007.stdout:(98/156): uuid-1.6.2-55.el9.x86_64.rpm 767 kB/s | 58 kB 00:00 2024-05-09T07:59:50.266 INFO:teuthology.orchestra.run.smithi007.stdout:(99/156): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.7 MB/s | 94 kB 00:00 2024-05-09T07:59:50.272 INFO:teuthology.orchestra.run.smithi001.stdout:(35/156): ceph-volume-19.0.0-2515.g96805dcf.el9 3.1 MB/s | 264 kB 00:00 2024-05-09T07:59:50.294 INFO:teuthology.orchestra.run.smithi007.stdout:(100/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-05-09T07:59:50.317 INFO:teuthology.orchestra.run.smithi007.stdout:(101/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 890 kB/s | 45 kB 00:00 2024-05-09T07:59:50.359 INFO:teuthology.orchestra.run.smithi007.stdout:(102/156): lua-devel-5.4.4-4.el9.x86_64.rpm 333 kB/s | 22 kB 00:00 2024-05-09T07:59:50.397 INFO:teuthology.orchestra.run.smithi001.stdout:(36/156): cephadm-19.0.0-2515.g96805dcf.el9.noa 6.0 MB/s | 763 kB 00:00 2024-05-09T07:59:50.426 INFO:teuthology.orchestra.run.smithi007.stdout:(103/156): protobuf-compiler-3.14.0-13.el9.x86_ 7.8 MB/s | 863 kB 00:00 2024-05-09T07:59:50.447 INFO:teuthology.orchestra.run.smithi001.stdout:(37/156): cephfs-top-19.0.0-2515.g96805dcf.el9. 368 kB/s | 18 kB 00:00 2024-05-09T07:59:50.627 INFO:teuthology.orchestra.run.smithi007.stdout:(104/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 23 MB/s | 16 MB 00:00 2024-05-09T07:59:50.664 INFO:teuthology.orchestra.run.smithi001.stdout:(38/156): ceph-mgr-diskprediction-local-19.0.0- 7.2 MB/s | 7.4 MB 00:01 2024-05-09T07:59:50.777 INFO:teuthology.orchestra.run.smithi007.stdout:(105/156): grpc-data-1.46.7-10.el9.noarch.rpm 130 kB/s | 19 kB 00:00 2024-05-09T07:59:50.885 INFO:teuthology.orchestra.run.smithi007.stdout:(106/156): abseil-cpp-20211102.0-3.el9.x86_64.r 1.0 MB/s | 548 kB 00:00 2024-05-09T07:59:50.927 INFO:teuthology.orchestra.run.smithi007.stdout:(107/156): gperftools-libs-2.9.1-2.el9.x86_64.r 617 kB/s | 309 kB 00:00 2024-05-09T07:59:50.953 INFO:teuthology.orchestra.run.smithi007.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 388 kB/s | 26 kB 00:00 2024-05-09T07:59:51.011 INFO:teuthology.orchestra.run.smithi007.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 581 kB/s | 49 kB 00:00 2024-05-09T07:59:51.028 INFO:teuthology.orchestra.run.smithi007.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 896 kB/s | 67 kB 00:00 2024-05-09T07:59:51.087 INFO:teuthology.orchestra.run.smithi007.stdout:(111/156): luarocks-3.9.2-1.el9.noarch.rpm 2.0 MB/s | 151 kB 00:00 2024-05-09T07:59:51.195 INFO:teuthology.orchestra.run.smithi007.stdout:(112/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 4.9 MB/s | 839 kB 00:00 2024-05-09T07:59:51.221 INFO:teuthology.orchestra.run.smithi007.stdout:(113/156): python3-asyncssh-2.13.2-5.el9.noarch 4.0 MB/s | 548 kB 00:00 2024-05-09T07:59:51.265 INFO:teuthology.orchestra.run.smithi001.stdout:(39/156): fuse-2.9.9-15.el9.x86_64.rpm 98 kB/s | 80 kB 00:00 2024-05-09T07:59:51.279 INFO:teuthology.orchestra.run.smithi007.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 520 kB/s | 43 kB 00:00 2024-05-09T07:59:51.305 INFO:teuthology.orchestra.run.smithi007.stdout:(115/156): python3-cachetools-4.2.4-1.el9.noarc 386 kB/s | 32 kB 00:00 2024-05-09T07:59:51.355 INFO:teuthology.orchestra.run.smithi007.stdout:(116/156): python3-certifi-2021.10.8-2.el9.noar 199 kB/s | 15 kB 00:00 2024-05-09T07:59:51.389 INFO:teuthology.orchestra.run.smithi007.stdout:(117/156): python3-cheroot-8.6.0-4.el9.noarch.r 2.0 MB/s | 172 kB 00:00 2024-05-09T07:59:51.439 INFO:teuthology.orchestra.run.smithi007.stdout:(118/156): python3-cherrypy-18.6.1-2.el9.noarch 4.2 MB/s | 358 kB 00:00 2024-05-09T07:59:51.498 INFO:teuthology.orchestra.run.smithi007.stdout:(119/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.7 MB/s | 303 kB 00:00 2024-05-09T07:59:51.526 INFO:teuthology.orchestra.run.smithi007.stdout:(120/156): python3-colorama-0.4.4-12.el9.noarch 420 kB/s | 35 kB 00:00 2024-05-09T07:59:51.632 INFO:teuthology.orchestra.run.smithi007.stdout:(121/156): python3-google-auth-2.29.0-1.el9.noa 1.5 MB/s | 206 kB 00:00 2024-05-09T07:59:51.707 INFO:teuthology.orchestra.run.smithi001.stdout:(40/156): libgfortran-11.4.1-3.el9.x86_64.rpm 772 kB/s | 803 kB 00:01 2024-05-09T07:59:51.715 INFO:teuthology.orchestra.run.smithi007.stdout:(122/156): python3-grpcio-1.46.7-10.el9.x86_64. 11 MB/s | 2.0 MB 00:00 2024-05-09T07:59:51.741 INFO:teuthology.orchestra.run.smithi007.stdout:(123/156): python3-grpcio-tools-1.46.7-10.el9.x 1.3 MB/s | 144 kB 00:00 2024-05-09T07:59:51.808 INFO:teuthology.orchestra.run.smithi007.stdout:(124/156): python3-jaraco-8.2.1-3.el9.noarch.rp 116 kB/s | 11 kB 00:00 2024-05-09T07:59:51.833 INFO:teuthology.orchestra.run.smithi007.stdout:(125/156): python3-jaraco-classes-3.2.1-5.el9.n 192 kB/s | 18 kB 00:00 2024-05-09T07:59:51.892 INFO:teuthology.orchestra.run.smithi007.stdout:(126/156): python3-jaraco-collections-3.0.0-8.e 277 kB/s | 23 kB 00:00 2024-05-09T07:59:51.917 INFO:teuthology.orchestra.run.smithi007.stdout:(127/156): python3-jaraco-functools-3.5.0-2.el9 232 kB/s | 19 kB 00:00 2024-05-09T07:59:51.967 INFO:teuthology.orchestra.run.smithi007.stdout:(128/156): python3-jaraco-text-3.2.0-6.el9.noar 262 kB/s | 20 kB 00:00 2024-05-09T07:59:52.051 INFO:teuthology.orchestra.run.smithi007.stdout:(129/156): python3-kubernetes-26.1.0-2.el9.noar 7.6 MB/s | 1.0 MB 00:00 2024-05-09T07:59:52.076 INFO:teuthology.orchestra.run.smithi007.stdout:(130/156): python3-logutils-0.3.5-21.el9.noarch 426 kB/s | 46 kB 00:00 2024-05-09T07:59:52.107 INFO:teuthology.orchestra.run.smithi001.stdout:(41/156): libquadmath-11.4.1-3.el9.x86_64.rpm 232 kB/s | 195 kB 00:00 2024-05-09T07:59:52.146 INFO:teuthology.orchestra.run.smithi007.stdout:(131/156): python3-more-itertools-8.12.0-2.el9. 856 kB/s | 79 kB 00:00 2024-05-09T07:59:52.169 INFO:teuthology.orchestra.run.smithi007.stdout:(132/156): python3-natsort-7.1.1-5.el9.noarch.r 627 kB/s | 58 kB 00:00 2024-05-09T07:59:52.235 INFO:teuthology.orchestra.run.smithi007.stdout:(133/156): python3-pecan-1.4.2-3.el9.noarch.rpm 2.9 MB/s | 272 kB 00:00 2024-05-09T07:59:52.261 INFO:teuthology.orchestra.run.smithi007.stdout:(134/156): python3-portend-3.1.0-2.el9.noarch.r 179 kB/s | 16 kB 00:00 2024-05-09T07:59:52.328 INFO:teuthology.orchestra.run.smithi007.stdout:(135/156): libarrow-9.0.0-11.el9.x86_64.rpm 2.8 MB/s | 4.4 MB 00:01 2024-05-09T07:59:52.354 INFO:teuthology.orchestra.run.smithi007.stdout:(136/156): python3-pyperclip-1.8.2-2.el9.noarch 307 kB/s | 28 kB 00:00 2024-05-09T07:59:52.378 INFO:teuthology.orchestra.run.smithi007.stdout:(137/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 630 kB/s | 90 kB 00:00 2024-05-09T07:59:52.412 INFO:teuthology.orchestra.run.smithi007.stdout:(138/156): python3-repoze-lru-0.7-16.el9.noarch 366 kB/s | 31 kB 00:00 2024-05-09T07:59:52.579 INFO:teuthology.orchestra.run.smithi007.stdout:(139/156): python3-routes-2.5.1-5.el9.noarch.rp 835 kB/s | 188 kB 00:00 2024-05-09T07:59:52.596 INFO:teuthology.orchestra.run.smithi007.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 195 kB/s | 36 kB 00:00 2024-05-09T07:59:52.613 INFO:teuthology.orchestra.run.smithi007.stdout:(141/156): python3-rsa-4.9-2.el9.noarch.rpm 252 kB/s | 59 kB 00:00 2024-05-09T07:59:52.647 INFO:teuthology.orchestra.run.smithi007.stdout:(142/156): python3-typing-extensions-4.4.0-2.el 759 kB/s | 51 kB 00:00 2024-05-09T07:59:52.674 INFO:teuthology.orchestra.run.smithi001.stdout:(42/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 262 kB/s | 253 kB 00:00 2024-05-09T07:59:52.680 INFO:teuthology.orchestra.run.smithi007.stdout:(143/156): python3-webob-1.8.7-6.el9.noarch.rpm 2.7 MB/s | 230 kB 00:00 2024-05-09T07:59:52.705 INFO:teuthology.orchestra.run.smithi007.stdout:(144/156): python3-websocket-client-1.2.3-2.el9 971 kB/s | 90 kB 00:00 2024-05-09T07:59:52.739 INFO:teuthology.orchestra.run.smithi007.stdout:(145/156): python3-werkzeug-2.0.3-3.el9.1.noarc 4.5 MB/s | 427 kB 00:00 2024-05-09T07:59:52.756 INFO:teuthology.orchestra.run.smithi007.stdout:(146/156): python3-zc-lockfile-2.0-10.el9.noarc 265 kB/s | 20 kB 00:00 2024-05-09T07:59:52.915 INFO:teuthology.orchestra.run.smithi007.stdout:(147/156): thrift-0.15.0-2.el9.x86_64.rpm 9.0 MB/s | 1.6 MB 00:00 2024-05-09T07:59:52.940 INFO:teuthology.orchestra.run.smithi007.stdout:(148/156): re2-20211101-3.el9.x86_64.rpm 817 kB/s | 192 kB 00:00 2024-05-09T07:59:53.065 INFO:teuthology.orchestra.run.smithi007.stdout:(149/156): librados2-19.0.0-2515.g96805dcf.el9. 11 MB/s | 3.4 MB 00:00 2024-05-09T07:59:53.341 INFO:teuthology.orchestra.run.smithi007.stdout:(150/156): librbd1-19.0.0-2515.g96805dcf.el9.x8 7.4 MB/s | 3.1 MB 00:00 2024-05-09T07:59:53.391 INFO:teuthology.orchestra.run.smithi007.stdout:(151/156): NetworkManager-1.47.5-1.el9.x86_64.r 5.1 MB/s | 2.3 MB 00:00 2024-05-09T07:59:53.458 INFO:teuthology.orchestra.run.smithi007.stdout:(152/156): NetworkManager-libnm-1.47.5-1.el9.x8 4.7 MB/s | 1.8 MB 00:00 2024-05-09T07:59:53.466 INFO:teuthology.orchestra.run.smithi001.stdout:(43/156): ceph-test-19.0.0-2515.g96805dcf.el9.x 8.9 MB/s | 48 MB 00:05 2024-05-09T07:59:53.484 INFO:teuthology.orchestra.run.smithi007.stdout:(153/156): NetworkManager-team-1.47.5-1.el9.x86 294 kB/s | 42 kB 00:00 2024-05-09T07:59:53.534 INFO:teuthology.orchestra.run.smithi007.stdout:(154/156): NetworkManager-tui-1.47.5-1.el9.x86_ 1.7 MB/s | 248 kB 00:00 2024-05-09T07:59:53.725 INFO:teuthology.orchestra.run.smithi007.stdout:(155/156): openssl-3.2.1-1.el9.x86_64.rpm 5.1 MB/s | 1.3 MB 00:00 2024-05-09T07:59:53.808 INFO:teuthology.orchestra.run.smithi001.stdout:(44/156): python3-cryptography-36.0.1-4.el9.x86 739 kB/s | 1.2 MB 00:01 2024-05-09T07:59:53.842 INFO:teuthology.orchestra.run.smithi007.stdout:(156/156): openssl-libs-3.2.1-1.el9.x86_64.rpm 6.7 MB/s | 2.4 MB 00:00 2024-05-09T07:59:53.847 INFO:teuthology.orchestra.run.smithi007.stdout:-------------------------------------------------------------------------------- 2024-05-09T07:59:53.847 INFO:teuthology.orchestra.run.smithi007.stdout:Total 16 MB/s | 225 MB 00:14 2024-05-09T07:59:53.908 INFO:teuthology.orchestra.run.smithi001.stdout:(45/156): python3-ply-3.11-14.el9.noarch.rpm 86 kB/s | 106 kB 00:01 2024-05-09T07:59:54.308 INFO:teuthology.orchestra.run.smithi001.stdout:(46/156): python3-pycparser-2.20-6.el9.noarch.r 161 kB/s | 135 kB 00:00 2024-05-09T07:59:54.417 INFO:teuthology.orchestra.run.smithi001.stdout:(47/156): automake-1.16.2-8.el9.noarch.rpm 6.2 MB/s | 689 kB 00:00 2024-05-09T07:59:54.526 INFO:teuthology.orchestra.run.smithi001.stdout:(48/156): bison-3.7.4-5.el9.x86_64.rpm 8.6 MB/s | 943 kB 00:00 2024-05-09T07:59:54.592 INFO:teuthology.orchestra.run.smithi001.stdout:(49/156): boost-program-options-1.75.0-8.el9.x8 1.6 MB/s | 107 kB 00:00 2024-05-09T07:59:54.626 INFO:teuthology.orchestra.run.smithi001.stdout:(50/156): python3-requests-2.25.1-8.el9.noarch. 153 kB/s | 125 kB 00:00 2024-05-09T07:59:54.668 INFO:teuthology.orchestra.run.smithi001.stdout:(51/156): elfutils-libelf-devel-0.190-2.el9.x86 287 kB/s | 21 kB 00:00 2024-05-09T07:59:54.719 INFO:teuthology.orchestra.run.smithi001.stdout:(52/156): flexiblas-3.0.4-8.el9.x86_64.rpm 665 kB/s | 33 kB 00:00 2024-05-09T07:59:54.752 INFO:teuthology.orchestra.run.smithi001.stdout:(53/156): flex-2.6.4-9.el9.x86_64.rpm 2.5 MB/s | 313 kB 00:00 2024-05-09T07:59:54.802 INFO:teuthology.orchestra.run.smithi001.stdout:(54/156): flexiblas-openblas-openmp-3.0.4-8.el9 371 kB/s | 18 kB 00:00 2024-05-09T07:59:54.845 INFO:teuthology.orchestra.run.smithi001.stdout:(55/156): libaio-devel-0.3.111-13.el9.x86_64.rp 274 kB/s | 11 kB 00:00 2024-05-09T07:59:54.878 INFO:teuthology.orchestra.run.smithi001.stdout:(56/156): python3-urllib3-1.26.5-5.el9.noarch.r 222 kB/s | 215 kB 00:00 2024-05-09T07:59:54.945 INFO:teuthology.orchestra.run.smithi001.stdout:(57/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 695 kB/s | 45 kB 00:00 2024-05-09T07:59:54.970 INFO:teuthology.orchestra.run.smithi001.stdout:(58/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.3 MB/s | 160 kB 00:00 2024-05-09T07:59:55.046 INFO:teuthology.orchestra.run.smithi001.stdout:(59/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 9.2 MB/s | 3.0 MB 00:00 2024-05-09T07:59:55.068 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction check 2024-05-09T07:59:55.096 INFO:teuthology.orchestra.run.smithi001.stdout:(60/156): libuuid-devel-2.37.4-18.el9.x86_64.rp 519 kB/s | 25 kB 00:00 2024-05-09T07:59:55.138 INFO:teuthology.orchestra.run.smithi001.stdout:(61/156): librdkafka-1.6.1-102.el9.x86_64.rpm 3.4 MB/s | 662 kB 00:00 2024-05-09T07:59:55.163 INFO:teuthology.orchestra.run.smithi001.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 720 kB/s | 47 kB 00:00 2024-05-09T07:59:55.205 INFO:teuthology.orchestra.run.smithi001.stdout:(63/156): libtool-2.4.6-45.el9.x86_64.rpm 2.4 MB/s | 582 kB 00:00 2024-05-09T07:59:55.236 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction check succeeded. 2024-05-09T07:59:55.237 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction test 2024-05-09T07:59:55.255 INFO:teuthology.orchestra.run.smithi001.stdout:(64/156): openblas-0.3.21-2.el9.x86_64.rpm 722 kB/s | 35 kB 00:00 2024-05-09T07:59:55.289 INFO:teuthology.orchestra.run.smithi001.stdout:(65/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.9 MB/s | 292 kB 00:00 2024-05-09T07:59:55.314 INFO:teuthology.orchestra.run.smithi001.stdout:(66/156): lua-5.4.4-4.el9.x86_64.rpm 1.2 MB/s | 188 kB 00:00 2024-05-09T07:59:55.465 INFO:teuthology.orchestra.run.smithi001.stdout:(67/156): postgresql-13.14-1.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-05-09T07:59:55.573 INFO:teuthology.orchestra.run.smithi001.stdout:(68/156): openblas-openmp-0.3.21-2.el9.x86_64.r 15 MB/s | 4.7 MB 00:00 2024-05-09T07:59:55.665 INFO:teuthology.orchestra.run.smithi001.stdout:(69/156): openssl-devel-3.2.1-1.el9.x86_64.rpm 12 MB/s | 4.4 MB 00:00 2024-05-09T07:59:55.715 INFO:teuthology.orchestra.run.smithi001.stdout:(70/156): postgresql-contrib-13.14-1.el9.x86_64 3.6 MB/s | 911 kB 00:00 2024-05-09T07:59:55.757 INFO:teuthology.orchestra.run.smithi001.stdout:(71/156): postgresql-private-libs-13.14-1.el9.x 730 kB/s | 133 kB 00:00 2024-05-09T07:59:55.835 INFO:teuthology.orchestra.run.smithi001.stdout:(72/156): python3-attrs-20.3.0-7.el9.noarch.rpm 1.1 MB/s | 87 kB 00:00 2024-05-09T07:59:55.883 INFO:teuthology.orchestra.run.smithi001.stdout:(73/156): protobuf-3.14.0-13.el9.x86_64.rpm 6.1 MB/s | 1.0 MB 00:00 2024-05-09T07:59:55.958 INFO:teuthology.orchestra.run.smithi001.stdout:(74/156): python3-devel-3.9.18-3.el9.x86_64.rpm 3.2 MB/s | 244 kB 00:00 2024-05-09T07:59:56.075 INFO:teuthology.orchestra.run.smithi001.stdout:(75/156): postgresql-server-13.14-1.el9.x86_64. 14 MB/s | 5.8 MB 00:00 2024-05-09T07:59:56.125 INFO:teuthology.orchestra.run.smithi001.stdout:(76/156): python3-jsonpatch-1.21-16.el9.noarch. 520 kB/s | 26 kB 00:00 2024-05-09T07:59:56.167 INFO:teuthology.orchestra.run.smithi001.stdout:(77/156): python3-jinja2-2.11.3-5.el9.noarch.rp 1.2 MB/s | 248 kB 00:00 2024-05-09T07:59:56.209 INFO:teuthology.orchestra.run.smithi001.stdout:(78/156): python3-jsonpointer-2.0-4.el9.noarch. 232 kB/s | 19 kB 00:00 2024-05-09T07:59:56.259 INFO:teuthology.orchestra.run.smithi001.stdout:(79/156): python3-mako-1.1.4-6.el9.noarch.rpm 1.8 MB/s | 172 kB 00:00 2024-05-09T07:59:56.285 INFO:teuthology.orchestra.run.smithi001.stdout:(80/156): python3-markupsafe-1.1.1-12.el9.x86_6 468 kB/s | 35 kB 00:00 2024-05-09T07:59:56.419 INFO:teuthology.orchestra.run.smithi001.stdout:(81/156): python3-babel-2.9.1-2.el9.noarch.rpm 10 MB/s | 6.0 MB 00:00 2024-05-09T07:59:56.477 INFO:teuthology.orchestra.run.smithi001.stdout:(82/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.3 MB/s | 250 kB 00:00 2024-05-09T07:59:56.519 INFO:teuthology.orchestra.run.smithi001.stdout:(83/156): python3-oauthlib-3.1.1-5.el9.noarch.r 2.2 MB/s | 222 kB 00:00 2024-05-09T07:59:56.553 INFO:teuthology.orchestra.run.smithi001.stdout:(84/156): python3-packaging-20.9-5.el9.noarch.r 1.0 MB/s | 77 kB 00:00 2024-05-09T07:59:56.603 INFO:teuthology.orchestra.run.smithi001.stdout:(85/156): python3-prettytable-0.7.2-27.el9.noar 514 kB/s | 42 kB 00:00 2024-05-09T07:59:56.653 INFO:teuthology.orchestra.run.smithi001.stdout:(86/156): python3-protobuf-3.14.0-13.el9.noarch 2.6 MB/s | 269 kB 00:00 2024-05-09T07:59:56.687 INFO:teuthology.orchestra.run.smithi001.stdout:(87/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.9 MB/s | 159 kB 00:00 2024-05-09T07:59:56.795 INFO:teuthology.orchestra.run.smithi001.stdout:(88/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 9.5 MB/s | 5.1 MB 00:00 2024-05-09T07:59:56.821 INFO:teuthology.orchestra.run.smithi001.stdout:(89/156): python3-pytz-2021.1-5.el9.noarch.rpm 384 kB/s | 51 kB 00:00 2024-05-09T07:59:56.854 INFO:teuthology.orchestra.run.smithi001.stdout:(90/156): python3-pyasn1-modules-0.4.8-6.el9.no 1.4 MB/s | 279 kB 00:00 2024-05-09T07:59:56.880 INFO:teuthology.orchestra.run.smithi001.stdout:(91/156): python3-requests-oauthlib-1.3.0-12.el 644 kB/s | 54 kB 00:00 2024-05-09T07:59:56.930 INFO:teuthology.orchestra.run.smithi001.stdout:(92/156): python3-toml-0.10.2-6.el9.noarch.rpm 561 kB/s | 42 kB 00:00 2024-05-09T07:59:56.955 INFO:teuthology.orchestra.run.smithi001.stdout:(93/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 596 kB/s | 44 kB 00:00 2024-05-09T07:59:57.005 INFO:teuthology.orchestra.run.smithi001.stdout:(94/156): qatlib-service-24.02.0-1.el9.x86_64.r 733 kB/s | 36 kB 00:00 2024-05-09T07:59:57.039 INFO:teuthology.orchestra.run.smithi001.stdout:(95/156): qatlib-24.02.0-1.el9.x86_64.rpm 2.0 MB/s | 221 kB 00:00 2024-05-09T07:59:57.098 INFO:teuthology.orchestra.run.smithi001.stdout:(96/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 522 kB/s | 47 kB 00:00 2024-05-09T07:59:57.144 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction test succeeded. 2024-05-09T07:59:57.145 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction 2024-05-09T07:59:57.156 INFO:teuthology.orchestra.run.smithi001.stdout:(97/156): socat-1.7.4.1-5.el9.x86_64.rpm 2.6 MB/s | 305 kB 00:00 2024-05-09T07:59:57.181 INFO:teuthology.orchestra.run.smithi001.stdout:(98/156): uuid-1.6.2-55.el9.x86_64.rpm 699 kB/s | 58 kB 00:00 2024-05-09T07:59:57.240 INFO:teuthology.orchestra.run.smithi001.stdout:(99/156): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.1 MB/s | 94 kB 00:00 2024-05-09T07:59:57.266 INFO:teuthology.orchestra.run.smithi001.stdout:(100/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 768 kB/s | 64 kB 00:00 2024-05-09T07:59:57.341 INFO:teuthology.orchestra.run.smithi001.stdout:(101/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 450 kB/s | 45 kB 00:00 2024-05-09T07:59:57.600 INFO:teuthology.orchestra.run.smithi001.stdout:(102/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 21 MB/s | 16 MB 00:00 2024-05-09T07:59:57.625 INFO:teuthology.orchestra.run.smithi001.stdout:(103/156): lua-devel-5.4.4-4.el9.x86_64.rpm 62 kB/s | 22 kB 00:00 2024-05-09T07:59:57.834 INFO:teuthology.orchestra.run.smithi001.stdout:(104/156): protobuf-compiler-3.14.0-13.el9.x86_ 1.7 MB/s | 863 kB 00:00 2024-05-09T07:59:58.194 INFO:teuthology.orchestra.run.smithi001.stdout:(105/156): grpc-data-1.46.7-10.el9.noarch.rpm 54 kB/s | 19 kB 00:00 2024-05-09T07:59:58.259 INFO:teuthology.orchestra.run.smithi001.stdout:(106/156): gperftools-libs-2.9.1-2.el9.x86_64.r 488 kB/s | 309 kB 00:00 2024-05-09T07:59:58.326 INFO:teuthology.orchestra.run.smithi001.stdout:(107/156): abseil-cpp-20211102.0-3.el9.x86_64.r 756 kB/s | 548 kB 00:00 2024-05-09T07:59:58.351 INFO:teuthology.orchestra.run.smithi001.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 286 kB/s | 26 kB 00:00 2024-05-09T07:59:58.401 INFO:teuthology.orchestra.run.smithi001.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 655 kB/s | 49 kB 00:00 2024-05-09T07:59:58.435 INFO:teuthology.orchestra.run.smithi001.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 811 kB/s | 67 kB 00:00 2024-05-09T07:59:58.486 INFO:teuthology.orchestra.run.smithi001.stdout:(111/156): luarocks-3.9.2-1.el9.noarch.rpm 1.8 MB/s | 151 kB 00:00 2024-05-09T07:59:58.611 INFO:teuthology.orchestra.run.smithi001.stdout:(112/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 4.7 MB/s | 839 kB 00:00 2024-05-09T07:59:58.648 INFO:teuthology.orchestra.run.smithi001.stdout:(113/156): python3-asyncssh-2.13.2-5.el9.noarch 3.4 MB/s | 548 kB 00:00 2024-05-09T07:59:58.770 INFO:teuthology.orchestra.run.smithi001.stdout:(114/156): libarrow-9.0.0-11.el9.x86_64.rpm 7.7 MB/s | 4.4 MB 00:00 2024-05-09T07:59:58.795 INFO:teuthology.orchestra.run.smithi001.stdout:(115/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 237 kB/s | 43 kB 00:00 2024-05-09T07:59:58.820 INFO:teuthology.orchestra.run.smithi001.stdout:(116/156): python3-cachetools-4.2.4-1.el9.noarc 184 kB/s | 32 kB 00:00 2024-05-09T07:59:58.846 INFO:teuthology.orchestra.run.smithi001.stdout:(117/156): python3-certifi-2021.10.8-2.el9.noar 202 kB/s | 15 kB 00:00 2024-05-09T07:59:58.904 INFO:teuthology.orchestra.run.smithi001.stdout:(118/156): python3-cheroot-8.6.0-4.el9.noarch.r 2.0 MB/s | 172 kB 00:00 2024-05-09T07:59:58.937 INFO:teuthology.orchestra.run.smithi001.stdout:(119/156): python3-cherrypy-18.6.1-2.el9.noarch 3.0 MB/s | 358 kB 00:00 2024-05-09T07:59:58.971 INFO:teuthology.orchestra.run.smithi001.stdout:(120/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.4 MB/s | 303 kB 00:00 2024-05-09T07:59:58.997 INFO:teuthology.orchestra.run.smithi001.stdout:(121/156): python3-colorama-0.4.4-12.el9.noarch 385 kB/s | 35 kB 00:00 2024-05-09T07:59:59.031 INFO:teuthology.orchestra.run.smithi001.stdout:(122/156): python3-google-auth-2.29.0-1.el9.noa 2.2 MB/s | 206 kB 00:00 2024-05-09T07:59:59.081 INFO:teuthology.orchestra.run.smithi001.stdout:(123/156): python3-grpcio-tools-1.46.7-10.el9.x 1.7 MB/s | 144 kB 00:00 2024-05-09T07:59:59.106 INFO:teuthology.orchestra.run.smithi001.stdout:(124/156): python3-jaraco-8.2.1-3.el9.noarch.rp 143 kB/s | 11 kB 00:00 2024-05-09T07:59:59.189 INFO:teuthology.orchestra.run.smithi001.stdout:(125/156): python3-grpcio-1.46.7-10.el9.x86_64. 9.4 MB/s | 2.0 MB 00:00 2024-05-09T07:59:59.215 INFO:teuthology.orchestra.run.smithi001.stdout:(126/156): python3-jaraco-classes-3.2.1-5.el9.n 133 kB/s | 18 kB 00:00 2024-05-09T07:59:59.265 INFO:teuthology.orchestra.run.smithi001.stdout:(127/156): python3-jaraco-collections-3.0.0-8.e 146 kB/s | 23 kB 00:00 2024-05-09T07:59:59.290 INFO:teuthology.orchestra.run.smithi001.stdout:(128/156): python3-jaraco-functools-3.5.0-2.el9 195 kB/s | 19 kB 00:00 2024-05-09T07:59:59.340 INFO:teuthology.orchestra.run.smithi001.stdout:(129/156): python3-jaraco-text-3.2.0-6.el9.noar 266 kB/s | 20 kB 00:00 2024-05-09T07:59:59.366 INFO:teuthology.orchestra.run.smithi001.stdout:(130/156): python3-logutils-0.3.5-21.el9.noarch 620 kB/s | 46 kB 00:00 2024-05-09T07:59:59.424 INFO:teuthology.orchestra.run.smithi001.stdout:(131/156): python3-kubernetes-26.1.0-2.el9.noar 6.5 MB/s | 1.0 MB 00:00 2024-05-09T07:59:59.450 INFO:teuthology.orchestra.run.smithi001.stdout:(132/156): python3-natsort-7.1.1-5.el9.noarch.r 697 kB/s | 58 kB 00:00 2024-05-09T07:59:59.475 INFO:teuthology.orchestra.run.smithi001.stdout:(133/156): python3-more-itertools-8.12.0-2.el9. 592 kB/s | 79 kB 00:00 2024-05-09T07:59:59.508 INFO:teuthology.orchestra.run.smithi001.stdout:(134/156): python3-pecan-1.4.2-3.el9.noarch.rpm 3.2 MB/s | 272 kB 00:00 2024-05-09T07:59:59.534 INFO:teuthology.orchestra.run.smithi001.stdout:(135/156): python3-portend-3.1.0-2.el9.noarch.r 198 kB/s | 16 kB 00:00 2024-05-09T07:59:59.559 INFO:teuthology.orchestra.run.smithi001.stdout:(136/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.1 MB/s | 90 kB 00:00 2024-05-09T07:59:59.584 INFO:teuthology.orchestra.run.smithi001.stdout:(137/156): python3-pyperclip-1.8.2-2.el9.noarch 378 kB/s | 28 kB 00:00 2024-05-09T07:59:59.609 INFO:teuthology.orchestra.run.smithi001.stdout:(138/156): python3-repoze-lru-0.7-16.el9.noarch 412 kB/s | 31 kB 00:00 2024-05-09T07:59:59.643 INFO:teuthology.orchestra.run.smithi001.stdout:(139/156): python3-routes-2.5.1-5.el9.noarch.rp 2.2 MB/s | 188 kB 00:00 2024-05-09T07:59:59.669 INFO:teuthology.orchestra.run.smithi001.stdout:(140/156): python3-rsa-4.9-2.el9.noarch.rpm 709 kB/s | 59 kB 00:00 2024-05-09T07:59:59.694 INFO:teuthology.orchestra.run.smithi001.stdout:(141/156): python3-tempora-5.0.0-2.el9.noarch.r 429 kB/s | 36 kB 00:00 2024-05-09T07:59:59.719 INFO:teuthology.orchestra.run.smithi001.stdout:(142/156): python3-typing-extensions-4.4.0-2.el 680 kB/s | 51 kB 00:00 2024-05-09T07:59:59.753 INFO:teuthology.orchestra.run.smithi001.stdout:(143/156): python3-webob-1.8.7-6.el9.noarch.rpm 2.7 MB/s | 230 kB 00:00 2024-05-09T07:59:59.778 INFO:teuthology.orchestra.run.smithi001.stdout:(144/156): python3-websocket-client-1.2.3-2.el9 1.1 MB/s | 90 kB 00:00 2024-05-09T07:59:59.820 INFO:teuthology.orchestra.run.smithi001.stdout:(145/156): python3-werkzeug-2.0.3-3.el9.1.noarc 4.2 MB/s | 427 kB 00:00 2024-05-09T07:59:59.845 INFO:teuthology.orchestra.run.smithi001.stdout:(146/156): python3-zc-lockfile-2.0-10.el9.noarc 218 kB/s | 20 kB 00:00 2024-05-09T07:59:59.870 INFO:teuthology.orchestra.run.smithi001.stdout:(147/156): re2-20211101-3.el9.x86_64.rpm 2.0 MB/s | 192 kB 00:00 2024-05-09T07:59:59.954 INFO:teuthology.orchestra.run.smithi001.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 12 MB/s | 1.6 MB 00:00 2024-05-09T08:00:00.329 INFO:teuthology.orchestra.run.smithi001.stdout:(149/156): librados2-19.0.0-2515.g96805dcf.el9. 7.0 MB/s | 3.4 MB 00:00 2024-05-09T08:00:00.405 INFO:teuthology.orchestra.run.smithi001.stdout:(150/156): librbd1-19.0.0-2515.g96805dcf.el9.x8 5.9 MB/s | 3.1 MB 00:00 2024-05-09T08:00:01.063 INFO:teuthology.orchestra.run.smithi001.stdout:(151/156): NetworkManager-team-1.47.5-1.el9.x86 63 kB/s | 42 kB 00:00 2024-05-09T08:00:01.130 INFO:teuthology.orchestra.run.smithi001.stdout:(152/156): NetworkManager-1.47.5-1.el9.x86_64.r 2.0 MB/s | 2.3 MB 00:01 2024-05-09T08:00:01.514 INFO:teuthology.orchestra.run.smithi001.stdout:(153/156): NetworkManager-libnm-1.47.5-1.el9.x8 1.5 MB/s | 1.8 MB 00:01 2024-05-09T08:00:01.939 INFO:teuthology.orchestra.run.smithi001.stdout:(154/156): NetworkManager-tui-1.47.5-1.el9.x86_ 284 kB/s | 248 kB 00:00 2024-05-09T08:00:02.281 INFO:teuthology.orchestra.run.smithi001.stdout:(155/156): openssl-3.2.1-1.el9.x86_64.rpm 1.2 MB/s | 1.3 MB 00:01 2024-05-09T08:00:02.722 INFO:teuthology.orchestra.run.smithi001.stdout:(156/156): openssl-libs-3.2.1-1.el9.x86_64.rpm 2.0 MB/s | 2.4 MB 00:01 2024-05-09T08:00:02.738 INFO:teuthology.orchestra.run.smithi001.stdout:-------------------------------------------------------------------------------- 2024-05-09T08:00:02.739 INFO:teuthology.orchestra.run.smithi001.stdout:Total 10 MB/s | 225 MB 00:22 2024-05-09T08:00:03.521 INFO:teuthology.orchestra.run.smithi007.stdout: Preparing : 1/1 2024-05-09T08:00:03.605 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/164 2024-05-09T08:00:03.645 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-05-09T08:00:03.698 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-05-09T08:00:03.732 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-05-09T08:00:03.757 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-05-09T08:00:03.809 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-05-09T08:00:04.160 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-05-09T08:00:04.331 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : librados2-2:19.0.0-2515.g96805dcf.el9.x86_64 8/164 2024-05-09T08:00:04.516 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: librados2-2:19.0.0-2515.g96805dcf.el9.x86_64 8/164 2024-05-09T08:00:04.542 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libcephfs2-2:19.0.0-2515.g96805dcf.el9.x86_64 9/164 2024-05-09T08:00:04.635 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: libcephfs2-2:19.0.0-2515.g96805dcf.el9.x86_64 9/164 2024-05-09T08:00:04.663 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-rados-2:19.0.0-2515.g96805dcf.el9.x86_64 10/164 2024-05-09T08:00:04.826 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/164 2024-05-09T08:00:04.886 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/164 2024-05-09T08:00:04.914 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libcephsqlite-2:19.0.0-2515.g96805dcf.el9.x86_64 13/164 2024-05-09T08:00:05.011 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2515.g96805dcf.el9.x86_64 13/164 2024-05-09T08:00:05.040 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libradosstriper1-2:19.0.0-2515.g96805dcf.el9.x86 14/164 2024-05-09T08:00:05.086 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2515.g96805dcf.el9.x86 14/164 2024-05-09T08:00:05.128 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-09T08:00:05.150 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-09T08:00:05.351 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-09T08:00:05.481 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-09T08:00:05.614 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : NetworkManager-libnm-1:1.47.5-1.el9.x86_64 17/164 2024-05-09T08:00:05.666 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.5-1.el9.x86_64 17/164 2024-05-09T08:00:05.773 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-09T08:00:05.842 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-09T08:00:05.977 INFO:teuthology.orchestra.run.smithi001.stdout:Running transaction check 2024-05-09T08:00:06.057 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-09T08:00:06.169 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : re2-1:20211101-3.el9.x86_64 19/164 2024-05-09T08:00:06.242 INFO:teuthology.orchestra.run.smithi001.stdout:Transaction check succeeded. 2024-05-09T08:00:06.242 INFO:teuthology.orchestra.run.smithi001.stdout:Running transaction test 2024-05-09T08:00:06.273 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-05-09T08:00:06.306 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-05-09T08:00:06.350 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-05-09T08:00:06.418 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-05-09T08:00:06.466 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-05-09T08:00:06.510 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 25/164 2024-05-09T08:00:06.542 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 26/164 2024-05-09T08:00:06.570 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : lua-5.4.4-4.el9.x86_64 27/164 2024-05-09T08:00:06.642 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 28/164 2024-05-09T08:00:06.694 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 29/164 2024-05-09T08:00:06.722 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 30/164 2024-05-09T08:00:06.758 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 31/164 2024-05-09T08:00:06.837 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 32/164 2024-05-09T08:00:06.869 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-ceph-common-2:19.0.0-2515.g96805dcf.el9. 33/164 2024-05-09T08:00:06.913 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-ceph-argparse-2:19.0.0-2515.g96805dcf.el 34/164 2024-05-09T08:00:06.941 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-cephfs-2:19.0.0-2515.g96805dcf.el9.x86_6 35/164 2024-05-09T08:00:07.015 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 36/164 2024-05-09T08:00:07.065 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : luarocks-3.9.2-1.el9.noarch 37/164 2024-05-09T08:00:07.143 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 38/164 2024-05-09T08:00:07.289 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 39/164 2024-05-09T08:00:07.352 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 40/164 2024-05-09T08:00:07.916 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-rsa-4.9-2.el9.noarch 41/164 2024-05-09T08:00:07.999 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 42/164 2024-05-09T08:00:08.042 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 43/164 2024-05-09T08:00:08.085 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 44/164 2024-05-09T08:00:08.123 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : librados-devel-2:19.0.0-2515.g96805dcf.el9.x86_6 45/164 2024-05-09T08:00:08.166 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 46/164 2024-05-09T08:00:08.323 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 47/164 2024-05-09T08:00:08.385 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 48/164 2024-05-09T08:00:08.442 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 49/164 2024-05-09T08:00:08.516 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 50/164 2024-05-09T08:00:08.552 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 51/164 2024-05-09T08:00:08.588 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 52/164 2024-05-09T08:00:08.637 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 53/164 2024-05-09T08:00:08.671 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 54/164 2024-05-09T08:00:08.715 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 55/164 2024-05-09T08:00:08.758 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 56/164 2024-05-09T08:00:08.904 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 57/164 2024-05-09T08:00:09.072 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 58/164 2024-05-09T08:00:09.109 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 59/164 2024-05-09T08:00:09.152 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 60/164 2024-05-09T08:00:09.252 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-05-09T08:00:09.292 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 62/164 2024-05-09T08:00:09.332 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-05-09T08:00:09.357 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-05-09T08:00:09.367 INFO:teuthology.orchestra.run.smithi001.stdout:Transaction test succeeded. 2024-05-09T08:00:09.369 INFO:teuthology.orchestra.run.smithi001.stdout:Running transaction 2024-05-09T08:00:09.382 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-05-09T08:00:09.462 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-05-09T08:00:09.592 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-05-09T08:00:09.632 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-05-09T08:00:09.660 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-05-09T08:00:09.703 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-05-09T08:00:09.743 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-05-09T08:00:09.799 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-05-09T08:00:09.828 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-05-09T08:00:09.872 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-05-09T08:00:09.905 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-05-09T08:00:09.947 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-05-09T08:00:09.981 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-05-09T08:00:10.064 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-05-09T08:00:10.112 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-05-09T08:00:10.143 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-05-09T08:00:10.191 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-05-09T08:00:10.303 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-packaging-20.9-5.el9.noarch 82/164 2024-05-09T08:00:10.371 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/164 2024-05-09T08:00:11.163 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/164 2024-05-09T08:00:11.208 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/164 2024-05-09T08:00:11.242 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/164 2024-05-09T08:00:11.351 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/164 2024-05-09T08:00:11.410 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 88/164 2024-05-09T08:00:11.502 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-05-09T08:00:11.544 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-05-09T08:00:11.731 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : openblas-0.3.21-2.el9.x86_64 91/164 2024-05-09T08:00:11.758 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 92/164 2024-05-09T08:00:11.841 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-05-09T08:00:12.548 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-05-09T08:00:12.681 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-05-09T08:00:14.015 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-05-09T08:00:14.089 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-05-09T08:00:14.117 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-05-09T08:00:14.150 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 99/164 2024-05-09T08:00:14.479 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-05-09T08:00:14.536 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : librbd1-2:19.0.0-2515.g96805dcf.el9.x86_64 101/164 2024-05-09T08:00:14.621 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: librbd1-2:19.0.0-2515.g96805dcf.el9.x86_64 101/164 2024-05-09T08:00:14.651 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-rbd-2:19.0.0-2515.g96805dcf.el9.x86_64 102/164 2024-05-09T08:00:14.689 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-05-09T08:00:15.188 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-05-09T08:00:15.299 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : librgw2-2:19.0.0-2515.g96805dcf.el9.x86_64 105/164 2024-05-09T08:00:15.370 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: librgw2-2:19.0.0-2515.g96805dcf.el9.x86_64 105/164 2024-05-09T08:00:15.401 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-rgw-2:19.0.0-2515.g96805dcf.el9.x86_64 106/164 2024-05-09T08:00:15.893 INFO:teuthology.orchestra.run.smithi001.stdout: Preparing : 1/1 2024-05-09T08:00:15.971 INFO:teuthology.orchestra.run.smithi001.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/164 2024-05-09T08:00:16.060 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-05-09T08:00:16.178 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-05-09T08:00:16.237 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-05-09T08:00:16.282 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-05-09T08:00:16.381 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-05-09T08:00:16.830 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-05-09T08:00:16.930 INFO:teuthology.orchestra.run.smithi001.stdout: Upgrading : librados2-2:19.0.0-2515.g96805dcf.el9.x86_64 8/164 2024-05-09T08:00:17.100 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: librados2-2:19.0.0-2515.g96805dcf.el9.x86_64 8/164 2024-05-09T08:00:17.131 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : libcephfs2-2:19.0.0-2515.g96805dcf.el9.x86_64 9/164 2024-05-09T08:00:17.259 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: libcephfs2-2:19.0.0-2515.g96805dcf.el9.x86_64 9/164 2024-05-09T08:00:17.296 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-rados-2:19.0.0-2515.g96805dcf.el9.x86_64 10/164 2024-05-09T08:00:17.613 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/164 2024-05-09T08:00:17.694 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/164 2024-05-09T08:00:17.717 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-common-2:19.0.0-2515.g96805dcf.el9.x86_64 107/164 2024-05-09T08:00:17.718 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : libcephsqlite-2:19.0.0-2515.g96805dcf.el9.x86_64 13/164 2024-05-09T08:00:17.747 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-common-2:19.0.0-2515.g96805dcf.el9.x86_64 107/164 2024-05-09T08:00:17.883 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2515.g96805dcf.el9.x86_64 13/164 2024-05-09T08:00:17.911 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : libradosstriper1-2:19.0.0-2515.g96805dcf.el9.x86 14/164 2024-05-09T08:00:17.998 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2515.g96805dcf.el9.x86 14/164 2024-05-09T08:00:18.029 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-09T08:00:18.086 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-09T08:00:18.317 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-common-2:19.0.0-2515.g96805dcf.el9.x86_64 107/164 2024-05-09T08:00:18.349 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-09T08:00:18.367 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-base-2:19.0.0-2515.g96805dcf.el9.x86_64 108/164 2024-05-09T08:00:18.448 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-base-2:19.0.0-2515.g96805dcf.el9.x86_64 108/164 2024-05-09T08:00:18.448 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-09T08:00:18.448 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-09T08:00:18.449 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-05-09T08:00:18.476 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-selinux-2:19.0.0-2515.g96805dcf.el9.x86_64 109/164 2024-05-09T08:00:18.716 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-09T08:00:18.885 INFO:teuthology.orchestra.run.smithi001.stdout: Upgrading : NetworkManager-libnm-1:1.47.5-1.el9.x86_64 17/164 2024-05-09T08:00:18.974 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.5-1.el9.x86_64 17/164 2024-05-09T08:00:19.213 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-09T08:00:19.297 INFO:teuthology.orchestra.run.smithi001.stdout: Upgrading : NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-09T08:00:19.636 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-09T08:00:19.786 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : re2-1:20211101-3.el9.x86_64 19/164 2024-05-09T08:00:19.914 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-05-09T08:00:19.957 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-05-09T08:00:20.010 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-05-09T08:00:20.115 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-05-09T08:00:20.166 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-05-09T08:00:20.231 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 25/164 2024-05-09T08:00:20.279 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 26/164 2024-05-09T08:00:20.318 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : lua-5.4.4-4.el9.x86_64 27/164 2024-05-09T08:00:20.459 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 28/164 2024-05-09T08:00:20.533 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 29/164 2024-05-09T08:00:20.571 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 30/164 2024-05-09T08:00:20.627 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 31/164 2024-05-09T08:00:20.741 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 32/164 2024-05-09T08:00:20.791 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-ceph-common-2:19.0.0-2515.g96805dcf.el9. 33/164 2024-05-09T08:00:20.851 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-ceph-argparse-2:19.0.0-2515.g96805dcf.el 34/164 2024-05-09T08:00:20.881 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-cephfs-2:19.0.0-2515.g96805dcf.el9.x86_6 35/164 2024-05-09T08:00:21.003 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 36/164 2024-05-09T08:00:21.067 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : luarocks-3.9.2-1.el9.noarch 37/164 2024-05-09T08:00:21.173 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 38/164 2024-05-09T08:00:21.351 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 39/164 2024-05-09T08:00:21.419 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 40/164 2024-05-09T08:00:22.099 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-rsa-4.9-2.el9.noarch 41/164 2024-05-09T08:00:22.211 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 42/164 2024-05-09T08:00:22.245 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 43/164 2024-05-09T08:00:22.308 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 44/164 2024-05-09T08:00:22.357 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : librados-devel-2:19.0.0-2515.g96805dcf.el9.x86_6 45/164 2024-05-09T08:00:22.420 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 46/164 2024-05-09T08:00:22.628 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 47/164 2024-05-09T08:00:22.689 INFO:teuthology.orchestra.run.smithi001.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 48/164 2024-05-09T08:00:22.764 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 49/164 2024-05-09T08:00:22.862 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 50/164 2024-05-09T08:00:22.906 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 51/164 2024-05-09T08:00:22.946 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 52/164 2024-05-09T08:00:23.016 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 53/164 2024-05-09T08:00:23.063 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 54/164 2024-05-09T08:00:23.125 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 55/164 2024-05-09T08:00:23.183 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 56/164 2024-05-09T08:00:23.381 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 57/164 2024-05-09T08:00:23.435 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 58/164 2024-05-09T08:00:23.472 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 59/164 2024-05-09T08:00:23.529 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 60/164 2024-05-09T08:00:23.650 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-05-09T08:00:23.688 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 62/164 2024-05-09T08:00:23.733 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-05-09T08:00:23.769 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-05-09T08:00:23.792 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-05-09T08:00:23.953 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-05-09T08:00:24.143 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-05-09T08:00:24.211 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-05-09T08:00:24.260 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-05-09T08:00:24.308 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-05-09T08:00:24.376 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-05-09T08:00:24.431 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-05-09T08:00:24.481 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-05-09T08:00:24.526 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-05-09T08:00:24.586 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-05-09T08:00:24.661 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-05-09T08:00:24.707 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-05-09T08:00:24.852 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-05-09T08:00:24.911 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-05-09T08:00:24.956 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-05-09T08:00:25.027 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-05-09T08:00:25.177 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-packaging-20.9-5.el9.noarch 82/164 2024-05-09T08:00:25.244 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/164 2024-05-09T08:00:26.234 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/164 2024-05-09T08:00:26.285 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/164 2024-05-09T08:00:26.318 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/164 2024-05-09T08:00:26.454 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/164 2024-05-09T08:00:26.523 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 88/164 2024-05-09T08:00:26.631 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-05-09T08:00:26.695 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-05-09T08:00:26.942 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : openblas-0.3.21-2.el9.x86_64 91/164 2024-05-09T08:00:26.975 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 92/164 2024-05-09T08:00:27.083 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-05-09T08:00:27.951 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-05-09T08:00:28.093 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-05-09T08:00:29.777 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-05-09T08:00:29.873 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-05-09T08:00:29.909 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-05-09T08:00:29.944 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 99/164 2024-05-09T08:00:30.358 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-05-09T08:00:30.401 INFO:teuthology.orchestra.run.smithi001.stdout: Upgrading : librbd1-2:19.0.0-2515.g96805dcf.el9.x86_64 101/164 2024-05-09T08:00:30.446 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2515.g96805dcf.el9.x86_64 109/164 2024-05-09T08:00:30.446 INFO:teuthology.orchestra.run.smithi007.stdout:skipping the directory /sys 2024-05-09T08:00:30.447 INFO:teuthology.orchestra.run.smithi007.stdout:skipping the directory /proc 2024-05-09T08:00:30.447 INFO:teuthology.orchestra.run.smithi007.stdout:skipping the directory /mnt 2024-05-09T08:00:30.447 INFO:teuthology.orchestra.run.smithi007.stdout:skipping the directory /var/tmp 2024-05-09T08:00:30.447 INFO:teuthology.orchestra.run.smithi007.stdout:skipping the directory /home 2024-05-09T08:00:30.447 INFO:teuthology.orchestra.run.smithi007.stdout:skipping the directory /root 2024-05-09T08:00:30.447 INFO:teuthology.orchestra.run.smithi007.stdout:skipping the directory /tmp 2024-05-09T08:00:30.447 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-05-09T08:00:30.526 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: librbd1-2:19.0.0-2515.g96805dcf.el9.x86_64 101/164 2024-05-09T08:00:30.558 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-rbd-2:19.0.0-2515.g96805dcf.el9.x86_64 102/164 2024-05-09T08:00:30.614 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-05-09T08:00:30.638 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-volume-2:19.0.0-2515.g96805dcf.el9.noarch 110/164 2024-05-09T08:00:30.654 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-volume-2:19.0.0-2515.g96805dcf.el9.noarch 110/164 2024-05-09T08:00:30.654 INFO:teuthology.orchestra.run.smithi007.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T08:00:30.654 INFO:teuthology.orchestra.run.smithi007.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-09T08:00:30.655 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-05-09T08:00:31.203 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-05-09T08:00:31.255 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : librgw2-2:19.0.0-2515.g96805dcf.el9.x86_64 105/164 2024-05-09T08:00:31.356 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: librgw2-2:19.0.0-2515.g96805dcf.el9.x86_64 105/164 2024-05-09T08:00:31.382 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-rgw-2:19.0.0-2515.g96805dcf.el9.x86_64 106/164 2024-05-09T08:00:32.174 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-osd-2:19.0.0-2515.g96805dcf.el9.x86_64 111/164 2024-05-09T08:00:32.209 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-osd-2:19.0.0-2515.g96805dcf.el9.x86_64 111/164 2024-05-09T08:00:32.209 INFO:teuthology.orchestra.run.smithi007.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T08:00:32.209 INFO:teuthology.orchestra.run.smithi007.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-09T08:00:32.209 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-09T08:00:32.209 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-09T08:00:32.209 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-05-09T08:00:32.443 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-mds-2:19.0.0-2515.g96805dcf.el9.x86_64 112/164 2024-05-09T08:00:32.473 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-mds-2:19.0.0-2515.g96805dcf.el9.x86_64 112/164 2024-05-09T08:00:32.473 INFO:teuthology.orchestra.run.smithi007.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T08:00:32.473 INFO:teuthology.orchestra.run.smithi007.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-09T08:00:32.473 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-09T08:00:32.474 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-09T08:00:32.474 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-05-09T08:00:32.936 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-mon-2:19.0.0-2515.g96805dcf.el9.x86_64 113/164 2024-05-09T08:00:32.960 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-mon-2:19.0.0-2515.g96805dcf.el9.x86_64 113/164 2024-05-09T08:00:32.960 INFO:teuthology.orchestra.run.smithi007.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T08:00:32.961 INFO:teuthology.orchestra.run.smithi007.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-09T08:00:32.961 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-09T08:00:32.961 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-09T08:00:32.961 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-05-09T08:00:33.084 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-05-09T08:00:33.143 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-05-09T08:00:33.206 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-05-09T08:00:33.406 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-05-09T08:00:33.463 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-05-09T08:00:33.536 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-05-09T08:00:33.635 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-05-09T08:00:33.779 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-05-09T08:00:33.821 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-05-09T08:00:33.852 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-05-09T08:00:33.879 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : fuse-2.9.9-15.el9.x86_64 124/164 2024-05-09T08:00:34.191 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: ceph-common-2:19.0.0-2515.g96805dcf.el9.x86_64 107/164 2024-05-09T08:00:34.236 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-common-2:19.0.0-2515.g96805dcf.el9.x86_64 107/164 2024-05-09T08:00:34.339 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: cephadm-2:19.0.0-2515.g96805dcf.el9.noarch 125/164 2024-05-09T08:00:34.370 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : cephadm-2:19.0.0-2515.g96805dcf.el9.noarch 125/164 2024-05-09T08:00:34.402 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2515.g96805dcf.e 126/164 2024-05-09T08:00:34.467 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2515.g96805dcf. 127/164 2024-05-09T08:00:34.493 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2515.g96805dcf.el9.noa 128/164 2024-05-09T08:00:34.909 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: ceph-common-2:19.0.0-2515.g96805dcf.el9.x86_64 107/164 2024-05-09T08:00:34.956 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-base-2:19.0.0-2515.g96805dcf.el9.x86_64 108/164 2024-05-09T08:00:35.112 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: ceph-base-2:19.0.0-2515.g96805dcf.el9.x86_64 108/164 2024-05-09T08:00:35.112 INFO:teuthology.orchestra.run.smithi001.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-09T08:00:35.112 INFO:teuthology.orchestra.run.smithi001.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-09T08:00:35.112 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T08:00:35.140 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-selinux-2:19.0.0-2515.g96805dcf.el9.x86_64 109/164 2024-05-09T08:00:35.600 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2515.g96805dcf.el9.noa 128/164 2024-05-09T08:00:35.636 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2515.g96805dcf.el9.n 129/164 2024-05-09T08:00:36.656 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2515.g96805dcf.el9.n 129/164 2024-05-09T08:00:36.678 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2515.g968 130/164 2024-05-09T08:00:36.791 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2515.g968 130/164 2024-05-09T08:00:36.957 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2515.g96805dcf.el 131/164 2024-05-09T08:00:36.988 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-mgr-2:19.0.0-2515.g96805dcf.el9.x86_64 132/164 2024-05-09T08:00:37.013 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2515.g96805dcf.el9.x86_64 132/164 2024-05-09T08:00:37.013 INFO:teuthology.orchestra.run.smithi007.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T08:00:37.013 INFO:teuthology.orchestra.run.smithi007.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-09T08:00:37.013 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-09T08:00:37.013 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-09T08:00:37.013 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-05-09T08:00:37.064 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-mgr-rook-2:19.0.0-2515.g96805dcf.el9.noarch 133/164 2024-05-09T08:00:37.076 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2515.g96805dcf.el9.noarch 133/164 2024-05-09T08:00:37.185 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-2:19.0.0-2515.g96805dcf.el9.x86_64 134/164 2024-05-09T08:00:37.239 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-fuse-2:19.0.0-2515.g96805dcf.el9.x86_64 135/164 2024-05-09T08:00:38.187 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libtool-2.4.6-45.el9.x86_64 136/164 2024-05-09T08:00:38.216 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-radosgw-2:19.0.0-2515.g96805dcf.el9.x86_64 137/164 2024-05-09T08:00:38.241 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2515.g96805dcf.el9.x86_64 137/164 2024-05-09T08:00:38.241 INFO:teuthology.orchestra.run.smithi007.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T08:00:38.241 INFO:teuthology.orchestra.run.smithi007.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-09T08:00:38.241 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-09T08:00:38.241 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-09T08:00:38.241 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-05-09T08:00:38.292 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2515.g96805 138/164 2024-05-09T08:00:38.316 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2515.g96805 138/164 2024-05-09T08:00:38.316 INFO:teuthology.orchestra.run.smithi007.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T08:00:38.316 INFO:teuthology.orchestra.run.smithi007.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-09T08:00:38.316 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-05-09T08:00:38.367 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : cephfs-mirror-2:19.0.0-2515.g96805dcf.el9.x86_64 139/164 2024-05-09T08:00:38.392 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2515.g96805dcf.el9.x86_64 139/164 2024-05-09T08:00:38.392 INFO:teuthology.orchestra.run.smithi007.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T08:00:38.392 INFO:teuthology.orchestra.run.smithi007.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-09T08:00:38.392 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-09T08:00:38.392 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-09T08:00:38.392 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-05-09T08:00:38.736 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : rbd-mirror-2:19.0.0-2515.g96805dcf.el9.x86_64 140/164 2024-05-09T08:00:38.760 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2515.g96805dcf.el9.x86_64 140/164 2024-05-09T08:00:38.760 INFO:teuthology.orchestra.run.smithi007.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T08:00:38.760 INFO:teuthology.orchestra.run.smithi007.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-09T08:00:38.760 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-09T08:00:38.760 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-09T08:00:38.760 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-05-09T08:00:43.531 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-test-2:19.0.0-2515.g96805dcf.el9.x86_64 141/164 2024-05-09T08:00:43.572 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : rbd-fuse-2:19.0.0-2515.g96805dcf.el9.x86_64 142/164 2024-05-09T08:00:43.614 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : rbd-nbd-2:19.0.0-2515.g96805dcf.el9.x86_64 143/164 2024-05-09T08:00:43.645 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 144/164 2024-05-09T08:00:43.672 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 145/164 2024-05-09T08:00:43.874 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : cephfs-shell-2:19.0.0-2515.g96805dcf.el9.x86_64 146/164 2024-05-09T08:00:43.918 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 147/164 2024-05-09T08:00:43.940 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libcephfs-devel-2:19.0.0-2515.g96805dcf.el9.x86_ 148/164 2024-05-09T08:00:43.969 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : NetworkManager-team-1:1.47.5-1.el9.x86_64 149/164 2024-05-09T08:00:44.007 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : NetworkManager-tui-1:1.47.5-1.el9.x86_64 150/164 2024-05-09T08:00:44.584 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-09T08:00:44.621 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-09T08:00:44.642 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-09T08:00:45.569 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : cephfs-top-2:19.0.0-2515.g96805dcf.el9.noarch 152/164 2024-05-09T08:00:45.659 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 153/164 2024-05-09T08:00:45.702 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-05-09T08:00:45.797 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-05-09T08:00:45.829 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-05-09T08:00:45.855 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 157/164 2024-05-09T08:00:45.855 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-09T08:00:45.939 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-09T08:00:45.972 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 159/164 2024-05-09T08:00:45.972 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-09T08:00:46.057 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-09T08:00:46.081 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 161/164 2024-05-09T08:00:46.087 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-09T08:00:46.087 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-09T08:00:46.275 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-09T08:00:46.275 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-09T08:00:46.360 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-09T08:00:46.426 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-09T08:00:48.579 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2515.g96805dcf.el9.x86_64 109/164 2024-05-09T08:00:48.579 INFO:teuthology.orchestra.run.smithi001.stdout:skipping the directory /sys 2024-05-09T08:00:48.579 INFO:teuthology.orchestra.run.smithi001.stdout:skipping the directory /proc 2024-05-09T08:00:48.579 INFO:teuthology.orchestra.run.smithi001.stdout:skipping the directory /mnt 2024-05-09T08:00:48.579 INFO:teuthology.orchestra.run.smithi001.stdout:skipping the directory /var/tmp 2024-05-09T08:00:48.579 INFO:teuthology.orchestra.run.smithi001.stdout:skipping the directory /home 2024-05-09T08:00:48.579 INFO:teuthology.orchestra.run.smithi001.stdout:skipping the directory /root 2024-05-09T08:00:48.580 INFO:teuthology.orchestra.run.smithi001.stdout:skipping the directory /tmp 2024-05-09T08:00:48.580 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T08:00:48.816 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-volume-2:19.0.0-2515.g96805dcf.el9.noarch 110/164 2024-05-09T08:00:48.871 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: ceph-volume-2:19.0.0-2515.g96805dcf.el9.noarch 110/164 2024-05-09T08:00:48.871 INFO:teuthology.orchestra.run.smithi001.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T08:00:48.871 INFO:teuthology.orchestra.run.smithi001.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-09T08:00:48.871 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T08:00:50.560 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-osd-2:19.0.0-2515.g96805dcf.el9.x86_64 111/164 2024-05-09T08:00:50.647 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: ceph-osd-2:19.0.0-2515.g96805dcf.el9.x86_64 111/164 2024-05-09T08:00:50.648 INFO:teuthology.orchestra.run.smithi001.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T08:00:50.648 INFO:teuthology.orchestra.run.smithi001.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-09T08:00:50.648 INFO:teuthology.orchestra.run.smithi001.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-09T08:00:50.648 INFO:teuthology.orchestra.run.smithi001.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-09T08:00:50.648 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T08:00:50.902 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-mds-2:19.0.0-2515.g96805dcf.el9.x86_64 112/164 2024-05-09T08:00:50.978 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: ceph-mds-2:19.0.0-2515.g96805dcf.el9.x86_64 112/164 2024-05-09T08:00:50.978 INFO:teuthology.orchestra.run.smithi001.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T08:00:50.979 INFO:teuthology.orchestra.run.smithi001.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-09T08:00:50.979 INFO:teuthology.orchestra.run.smithi001.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-09T08:00:50.979 INFO:teuthology.orchestra.run.smithi001.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-09T08:00:50.979 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T08:00:51.503 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-mon-2:19.0.0-2515.g96805dcf.el9.x86_64 113/164 2024-05-09T08:00:51.589 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: ceph-mon-2:19.0.0-2515.g96805dcf.el9.x86_64 113/164 2024-05-09T08:00:51.589 INFO:teuthology.orchestra.run.smithi001.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T08:00:51.589 INFO:teuthology.orchestra.run.smithi001.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-09T08:00:51.589 INFO:teuthology.orchestra.run.smithi001.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-09T08:00:51.589 INFO:teuthology.orchestra.run.smithi001.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-09T08:00:51.589 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T08:00:51.738 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-05-09T08:00:51.804 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-05-09T08:00:51.880 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-05-09T08:00:51.961 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-09T08:00:51.961 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-2:19.0.0-2515.g96805dcf.el9.x86_64 1/164 2024-05-09T08:00:51.961 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-base-2:19.0.0-2515.g96805dcf.el9.x86_64 2/164 2024-05-09T08:00:51.961 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-common-2:19.0.0-2515.g96805dcf.el9.x86_64 3/164 2024-05-09T08:00:51.961 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-fuse-2:19.0.0-2515.g96805dcf.el9.x86_64 4/164 2024-05-09T08:00:51.961 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2515.g96805 5/164 2024-05-09T08:00:51.961 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-mds-2:19.0.0-2515.g96805dcf.el9.x86_64 6/164 2024-05-09T08:00:51.961 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-mgr-2:19.0.0-2515.g96805dcf.el9.x86_64 7/164 2024-05-09T08:00:51.962 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-mon-2:19.0.0-2515.g96805dcf.el9.x86_64 8/164 2024-05-09T08:00:51.962 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-osd-2:19.0.0-2515.g96805dcf.el9.x86_64 9/164 2024-05-09T08:00:51.962 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-radosgw-2:19.0.0-2515.g96805dcf.el9.x86_64 10/164 2024-05-09T08:00:51.962 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-selinux-2:19.0.0-2515.g96805dcf.el9.x86_64 11/164 2024-05-09T08:00:51.962 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-test-2:19.0.0-2515.g96805dcf.el9.x86_64 12/164 2024-05-09T08:00:51.962 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : cephfs-mirror-2:19.0.0-2515.g96805dcf.el9.x86_64 13/164 2024-05-09T08:00:51.962 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : cephfs-shell-2:19.0.0-2515.g96805dcf.el9.x86_64 14/164 2024-05-09T08:00:51.962 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libcephfs-devel-2:19.0.0-2515.g96805dcf.el9.x86_ 15/164 2024-05-09T08:00:51.962 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libcephfs2-2:19.0.0-2515.g96805dcf.el9.x86_64 16/164 2024-05-09T08:00:51.962 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libcephsqlite-2:19.0.0-2515.g96805dcf.el9.x86_64 17/164 2024-05-09T08:00:51.962 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : librados-devel-2:19.0.0-2515.g96805dcf.el9.x86_6 18/164 2024-05-09T08:00:51.963 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libradosstriper1-2:19.0.0-2515.g96805dcf.el9.x86 19/164 2024-05-09T08:00:51.963 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : librgw2-2:19.0.0-2515.g96805dcf.el9.x86_64 20/164 2024-05-09T08:00:51.963 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2515.g96805dcf.el 21/164 2024-05-09T08:00:51.963 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-ceph-common-2:19.0.0-2515.g96805dcf.el9. 22/164 2024-05-09T08:00:51.963 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-cephfs-2:19.0.0-2515.g96805dcf.el9.x86_6 23/164 2024-05-09T08:00:51.963 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-rados-2:19.0.0-2515.g96805dcf.el9.x86_64 24/164 2024-05-09T08:00:51.963 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-rbd-2:19.0.0-2515.g96805dcf.el9.x86_64 25/164 2024-05-09T08:00:51.963 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-rgw-2:19.0.0-2515.g96805dcf.el9.x86_64 26/164 2024-05-09T08:00:51.963 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : rbd-fuse-2:19.0.0-2515.g96805dcf.el9.x86_64 27/164 2024-05-09T08:00:51.963 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : rbd-mirror-2:19.0.0-2515.g96805dcf.el9.x86_64 28/164 2024-05-09T08:00:51.963 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : rbd-nbd-2:19.0.0-2515.g96805dcf.el9.x86_64 29/164 2024-05-09T08:00:51.963 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2515.g96805dcf. 30/164 2024-05-09T08:00:51.964 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2515.g96805dcf.el9.noa 31/164 2024-05-09T08:00:51.964 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2515.g96805dcf.el9.n 32/164 2024-05-09T08:00:51.964 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2515.g968 33/164 2024-05-09T08:00:51.964 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2515.g96805dcf.el 34/164 2024-05-09T08:00:51.964 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2515.g96805dcf.el9.noarch 35/164 2024-05-09T08:00:51.964 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2515.g96805dcf.e 36/164 2024-05-09T08:00:51.964 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-volume-2:19.0.0-2515.g96805dcf.el9.noarch 37/164 2024-05-09T08:00:51.964 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : cephadm-2:19.0.0-2515.g96805dcf.el9.noarch 38/164 2024-05-09T08:00:51.964 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : cephfs-top-2:19.0.0-2515.g96805dcf.el9.noarch 39/164 2024-05-09T08:00:51.964 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/164 2024-05-09T08:00:51.964 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/164 2024-05-09T08:00:51.964 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/164 2024-05-09T08:00:51.964 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-05-09T08:00:51.965 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-05-09T08:00:51.965 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-05-09T08:00:51.965 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-05-09T08:00:51.965 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-05-09T08:00:51.965 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/164 2024-05-09T08:00:51.965 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-05-09T08:00:51.965 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-05-09T08:00:51.965 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-05-09T08:00:51.965 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/164 2024-05-09T08:00:51.965 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-05-09T08:00:51.966 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-05-09T08:00:51.966 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-05-09T08:00:51.966 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-05-09T08:00:51.966 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-05-09T08:00:51.966 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-05-09T08:00:51.966 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-05-09T08:00:51.966 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-05-09T08:00:51.966 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/164 2024-05-09T08:00:51.966 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/164 2024-05-09T08:00:51.966 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-05-09T08:00:51.966 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-05-09T08:00:51.966 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-05-09T08:00:51.967 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/164 2024-05-09T08:00:51.967 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/164 2024-05-09T08:00:51.967 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/164 2024-05-09T08:00:51.967 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/164 2024-05-09T08:00:51.967 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/164 2024-05-09T08:00:51.967 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/164 2024-05-09T08:00:51.967 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/164 2024-05-09T08:00:51.967 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-05-09T08:00:51.967 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-05-09T08:00:51.967 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-05-09T08:00:51.967 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/164 2024-05-09T08:00:51.967 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/164 2024-05-09T08:00:51.968 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/164 2024-05-09T08:00:51.968 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/164 2024-05-09T08:00:51.968 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/164 2024-05-09T08:00:51.968 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/164 2024-05-09T08:00:51.968 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/164 2024-05-09T08:00:51.968 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/164 2024-05-09T08:00:51.968 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/164 2024-05-09T08:00:51.968 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/164 2024-05-09T08:00:51.968 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/164 2024-05-09T08:00:51.968 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/164 2024-05-09T08:00:51.968 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/164 2024-05-09T08:00:51.968 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/164 2024-05-09T08:00:51.968 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/164 2024-05-09T08:00:51.969 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/164 2024-05-09T08:00:51.969 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/164 2024-05-09T08:00:51.969 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/164 2024-05-09T08:00:51.969 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/164 2024-05-09T08:00:51.969 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/164 2024-05-09T08:00:51.969 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/164 2024-05-09T08:00:51.969 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/164 2024-05-09T08:00:51.969 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/164 2024-05-09T08:00:51.969 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/164 2024-05-09T08:00:51.969 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/164 2024-05-09T08:00:51.969 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/164 2024-05-09T08:00:51.969 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/164 2024-05-09T08:00:51.969 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/164 2024-05-09T08:00:51.970 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/164 2024-05-09T08:00:51.970 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/164 2024-05-09T08:00:51.970 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/164 2024-05-09T08:00:51.970 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/164 2024-05-09T08:00:51.970 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/164 2024-05-09T08:00:51.970 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/164 2024-05-09T08:00:51.970 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/164 2024-05-09T08:00:51.970 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/164 2024-05-09T08:00:51.970 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/164 2024-05-09T08:00:51.970 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/164 2024-05-09T08:00:51.970 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/164 2024-05-09T08:00:51.971 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/164 2024-05-09T08:00:51.971 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/164 2024-05-09T08:00:51.971 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/164 2024-05-09T08:00:51.971 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/164 2024-05-09T08:00:51.971 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/164 2024-05-09T08:00:51.971 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/164 2024-05-09T08:00:51.971 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/164 2024-05-09T08:00:51.971 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/164 2024-05-09T08:00:51.971 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-05-09T08:00:51.971 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-05-09T08:00:51.971 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-05-09T08:00:51.971 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-05-09T08:00:51.971 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-05-09T08:00:51.972 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-05-09T08:00:51.972 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-05-09T08:00:51.972 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-05-09T08:00:51.972 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-05-09T08:00:51.972 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-05-09T08:00:51.972 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-05-09T08:00:51.972 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-05-09T08:00:51.972 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-05-09T08:00:51.972 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-05-09T08:00:51.972 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-05-09T08:00:51.972 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-05-09T08:00:51.972 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-05-09T08:00:51.973 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-05-09T08:00:51.973 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-05-09T08:00:51.973 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/164 2024-05-09T08:00:51.973 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/164 2024-05-09T08:00:51.973 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-05-09T08:00:51.973 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-05-09T08:00:51.973 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-05-09T08:00:51.973 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/164 2024-05-09T08:00:51.973 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-05-09T08:00:51.973 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : librados2-2:19.0.0-2515.g96805dcf.el9.x86_64 149/164 2024-05-09T08:00:51.973 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-05-09T08:00:51.974 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : librbd1-2:19.0.0-2515.g96805dcf.el9.x86_64 151/164 2024-05-09T08:00:51.974 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-05-09T08:00:51.974 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : NetworkManager-1:1.47.5-1.el9.x86_64 153/164 2024-05-09T08:00:51.974 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/164 2024-05-09T08:00:51.974 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : NetworkManager-libnm-1:1.47.5-1.el9.x86_64 155/164 2024-05-09T08:00:51.974 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/164 2024-05-09T08:00:51.974 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : NetworkManager-team-1:1.47.5-1.el9.x86_64 157/164 2024-05-09T08:00:51.974 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/164 2024-05-09T08:00:51.974 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : NetworkManager-tui-1:1.47.5-1.el9.x86_64 159/164 2024-05-09T08:00:51.974 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/164 2024-05-09T08:00:51.974 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 161/164 2024-05-09T08:00:51.974 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 162/164 2024-05-09T08:00:51.975 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 163/164 2024-05-09T08:00:52.141 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-05-09T08:00:52.209 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-05-09T08:00:52.310 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-05-09T08:00:52.421 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-05-09T08:00:52.611 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-05-09T08:00:52.666 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-05-09T08:00:52.726 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-05-09T08:00:52.755 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : fuse-2.9.9-15.el9.x86_64 124/164 2024-05-09T08:00:53.401 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: cephadm-2:19.0.0-2515.g96805dcf.el9.noarch 125/164 2024-05-09T08:00:53.451 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : cephadm-2:19.0.0-2515.g96805dcf.el9.noarch 125/164 2024-05-09T08:00:53.509 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2515.g96805dcf.e 126/164 2024-05-09T08:00:53.634 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2515.g96805dcf. 127/164 2024-05-09T08:00:53.668 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2515.g96805dcf.el9.noa 128/164 2024-05-09T08:00:54.733 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-09T08:00:54.733 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-05-09T08:00:54.733 INFO:teuthology.orchestra.run.smithi007.stdout:Upgraded: 2024-05-09T08:00:54.733 INFO:teuthology.orchestra.run.smithi007.stdout: NetworkManager-1:1.47.5-1.el9.x86_64 2024-05-09T08:00:54.733 INFO:teuthology.orchestra.run.smithi007.stdout: NetworkManager-libnm-1:1.47.5-1.el9.x86_64 2024-05-09T08:00:54.733 INFO:teuthology.orchestra.run.smithi007.stdout: NetworkManager-team-1:1.47.5-1.el9.x86_64 2024-05-09T08:00:54.733 INFO:teuthology.orchestra.run.smithi007.stdout: NetworkManager-tui-1:1.47.5-1.el9.x86_64 2024-05-09T08:00:54.734 INFO:teuthology.orchestra.run.smithi007.stdout: librados2-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:00:54.734 INFO:teuthology.orchestra.run.smithi007.stdout: librbd1-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:00:54.734 INFO:teuthology.orchestra.run.smithi007.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-09T08:00:54.734 INFO:teuthology.orchestra.run.smithi007.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-09T08:00:54.734 INFO:teuthology.orchestra.run.smithi007.stdout:Installed: 2024-05-09T08:00:54.734 INFO:teuthology.orchestra.run.smithi007.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-09T08:00:54.734 INFO:teuthology.orchestra.run.smithi007.stdout: automake-1.16.2-8.el9.noarch 2024-05-09T08:00:54.734 INFO:teuthology.orchestra.run.smithi007.stdout: bison-3.7.4-5.el9.x86_64 2024-05-09T08:00:54.734 INFO:teuthology.orchestra.run.smithi007.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-09T08:00:54.734 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:00:54.734 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-base-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:00:54.734 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-common-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:00:54.735 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-fuse-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:00:54.735 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-grafana-dashboards-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T08:00:54.735 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-immutable-object-cache-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:00:54.735 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mds-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:00:54.735 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:00:54.735 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-cephadm-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T08:00:54.735 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-dashboard-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T08:00:54.735 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T08:00:54.735 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-modules-core-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T08:00:54.735 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-rook-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T08:00:54.735 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mon-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:00:54.735 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-osd-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:00:54.735 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-prometheus-alerts-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T08:00:54.736 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-radosgw-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:00:54.736 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-selinux-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:00:54.736 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-test-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:00:54.736 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-volume-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T08:00:54.736 INFO:teuthology.orchestra.run.smithi007.stdout: cephadm-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T08:00:54.736 INFO:teuthology.orchestra.run.smithi007.stdout: cephfs-mirror-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:00:54.736 INFO:teuthology.orchestra.run.smithi007.stdout: cephfs-shell-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:00:54.736 INFO:teuthology.orchestra.run.smithi007.stdout: cephfs-top-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T08:00:54.736 INFO:teuthology.orchestra.run.smithi007.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-05-09T08:00:54.736 INFO:teuthology.orchestra.run.smithi007.stdout: flex-2.6.4-9.el9.x86_64 2024-05-09T08:00:54.736 INFO:teuthology.orchestra.run.smithi007.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-09T08:00:54.736 INFO:teuthology.orchestra.run.smithi007.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-09T08:00:54.736 INFO:teuthology.orchestra.run.smithi007.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-09T08:00:54.737 INFO:teuthology.orchestra.run.smithi007.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-09T08:00:54.737 INFO:teuthology.orchestra.run.smithi007.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-09T08:00:54.737 INFO:teuthology.orchestra.run.smithi007.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-09T08:00:54.737 INFO:teuthology.orchestra.run.smithi007.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-09T08:00:54.737 INFO:teuthology.orchestra.run.smithi007.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-09T08:00:54.737 INFO:teuthology.orchestra.run.smithi007.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-09T08:00:54.737 INFO:teuthology.orchestra.run.smithi007.stdout: libcephfs-devel-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:00:54.737 INFO:teuthology.orchestra.run.smithi007.stdout: libcephfs2-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:00:54.737 INFO:teuthology.orchestra.run.smithi007.stdout: libcephsqlite-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:00:54.737 INFO:teuthology.orchestra.run.smithi007.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-09T08:00:54.737 INFO:teuthology.orchestra.run.smithi007.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-09T08:00:54.737 INFO:teuthology.orchestra.run.smithi007.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-09T08:00:54.737 INFO:teuthology.orchestra.run.smithi007.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-09T08:00:54.738 INFO:teuthology.orchestra.run.smithi007.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-09T08:00:54.738 INFO:teuthology.orchestra.run.smithi007.stdout: librados-devel-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:00:54.738 INFO:teuthology.orchestra.run.smithi007.stdout: libradosstriper1-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:00:54.738 INFO:teuthology.orchestra.run.smithi007.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-09T08:00:54.738 INFO:teuthology.orchestra.run.smithi007.stdout: librgw2-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:00:54.738 INFO:teuthology.orchestra.run.smithi007.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-09T08:00:54.738 INFO:teuthology.orchestra.run.smithi007.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-09T08:00:54.738 INFO:teuthology.orchestra.run.smithi007.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-09T08:00:54.738 INFO:teuthology.orchestra.run.smithi007.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-09T08:00:54.738 INFO:teuthology.orchestra.run.smithi007.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-09T08:00:54.738 INFO:teuthology.orchestra.run.smithi007.stdout: lua-5.4.4-4.el9.x86_64 2024-05-09T08:00:54.738 INFO:teuthology.orchestra.run.smithi007.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-09T08:00:54.739 INFO:teuthology.orchestra.run.smithi007.stdout: luarocks-3.9.2-1.el9.noarch 2024-05-09T08:00:54.739 INFO:teuthology.orchestra.run.smithi007.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-09T08:00:54.739 INFO:teuthology.orchestra.run.smithi007.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-09T08:00:54.739 INFO:teuthology.orchestra.run.smithi007.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-09T08:00:54.739 INFO:teuthology.orchestra.run.smithi007.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-09T08:00:54.739 INFO:teuthology.orchestra.run.smithi007.stdout: postgresql-13.14-1.el9.x86_64 2024-05-09T08:00:54.739 INFO:teuthology.orchestra.run.smithi007.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-09T08:00:54.739 INFO:teuthology.orchestra.run.smithi007.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-09T08:00:54.739 INFO:teuthology.orchestra.run.smithi007.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-09T08:00:54.739 INFO:teuthology.orchestra.run.smithi007.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-09T08:00:54.739 INFO:teuthology.orchestra.run.smithi007.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-09T08:00:54.739 INFO:teuthology.orchestra.run.smithi007.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-09T08:00:54.739 INFO:teuthology.orchestra.run.smithi007.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-09T08:00:54.740 INFO:teuthology.orchestra.run.smithi007.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-09T08:00:54.740 INFO:teuthology.orchestra.run.smithi007.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-09T08:00:54.740 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-09T08:00:54.740 INFO:teuthology.orchestra.run.smithi007.stdout: python3-ceph-argparse-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:00:54.740 INFO:teuthology.orchestra.run.smithi007.stdout: python3-ceph-common-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:00:54.740 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cephfs-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:00:54.740 INFO:teuthology.orchestra.run.smithi007.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-09T08:00:54.740 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-09T08:00:54.740 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-09T08:00:54.740 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-09T08:00:54.740 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-09T08:00:54.740 INFO:teuthology.orchestra.run.smithi007.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-05-09T08:00:54.740 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-09T08:00:54.741 INFO:teuthology.orchestra.run.smithi007.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-09T08:00:54.741 INFO:teuthology.orchestra.run.smithi007.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-09T08:00:54.741 INFO:teuthology.orchestra.run.smithi007.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-09T08:00:54.741 INFO:teuthology.orchestra.run.smithi007.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-09T08:00:54.741 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-09T08:00:54.741 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-09T08:00:54.741 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-09T08:00:54.741 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-09T08:00:54.741 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-09T08:00:54.741 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-05-09T08:00:54.741 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-09T08:00:54.741 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-09T08:00:54.741 INFO:teuthology.orchestra.run.smithi007.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-09T08:00:54.742 INFO:teuthology.orchestra.run.smithi007.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-09T08:00:54.742 INFO:teuthology.orchestra.run.smithi007.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-09T08:00:54.742 INFO:teuthology.orchestra.run.smithi007.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-09T08:00:54.742 INFO:teuthology.orchestra.run.smithi007.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-09T08:00:54.742 INFO:teuthology.orchestra.run.smithi007.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-09T08:00:54.742 INFO:teuthology.orchestra.run.smithi007.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-09T08:00:54.742 INFO:teuthology.orchestra.run.smithi007.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-09T08:00:54.742 INFO:teuthology.orchestra.run.smithi007.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-09T08:00:54.742 INFO:teuthology.orchestra.run.smithi007.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-09T08:00:54.742 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-09T08:00:54.742 INFO:teuthology.orchestra.run.smithi007.stdout: python3-ply-3.11-14.el9.noarch 2024-05-09T08:00:54.742 INFO:teuthology.orchestra.run.smithi007.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-09T08:00:54.742 INFO:teuthology.orchestra.run.smithi007.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-09T08:00:54.743 INFO:teuthology.orchestra.run.smithi007.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-09T08:00:54.743 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-09T08:00:54.743 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-09T08:00:54.743 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-09T08:00:54.743 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-09T08:00:54.743 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-09T08:00:54.743 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-09T08:00:54.743 INFO:teuthology.orchestra.run.smithi007.stdout: python3-rados-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:00:54.743 INFO:teuthology.orchestra.run.smithi007.stdout: python3-rbd-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:00:54.743 INFO:teuthology.orchestra.run.smithi007.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-09T08:00:54.743 INFO:teuthology.orchestra.run.smithi007.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-09T08:00:54.743 INFO:teuthology.orchestra.run.smithi007.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-09T08:00:54.744 INFO:teuthology.orchestra.run.smithi007.stdout: python3-rgw-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:00:54.744 INFO:teuthology.orchestra.run.smithi007.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-09T08:00:54.744 INFO:teuthology.orchestra.run.smithi007.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-09T08:00:54.744 INFO:teuthology.orchestra.run.smithi007.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-09T08:00:54.744 INFO:teuthology.orchestra.run.smithi007.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-09T08:00:54.744 INFO:teuthology.orchestra.run.smithi007.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-09T08:00:54.744 INFO:teuthology.orchestra.run.smithi007.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-09T08:00:54.744 INFO:teuthology.orchestra.run.smithi007.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-09T08:00:54.744 INFO:teuthology.orchestra.run.smithi007.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-09T08:00:54.744 INFO:teuthology.orchestra.run.smithi007.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-09T08:00:54.744 INFO:teuthology.orchestra.run.smithi007.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-09T08:00:54.744 INFO:teuthology.orchestra.run.smithi007.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-09T08:00:54.744 INFO:teuthology.orchestra.run.smithi007.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-09T08:00:54.744 INFO:teuthology.orchestra.run.smithi007.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-09T08:00:54.745 INFO:teuthology.orchestra.run.smithi007.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-09T08:00:54.745 INFO:teuthology.orchestra.run.smithi007.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-09T08:00:54.745 INFO:teuthology.orchestra.run.smithi007.stdout: rbd-fuse-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:00:54.745 INFO:teuthology.orchestra.run.smithi007.stdout: rbd-mirror-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:00:54.745 INFO:teuthology.orchestra.run.smithi007.stdout: rbd-nbd-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:00:54.745 INFO:teuthology.orchestra.run.smithi007.stdout: re2-1:20211101-3.el9.x86_64 2024-05-09T08:00:54.745 INFO:teuthology.orchestra.run.smithi007.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-09T08:00:54.745 INFO:teuthology.orchestra.run.smithi007.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-09T08:00:54.745 INFO:teuthology.orchestra.run.smithi007.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-09T08:00:54.745 INFO:teuthology.orchestra.run.smithi007.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-09T08:00:54.745 INFO:teuthology.orchestra.run.smithi007.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-09T08:00:54.746 INFO:teuthology.orchestra.run.smithi007.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-09T08:00:54.746 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-05-09T08:00:54.746 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-05-09T08:00:55.024 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2515.g96805dcf.el9.noa 128/164 2024-05-09T08:00:55.079 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2515.g96805dcf.el9.n 129/164 2024-05-09T08:00:55.128 DEBUG:teuthology.parallel:result is None 2024-05-09T08:00:56.243 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2515.g96805dcf.el9.n 129/164 2024-05-09T08:00:56.270 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2515.g968 130/164 2024-05-09T08:00:56.425 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2515.g968 130/164 2024-05-09T08:00:56.636 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2515.g96805dcf.el 131/164 2024-05-09T08:00:56.672 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-mgr-2:19.0.0-2515.g96805dcf.el9.x86_64 132/164 2024-05-09T08:00:56.750 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2515.g96805dcf.el9.x86_64 132/164 2024-05-09T08:00:56.750 INFO:teuthology.orchestra.run.smithi001.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T08:00:56.750 INFO:teuthology.orchestra.run.smithi001.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-09T08:00:56.750 INFO:teuthology.orchestra.run.smithi001.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-09T08:00:56.750 INFO:teuthology.orchestra.run.smithi001.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-09T08:00:56.751 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T08:00:56.814 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-mgr-rook-2:19.0.0-2515.g96805dcf.el9.noarch 133/164 2024-05-09T08:00:56.857 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2515.g96805dcf.el9.noarch 133/164 2024-05-09T08:00:57.001 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-2:19.0.0-2515.g96805dcf.el9.x86_64 134/164 2024-05-09T08:00:57.078 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-fuse-2:19.0.0-2515.g96805dcf.el9.x86_64 135/164 2024-05-09T08:00:58.144 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : libtool-2.4.6-45.el9.x86_64 136/164 2024-05-09T08:00:58.184 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-radosgw-2:19.0.0-2515.g96805dcf.el9.x86_64 137/164 2024-05-09T08:00:58.270 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2515.g96805dcf.el9.x86_64 137/164 2024-05-09T08:00:58.270 INFO:teuthology.orchestra.run.smithi001.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T08:00:58.270 INFO:teuthology.orchestra.run.smithi001.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-09T08:00:58.270 INFO:teuthology.orchestra.run.smithi001.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-09T08:00:58.270 INFO:teuthology.orchestra.run.smithi001.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-09T08:00:58.270 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T08:00:58.318 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2515.g96805 138/164 2024-05-09T08:00:58.415 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2515.g96805 138/164 2024-05-09T08:00:58.415 INFO:teuthology.orchestra.run.smithi001.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T08:00:58.415 INFO:teuthology.orchestra.run.smithi001.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-09T08:00:58.415 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T08:00:58.502 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : cephfs-mirror-2:19.0.0-2515.g96805dcf.el9.x86_64 139/164 2024-05-09T08:00:58.590 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2515.g96805dcf.el9.x86_64 139/164 2024-05-09T08:00:58.590 INFO:teuthology.orchestra.run.smithi001.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T08:00:58.590 INFO:teuthology.orchestra.run.smithi001.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-09T08:00:58.590 INFO:teuthology.orchestra.run.smithi001.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-09T08:00:58.590 INFO:teuthology.orchestra.run.smithi001.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-09T08:00:58.590 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T08:00:58.994 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : rbd-mirror-2:19.0.0-2515.g96805dcf.el9.x86_64 140/164 2024-05-09T08:00:59.068 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2515.g96805dcf.el9.x86_64 140/164 2024-05-09T08:00:59.068 INFO:teuthology.orchestra.run.smithi001.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T08:00:59.068 INFO:teuthology.orchestra.run.smithi001.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-09T08:00:59.068 INFO:teuthology.orchestra.run.smithi001.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-09T08:00:59.068 INFO:teuthology.orchestra.run.smithi001.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-09T08:00:59.068 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T08:01:04.250 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-test-2:19.0.0-2515.g96805dcf.el9.x86_64 141/164 2024-05-09T08:01:04.293 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : rbd-fuse-2:19.0.0-2515.g96805dcf.el9.x86_64 142/164 2024-05-09T08:01:04.347 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : rbd-nbd-2:19.0.0-2515.g96805dcf.el9.x86_64 143/164 2024-05-09T08:01:04.382 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 144/164 2024-05-09T08:01:04.419 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 145/164 2024-05-09T08:01:04.710 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : cephfs-shell-2:19.0.0-2515.g96805dcf.el9.x86_64 146/164 2024-05-09T08:01:04.762 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 147/164 2024-05-09T08:01:04.792 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : libcephfs-devel-2:19.0.0-2515.g96805dcf.el9.x86_ 148/164 2024-05-09T08:01:04.834 INFO:teuthology.orchestra.run.smithi001.stdout: Upgrading : NetworkManager-team-1:1.47.5-1.el9.x86_64 149/164 2024-05-09T08:01:04.876 INFO:teuthology.orchestra.run.smithi001.stdout: Upgrading : NetworkManager-tui-1:1.47.5-1.el9.x86_64 150/164 2024-05-09T08:01:05.769 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-09T08:01:05.823 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-09T08:01:05.889 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-09T08:01:07.328 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : cephfs-top-2:19.0.0-2515.g96805dcf.el9.noarch 152/164 2024-05-09T08:01:07.438 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 153/164 2024-05-09T08:01:07.507 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-05-09T08:01:07.654 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-05-09T08:01:07.698 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-05-09T08:01:07.736 INFO:teuthology.orchestra.run.smithi001.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 157/164 2024-05-09T08:01:07.736 INFO:teuthology.orchestra.run.smithi001.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-09T08:01:07.864 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-09T08:01:07.910 INFO:teuthology.orchestra.run.smithi001.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 159/164 2024-05-09T08:01:07.911 INFO:teuthology.orchestra.run.smithi001.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-09T08:01:08.072 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-09T08:01:08.112 INFO:teuthology.orchestra.run.smithi001.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 161/164 2024-05-09T08:01:08.146 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-09T08:01:08.147 INFO:teuthology.orchestra.run.smithi001.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-09T08:01:08.491 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-09T08:01:08.491 INFO:teuthology.orchestra.run.smithi001.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-09T08:01:08.635 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-09T08:01:08.781 INFO:teuthology.orchestra.run.smithi001.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-09T08:01:09.060 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-09T08:01:09.061 INFO:teuthology.orchestra.run.smithi001.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-05-09T08:01:09.061 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T08:01:14.566 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-2:19.0.0-2515.g96805dcf.el9.x86_64 1/164 2024-05-09T08:01:14.566 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-base-2:19.0.0-2515.g96805dcf.el9.x86_64 2/164 2024-05-09T08:01:14.566 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-common-2:19.0.0-2515.g96805dcf.el9.x86_64 3/164 2024-05-09T08:01:14.567 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-fuse-2:19.0.0-2515.g96805dcf.el9.x86_64 4/164 2024-05-09T08:01:14.567 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2515.g96805 5/164 2024-05-09T08:01:14.567 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-mds-2:19.0.0-2515.g96805dcf.el9.x86_64 6/164 2024-05-09T08:01:14.567 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-mgr-2:19.0.0-2515.g96805dcf.el9.x86_64 7/164 2024-05-09T08:01:14.567 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-mon-2:19.0.0-2515.g96805dcf.el9.x86_64 8/164 2024-05-09T08:01:14.567 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-osd-2:19.0.0-2515.g96805dcf.el9.x86_64 9/164 2024-05-09T08:01:14.567 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-radosgw-2:19.0.0-2515.g96805dcf.el9.x86_64 10/164 2024-05-09T08:01:14.567 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-selinux-2:19.0.0-2515.g96805dcf.el9.x86_64 11/164 2024-05-09T08:01:14.567 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-test-2:19.0.0-2515.g96805dcf.el9.x86_64 12/164 2024-05-09T08:01:14.567 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : cephfs-mirror-2:19.0.0-2515.g96805dcf.el9.x86_64 13/164 2024-05-09T08:01:14.567 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : cephfs-shell-2:19.0.0-2515.g96805dcf.el9.x86_64 14/164 2024-05-09T08:01:14.567 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : libcephfs-devel-2:19.0.0-2515.g96805dcf.el9.x86_ 15/164 2024-05-09T08:01:14.568 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : libcephfs2-2:19.0.0-2515.g96805dcf.el9.x86_64 16/164 2024-05-09T08:01:14.568 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : libcephsqlite-2:19.0.0-2515.g96805dcf.el9.x86_64 17/164 2024-05-09T08:01:14.568 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : librados-devel-2:19.0.0-2515.g96805dcf.el9.x86_6 18/164 2024-05-09T08:01:14.568 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : libradosstriper1-2:19.0.0-2515.g96805dcf.el9.x86 19/164 2024-05-09T08:01:14.568 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : librgw2-2:19.0.0-2515.g96805dcf.el9.x86_64 20/164 2024-05-09T08:01:14.568 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2515.g96805dcf.el 21/164 2024-05-09T08:01:14.568 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-ceph-common-2:19.0.0-2515.g96805dcf.el9. 22/164 2024-05-09T08:01:14.568 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-cephfs-2:19.0.0-2515.g96805dcf.el9.x86_6 23/164 2024-05-09T08:01:14.568 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-rados-2:19.0.0-2515.g96805dcf.el9.x86_64 24/164 2024-05-09T08:01:14.568 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-rbd-2:19.0.0-2515.g96805dcf.el9.x86_64 25/164 2024-05-09T08:01:14.582 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-rgw-2:19.0.0-2515.g96805dcf.el9.x86_64 26/164 2024-05-09T08:01:14.582 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : rbd-fuse-2:19.0.0-2515.g96805dcf.el9.x86_64 27/164 2024-05-09T08:01:14.583 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : rbd-mirror-2:19.0.0-2515.g96805dcf.el9.x86_64 28/164 2024-05-09T08:01:14.583 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : rbd-nbd-2:19.0.0-2515.g96805dcf.el9.x86_64 29/164 2024-05-09T08:01:14.583 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2515.g96805dcf. 30/164 2024-05-09T08:01:14.583 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2515.g96805dcf.el9.noa 31/164 2024-05-09T08:01:14.583 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2515.g96805dcf.el9.n 32/164 2024-05-09T08:01:14.583 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2515.g968 33/164 2024-05-09T08:01:14.583 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2515.g96805dcf.el 34/164 2024-05-09T08:01:14.583 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2515.g96805dcf.el9.noarch 35/164 2024-05-09T08:01:14.583 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2515.g96805dcf.e 36/164 2024-05-09T08:01:14.583 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-volume-2:19.0.0-2515.g96805dcf.el9.noarch 37/164 2024-05-09T08:01:14.583 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : cephadm-2:19.0.0-2515.g96805dcf.el9.noarch 38/164 2024-05-09T08:01:14.583 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : cephfs-top-2:19.0.0-2515.g96805dcf.el9.noarch 39/164 2024-05-09T08:01:14.583 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/164 2024-05-09T08:01:14.584 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/164 2024-05-09T08:01:14.584 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/164 2024-05-09T08:01:14.584 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-05-09T08:01:14.584 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-05-09T08:01:14.584 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-05-09T08:01:14.584 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-05-09T08:01:14.584 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-05-09T08:01:14.584 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/164 2024-05-09T08:01:14.584 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-05-09T08:01:14.584 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-05-09T08:01:14.584 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-05-09T08:01:14.584 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/164 2024-05-09T08:01:14.584 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-05-09T08:01:14.585 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-05-09T08:01:14.585 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-05-09T08:01:14.585 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-05-09T08:01:14.585 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-05-09T08:01:14.585 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-05-09T08:01:14.585 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-05-09T08:01:14.585 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-05-09T08:01:14.585 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/164 2024-05-09T08:01:14.586 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/164 2024-05-09T08:01:14.586 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-05-09T08:01:14.586 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-05-09T08:01:14.586 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-05-09T08:01:14.586 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/164 2024-05-09T08:01:14.587 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/164 2024-05-09T08:01:14.587 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/164 2024-05-09T08:01:14.587 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/164 2024-05-09T08:01:14.587 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/164 2024-05-09T08:01:14.587 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/164 2024-05-09T08:01:14.587 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/164 2024-05-09T08:01:14.588 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-05-09T08:01:14.588 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-05-09T08:01:14.588 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-05-09T08:01:14.588 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/164 2024-05-09T08:01:14.588 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/164 2024-05-09T08:01:14.588 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/164 2024-05-09T08:01:14.589 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/164 2024-05-09T08:01:14.589 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/164 2024-05-09T08:01:14.589 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/164 2024-05-09T08:01:14.589 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/164 2024-05-09T08:01:14.589 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/164 2024-05-09T08:01:14.589 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/164 2024-05-09T08:01:14.589 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/164 2024-05-09T08:01:14.590 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/164 2024-05-09T08:01:14.590 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/164 2024-05-09T08:01:14.590 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/164 2024-05-09T08:01:14.590 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/164 2024-05-09T08:01:14.590 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/164 2024-05-09T08:01:14.590 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/164 2024-05-09T08:01:14.591 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/164 2024-05-09T08:01:14.591 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/164 2024-05-09T08:01:14.591 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/164 2024-05-09T08:01:14.591 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/164 2024-05-09T08:01:14.591 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/164 2024-05-09T08:01:14.591 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/164 2024-05-09T08:01:14.592 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/164 2024-05-09T08:01:14.592 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/164 2024-05-09T08:01:14.592 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/164 2024-05-09T08:01:14.592 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/164 2024-05-09T08:01:14.592 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/164 2024-05-09T08:01:14.592 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/164 2024-05-09T08:01:14.593 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/164 2024-05-09T08:01:14.593 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/164 2024-05-09T08:01:14.593 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/164 2024-05-09T08:01:14.593 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/164 2024-05-09T08:01:14.593 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/164 2024-05-09T08:01:14.593 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/164 2024-05-09T08:01:14.593 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/164 2024-05-09T08:01:14.593 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/164 2024-05-09T08:01:14.593 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/164 2024-05-09T08:01:14.593 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/164 2024-05-09T08:01:14.593 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/164 2024-05-09T08:01:14.593 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/164 2024-05-09T08:01:14.593 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/164 2024-05-09T08:01:14.593 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/164 2024-05-09T08:01:14.594 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/164 2024-05-09T08:01:14.594 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/164 2024-05-09T08:01:14.594 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/164 2024-05-09T08:01:14.594 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/164 2024-05-09T08:01:14.594 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/164 2024-05-09T08:01:14.594 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-05-09T08:01:14.594 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-05-09T08:01:14.595 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-05-09T08:01:14.595 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-05-09T08:01:14.595 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-05-09T08:01:14.595 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-05-09T08:01:14.595 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-05-09T08:01:14.595 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-05-09T08:01:14.595 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-05-09T08:01:14.595 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-05-09T08:01:14.595 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-05-09T08:01:14.595 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-05-09T08:01:14.595 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-05-09T08:01:14.596 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-05-09T08:01:14.596 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-05-09T08:01:14.596 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-05-09T08:01:14.596 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-05-09T08:01:14.596 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-05-09T08:01:14.596 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-05-09T08:01:14.596 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/164 2024-05-09T08:01:14.596 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/164 2024-05-09T08:01:14.596 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-05-09T08:01:14.596 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-05-09T08:01:14.596 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-05-09T08:01:14.596 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/164 2024-05-09T08:01:14.596 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-05-09T08:01:14.597 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : librados2-2:19.0.0-2515.g96805dcf.el9.x86_64 149/164 2024-05-09T08:01:14.597 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-05-09T08:01:14.597 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : librbd1-2:19.0.0-2515.g96805dcf.el9.x86_64 151/164 2024-05-09T08:01:14.597 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-05-09T08:01:14.597 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : NetworkManager-1:1.47.5-1.el9.x86_64 153/164 2024-05-09T08:01:14.597 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/164 2024-05-09T08:01:14.597 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : NetworkManager-libnm-1:1.47.5-1.el9.x86_64 155/164 2024-05-09T08:01:14.597 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/164 2024-05-09T08:01:14.597 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : NetworkManager-team-1:1.47.5-1.el9.x86_64 157/164 2024-05-09T08:01:14.597 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/164 2024-05-09T08:01:14.597 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : NetworkManager-tui-1:1.47.5-1.el9.x86_64 159/164 2024-05-09T08:01:14.597 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/164 2024-05-09T08:01:14.597 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 161/164 2024-05-09T08:01:14.597 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 162/164 2024-05-09T08:01:14.597 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 163/164 2024-05-09T08:01:16.739 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-09T08:01:16.739 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T08:01:16.739 INFO:teuthology.orchestra.run.smithi001.stdout:Upgraded: 2024-05-09T08:01:16.739 INFO:teuthology.orchestra.run.smithi001.stdout: NetworkManager-1:1.47.5-1.el9.x86_64 2024-05-09T08:01:16.739 INFO:teuthology.orchestra.run.smithi001.stdout: NetworkManager-libnm-1:1.47.5-1.el9.x86_64 2024-05-09T08:01:16.739 INFO:teuthology.orchestra.run.smithi001.stdout: NetworkManager-team-1:1.47.5-1.el9.x86_64 2024-05-09T08:01:16.740 INFO:teuthology.orchestra.run.smithi001.stdout: NetworkManager-tui-1:1.47.5-1.el9.x86_64 2024-05-09T08:01:16.740 INFO:teuthology.orchestra.run.smithi001.stdout: librados2-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:01:16.740 INFO:teuthology.orchestra.run.smithi001.stdout: librbd1-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:01:16.740 INFO:teuthology.orchestra.run.smithi001.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-09T08:01:16.740 INFO:teuthology.orchestra.run.smithi001.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-09T08:01:16.740 INFO:teuthology.orchestra.run.smithi001.stdout:Installed: 2024-05-09T08:01:16.740 INFO:teuthology.orchestra.run.smithi001.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-09T08:01:16.740 INFO:teuthology.orchestra.run.smithi001.stdout: automake-1.16.2-8.el9.noarch 2024-05-09T08:01:16.740 INFO:teuthology.orchestra.run.smithi001.stdout: bison-3.7.4-5.el9.x86_64 2024-05-09T08:01:16.740 INFO:teuthology.orchestra.run.smithi001.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-09T08:01:16.740 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:01:16.740 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-base-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:01:16.741 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-common-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:01:16.741 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-fuse-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:01:16.741 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-grafana-dashboards-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T08:01:16.741 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-immutable-object-cache-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:01:16.741 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-mds-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:01:16.741 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-mgr-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:01:16.741 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-mgr-cephadm-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T08:01:16.741 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-mgr-dashboard-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T08:01:16.741 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T08:01:16.741 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-mgr-modules-core-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T08:01:16.741 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-mgr-rook-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T08:01:16.741 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-mon-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:01:16.741 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-osd-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:01:16.742 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-prometheus-alerts-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T08:01:16.742 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-radosgw-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:01:16.742 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-selinux-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:01:16.742 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-test-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:01:16.742 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-volume-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T08:01:16.742 INFO:teuthology.orchestra.run.smithi001.stdout: cephadm-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T08:01:16.742 INFO:teuthology.orchestra.run.smithi001.stdout: cephfs-mirror-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:01:16.742 INFO:teuthology.orchestra.run.smithi001.stdout: cephfs-shell-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:01:16.742 INFO:teuthology.orchestra.run.smithi001.stdout: cephfs-top-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T08:01:16.742 INFO:teuthology.orchestra.run.smithi001.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-05-09T08:01:16.742 INFO:teuthology.orchestra.run.smithi001.stdout: flex-2.6.4-9.el9.x86_64 2024-05-09T08:01:16.742 INFO:teuthology.orchestra.run.smithi001.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-09T08:01:16.742 INFO:teuthology.orchestra.run.smithi001.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-09T08:01:16.743 INFO:teuthology.orchestra.run.smithi001.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-09T08:01:16.743 INFO:teuthology.orchestra.run.smithi001.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-09T08:01:16.743 INFO:teuthology.orchestra.run.smithi001.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-09T08:01:16.743 INFO:teuthology.orchestra.run.smithi001.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-09T08:01:16.743 INFO:teuthology.orchestra.run.smithi001.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-09T08:01:16.743 INFO:teuthology.orchestra.run.smithi001.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-09T08:01:16.743 INFO:teuthology.orchestra.run.smithi001.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-09T08:01:16.743 INFO:teuthology.orchestra.run.smithi001.stdout: libcephfs-devel-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:01:16.743 INFO:teuthology.orchestra.run.smithi001.stdout: libcephfs2-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:01:16.743 INFO:teuthology.orchestra.run.smithi001.stdout: libcephsqlite-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:01:16.743 INFO:teuthology.orchestra.run.smithi001.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-09T08:01:16.743 INFO:teuthology.orchestra.run.smithi001.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-09T08:01:16.743 INFO:teuthology.orchestra.run.smithi001.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-09T08:01:16.744 INFO:teuthology.orchestra.run.smithi001.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-09T08:01:16.744 INFO:teuthology.orchestra.run.smithi001.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-09T08:01:16.744 INFO:teuthology.orchestra.run.smithi001.stdout: librados-devel-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:01:16.744 INFO:teuthology.orchestra.run.smithi001.stdout: libradosstriper1-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:01:16.744 INFO:teuthology.orchestra.run.smithi001.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-09T08:01:16.744 INFO:teuthology.orchestra.run.smithi001.stdout: librgw2-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:01:16.744 INFO:teuthology.orchestra.run.smithi001.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-09T08:01:16.744 INFO:teuthology.orchestra.run.smithi001.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-09T08:01:16.744 INFO:teuthology.orchestra.run.smithi001.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-09T08:01:16.744 INFO:teuthology.orchestra.run.smithi001.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-09T08:01:16.744 INFO:teuthology.orchestra.run.smithi001.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-09T08:01:16.744 INFO:teuthology.orchestra.run.smithi001.stdout: lua-5.4.4-4.el9.x86_64 2024-05-09T08:01:16.744 INFO:teuthology.orchestra.run.smithi001.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-09T08:01:16.744 INFO:teuthology.orchestra.run.smithi001.stdout: luarocks-3.9.2-1.el9.noarch 2024-05-09T08:01:16.745 INFO:teuthology.orchestra.run.smithi001.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-09T08:01:16.745 INFO:teuthology.orchestra.run.smithi001.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-09T08:01:16.745 INFO:teuthology.orchestra.run.smithi001.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-09T08:01:16.745 INFO:teuthology.orchestra.run.smithi001.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-09T08:01:16.745 INFO:teuthology.orchestra.run.smithi001.stdout: postgresql-13.14-1.el9.x86_64 2024-05-09T08:01:16.745 INFO:teuthology.orchestra.run.smithi001.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-09T08:01:16.745 INFO:teuthology.orchestra.run.smithi001.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-09T08:01:16.745 INFO:teuthology.orchestra.run.smithi001.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-09T08:01:16.745 INFO:teuthology.orchestra.run.smithi001.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-09T08:01:16.745 INFO:teuthology.orchestra.run.smithi001.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-09T08:01:16.745 INFO:teuthology.orchestra.run.smithi001.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-09T08:01:16.745 INFO:teuthology.orchestra.run.smithi001.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-09T08:01:16.745 INFO:teuthology.orchestra.run.smithi001.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-09T08:01:16.746 INFO:teuthology.orchestra.run.smithi001.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-09T08:01:16.746 INFO:teuthology.orchestra.run.smithi001.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-09T08:01:16.746 INFO:teuthology.orchestra.run.smithi001.stdout: python3-ceph-argparse-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:01:16.746 INFO:teuthology.orchestra.run.smithi001.stdout: python3-ceph-common-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:01:16.746 INFO:teuthology.orchestra.run.smithi001.stdout: python3-cephfs-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:01:16.746 INFO:teuthology.orchestra.run.smithi001.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-09T08:01:16.746 INFO:teuthology.orchestra.run.smithi001.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-09T08:01:16.746 INFO:teuthology.orchestra.run.smithi001.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-09T08:01:16.746 INFO:teuthology.orchestra.run.smithi001.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-09T08:01:16.746 INFO:teuthology.orchestra.run.smithi001.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-09T08:01:16.746 INFO:teuthology.orchestra.run.smithi001.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-05-09T08:01:16.746 INFO:teuthology.orchestra.run.smithi001.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-09T08:01:16.746 INFO:teuthology.orchestra.run.smithi001.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-09T08:01:16.746 INFO:teuthology.orchestra.run.smithi001.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-09T08:01:16.747 INFO:teuthology.orchestra.run.smithi001.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-09T08:01:16.747 INFO:teuthology.orchestra.run.smithi001.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-09T08:01:16.747 INFO:teuthology.orchestra.run.smithi001.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-09T08:01:16.747 INFO:teuthology.orchestra.run.smithi001.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-09T08:01:16.747 INFO:teuthology.orchestra.run.smithi001.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-09T08:01:16.747 INFO:teuthology.orchestra.run.smithi001.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-09T08:01:16.747 INFO:teuthology.orchestra.run.smithi001.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-09T08:01:16.747 INFO:teuthology.orchestra.run.smithi001.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-05-09T08:01:16.747 INFO:teuthology.orchestra.run.smithi001.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-09T08:01:16.747 INFO:teuthology.orchestra.run.smithi001.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-09T08:01:16.747 INFO:teuthology.orchestra.run.smithi001.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-09T08:01:16.747 INFO:teuthology.orchestra.run.smithi001.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-09T08:01:16.747 INFO:teuthology.orchestra.run.smithi001.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-09T08:01:16.747 INFO:teuthology.orchestra.run.smithi001.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-09T08:01:16.748 INFO:teuthology.orchestra.run.smithi001.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-09T08:01:16.748 INFO:teuthology.orchestra.run.smithi001.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-09T08:01:16.748 INFO:teuthology.orchestra.run.smithi001.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-09T08:01:16.748 INFO:teuthology.orchestra.run.smithi001.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-09T08:01:16.748 INFO:teuthology.orchestra.run.smithi001.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-09T08:01:16.748 INFO:teuthology.orchestra.run.smithi001.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-09T08:01:16.748 INFO:teuthology.orchestra.run.smithi001.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-09T08:01:16.748 INFO:teuthology.orchestra.run.smithi001.stdout: python3-ply-3.11-14.el9.noarch 2024-05-09T08:01:16.748 INFO:teuthology.orchestra.run.smithi001.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-09T08:01:16.748 INFO:teuthology.orchestra.run.smithi001.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-09T08:01:16.748 INFO:teuthology.orchestra.run.smithi001.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-09T08:01:16.748 INFO:teuthology.orchestra.run.smithi001.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-09T08:01:16.748 INFO:teuthology.orchestra.run.smithi001.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-09T08:01:16.748 INFO:teuthology.orchestra.run.smithi001.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-09T08:01:16.749 INFO:teuthology.orchestra.run.smithi001.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-09T08:01:16.749 INFO:teuthology.orchestra.run.smithi001.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-09T08:01:16.749 INFO:teuthology.orchestra.run.smithi001.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-09T08:01:16.749 INFO:teuthology.orchestra.run.smithi001.stdout: python3-rados-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:01:16.749 INFO:teuthology.orchestra.run.smithi001.stdout: python3-rbd-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:01:16.749 INFO:teuthology.orchestra.run.smithi001.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-09T08:01:16.749 INFO:teuthology.orchestra.run.smithi001.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-09T08:01:16.749 INFO:teuthology.orchestra.run.smithi001.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-09T08:01:16.749 INFO:teuthology.orchestra.run.smithi001.stdout: python3-rgw-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:01:16.749 INFO:teuthology.orchestra.run.smithi001.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-09T08:01:16.749 INFO:teuthology.orchestra.run.smithi001.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-09T08:01:16.749 INFO:teuthology.orchestra.run.smithi001.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-09T08:01:16.749 INFO:teuthology.orchestra.run.smithi001.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-09T08:01:16.749 INFO:teuthology.orchestra.run.smithi001.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-09T08:01:16.750 INFO:teuthology.orchestra.run.smithi001.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-09T08:01:16.750 INFO:teuthology.orchestra.run.smithi001.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-09T08:01:16.750 INFO:teuthology.orchestra.run.smithi001.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-09T08:01:16.750 INFO:teuthology.orchestra.run.smithi001.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-09T08:01:16.750 INFO:teuthology.orchestra.run.smithi001.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-09T08:01:16.750 INFO:teuthology.orchestra.run.smithi001.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-09T08:01:16.750 INFO:teuthology.orchestra.run.smithi001.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-09T08:01:16.750 INFO:teuthology.orchestra.run.smithi001.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-09T08:01:16.750 INFO:teuthology.orchestra.run.smithi001.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-09T08:01:16.750 INFO:teuthology.orchestra.run.smithi001.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-09T08:01:16.750 INFO:teuthology.orchestra.run.smithi001.stdout: rbd-fuse-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:01:16.750 INFO:teuthology.orchestra.run.smithi001.stdout: rbd-mirror-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:01:16.751 INFO:teuthology.orchestra.run.smithi001.stdout: rbd-nbd-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:01:16.751 INFO:teuthology.orchestra.run.smithi001.stdout: re2-1:20211101-3.el9.x86_64 2024-05-09T08:01:16.751 INFO:teuthology.orchestra.run.smithi001.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-09T08:01:16.751 INFO:teuthology.orchestra.run.smithi001.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-09T08:01:16.751 INFO:teuthology.orchestra.run.smithi001.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-09T08:01:16.751 INFO:teuthology.orchestra.run.smithi001.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-09T08:01:16.751 INFO:teuthology.orchestra.run.smithi001.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-09T08:01:16.751 INFO:teuthology.orchestra.run.smithi001.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-09T08:01:16.752 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T08:01:16.752 INFO:teuthology.orchestra.run.smithi001.stdout:Complete! 2024-05-09T08:01:17.372 DEBUG:teuthology.parallel:result is None 2024-05-09T08:01:17.373 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=96805dcf909541c4ca9a1e90258de8ae5b6fa1bc 2024-05-09T08:01:17.589 DEBUG:teuthology.orchestra.run.smithi001:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-09T08:01:17.650 INFO:teuthology.orchestra.run.smithi001.stdout:19.0.0-2515.g96805dcf.el9 2024-05-09T08:01:17.651 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2515.g96805dcf.el9 2024-05-09T08:01:17.651 INFO:teuthology.task.install:The correct ceph version 19.0.0-2515.g96805dcf is installed. 2024-05-09T08:01:17.653 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=96805dcf909541c4ca9a1e90258de8ae5b6fa1bc 2024-05-09T08:01:17.790 DEBUG:teuthology.orchestra.run.smithi007:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-09T08:01:17.817 INFO:teuthology.orchestra.run.smithi007.stdout:19.0.0-2515.g96805dcf.el9 2024-05-09T08:01:17.817 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2515.g96805dcf.el9 2024-05-09T08:01:17.817 INFO:teuthology.task.install:The correct ceph version 19.0.0-2515.g96805dcf is installed. 2024-05-09T08:01:17.819 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-05-09T08:01:17.819 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-05-09T08:01:17.819 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-09T08:01:17.891 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-05-09T08:01:17.891 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-09T08:01:17.924 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-05-09T08:01:17.924 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-05-09T08:01:17.924 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd of=/usr/bin/daemon-helper 2024-05-09T08:01:17.999 DEBUG:teuthology.orchestra.run.smithi001:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-09T08:01:18.102 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-05-09T08:01:18.102 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/usr/bin/daemon-helper 2024-05-09T08:01:18.131 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-09T08:01:18.200 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-05-09T08:01:18.201 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-05-09T08:01:18.201 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-09T08:01:18.271 DEBUG:teuthology.orchestra.run.smithi001:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-09T08:01:18.374 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-05-09T08:01:18.374 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-09T08:01:18.399 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-09T08:01:18.465 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-05-09T08:01:18.465 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-05-09T08:01:18.465 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd of=/usr/bin/stdin-killer 2024-05-09T08:01:18.537 DEBUG:teuthology.orchestra.run.smithi001:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-09T08:01:18.649 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-05-09T08:01:18.649 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/usr/bin/stdin-killer 2024-05-09T08:01:18.674 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-09T08:01:18.739 INFO:teuthology.run_tasks:Running task ceph... 2024-05-09T08:01:18.816 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-05-09T08:01:18.816 DEBUG:teuthology.orchestra.run.smithi001:> sudo chmod 777 /var/log/ceph 2024-05-09T08:01:18.821 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 777 /var/log/ceph 2024-05-09T08:01:18.893 INFO:tasks.ceph:Disabling ceph logrotate... 2024-05-09T08:01:18.893 DEBUG:teuthology.orchestra.run.smithi001:> sudo rm -f -- /etc/logrotate.d/ceph 2024-05-09T08:01:18.897 DEBUG:teuthology.orchestra.run.smithi007:> sudo rm -f -- /etc/logrotate.d/ceph 2024-05-09T08:01:18.955 INFO:tasks.ceph:Creating extra log directories... 2024-05-09T08:01:18.955 DEBUG:teuthology.orchestra.run.smithi001:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-05-09T08:01:18.959 DEBUG:teuthology.orchestra.run.smithi007:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-05-09T08:01:19.065 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-05-09T08:01:19.065 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-05-09T08:01:19.065 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-05-09T08:01:19.065 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-05-09T08:01:19.066 DEBUG:teuthology.orchestra.run.smithi001:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-05-09T08:01:19.108 DEBUG:teuthology.orchestra.run.smithi001:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-05-09T08:01:19.301 DEBUG:teuthology.orchestra.run.smithi001:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-05-09T08:01:19.361 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-05-09T08:01:19.362 DEBUG:teuthology.orchestra.run.smithi007:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-05-09T08:01:19.376 DEBUG:teuthology.orchestra.run.smithi007:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-05-09T08:01:19.486 DEBUG:teuthology.orchestra.run.smithi007:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-05-09T08:01:19.515 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-05-09T08:01:19.515 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'client_shutdown_timeout': 120, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds oft prefetch dirfrags': False, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_dir_prefetch': False, 'mds_sleep_rank_change': 5000000.0, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', '\\(MON_DOWN\\)'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-05-09T08:01:19.515 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/teuthology-2024-05-08_21:24:04-fs-squid-distro-default-smithi/7698643', 'branch': 'squid', 'description': 'fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a5s-mds-1c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-ec-root overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/no prefetch_entire_dirfrags/no races session_timeout thrashosds-health} ranks/1 tasks/{1-thrash/mon 2-workunit/suites/iozone}}', 'email': 'ceph-qa@ceph.com', 'first_in_suite': False, 'job_id': '7698643', 'kernel': {'client': {'branch': 'testing'}, 'kdb': 1, 'sha1': 'distro'}, 'ktype': 'upstream', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'mgrmodules': {'sequential': [{'print': 'Enabling mgr modules'}]}, 'name': 'teuthology-2024-05-08_21:24:04-fs-squid-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'squid'}, 'ceph': {'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'session_timeout': 300}, 'conf': {'client': {'client mount timeout': 600, 'client_shutdown_timeout': 120, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds oft prefetch dirfrags': False, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_dir_prefetch': False, 'mds_sleep_rank_change': 5000000.0, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', '\\(MON_DOWN\\)'], 'sha1': '96805dcf909541c4ca9a1e90258de8ae5b6fa1bc'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '96805dcf909541c4ca9a1e90258de8ae5b6fa1bc'}}, 'kclient': {'syntax': 'v1'}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'squid', 'sha1': 'be85c202969d25af6e130678e7d73a20e9c4ca11'}}, 'owner': 'scheduled_teuthology@teuthology', 'priority': 100, 'repo': 'https://git.ceph.com/ceph.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'mds.e', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'mds.f', 'osd.4', 'osd.5', 'osd.6', 'osd.7']], 'seed': 2334, 'sha1': '96805dcf909541c4ca9a1e90258de8ae5b6fa1bc', 'sleep_before_teardown': 0, 'subset': '359/512', 'suite': 'fs', 'suite_branch': 'squid', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph_be85c202969d25af6e130678e7d73a20e9c4ca11/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph.git', 'suite_sha1': 'be85c202969d25af6e130678e7d73a20e9c4ca11', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi001.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOuQ3EjDSvcgqGy6BS5Ig9vVHLkMq6JjkSO277ytzuWqwDKQ91Dn6OR3QzTzTVxopO6YAyuYfxePNnGMHVaZS2M=', 'smithi007.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBANhHY74xUlJgJLHFh1AMaT07n0TE4qfCorQtgll2/4p7Jtmzv31b/1Z4GDoHSfqANJQPCCv2wQY8lvidNaPGfw='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'branch': 'testing'}, 'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '96805dcf909541c4ca9a1e90258de8ae5b6fa1bc'}}, {'ceph': {'create_rbd_pool': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'session_timeout': 300}, 'conf': {'client': {'client mount timeout': 600, 'client_shutdown_timeout': 120, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds oft prefetch dirfrags': False, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_dir_prefetch': False, 'mds_sleep_rank_change': 5000000.0, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', '\\(MON_DOWN\\)'], 'sha1': '96805dcf909541c4ca9a1e90258de8ae5b6fa1bc', 'cluster': 'ceph'}}, {'sequential': ['mgrmodules']}, {'kclient': None}, {'mon_thrash': {'check_mds_failover': True, 'revive_delay': 20, 'thrash_delay': 10}}, {'workunit': {'clients': {'all': ['suites/iozone.sh']}}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': '1ae7ad82388e92a475afff437d49054826c019a1', 'timestamp': '2024-05-08_21:24:04', 'tube': 'smithi', 'user': 'teuthology', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-05-09T08:01:19.516 DEBUG:teuthology.orchestra.run.smithi001:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-05-09T08:01:19.521 DEBUG:teuthology.orchestra.run.smithi007:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-05-09T08:01:19.565 DEBUG:teuthology.orchestra.run.smithi001:> sudo install -d -m0777 -- /var/run/ceph 2024-05-09T08:01:19.599 DEBUG:teuthology.orchestra.run.smithi007:> sudo install -d -m0777 -- /var/run/ceph 2024-05-09T08:01:19.666 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-05-09T08:01:19.666 DEBUG:teuthology.orchestra.run.smithi001:> dd if=/scratch_devs of=/dev/stdout 2024-05-09T08:01:19.706 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-09T08:01:19.706 DEBUG:teuthology.orchestra.run.smithi001:> stat /dev/vg_nvme/lv_1 2024-05-09T08:01:19.785 INFO:teuthology.orchestra.run.smithi001.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-09T08:01:19.785 INFO:teuthology.orchestra.run.smithi001.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T08:01:19.785 INFO:teuthology.orchestra.run.smithi001.stdout:Device: 6h/6d Inode: 634 Links: 1 2024-05-09T08:01:19.785 INFO:teuthology.orchestra.run.smithi001.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T08:01:19.786 INFO:teuthology.orchestra.run.smithi001.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T08:01:19.786 INFO:teuthology.orchestra.run.smithi001.stdout:Access: 2024-05-09 08:01:09.504898335 +0000 2024-05-09T08:01:19.786 INFO:teuthology.orchestra.run.smithi001.stdout:Modify: 2024-05-09 07:58:41.098346519 +0000 2024-05-09T08:01:19.786 INFO:teuthology.orchestra.run.smithi001.stdout:Change: 2024-05-09 07:58:41.098346519 +0000 2024-05-09T08:01:19.786 INFO:teuthology.orchestra.run.smithi001.stdout: Birth: 2024-05-09 07:58:41.098346519 +0000 2024-05-09T08:01:19.786 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-09T08:01:19.882 INFO:teuthology.orchestra.run.smithi001.stderr:1+0 records in 2024-05-09T08:01:19.882 INFO:teuthology.orchestra.run.smithi001.stderr:1+0 records out 2024-05-09T08:01:19.882 INFO:teuthology.orchestra.run.smithi001.stderr:512 bytes copied, 0.000243022 s, 2.1 MB/s 2024-05-09T08:01:19.885 DEBUG:teuthology.orchestra.run.smithi001:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-09T08:01:19.925 DEBUG:teuthology.orchestra.run.smithi001:> stat /dev/vg_nvme/lv_2 2024-05-09T08:01:20.001 INFO:teuthology.orchestra.run.smithi001.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-09T08:01:20.001 INFO:teuthology.orchestra.run.smithi001.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T08:01:20.001 INFO:teuthology.orchestra.run.smithi001.stdout:Device: 6h/6d Inode: 652 Links: 1 2024-05-09T08:01:20.001 INFO:teuthology.orchestra.run.smithi001.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T08:01:20.001 INFO:teuthology.orchestra.run.smithi001.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T08:01:20.001 INFO:teuthology.orchestra.run.smithi001.stdout:Access: 2024-05-09 08:01:09.504898335 +0000 2024-05-09T08:01:20.002 INFO:teuthology.orchestra.run.smithi001.stdout:Modify: 2024-05-09 07:58:41.731337783 +0000 2024-05-09T08:01:20.002 INFO:teuthology.orchestra.run.smithi001.stdout:Change: 2024-05-09 07:58:41.731337783 +0000 2024-05-09T08:01:20.002 INFO:teuthology.orchestra.run.smithi001.stdout: Birth: 2024-05-09 07:58:41.731337783 +0000 2024-05-09T08:01:20.002 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-09T08:01:20.098 INFO:teuthology.orchestra.run.smithi001.stderr:1+0 records in 2024-05-09T08:01:20.098 INFO:teuthology.orchestra.run.smithi001.stderr:1+0 records out 2024-05-09T08:01:20.098 INFO:teuthology.orchestra.run.smithi001.stderr:512 bytes copied, 0.000193766 s, 2.6 MB/s 2024-05-09T08:01:20.102 DEBUG:teuthology.orchestra.run.smithi001:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-09T08:01:20.141 DEBUG:teuthology.orchestra.run.smithi001:> stat /dev/vg_nvme/lv_3 2024-05-09T08:01:20.222 INFO:teuthology.orchestra.run.smithi001.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-09T08:01:20.222 INFO:teuthology.orchestra.run.smithi001.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T08:01:20.223 INFO:teuthology.orchestra.run.smithi001.stdout:Device: 6h/6d Inode: 670 Links: 1 2024-05-09T08:01:20.223 INFO:teuthology.orchestra.run.smithi001.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T08:01:20.223 INFO:teuthology.orchestra.run.smithi001.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T08:01:20.223 INFO:teuthology.orchestra.run.smithi001.stdout:Access: 2024-05-09 08:01:09.505898322 +0000 2024-05-09T08:01:20.223 INFO:teuthology.orchestra.run.smithi001.stdout:Modify: 2024-05-09 07:58:42.326329572 +0000 2024-05-09T08:01:20.223 INFO:teuthology.orchestra.run.smithi001.stdout:Change: 2024-05-09 07:58:42.326329572 +0000 2024-05-09T08:01:20.223 INFO:teuthology.orchestra.run.smithi001.stdout: Birth: 2024-05-09 07:58:42.326329572 +0000 2024-05-09T08:01:20.223 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-09T08:01:20.319 INFO:teuthology.orchestra.run.smithi001.stderr:1+0 records in 2024-05-09T08:01:20.319 INFO:teuthology.orchestra.run.smithi001.stderr:1+0 records out 2024-05-09T08:01:20.319 INFO:teuthology.orchestra.run.smithi001.stderr:512 bytes copied, 0.000193128 s, 2.7 MB/s 2024-05-09T08:01:20.323 DEBUG:teuthology.orchestra.run.smithi001:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-09T08:01:20.362 DEBUG:teuthology.orchestra.run.smithi001:> stat /dev/vg_nvme/lv_4 2024-05-09T08:01:20.441 INFO:teuthology.orchestra.run.smithi001.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-09T08:01:20.441 INFO:teuthology.orchestra.run.smithi001.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T08:01:20.441 INFO:teuthology.orchestra.run.smithi001.stdout:Device: 6h/6d Inode: 689 Links: 1 2024-05-09T08:01:20.441 INFO:teuthology.orchestra.run.smithi001.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T08:01:20.441 INFO:teuthology.orchestra.run.smithi001.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T08:01:20.442 INFO:teuthology.orchestra.run.smithi001.stdout:Access: 2024-05-09 08:01:09.506898308 +0000 2024-05-09T08:01:20.442 INFO:teuthology.orchestra.run.smithi001.stdout:Modify: 2024-05-09 07:58:42.946321016 +0000 2024-05-09T08:01:20.442 INFO:teuthology.orchestra.run.smithi001.stdout:Change: 2024-05-09 07:58:42.946321016 +0000 2024-05-09T08:01:20.442 INFO:teuthology.orchestra.run.smithi001.stdout: Birth: 2024-05-09 07:58:42.946321016 +0000 2024-05-09T08:01:20.442 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-09T08:01:20.537 INFO:teuthology.orchestra.run.smithi001.stderr:1+0 records in 2024-05-09T08:01:20.538 INFO:teuthology.orchestra.run.smithi001.stderr:1+0 records out 2024-05-09T08:01:20.538 INFO:teuthology.orchestra.run.smithi001.stderr:512 bytes copied, 0.000190824 s, 2.7 MB/s 2024-05-09T08:01:20.540 DEBUG:teuthology.orchestra.run.smithi001:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-09T08:01:20.580 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-09T08:01:20.580 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-05-09T08:01:20.580 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/scratch_devs of=/dev/stdout 2024-05-09T08:01:20.599 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-09T08:01:20.600 DEBUG:teuthology.orchestra.run.smithi007:> stat /dev/vg_nvme/lv_1 2024-05-09T08:01:20.658 INFO:teuthology.orchestra.run.smithi007.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-09T08:01:20.659 INFO:teuthology.orchestra.run.smithi007.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T08:01:20.659 INFO:teuthology.orchestra.run.smithi007.stdout:Device: 5h/5d Inode: 664 Links: 1 2024-05-09T08:01:20.659 INFO:teuthology.orchestra.run.smithi007.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T08:01:20.659 INFO:teuthology.orchestra.run.smithi007.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T08:01:20.659 INFO:teuthology.orchestra.run.smithi007.stdout:Access: 2024-05-09 08:00:46.786804817 +0000 2024-05-09T08:01:20.659 INFO:teuthology.orchestra.run.smithi007.stdout:Modify: 2024-05-09 07:57:28.670425648 +0000 2024-05-09T08:01:20.659 INFO:teuthology.orchestra.run.smithi007.stdout:Change: 2024-05-09 07:57:28.670425648 +0000 2024-05-09T08:01:20.659 INFO:teuthology.orchestra.run.smithi007.stdout: Birth: 2024-05-09 07:57:28.670425648 +0000 2024-05-09T08:01:20.659 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-09T08:01:20.725 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records in 2024-05-09T08:01:20.725 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records out 2024-05-09T08:01:20.725 INFO:teuthology.orchestra.run.smithi007.stderr:512 bytes copied, 0.000117429 s, 4.4 MB/s 2024-05-09T08:01:20.727 DEBUG:teuthology.orchestra.run.smithi007:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-09T08:01:20.786 DEBUG:teuthology.orchestra.run.smithi007:> stat /dev/vg_nvme/lv_2 2024-05-09T08:01:20.843 INFO:teuthology.orchestra.run.smithi007.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-09T08:01:20.843 INFO:teuthology.orchestra.run.smithi007.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T08:01:20.843 INFO:teuthology.orchestra.run.smithi007.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-05-09T08:01:20.843 INFO:teuthology.orchestra.run.smithi007.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T08:01:20.843 INFO:teuthology.orchestra.run.smithi007.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T08:01:20.843 INFO:teuthology.orchestra.run.smithi007.stdout:Access: 2024-05-09 08:00:46.786804817 +0000 2024-05-09T08:01:20.844 INFO:teuthology.orchestra.run.smithi007.stdout:Modify: 2024-05-09 07:57:29.019420555 +0000 2024-05-09T08:01:20.844 INFO:teuthology.orchestra.run.smithi007.stdout:Change: 2024-05-09 07:57:29.019420555 +0000 2024-05-09T08:01:20.844 INFO:teuthology.orchestra.run.smithi007.stdout: Birth: 2024-05-09 07:57:29.019420555 +0000 2024-05-09T08:01:20.844 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-09T08:01:20.908 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records in 2024-05-09T08:01:20.908 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records out 2024-05-09T08:01:20.909 INFO:teuthology.orchestra.run.smithi007.stderr:512 bytes copied, 0.000109214 s, 4.7 MB/s 2024-05-09T08:01:20.910 DEBUG:teuthology.orchestra.run.smithi007:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-09T08:01:20.968 DEBUG:teuthology.orchestra.run.smithi007:> stat /dev/vg_nvme/lv_3 2024-05-09T08:01:21.025 INFO:teuthology.orchestra.run.smithi007.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-09T08:01:21.025 INFO:teuthology.orchestra.run.smithi007.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T08:01:21.025 INFO:teuthology.orchestra.run.smithi007.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-05-09T08:01:21.025 INFO:teuthology.orchestra.run.smithi007.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T08:01:21.025 INFO:teuthology.orchestra.run.smithi007.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T08:01:21.025 INFO:teuthology.orchestra.run.smithi007.stdout:Access: 2024-05-09 08:00:46.786804817 +0000 2024-05-09T08:01:21.026 INFO:teuthology.orchestra.run.smithi007.stdout:Modify: 2024-05-09 07:57:29.382415259 +0000 2024-05-09T08:01:21.026 INFO:teuthology.orchestra.run.smithi007.stdout:Change: 2024-05-09 07:57:29.382415259 +0000 2024-05-09T08:01:21.026 INFO:teuthology.orchestra.run.smithi007.stdout: Birth: 2024-05-09 07:57:29.382415259 +0000 2024-05-09T08:01:21.026 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-09T08:01:21.090 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records in 2024-05-09T08:01:21.090 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records out 2024-05-09T08:01:21.090 INFO:teuthology.orchestra.run.smithi007.stderr:512 bytes copied, 0.00010858 s, 4.7 MB/s 2024-05-09T08:01:21.092 DEBUG:teuthology.orchestra.run.smithi007:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-09T08:01:21.149 DEBUG:teuthology.orchestra.run.smithi007:> stat /dev/vg_nvme/lv_4 2024-05-09T08:01:21.205 INFO:teuthology.orchestra.run.smithi007.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-09T08:01:21.205 INFO:teuthology.orchestra.run.smithi007.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T08:01:21.206 INFO:teuthology.orchestra.run.smithi007.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-05-09T08:01:21.206 INFO:teuthology.orchestra.run.smithi007.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T08:01:21.206 INFO:teuthology.orchestra.run.smithi007.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T08:01:21.206 INFO:teuthology.orchestra.run.smithi007.stdout:Access: 2024-05-09 08:00:46.787804803 +0000 2024-05-09T08:01:21.206 INFO:teuthology.orchestra.run.smithi007.stdout:Modify: 2024-05-09 07:57:29.735410108 +0000 2024-05-09T08:01:21.206 INFO:teuthology.orchestra.run.smithi007.stdout:Change: 2024-05-09 07:57:29.735410108 +0000 2024-05-09T08:01:21.206 INFO:teuthology.orchestra.run.smithi007.stdout: Birth: 2024-05-09 07:57:29.735410108 +0000 2024-05-09T08:01:21.206 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-09T08:01:21.270 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records in 2024-05-09T08:01:21.271 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records out 2024-05-09T08:01:21.271 INFO:teuthology.orchestra.run.smithi007.stderr:512 bytes copied, 0.000124996 s, 4.1 MB/s 2024-05-09T08:01:21.272 DEBUG:teuthology.orchestra.run.smithi007:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-09T08:01:21.329 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-09T08:01:21.329 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi001.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi007.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-05-09T08:01:21.329 INFO:tasks.ceph:Generating config... 2024-05-09T08:01:21.331 INFO:tasks.ceph:[client] client mount timeout = 600 2024-05-09T08:01:21.331 INFO:tasks.ceph:[client] client_shutdown_timeout = 120 2024-05-09T08:01:21.331 INFO:tasks.ceph:[client] debug client = 20 2024-05-09T08:01:21.331 INFO:tasks.ceph:[client] debug ms = 1 2024-05-09T08:01:21.331 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-05-09T08:01:21.331 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-05-09T08:01:21.331 INFO:tasks.ceph:[global] ms die on skipped message = False 2024-05-09T08:01:21.331 INFO:tasks.ceph:[mds] debug mds = 20 2024-05-09T08:01:21.331 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-05-09T08:01:21.331 INFO:tasks.ceph:[mds] debug ms = 1 2024-05-09T08:01:21.331 INFO:tasks.ceph:[mds] mds bal fragment size max = 10000 2024-05-09T08:01:21.331 INFO:tasks.ceph:[mds] mds bal merge size = 5 2024-05-09T08:01:21.331 INFO:tasks.ceph:[mds] mds bal split bits = 3 2024-05-09T08:01:21.332 INFO:tasks.ceph:[mds] mds bal split size = 100 2024-05-09T08:01:21.332 INFO:tasks.ceph:[mds] mds debug frag = True 2024-05-09T08:01:21.332 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-05-09T08:01:21.332 INFO:tasks.ceph:[mds] mds oft prefetch dirfrags = False 2024-05-09T08:01:21.332 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-05-09T08:01:21.332 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-05-09T08:01:21.332 INFO:tasks.ceph:[mds] mds_dir_prefetch = False 2024-05-09T08:01:21.332 INFO:tasks.ceph:[mds] mds_sleep_rank_change = 5000000.0 2024-05-09T08:01:21.332 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-05-09T08:01:21.332 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-05-09T08:01:21.332 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-05-09T08:01:21.332 INFO:tasks.ceph:[mgr] debug client = 20 2024-05-09T08:01:21.332 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-05-09T08:01:21.332 INFO:tasks.ceph:[mgr] debug ms = 1 2024-05-09T08:01:21.332 INFO:tasks.ceph:[mon] debug mon = 20 2024-05-09T08:01:21.333 INFO:tasks.ceph:[mon] debug ms = 1 2024-05-09T08:01:21.333 INFO:tasks.ceph:[mon] debug paxos = 20 2024-05-09T08:01:21.333 INFO:tasks.ceph:[mon] mon down mkfs grace = 300 2024-05-09T08:01:21.333 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-05-09T08:01:21.333 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-05-09T08:01:21.333 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-05-09T08:01:21.333 INFO:tasks.ceph:[osd] debug bluefs = 20 2024-05-09T08:01:21.333 INFO:tasks.ceph:[osd] debug bluestore = 20 2024-05-09T08:01:21.333 INFO:tasks.ceph:[osd] debug ms = 1 2024-05-09T08:01:21.333 INFO:tasks.ceph:[osd] debug osd = 20 2024-05-09T08:01:21.333 INFO:tasks.ceph:[osd] debug rocksdb = 10 2024-05-09T08:01:21.333 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-05-09T08:01:21.333 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-05-09T08:01:21.333 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-05-09T08:01:21.333 INFO:tasks.ceph:[osd] osd blocked scrub grace period = 3600 2024-05-09T08:01:21.334 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-05-09T08:01:21.334 INFO:tasks.ceph:[osd] osd max markdown count = 1000 2024-05-09T08:01:21.334 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-05-09T08:01:21.334 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-05-09T08:01:21.334 INFO:tasks.ceph:Setting up mon.a... 2024-05-09T08:01:21.334 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-05-09T08:01:21.470 INFO:teuthology.orchestra.run.smithi001.stdout:creating /etc/ceph/ceph.keyring 2024-05-09T08:01:21.479 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-05-09T08:01:21.592 DEBUG:teuthology.orchestra.run.smithi001:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-05-09T08:01:21.656 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.1'), ('mon.b', '172.21.15.7'), ('mon.c', '[v2:172.21.15.7:3301,v1:172.21.15.7:6790]')] 2024-05-09T08:01:21.657 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.1,172.21.15.7,[v2:172.21.15.7:3301,v1:172.21.15.7:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'client_shutdown_timeout': 120, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.e': {}, 'mds.b': {}, 'mds.d': {}, 'mds.f': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds oft prefetch dirfrags': False, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_dir_prefetch': False, 'mds_sleep_rank_change': 5000000.0, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-05-09T08:01:21.657 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-05-09T08:01:21.658 DEBUG:teuthology.orchestra.run.smithi001:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-05-09T08:01:21.698 DEBUG:teuthology.orchestra.run.smithi001:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.1 --add b 172.21.15.7 --addv c '[v2:172.21.15.7:3301,v1:172.21.15.7:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-05-09T08:01:21.834 INFO:teuthology.orchestra.run.smithi001.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-05-09T08:01:21.835 INFO:teuthology.orchestra.run.smithi001.stdout:monmaptool: generated fsid 54329ec8-3270-4043-9a82-c6cd7420aa7f 2024-05-09T08:01:21.835 INFO:teuthology.orchestra.run.smithi001.stdout:setting min_mon_release = quincy 2024-05-09T08:01:21.835 INFO:teuthology.orchestra.run.smithi001.stdout:epoch 0 2024-05-09T08:01:21.835 INFO:teuthology.orchestra.run.smithi001.stdout:fsid 54329ec8-3270-4043-9a82-c6cd7420aa7f 2024-05-09T08:01:21.835 INFO:teuthology.orchestra.run.smithi001.stdout:last_changed 2024-05-09T08:01:21.819270+0000 2024-05-09T08:01:21.835 INFO:teuthology.orchestra.run.smithi001.stdout:created 2024-05-09T08:01:21.819270+0000 2024-05-09T08:01:21.835 INFO:teuthology.orchestra.run.smithi001.stdout:min_mon_release 17 (quincy) 2024-05-09T08:01:21.835 INFO:teuthology.orchestra.run.smithi001.stdout:election_strategy: 1 2024-05-09T08:01:21.835 INFO:teuthology.orchestra.run.smithi001.stdout:0: [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] mon.a 2024-05-09T08:01:21.835 INFO:teuthology.orchestra.run.smithi001.stdout:1: [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] mon.b 2024-05-09T08:01:21.835 INFO:teuthology.orchestra.run.smithi001.stdout:2: [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] mon.c 2024-05-09T08:01:21.836 INFO:teuthology.orchestra.run.smithi001.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-05-09T08:01:21.838 DEBUG:teuthology.orchestra.run.smithi001:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-05-09T08:01:21.887 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 54329ec8-3270-4043-9a82-c6cd7420aa7f... 2024-05-09T08:01:21.888 DEBUG:teuthology.orchestra.run.smithi001:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-05-09T08:01:21.893 DEBUG:teuthology.orchestra.run.smithi007:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-05-09T08:01:21.942 INFO:teuthology.orchestra.run.smithi007.stdout:[global] 2024-05-09T08:01:21.942 INFO:teuthology.orchestra.run.smithi007.stdout: chdir = "" 2024-05-09T08:01:21.942 INFO:teuthology.orchestra.run.smithi007.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-05-09T08:01:21.943 INFO:teuthology.orchestra.run.smithi007.stdout: auth supported = cephx 2024-05-09T08:01:21.943 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-05-09T08:01:21.943 INFO:teuthology.orchestra.run.smithi007.stdout: filestore xattr use omap = true 2024-05-09T08:01:21.943 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-05-09T08:01:21.943 INFO:teuthology.orchestra.run.smithi007.stdout: mon clock drift allowed = 1.000 2024-05-09T08:01:21.943 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-05-09T08:01:21.943 INFO:teuthology.orchestra.run.smithi007.stdout: osd crush chooseleaf type = 0 2024-05-09T08:01:21.943 INFO:teuthology.orchestra.run.smithi007.stdout: auth debug = true 2024-05-09T08:01:21.943 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-05-09T08:01:21.943 INFO:teuthology.orchestra.run.smithi007.stdout: ms die on old message = true 2024-05-09T08:01:21.943 INFO:teuthology.orchestra.run.smithi007.stdout: ms die on bug = true 2024-05-09T08:01:21.943 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-05-09T08:01:21.943 INFO:teuthology.orchestra.run.smithi007.stdout: mon max pg per osd = 10000 # >= luminous 2024-05-09T08:01:21.943 INFO:teuthology.orchestra.run.smithi007.stdout: mon pg warn max object skew = 0 2024-05-09T08:01:21.944 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-05-09T08:01:21.944 INFO:teuthology.orchestra.run.smithi007.stdout: # disable pg_autoscaler by default for new pools 2024-05-09T08:01:21.944 INFO:teuthology.orchestra.run.smithi007.stdout: osd_pool_default_pg_autoscale_mode = off 2024-05-09T08:01:21.944 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-05-09T08:01:21.944 INFO:teuthology.orchestra.run.smithi007.stdout: osd pool default size = 2 2024-05-09T08:01:21.944 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-05-09T08:01:21.944 INFO:teuthology.orchestra.run.smithi007.stdout: mon osd allow primary affinity = true 2024-05-09T08:01:21.944 INFO:teuthology.orchestra.run.smithi007.stdout: mon osd allow pg remap = true 2024-05-09T08:01:21.944 INFO:teuthology.orchestra.run.smithi007.stdout: mon warn on legacy crush tunables = false 2024-05-09T08:01:21.944 INFO:teuthology.orchestra.run.smithi007.stdout: mon warn on crush straw calc version zero = false 2024-05-09T08:01:21.944 INFO:teuthology.orchestra.run.smithi007.stdout: mon warn on no sortbitwise = false 2024-05-09T08:01:21.944 INFO:teuthology.orchestra.run.smithi007.stdout: mon warn on osd down out interval zero = false 2024-05-09T08:01:21.944 INFO:teuthology.orchestra.run.smithi007.stdout: mon warn on too few osds = false 2024-05-09T08:01:21.944 INFO:teuthology.orchestra.run.smithi007.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-05-09T08:01:21.944 INFO:teuthology.orchestra.run.smithi007.stdout: mon_warn_on_pool_no_redundancy = false 2024-05-09T08:01:21.945 INFO:teuthology.orchestra.run.smithi007.stdout: mon_allow_pool_size_one = true 2024-05-09T08:01:21.945 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-05-09T08:01:21.945 INFO:teuthology.orchestra.run.smithi007.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-05-09T08:01:21.945 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-05-09T08:01:21.945 INFO:teuthology.orchestra.run.smithi007.stdout: osd default data pool replay window = 5 2024-05-09T08:01:21.945 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-05-09T08:01:21.945 INFO:teuthology.orchestra.run.smithi007.stdout: mon allow pool delete = true 2024-05-09T08:01:21.945 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-05-09T08:01:21.945 INFO:teuthology.orchestra.run.smithi007.stdout: mon cluster log file level = debug 2024-05-09T08:01:21.945 INFO:teuthology.orchestra.run.smithi007.stdout: debug asserts on shutdown = true 2024-05-09T08:01:21.945 INFO:teuthology.orchestra.run.smithi007.stdout: mon health detail to clog = false 2024-05-09T08:01:21.945 INFO:teuthology.orchestra.run.smithi007.stdout: mon host = "172.21.15.1,172.21.15.7,[v2:172.21.15.7:3301,v1:172.21.15.7:6790]" 2024-05-09T08:01:21.945 INFO:teuthology.orchestra.run.smithi007.stdout: ms die on skipped message = False 2024-05-09T08:01:21.945 INFO:teuthology.orchestra.run.smithi007.stdout: fsid = 54329ec8-3270-4043-9a82-c6cd7420aa7f 2024-05-09T08:01:21.945 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-05-09T08:01:21.946 INFO:teuthology.orchestra.run.smithi007.stdout:[osd] 2024-05-09T08:01:21.946 INFO:teuthology.orchestra.run.smithi007.stdout: osd journal size = 100 2024-05-09T08:01:21.946 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-05-09T08:01:21.946 INFO:teuthology.orchestra.run.smithi007.stdout: osd scrub load threshold = 5.0 2024-05-09T08:01:21.946 INFO:teuthology.orchestra.run.smithi007.stdout: osd scrub max interval = 600 2024-05-09T08:01:21.946 INFO:teuthology.orchestra.run.smithi007.stdout: osd mclock profile = high_recovery_ops 2024-05-09T08:01:21.946 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-05-09T08:01:21.946 INFO:teuthology.orchestra.run.smithi007.stdout: osd recover clone overlap = true 2024-05-09T08:01:21.946 INFO:teuthology.orchestra.run.smithi007.stdout: osd recovery max chunk = 1048576 2024-05-09T08:01:21.946 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-05-09T08:01:21.946 INFO:teuthology.orchestra.run.smithi007.stdout: osd debug shutdown = true 2024-05-09T08:01:21.946 INFO:teuthology.orchestra.run.smithi007.stdout: osd debug op order = true 2024-05-09T08:01:21.946 INFO:teuthology.orchestra.run.smithi007.stdout: osd debug verify stray on activate = true 2024-05-09T08:01:21.947 INFO:teuthology.orchestra.run.smithi007.stdout: osd debug trim objects = true 2024-05-09T08:01:21.947 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-05-09T08:01:21.947 INFO:teuthology.orchestra.run.smithi007.stdout: osd open classes on start = true 2024-05-09T08:01:21.947 INFO:teuthology.orchestra.run.smithi007.stdout: osd debug pg log writeout = true 2024-05-09T08:01:21.947 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-05-09T08:01:21.947 INFO:teuthology.orchestra.run.smithi007.stdout: osd deep scrub update digest min age = 30 2024-05-09T08:01:21.947 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-05-09T08:01:21.947 INFO:teuthology.orchestra.run.smithi007.stdout: osd map max advance = 10 2024-05-09T08:01:21.947 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-05-09T08:01:21.947 INFO:teuthology.orchestra.run.smithi007.stdout: journal zero on create = true 2024-05-09T08:01:21.947 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-05-09T08:01:21.947 INFO:teuthology.orchestra.run.smithi007.stdout: filestore ondisk finisher threads = 3 2024-05-09T08:01:21.947 INFO:teuthology.orchestra.run.smithi007.stdout: filestore apply finisher threads = 3 2024-05-09T08:01:21.947 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-05-09T08:01:21.948 INFO:teuthology.orchestra.run.smithi007.stdout: bdev debug aio = true 2024-05-09T08:01:21.948 INFO:teuthology.orchestra.run.smithi007.stdout: osd debug misdirected ops = true 2024-05-09T08:01:21.948 INFO:teuthology.orchestra.run.smithi007.stdout: bluestore block size = 96636764160 2024-05-09T08:01:21.948 INFO:teuthology.orchestra.run.smithi007.stdout: bluestore fsck on mount = True 2024-05-09T08:01:21.948 INFO:teuthology.orchestra.run.smithi007.stdout: debug bluefs = 20 2024-05-09T08:01:21.948 INFO:teuthology.orchestra.run.smithi007.stdout: debug bluestore = 20 2024-05-09T08:01:21.948 INFO:teuthology.orchestra.run.smithi007.stdout: debug ms = 1 2024-05-09T08:01:21.948 INFO:teuthology.orchestra.run.smithi007.stdout: debug osd = 20 2024-05-09T08:01:21.948 INFO:teuthology.orchestra.run.smithi007.stdout: debug rocksdb = 10 2024-05-09T08:01:21.948 INFO:teuthology.orchestra.run.smithi007.stdout: mon osd backfillfull_ratio = 0.85 2024-05-09T08:01:21.948 INFO:teuthology.orchestra.run.smithi007.stdout: mon osd full ratio = 0.9 2024-05-09T08:01:21.948 INFO:teuthology.orchestra.run.smithi007.stdout: mon osd nearfull ratio = 0.8 2024-05-09T08:01:21.948 INFO:teuthology.orchestra.run.smithi007.stdout: osd blocked scrub grace period = 3600 2024-05-09T08:01:21.948 INFO:teuthology.orchestra.run.smithi007.stdout: osd failsafe full ratio = 0.95 2024-05-09T08:01:21.949 INFO:teuthology.orchestra.run.smithi007.stdout: osd max markdown count = 1000 2024-05-09T08:01:21.949 INFO:teuthology.orchestra.run.smithi007.stdout: osd objectstore = bluestore 2024-05-09T08:01:21.949 INFO:teuthology.orchestra.run.smithi007.stdout: osd op complaint time = 180 2024-05-09T08:01:21.949 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-05-09T08:01:21.949 INFO:teuthology.orchestra.run.smithi007.stdout:[mgr] 2024-05-09T08:01:21.949 INFO:teuthology.orchestra.run.smithi007.stdout: debug ms = 1 2024-05-09T08:01:21.949 INFO:teuthology.orchestra.run.smithi007.stdout: debug mgr = 20 2024-05-09T08:01:21.949 INFO:teuthology.orchestra.run.smithi007.stdout: debug mon = 20 2024-05-09T08:01:21.949 INFO:teuthology.orchestra.run.smithi007.stdout: debug auth = 20 2024-05-09T08:01:21.949 INFO:teuthology.orchestra.run.smithi007.stdout: mon reweight min pgs per osd = 4 2024-05-09T08:01:21.949 INFO:teuthology.orchestra.run.smithi007.stdout: mon reweight min bytes per osd = 10 2024-05-09T08:01:21.949 INFO:teuthology.orchestra.run.smithi007.stdout: mgr/telemetry/nag = false 2024-05-09T08:01:21.949 INFO:teuthology.orchestra.run.smithi007.stdout: debug client = 20 2024-05-09T08:01:21.949 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-05-09T08:01:21.949 INFO:teuthology.orchestra.run.smithi007.stdout:[mon] 2024-05-09T08:01:21.950 INFO:teuthology.orchestra.run.smithi007.stdout: debug ms = 1 2024-05-09T08:01:21.950 INFO:teuthology.orchestra.run.smithi007.stdout: debug mon = 20 2024-05-09T08:01:21.950 INFO:teuthology.orchestra.run.smithi007.stdout: debug paxos = 20 2024-05-09T08:01:21.950 INFO:teuthology.orchestra.run.smithi007.stdout: debug auth = 20 2024-05-09T08:01:21.950 INFO:teuthology.orchestra.run.smithi007.stdout: mon data avail warn = 5 2024-05-09T08:01:21.950 INFO:teuthology.orchestra.run.smithi007.stdout: mon mgr mkfs grace = 240 2024-05-09T08:01:21.950 INFO:teuthology.orchestra.run.smithi007.stdout: mon reweight min pgs per osd = 4 2024-05-09T08:01:21.950 INFO:teuthology.orchestra.run.smithi007.stdout: mon osd reporter subtree level = osd 2024-05-09T08:01:21.950 INFO:teuthology.orchestra.run.smithi007.stdout: mon osd prime pg temp = true 2024-05-09T08:01:21.950 INFO:teuthology.orchestra.run.smithi007.stdout: mon reweight min bytes per osd = 10 2024-05-09T08:01:21.950 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-05-09T08:01:21.950 INFO:teuthology.orchestra.run.smithi007.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-05-09T08:01:21.951 INFO:teuthology.orchestra.run.smithi007.stdout: auth mon ticket ttl = 660 # 11m 2024-05-09T08:01:21.951 INFO:teuthology.orchestra.run.smithi007.stdout: auth service ticket ttl = 240 # 4m 2024-05-09T08:01:21.951 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-05-09T08:01:21.951 INFO:teuthology.orchestra.run.smithi007.stdout: # don't complain about insecure global_id in the test suite 2024-05-09T08:01:21.951 INFO:teuthology.orchestra.run.smithi007.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-05-09T08:01:21.951 INFO:teuthology.orchestra.run.smithi007.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-05-09T08:01:21.951 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-05-09T08:01:21.951 INFO:teuthology.orchestra.run.smithi007.stdout: # 1m isn't quite enough 2024-05-09T08:01:21.951 INFO:teuthology.orchestra.run.smithi007.stdout: mon_down_mkfs_grace = 2m 2024-05-09T08:01:21.951 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-05-09T08:01:21.951 INFO:teuthology.orchestra.run.smithi007.stdout: mon_warn_on_filestore_osds = false 2024-05-09T08:01:21.951 INFO:teuthology.orchestra.run.smithi007.stdout: mon down mkfs grace = 300 2024-05-09T08:01:21.951 INFO:teuthology.orchestra.run.smithi007.stdout: mon op complaint time = 120 2024-05-09T08:01:21.951 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-05-09T08:01:21.952 INFO:teuthology.orchestra.run.smithi007.stdout:[client] 2024-05-09T08:01:21.952 INFO:teuthology.orchestra.run.smithi007.stdout: rgw cache enabled = true 2024-05-09T08:01:21.952 INFO:teuthology.orchestra.run.smithi007.stdout: rgw enable ops log = true 2024-05-09T08:01:21.952 INFO:teuthology.orchestra.run.smithi007.stdout: rgw enable usage log = true 2024-05-09T08:01:21.952 INFO:teuthology.orchestra.run.smithi007.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-05-09T08:01:21.952 INFO:teuthology.orchestra.run.smithi007.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-05-09T08:01:21.952 INFO:teuthology.orchestra.run.smithi007.stdout: client mount timeout = 600 2024-05-09T08:01:21.952 INFO:teuthology.orchestra.run.smithi007.stdout: client_shutdown_timeout = 120 2024-05-09T08:01:21.952 INFO:teuthology.orchestra.run.smithi007.stdout: debug client = 20 2024-05-09T08:01:21.952 INFO:teuthology.orchestra.run.smithi007.stdout: debug ms = 1 2024-05-09T08:01:21.952 INFO:teuthology.orchestra.run.smithi007.stdout: rados mon op timeout = 900 2024-05-09T08:01:21.952 INFO:teuthology.orchestra.run.smithi007.stdout: rados osd op timeout = 900 2024-05-09T08:01:21.952 INFO:teuthology.orchestra.run.smithi007.stdout:[mon.a] 2024-05-09T08:01:21.952 INFO:teuthology.orchestra.run.smithi007.stdout:[mon.b] 2024-05-09T08:01:21.952 INFO:teuthology.orchestra.run.smithi007.stdout:[mon.c] 2024-05-09T08:01:21.953 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.a] 2024-05-09T08:01:21.953 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.c] 2024-05-09T08:01:21.953 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.e] 2024-05-09T08:01:21.953 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.b] 2024-05-09T08:01:21.953 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.d] 2024-05-09T08:01:21.953 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.f] 2024-05-09T08:01:21.953 INFO:teuthology.orchestra.run.smithi007.stdout:[mds] 2024-05-09T08:01:21.953 INFO:teuthology.orchestra.run.smithi007.stdout: debug mds = 20 2024-05-09T08:01:21.953 INFO:teuthology.orchestra.run.smithi007.stdout: debug mds balancer = 20 2024-05-09T08:01:21.953 INFO:teuthology.orchestra.run.smithi007.stdout: debug ms = 1 2024-05-09T08:01:21.953 INFO:teuthology.orchestra.run.smithi007.stdout: mds bal fragment size max = 10000 2024-05-09T08:01:21.953 INFO:teuthology.orchestra.run.smithi007.stdout: mds bal merge size = 5 2024-05-09T08:01:21.953 INFO:teuthology.orchestra.run.smithi007.stdout: mds bal split bits = 3 2024-05-09T08:01:21.953 INFO:teuthology.orchestra.run.smithi007.stdout: mds bal split size = 100 2024-05-09T08:01:21.954 INFO:teuthology.orchestra.run.smithi007.stdout: mds debug frag = True 2024-05-09T08:01:21.954 INFO:teuthology.orchestra.run.smithi007.stdout: mds debug scatterstat = True 2024-05-09T08:01:21.954 INFO:teuthology.orchestra.run.smithi007.stdout: mds oft prefetch dirfrags = False 2024-05-09T08:01:21.954 INFO:teuthology.orchestra.run.smithi007.stdout: mds op complaint time = 180 2024-05-09T08:01:21.954 INFO:teuthology.orchestra.run.smithi007.stdout: mds verify scatter = True 2024-05-09T08:01:21.954 INFO:teuthology.orchestra.run.smithi007.stdout: mds_dir_prefetch = False 2024-05-09T08:01:21.954 INFO:teuthology.orchestra.run.smithi007.stdout: mds_sleep_rank_change = 5000000.0 2024-05-09T08:01:21.954 INFO:teuthology.orchestra.run.smithi007.stdout: osd op complaint time = 180 2024-05-09T08:01:21.954 INFO:teuthology.orchestra.run.smithi007.stdout: rados mon op timeout = 900 2024-05-09T08:01:21.954 INFO:teuthology.orchestra.run.smithi007.stdout: rados osd op timeout = 900 2024-05-09T08:01:22.032 INFO:teuthology.orchestra.run.smithi001.stdout:[global] 2024-05-09T08:01:22.032 INFO:teuthology.orchestra.run.smithi001.stdout: chdir = "" 2024-05-09T08:01:22.032 INFO:teuthology.orchestra.run.smithi001.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-05-09T08:01:22.032 INFO:teuthology.orchestra.run.smithi001.stdout: auth supported = cephx 2024-05-09T08:01:22.032 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T08:01:22.032 INFO:teuthology.orchestra.run.smithi001.stdout: filestore xattr use omap = true 2024-05-09T08:01:22.032 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T08:01:22.032 INFO:teuthology.orchestra.run.smithi001.stdout: mon clock drift allowed = 1.000 2024-05-09T08:01:22.032 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T08:01:22.032 INFO:teuthology.orchestra.run.smithi001.stdout: osd crush chooseleaf type = 0 2024-05-09T08:01:22.032 INFO:teuthology.orchestra.run.smithi001.stdout: auth debug = true 2024-05-09T08:01:22.032 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T08:01:22.033 INFO:teuthology.orchestra.run.smithi001.stdout: ms die on old message = true 2024-05-09T08:01:22.033 INFO:teuthology.orchestra.run.smithi001.stdout: ms die on bug = true 2024-05-09T08:01:22.033 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T08:01:22.033 INFO:teuthology.orchestra.run.smithi001.stdout: mon max pg per osd = 10000 # >= luminous 2024-05-09T08:01:22.033 INFO:teuthology.orchestra.run.smithi001.stdout: mon pg warn max object skew = 0 2024-05-09T08:01:22.033 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T08:01:22.033 INFO:teuthology.orchestra.run.smithi001.stdout: # disable pg_autoscaler by default for new pools 2024-05-09T08:01:22.033 INFO:teuthology.orchestra.run.smithi001.stdout: osd_pool_default_pg_autoscale_mode = off 2024-05-09T08:01:22.033 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T08:01:22.033 INFO:teuthology.orchestra.run.smithi001.stdout: osd pool default size = 2 2024-05-09T08:01:22.033 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T08:01:22.033 INFO:teuthology.orchestra.run.smithi001.stdout: mon osd allow primary affinity = true 2024-05-09T08:01:22.033 INFO:teuthology.orchestra.run.smithi001.stdout: mon osd allow pg remap = true 2024-05-09T08:01:22.034 INFO:teuthology.orchestra.run.smithi001.stdout: mon warn on legacy crush tunables = false 2024-05-09T08:01:22.034 INFO:teuthology.orchestra.run.smithi001.stdout: mon warn on crush straw calc version zero = false 2024-05-09T08:01:22.034 INFO:teuthology.orchestra.run.smithi001.stdout: mon warn on no sortbitwise = false 2024-05-09T08:01:22.034 INFO:teuthology.orchestra.run.smithi001.stdout: mon warn on osd down out interval zero = false 2024-05-09T08:01:22.034 INFO:teuthology.orchestra.run.smithi001.stdout: mon warn on too few osds = false 2024-05-09T08:01:22.034 INFO:teuthology.orchestra.run.smithi001.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-05-09T08:01:22.034 INFO:teuthology.orchestra.run.smithi001.stdout: mon_warn_on_pool_no_redundancy = false 2024-05-09T08:01:22.034 INFO:teuthology.orchestra.run.smithi001.stdout: mon_allow_pool_size_one = true 2024-05-09T08:01:22.034 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T08:01:22.034 INFO:teuthology.orchestra.run.smithi001.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-05-09T08:01:22.034 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T08:01:22.034 INFO:teuthology.orchestra.run.smithi001.stdout: osd default data pool replay window = 5 2024-05-09T08:01:22.034 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T08:01:22.035 INFO:teuthology.orchestra.run.smithi001.stdout: mon allow pool delete = true 2024-05-09T08:01:22.035 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T08:01:22.035 INFO:teuthology.orchestra.run.smithi001.stdout: mon cluster log file level = debug 2024-05-09T08:01:22.035 INFO:teuthology.orchestra.run.smithi001.stdout: debug asserts on shutdown = true 2024-05-09T08:01:22.035 INFO:teuthology.orchestra.run.smithi001.stdout: mon health detail to clog = false 2024-05-09T08:01:22.035 INFO:teuthology.orchestra.run.smithi001.stdout: mon host = "172.21.15.1,172.21.15.7,[v2:172.21.15.7:3301,v1:172.21.15.7:6790]" 2024-05-09T08:01:22.035 INFO:teuthology.orchestra.run.smithi001.stdout: ms die on skipped message = False 2024-05-09T08:01:22.035 INFO:teuthology.orchestra.run.smithi001.stdout: fsid = 54329ec8-3270-4043-9a82-c6cd7420aa7f 2024-05-09T08:01:22.035 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T08:01:22.035 INFO:teuthology.orchestra.run.smithi001.stdout:[osd] 2024-05-09T08:01:22.035 INFO:teuthology.orchestra.run.smithi001.stdout: osd journal size = 100 2024-05-09T08:01:22.035 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T08:01:22.035 INFO:teuthology.orchestra.run.smithi001.stdout: osd scrub load threshold = 5.0 2024-05-09T08:01:22.035 INFO:teuthology.orchestra.run.smithi001.stdout: osd scrub max interval = 600 2024-05-09T08:01:22.036 INFO:teuthology.orchestra.run.smithi001.stdout: osd mclock profile = high_recovery_ops 2024-05-09T08:01:22.036 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T08:01:22.036 INFO:teuthology.orchestra.run.smithi001.stdout: osd recover clone overlap = true 2024-05-09T08:01:22.036 INFO:teuthology.orchestra.run.smithi001.stdout: osd recovery max chunk = 1048576 2024-05-09T08:01:22.036 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T08:01:22.036 INFO:teuthology.orchestra.run.smithi001.stdout: osd debug shutdown = true 2024-05-09T08:01:22.036 INFO:teuthology.orchestra.run.smithi001.stdout: osd debug op order = true 2024-05-09T08:01:22.036 INFO:teuthology.orchestra.run.smithi001.stdout: osd debug verify stray on activate = true 2024-05-09T08:01:22.036 INFO:teuthology.orchestra.run.smithi001.stdout: osd debug trim objects = true 2024-05-09T08:01:22.036 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T08:01:22.036 INFO:teuthology.orchestra.run.smithi001.stdout: osd open classes on start = true 2024-05-09T08:01:22.036 INFO:teuthology.orchestra.run.smithi001.stdout: osd debug pg log writeout = true 2024-05-09T08:01:22.036 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T08:01:22.036 INFO:teuthology.orchestra.run.smithi001.stdout: osd deep scrub update digest min age = 30 2024-05-09T08:01:22.036 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T08:01:22.037 INFO:teuthology.orchestra.run.smithi001.stdout: osd map max advance = 10 2024-05-09T08:01:22.037 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T08:01:22.037 INFO:teuthology.orchestra.run.smithi001.stdout: journal zero on create = true 2024-05-09T08:01:22.037 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T08:01:22.037 INFO:teuthology.orchestra.run.smithi001.stdout: filestore ondisk finisher threads = 3 2024-05-09T08:01:22.037 INFO:teuthology.orchestra.run.smithi001.stdout: filestore apply finisher threads = 3 2024-05-09T08:01:22.037 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T08:01:22.037 INFO:teuthology.orchestra.run.smithi001.stdout: bdev debug aio = true 2024-05-09T08:01:22.037 INFO:teuthology.orchestra.run.smithi001.stdout: osd debug misdirected ops = true 2024-05-09T08:01:22.037 INFO:teuthology.orchestra.run.smithi001.stdout: bluestore block size = 96636764160 2024-05-09T08:01:22.037 INFO:teuthology.orchestra.run.smithi001.stdout: bluestore fsck on mount = True 2024-05-09T08:01:22.037 INFO:teuthology.orchestra.run.smithi001.stdout: debug bluefs = 20 2024-05-09T08:01:22.037 INFO:teuthology.orchestra.run.smithi001.stdout: debug bluestore = 20 2024-05-09T08:01:22.037 INFO:teuthology.orchestra.run.smithi001.stdout: debug ms = 1 2024-05-09T08:01:22.038 INFO:teuthology.orchestra.run.smithi001.stdout: debug osd = 20 2024-05-09T08:01:22.038 INFO:teuthology.orchestra.run.smithi001.stdout: debug rocksdb = 10 2024-05-09T08:01:22.038 INFO:teuthology.orchestra.run.smithi001.stdout: mon osd backfillfull_ratio = 0.85 2024-05-09T08:01:22.038 INFO:teuthology.orchestra.run.smithi001.stdout: mon osd full ratio = 0.9 2024-05-09T08:01:22.038 INFO:teuthology.orchestra.run.smithi001.stdout: mon osd nearfull ratio = 0.8 2024-05-09T08:01:22.038 INFO:teuthology.orchestra.run.smithi001.stdout: osd blocked scrub grace period = 3600 2024-05-09T08:01:22.038 INFO:teuthology.orchestra.run.smithi001.stdout: osd failsafe full ratio = 0.95 2024-05-09T08:01:22.038 INFO:teuthology.orchestra.run.smithi001.stdout: osd max markdown count = 1000 2024-05-09T08:01:22.038 INFO:teuthology.orchestra.run.smithi001.stdout: osd objectstore = bluestore 2024-05-09T08:01:22.038 INFO:teuthology.orchestra.run.smithi001.stdout: osd op complaint time = 180 2024-05-09T08:01:22.038 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T08:01:22.038 INFO:teuthology.orchestra.run.smithi001.stdout:[mgr] 2024-05-09T08:01:22.038 INFO:teuthology.orchestra.run.smithi001.stdout: debug ms = 1 2024-05-09T08:01:22.039 INFO:teuthology.orchestra.run.smithi001.stdout: debug mgr = 20 2024-05-09T08:01:22.039 INFO:teuthology.orchestra.run.smithi001.stdout: debug mon = 20 2024-05-09T08:01:22.039 INFO:teuthology.orchestra.run.smithi001.stdout: debug auth = 20 2024-05-09T08:01:22.039 INFO:teuthology.orchestra.run.smithi001.stdout: mon reweight min pgs per osd = 4 2024-05-09T08:01:22.039 INFO:teuthology.orchestra.run.smithi001.stdout: mon reweight min bytes per osd = 10 2024-05-09T08:01:22.039 INFO:teuthology.orchestra.run.smithi001.stdout: mgr/telemetry/nag = false 2024-05-09T08:01:22.039 INFO:teuthology.orchestra.run.smithi001.stdout: debug client = 20 2024-05-09T08:01:22.039 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T08:01:22.039 INFO:teuthology.orchestra.run.smithi001.stdout:[mon] 2024-05-09T08:01:22.039 INFO:teuthology.orchestra.run.smithi001.stdout: debug ms = 1 2024-05-09T08:01:22.039 INFO:teuthology.orchestra.run.smithi001.stdout: debug mon = 20 2024-05-09T08:01:22.039 INFO:teuthology.orchestra.run.smithi001.stdout: debug paxos = 20 2024-05-09T08:01:22.039 INFO:teuthology.orchestra.run.smithi001.stdout: debug auth = 20 2024-05-09T08:01:22.040 INFO:teuthology.orchestra.run.smithi001.stdout: mon data avail warn = 5 2024-05-09T08:01:22.040 INFO:teuthology.orchestra.run.smithi001.stdout: mon mgr mkfs grace = 240 2024-05-09T08:01:22.040 INFO:teuthology.orchestra.run.smithi001.stdout: mon reweight min pgs per osd = 4 2024-05-09T08:01:22.040 INFO:teuthology.orchestra.run.smithi001.stdout: mon osd reporter subtree level = osd 2024-05-09T08:01:22.040 INFO:teuthology.orchestra.run.smithi001.stdout: mon osd prime pg temp = true 2024-05-09T08:01:22.040 INFO:teuthology.orchestra.run.smithi001.stdout: mon reweight min bytes per osd = 10 2024-05-09T08:01:22.040 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T08:01:22.040 INFO:teuthology.orchestra.run.smithi001.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-05-09T08:01:22.040 INFO:teuthology.orchestra.run.smithi001.stdout: auth mon ticket ttl = 660 # 11m 2024-05-09T08:01:22.040 INFO:teuthology.orchestra.run.smithi001.stdout: auth service ticket ttl = 240 # 4m 2024-05-09T08:01:22.040 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T08:01:22.040 INFO:teuthology.orchestra.run.smithi001.stdout: # don't complain about insecure global_id in the test suite 2024-05-09T08:01:22.040 INFO:teuthology.orchestra.run.smithi001.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-05-09T08:01:22.041 INFO:teuthology.orchestra.run.smithi001.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-05-09T08:01:22.041 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T08:01:22.041 INFO:teuthology.orchestra.run.smithi001.stdout: # 1m isn't quite enough 2024-05-09T08:01:22.041 INFO:teuthology.orchestra.run.smithi001.stdout: mon_down_mkfs_grace = 2m 2024-05-09T08:01:22.041 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T08:01:22.041 INFO:teuthology.orchestra.run.smithi001.stdout: mon_warn_on_filestore_osds = false 2024-05-09T08:01:22.041 INFO:teuthology.orchestra.run.smithi001.stdout: mon down mkfs grace = 300 2024-05-09T08:01:22.041 INFO:teuthology.orchestra.run.smithi001.stdout: mon op complaint time = 120 2024-05-09T08:01:22.041 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T08:01:22.041 INFO:teuthology.orchestra.run.smithi001.stdout:[client] 2024-05-09T08:01:22.041 INFO:teuthology.orchestra.run.smithi001.stdout: rgw cache enabled = true 2024-05-09T08:01:22.041 INFO:teuthology.orchestra.run.smithi001.stdout: rgw enable ops log = true 2024-05-09T08:01:22.041 INFO:teuthology.orchestra.run.smithi001.stdout: rgw enable usage log = true 2024-05-09T08:01:22.041 INFO:teuthology.orchestra.run.smithi001.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-05-09T08:01:22.041 INFO:teuthology.orchestra.run.smithi001.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-05-09T08:01:22.042 INFO:teuthology.orchestra.run.smithi001.stdout: client mount timeout = 600 2024-05-09T08:01:22.042 INFO:teuthology.orchestra.run.smithi001.stdout: client_shutdown_timeout = 120 2024-05-09T08:01:22.042 INFO:teuthology.orchestra.run.smithi001.stdout: debug client = 20 2024-05-09T08:01:22.042 INFO:teuthology.orchestra.run.smithi001.stdout: debug ms = 1 2024-05-09T08:01:22.042 INFO:teuthology.orchestra.run.smithi001.stdout: rados mon op timeout = 900 2024-05-09T08:01:22.042 INFO:teuthology.orchestra.run.smithi001.stdout: rados osd op timeout = 900 2024-05-09T08:01:22.042 INFO:teuthology.orchestra.run.smithi001.stdout:[mon.a] 2024-05-09T08:01:22.042 INFO:teuthology.orchestra.run.smithi001.stdout:[mon.b] 2024-05-09T08:01:22.042 INFO:teuthology.orchestra.run.smithi001.stdout:[mon.c] 2024-05-09T08:01:22.042 INFO:teuthology.orchestra.run.smithi001.stdout:[mds.a] 2024-05-09T08:01:22.042 INFO:teuthology.orchestra.run.smithi001.stdout:[mds.c] 2024-05-09T08:01:22.042 INFO:teuthology.orchestra.run.smithi001.stdout:[mds.e] 2024-05-09T08:01:22.042 INFO:teuthology.orchestra.run.smithi001.stdout:[mds.b] 2024-05-09T08:01:22.042 INFO:teuthology.orchestra.run.smithi001.stdout:[mds.d] 2024-05-09T08:01:22.043 INFO:teuthology.orchestra.run.smithi001.stdout:[mds.f] 2024-05-09T08:01:22.043 INFO:teuthology.orchestra.run.smithi001.stdout:[mds] 2024-05-09T08:01:22.043 INFO:teuthology.orchestra.run.smithi001.stdout: debug mds = 20 2024-05-09T08:01:22.043 INFO:teuthology.orchestra.run.smithi001.stdout: debug mds balancer = 20 2024-05-09T08:01:22.043 INFO:teuthology.orchestra.run.smithi001.stdout: debug ms = 1 2024-05-09T08:01:22.043 INFO:teuthology.orchestra.run.smithi001.stdout: mds bal fragment size max = 10000 2024-05-09T08:01:22.043 INFO:teuthology.orchestra.run.smithi001.stdout: mds bal merge size = 5 2024-05-09T08:01:22.043 INFO:teuthology.orchestra.run.smithi001.stdout: mds bal split bits = 3 2024-05-09T08:01:22.043 INFO:teuthology.orchestra.run.smithi001.stdout: mds bal split size = 100 2024-05-09T08:01:22.043 INFO:teuthology.orchestra.run.smithi001.stdout: mds debug frag = True 2024-05-09T08:01:22.043 INFO:teuthology.orchestra.run.smithi001.stdout: mds debug scatterstat = True 2024-05-09T08:01:22.043 INFO:teuthology.orchestra.run.smithi001.stdout: mds oft prefetch dirfrags = False 2024-05-09T08:01:22.043 INFO:teuthology.orchestra.run.smithi001.stdout: mds op complaint time = 180 2024-05-09T08:01:22.044 INFO:teuthology.orchestra.run.smithi001.stdout: mds verify scatter = True 2024-05-09T08:01:22.044 INFO:teuthology.orchestra.run.smithi001.stdout: mds_dir_prefetch = False 2024-05-09T08:01:22.044 INFO:teuthology.orchestra.run.smithi001.stdout: mds_sleep_rank_change = 5000000.0 2024-05-09T08:01:22.044 INFO:teuthology.orchestra.run.smithi001.stdout: osd op complaint time = 180 2024-05-09T08:01:22.044 INFO:teuthology.orchestra.run.smithi001.stdout: rados mon op timeout = 900 2024-05-09T08:01:22.044 INFO:teuthology.orchestra.run.smithi001.stdout: rados osd op timeout = 900 2024-05-09T08:01:22.070 INFO:tasks.ceph:Creating admin key on mon.a... 2024-05-09T08:01:22.071 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-05-09T08:01:22.209 INFO:tasks.ceph:Copying monmap to all nodes... 2024-05-09T08:01:22.210 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-05-09T08:01:22.210 DEBUG:teuthology.orchestra.run.smithi001:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-05-09T08:01:22.261 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-05-09T08:01:22.261 DEBUG:teuthology.orchestra.run.smithi001:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-05-09T08:01:22.313 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi001.front.sepia.ceph.com 2024-05-09T08:01:22.313 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-05-09T08:01:22.313 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd of=/etc/ceph/ceph.keyring 2024-05-09T08:01:22.313 DEBUG:teuthology.orchestra.run.smithi001:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-05-09T08:01:22.440 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-05-09T08:01:22.440 DEBUG:teuthology.orchestra.run.smithi001:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-05-09T08:01:22.541 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi007.front.sepia.ceph.com 2024-05-09T08:01:22.541 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-05-09T08:01:22.541 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/etc/ceph/ceph.keyring 2024-05-09T08:01:22.541 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-05-09T08:01:22.582 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-05-09T08:01:22.582 DEBUG:teuthology.orchestra.run.smithi007:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-05-09T08:01:22.640 INFO:tasks.ceph:Setting up mon nodes... 2024-05-09T08:01:22.640 INFO:tasks.ceph:Setting up mgr nodes... 2024-05-09T08:01:22.640 DEBUG:teuthology.orchestra.run.smithi001:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-05-09T08:01:22.812 INFO:teuthology.orchestra.run.smithi001.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-05-09T08:01:22.821 DEBUG:teuthology.orchestra.run.smithi007:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-05-09T08:01:22.891 INFO:teuthology.orchestra.run.smithi007.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-05-09T08:01:22.894 INFO:tasks.ceph:Setting up mds nodes... 2024-05-09T08:01:22.895 DEBUG:teuthology.orchestra.run.smithi001:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-05-09T08:01:23.054 INFO:teuthology.orchestra.run.smithi001.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-05-09T08:01:23.064 DEBUG:teuthology.orchestra.run.smithi001:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-05-09T08:01:23.144 DEBUG:teuthology.orchestra.run.smithi001:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-05-09T08:01:23.309 INFO:teuthology.orchestra.run.smithi001.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-05-09T08:01:23.319 DEBUG:teuthology.orchestra.run.smithi001:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-05-09T08:01:23.407 DEBUG:teuthology.orchestra.run.smithi001:> sudo mkdir -p /var/lib/ceph/mds/ceph-e && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.e /var/lib/ceph/mds/ceph-e/keyring 2024-05-09T08:01:23.568 INFO:teuthology.orchestra.run.smithi001.stdout:creating /var/lib/ceph/mds/ceph-e/keyring 2024-05-09T08:01:23.577 DEBUG:teuthology.orchestra.run.smithi001:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-e 2024-05-09T08:01:23.664 DEBUG:teuthology.orchestra.run.smithi007:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-05-09T08:01:23.723 INFO:teuthology.orchestra.run.smithi007.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-05-09T08:01:23.725 DEBUG:teuthology.orchestra.run.smithi007:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-05-09T08:01:23.752 DEBUG:teuthology.orchestra.run.smithi007:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-05-09T08:01:23.846 INFO:teuthology.orchestra.run.smithi007.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-05-09T08:01:23.849 DEBUG:teuthology.orchestra.run.smithi007:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-05-09T08:01:23.874 DEBUG:teuthology.orchestra.run.smithi007:> sudo mkdir -p /var/lib/ceph/mds/ceph-f && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.f /var/lib/ceph/mds/ceph-f/keyring 2024-05-09T08:01:23.965 INFO:teuthology.orchestra.run.smithi007.stdout:creating /var/lib/ceph/mds/ceph-f/keyring 2024-05-09T08:01:23.967 DEBUG:teuthology.orchestra.run.smithi007:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-f 2024-05-09T08:01:23.991 INFO:tasks.ceph_client:Setting up client nodes... 2024-05-09T08:01:23.991 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-05-09T08:01:24.121 INFO:teuthology.orchestra.run.smithi001.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-05-09T08:01:24.163 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-05-09T08:01:24.163 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi001.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi007.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-05-09T08:01:24.164 DEBUG:teuthology.orchestra.run.smithi001:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-05-09T08:01:24.254 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-09T08:01:24.254 INFO:tasks.ceph:role: osd.0 2024-05-09T08:01:24.254 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi001.front.sepia.ceph.com 2024-05-09T08:01:24.254 DEBUG:teuthology.orchestra.run.smithi001:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-05-09T08:01:24.336 INFO:teuthology.orchestra.run.smithi001.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-05-09T08:01:24.337 INFO:teuthology.orchestra.run.smithi001.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-09T08:01:24.337 INFO:teuthology.orchestra.run.smithi001.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-09T08:01:24.337 INFO:teuthology.orchestra.run.smithi001.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-09T08:01:24.337 INFO:teuthology.orchestra.run.smithi001.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-09T08:01:24.337 INFO:teuthology.orchestra.run.smithi001.stdout: = sunit=0 swidth=0 blks 2024-05-09T08:01:24.337 INFO:teuthology.orchestra.run.smithi001.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-09T08:01:24.337 INFO:teuthology.orchestra.run.smithi001.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-09T08:01:24.337 INFO:teuthology.orchestra.run.smithi001.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-09T08:01:24.337 INFO:teuthology.orchestra.run.smithi001.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-09T08:01:24.370 INFO:teuthology.orchestra.run.smithi001.stdout:Discarding blocks...Done. 2024-05-09T08:01:24.376 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi001.front.sepia.ceph.com -o noatime 2024-05-09T08:01:24.376 DEBUG:teuthology.orchestra.run.smithi001:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-05-09T08:01:24.475 DEBUG:teuthology.orchestra.run.smithi001:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-05-09T08:01:24.563 DEBUG:teuthology.orchestra.run.smithi001:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-05-09T08:01:24.648 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-09T08:01:24.648 INFO:tasks.ceph:role: osd.1 2024-05-09T08:01:24.648 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi001.front.sepia.ceph.com 2024-05-09T08:01:24.648 DEBUG:teuthology.orchestra.run.smithi001:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-05-09T08:01:24.722 INFO:teuthology.orchestra.run.smithi001.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-05-09T08:01:24.722 INFO:teuthology.orchestra.run.smithi001.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-09T08:01:24.722 INFO:teuthology.orchestra.run.smithi001.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-09T08:01:24.722 INFO:teuthology.orchestra.run.smithi001.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-09T08:01:24.722 INFO:teuthology.orchestra.run.smithi001.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-09T08:01:24.722 INFO:teuthology.orchestra.run.smithi001.stdout: = sunit=0 swidth=0 blks 2024-05-09T08:01:24.723 INFO:teuthology.orchestra.run.smithi001.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-09T08:01:24.723 INFO:teuthology.orchestra.run.smithi001.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-09T08:01:24.723 INFO:teuthology.orchestra.run.smithi001.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-09T08:01:24.723 INFO:teuthology.orchestra.run.smithi001.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-09T08:01:24.756 INFO:teuthology.orchestra.run.smithi001.stdout:Discarding blocks...Done. 2024-05-09T08:01:24.761 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi001.front.sepia.ceph.com -o noatime 2024-05-09T08:01:24.761 DEBUG:teuthology.orchestra.run.smithi001:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-05-09T08:01:24.862 DEBUG:teuthology.orchestra.run.smithi001:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-05-09T08:01:24.943 DEBUG:teuthology.orchestra.run.smithi001:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-05-09T08:01:25.027 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-09T08:01:25.028 INFO:tasks.ceph:role: osd.2 2024-05-09T08:01:25.028 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi001.front.sepia.ceph.com 2024-05-09T08:01:25.028 DEBUG:teuthology.orchestra.run.smithi001:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-05-09T08:01:25.110 INFO:teuthology.orchestra.run.smithi001.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-05-09T08:01:25.111 INFO:teuthology.orchestra.run.smithi001.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-09T08:01:25.111 INFO:teuthology.orchestra.run.smithi001.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-09T08:01:25.111 INFO:teuthology.orchestra.run.smithi001.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-09T08:01:25.111 INFO:teuthology.orchestra.run.smithi001.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-09T08:01:25.111 INFO:teuthology.orchestra.run.smithi001.stdout: = sunit=0 swidth=0 blks 2024-05-09T08:01:25.111 INFO:teuthology.orchestra.run.smithi001.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-09T08:01:25.111 INFO:teuthology.orchestra.run.smithi001.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-09T08:01:25.111 INFO:teuthology.orchestra.run.smithi001.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-09T08:01:25.111 INFO:teuthology.orchestra.run.smithi001.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-09T08:01:25.144 INFO:teuthology.orchestra.run.smithi001.stdout:Discarding blocks...Done. 2024-05-09T08:01:25.149 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi001.front.sepia.ceph.com -o noatime 2024-05-09T08:01:25.150 DEBUG:teuthology.orchestra.run.smithi001:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-05-09T08:01:25.262 DEBUG:teuthology.orchestra.run.smithi001:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-05-09T08:01:25.341 DEBUG:teuthology.orchestra.run.smithi001:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-05-09T08:01:25.413 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-09T08:01:25.413 INFO:tasks.ceph:role: osd.3 2024-05-09T08:01:25.413 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi001.front.sepia.ceph.com 2024-05-09T08:01:25.414 DEBUG:teuthology.orchestra.run.smithi001:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-05-09T08:01:25.494 INFO:teuthology.orchestra.run.smithi001.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-05-09T08:01:25.494 INFO:teuthology.orchestra.run.smithi001.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-09T08:01:25.494 INFO:teuthology.orchestra.run.smithi001.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-09T08:01:25.494 INFO:teuthology.orchestra.run.smithi001.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-09T08:01:25.494 INFO:teuthology.orchestra.run.smithi001.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-09T08:01:25.494 INFO:teuthology.orchestra.run.smithi001.stdout: = sunit=0 swidth=0 blks 2024-05-09T08:01:25.494 INFO:teuthology.orchestra.run.smithi001.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-09T08:01:25.494 INFO:teuthology.orchestra.run.smithi001.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-09T08:01:25.494 INFO:teuthology.orchestra.run.smithi001.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-09T08:01:25.494 INFO:teuthology.orchestra.run.smithi001.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-09T08:01:25.527 INFO:teuthology.orchestra.run.smithi001.stdout:Discarding blocks...Done. 2024-05-09T08:01:25.534 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi001.front.sepia.ceph.com -o noatime 2024-05-09T08:01:25.535 DEBUG:teuthology.orchestra.run.smithi001:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-05-09T08:01:25.639 DEBUG:teuthology.orchestra.run.smithi001:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-05-09T08:01:25.711 DEBUG:teuthology.orchestra.run.smithi001:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-09T08:01:25.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:01:25.836+0000 7f7a02a16640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-05-09T08:01:25.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:01:25.837+0000 7f7a02a16640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-05-09T08:01:25.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:01:25.837+0000 7f7a02a16640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-05-09T08:01:25.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:01:25.837+0000 7f7a02a16640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-05-09T08:01:25.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:01:25.837+0000 7f7a02a16640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-05-09T08:01:25.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:01:25.841+0000 7f7a02a16640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-05-09T08:01:28.821 DEBUG:teuthology.orchestra.run.smithi001:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-05-09T08:01:28.914 DEBUG:teuthology.orchestra.run.smithi001:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-09T08:01:29.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:01:29.032+0000 7f0baeed9640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-05-09T08:01:29.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:01:29.032+0000 7f0baeed9640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-05-09T08:01:29.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:01:29.033+0000 7f0baeed9640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-05-09T08:01:29.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:01:29.033+0000 7f0baeed9640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-05-09T08:01:29.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:01:29.033+0000 7f0baeed9640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-05-09T08:01:29.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:01:29.035+0000 7f0baeed9640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-05-09T08:01:32.258 DEBUG:teuthology.orchestra.run.smithi001:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-05-09T08:01:32.345 DEBUG:teuthology.orchestra.run.smithi001:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-09T08:01:32.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:01:32.464+0000 7f05e2a19640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-05-09T08:01:32.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:01:32.465+0000 7f05e2a19640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-05-09T08:01:32.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:01:32.465+0000 7f05e2a19640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-05-09T08:01:32.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:01:32.465+0000 7f05e2a19640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-05-09T08:01:32.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:01:32.465+0000 7f05e2a19640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-05-09T08:01:32.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:01:32.467+0000 7f05e2a19640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-05-09T08:01:35.468 DEBUG:teuthology.orchestra.run.smithi001:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-05-09T08:01:35.556 DEBUG:teuthology.orchestra.run.smithi001:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-09T08:01:35.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:01:35.668+0000 7f625981c640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-05-09T08:01:35.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:01:35.669+0000 7f625981c640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-05-09T08:01:35.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:01:35.669+0000 7f625981c640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-05-09T08:01:35.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:01:35.669+0000 7f625981c640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-05-09T08:01:35.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:01:35.669+0000 7f625981c640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-05-09T08:01:35.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:01:35.671+0000 7f625981c640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-05-09T08:01:38.910 DEBUG:teuthology.orchestra.run.smithi001:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-05-09T08:01:38.996 DEBUG:teuthology.orchestra.run.smithi007:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-05-09T08:01:39.027 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-09T08:01:39.027 INFO:tasks.ceph:role: osd.4 2024-05-09T08:01:39.027 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi007.front.sepia.ceph.com 2024-05-09T08:01:39.027 DEBUG:teuthology.orchestra.run.smithi007:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-05-09T08:01:39.095 INFO:teuthology.orchestra.run.smithi007.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-05-09T08:01:39.095 INFO:teuthology.orchestra.run.smithi007.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-09T08:01:39.096 INFO:teuthology.orchestra.run.smithi007.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-09T08:01:39.096 INFO:teuthology.orchestra.run.smithi007.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-09T08:01:39.096 INFO:teuthology.orchestra.run.smithi007.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-09T08:01:39.096 INFO:teuthology.orchestra.run.smithi007.stdout: = sunit=0 swidth=0 blks 2024-05-09T08:01:39.096 INFO:teuthology.orchestra.run.smithi007.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-09T08:01:39.096 INFO:teuthology.orchestra.run.smithi007.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-09T08:01:39.096 INFO:teuthology.orchestra.run.smithi007.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-09T08:01:39.096 INFO:teuthology.orchestra.run.smithi007.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-09T08:01:39.127 INFO:teuthology.orchestra.run.smithi007.stdout:Discarding blocks...Done. 2024-05-09T08:01:39.129 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi007.front.sepia.ceph.com -o noatime 2024-05-09T08:01:39.129 DEBUG:teuthology.orchestra.run.smithi007:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-05-09T08:01:39.202 DEBUG:teuthology.orchestra.run.smithi007:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-05-09T08:01:39.273 DEBUG:teuthology.orchestra.run.smithi007:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-05-09T08:01:39.339 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-09T08:01:39.339 INFO:tasks.ceph:role: osd.5 2024-05-09T08:01:39.339 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi007.front.sepia.ceph.com 2024-05-09T08:01:39.339 DEBUG:teuthology.orchestra.run.smithi007:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-05-09T08:01:39.405 INFO:teuthology.orchestra.run.smithi007.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-05-09T08:01:39.405 INFO:teuthology.orchestra.run.smithi007.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-09T08:01:39.405 INFO:teuthology.orchestra.run.smithi007.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-09T08:01:39.405 INFO:teuthology.orchestra.run.smithi007.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-09T08:01:39.405 INFO:teuthology.orchestra.run.smithi007.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-09T08:01:39.405 INFO:teuthology.orchestra.run.smithi007.stdout: = sunit=0 swidth=0 blks 2024-05-09T08:01:39.405 INFO:teuthology.orchestra.run.smithi007.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-09T08:01:39.405 INFO:teuthology.orchestra.run.smithi007.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-09T08:01:39.405 INFO:teuthology.orchestra.run.smithi007.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-09T08:01:39.405 INFO:teuthology.orchestra.run.smithi007.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-09T08:01:39.437 INFO:teuthology.orchestra.run.smithi007.stdout:Discarding blocks...Done. 2024-05-09T08:01:39.438 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi007.front.sepia.ceph.com -o noatime 2024-05-09T08:01:39.439 DEBUG:teuthology.orchestra.run.smithi007:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-05-09T08:01:39.507 DEBUG:teuthology.orchestra.run.smithi007:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-05-09T08:01:39.575 DEBUG:teuthology.orchestra.run.smithi007:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-05-09T08:01:39.639 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-09T08:01:39.640 INFO:tasks.ceph:role: osd.6 2024-05-09T08:01:39.640 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi007.front.sepia.ceph.com 2024-05-09T08:01:39.640 DEBUG:teuthology.orchestra.run.smithi007:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-05-09T08:01:39.705 INFO:teuthology.orchestra.run.smithi007.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-05-09T08:01:39.705 INFO:teuthology.orchestra.run.smithi007.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-09T08:01:39.705 INFO:teuthology.orchestra.run.smithi007.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-09T08:01:39.705 INFO:teuthology.orchestra.run.smithi007.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-09T08:01:39.705 INFO:teuthology.orchestra.run.smithi007.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-09T08:01:39.705 INFO:teuthology.orchestra.run.smithi007.stdout: = sunit=0 swidth=0 blks 2024-05-09T08:01:39.705 INFO:teuthology.orchestra.run.smithi007.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-09T08:01:39.705 INFO:teuthology.orchestra.run.smithi007.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-09T08:01:39.705 INFO:teuthology.orchestra.run.smithi007.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-09T08:01:39.705 INFO:teuthology.orchestra.run.smithi007.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-09T08:01:39.738 INFO:teuthology.orchestra.run.smithi007.stdout:Discarding blocks...Done. 2024-05-09T08:01:39.740 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi007.front.sepia.ceph.com -o noatime 2024-05-09T08:01:39.740 DEBUG:teuthology.orchestra.run.smithi007:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-05-09T08:01:39.809 DEBUG:teuthology.orchestra.run.smithi007:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-05-09T08:01:39.877 DEBUG:teuthology.orchestra.run.smithi007:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-05-09T08:01:39.941 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-09T08:01:39.941 INFO:tasks.ceph:role: osd.7 2024-05-09T08:01:39.941 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi007.front.sepia.ceph.com 2024-05-09T08:01:39.942 DEBUG:teuthology.orchestra.run.smithi007:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-05-09T08:01:40.006 INFO:teuthology.orchestra.run.smithi007.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-05-09T08:01:40.006 INFO:teuthology.orchestra.run.smithi007.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-09T08:01:40.006 INFO:teuthology.orchestra.run.smithi007.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-09T08:01:40.006 INFO:teuthology.orchestra.run.smithi007.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-09T08:01:40.006 INFO:teuthology.orchestra.run.smithi007.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-09T08:01:40.006 INFO:teuthology.orchestra.run.smithi007.stdout: = sunit=0 swidth=0 blks 2024-05-09T08:01:40.006 INFO:teuthology.orchestra.run.smithi007.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-09T08:01:40.006 INFO:teuthology.orchestra.run.smithi007.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-09T08:01:40.006 INFO:teuthology.orchestra.run.smithi007.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-09T08:01:40.006 INFO:teuthology.orchestra.run.smithi007.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-09T08:01:40.038 INFO:teuthology.orchestra.run.smithi007.stdout:Discarding blocks...Done. 2024-05-09T08:01:40.040 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi007.front.sepia.ceph.com -o noatime 2024-05-09T08:01:40.040 DEBUG:teuthology.orchestra.run.smithi007:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-05-09T08:01:40.109 DEBUG:teuthology.orchestra.run.smithi007:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-05-09T08:01:40.177 DEBUG:teuthology.orchestra.run.smithi007:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-09T08:01:40.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:40.251+0000 7f7cd7012640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-05-09T08:01:40.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:40.251+0000 7f7cd7012640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-05-09T08:01:40.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:40.251+0000 7f7cd7012640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-05-09T08:01:40.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:40.251+0000 7f7cd7012640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-05-09T08:01:40.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:40.251+0000 7f7cd7012640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-05-09T08:01:40.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:40.251+0000 7f7cd7012640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-05-09T08:01:43.121 DEBUG:teuthology.orchestra.run.smithi007:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-05-09T08:01:43.150 DEBUG:teuthology.orchestra.run.smithi007:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-09T08:01:43.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:43.223+0000 7fb8c92f0640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-05-09T08:01:43.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:43.224+0000 7fb8c92f0640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-05-09T08:01:43.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:43.224+0000 7fb8c92f0640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-05-09T08:01:43.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:43.224+0000 7fb8c92f0640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-05-09T08:01:43.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:43.224+0000 7fb8c92f0640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-05-09T08:01:43.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:43.224+0000 7fb8c92f0640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-05-09T08:01:46.084 DEBUG:teuthology.orchestra.run.smithi007:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-05-09T08:01:46.113 DEBUG:teuthology.orchestra.run.smithi007:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-09T08:01:46.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:46.188+0000 7fd735224640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-05-09T08:01:46.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:46.189+0000 7fd735224640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-05-09T08:01:46.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:46.189+0000 7fd735224640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-05-09T08:01:46.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:46.189+0000 7fd735224640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-05-09T08:01:46.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:46.189+0000 7fd735224640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-05-09T08:01:46.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:46.189+0000 7fd735224640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-05-09T08:01:49.056 DEBUG:teuthology.orchestra.run.smithi007:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-05-09T08:01:49.085 DEBUG:teuthology.orchestra.run.smithi007:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-09T08:01:49.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:49.157+0000 7f122392c640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-05-09T08:01:49.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:49.157+0000 7f122392c640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-05-09T08:01:49.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:49.157+0000 7f122392c640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-05-09T08:01:49.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:49.157+0000 7f122392c640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-05-09T08:01:49.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:49.157+0000 7f122392c640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-05-09T08:01:49.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:49.157+0000 7f122392c640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-05-09T08:01:49.517 DEBUG:teuthology.orchestra.run.smithi001:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T08:01:49.525 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T08:01:52.027 DEBUG:teuthology.orchestra.run.smithi007:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-05-09T08:01:52.056 INFO:tasks.ceph:Reading keys from all nodes... 2024-05-09T08:01:52.056 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-05-09T08:01:52.056 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-05-09T08:01:52.151 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-05-09T08:01:52.151 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-05-09T08:01:52.244 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-05-09T08:01:52.244 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-05-09T08:01:52.328 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-05-09T08:01:52.328 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd if=/var/lib/ceph/mds/ceph-e/keyring of=/dev/stdout 2024-05-09T08:01:52.414 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-05-09T08:01:52.414 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-05-09T08:01:52.501 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-05-09T08:01:52.501 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-05-09T08:01:52.582 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-05-09T08:01:52.582 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-05-09T08:01:52.666 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-05-09T08:01:52.666 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-05-09T08:01:52.756 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-05-09T08:01:52.757 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-05-09T08:01:52.786 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-05-09T08:01:52.786 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-05-09T08:01:52.854 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-05-09T08:01:52.854 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-05-09T08:01:52.922 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-05-09T08:01:52.922 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/var/lib/ceph/mds/ceph-f/keyring of=/dev/stdout 2024-05-09T08:01:52.988 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-05-09T08:01:52.988 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-05-09T08:01:53.054 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-05-09T08:01:53.054 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-05-09T08:01:53.120 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-05-09T08:01:53.120 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-05-09T08:01:53.186 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-05-09T08:01:53.186 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-05-09T08:01:53.251 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-05-09T08:01:53.251 DEBUG:teuthology.orchestra.run.smithi001:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-05-09T08:01:53.311 INFO:tasks.ceph:Adding keys to all mons... 2024-05-09T08:01:53.312 DEBUG:teuthology.orchestra.run.smithi001:> sudo tee -a /etc/ceph/ceph.keyring 2024-05-09T08:01:53.316 DEBUG:teuthology.orchestra.run.smithi007:> sudo tee -a /etc/ceph/ceph.keyring 2024-05-09T08:01:53.339 INFO:teuthology.orchestra.run.smithi007.stdout:[mgr.y] 2024-05-09T08:01:53.340 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQDSgjxmEL2FLxAAgnbSy8rR3mKk5F8MPPe4QA== 2024-05-09T08:01:53.340 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.a] 2024-05-09T08:01:53.340 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQDTgjxmpTpbAhAA54EwSqyvKxO3d2TMTtyjTw== 2024-05-09T08:01:53.340 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.c] 2024-05-09T08:01:53.340 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQDTgjxmPKCGERAAhl3IIV89JS8PlR7rWe4UGg== 2024-05-09T08:01:53.340 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.e] 2024-05-09T08:01:53.340 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQDTgjxmD470IBAAp0XZbd21UhBuvzd0ZJLzzQ== 2024-05-09T08:01:53.340 INFO:teuthology.orchestra.run.smithi007.stdout:[osd.0] 2024-05-09T08:01:53.340 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQDVgjxmdODxMRAAzIfbYEQiiHhK9bYJHAprew== 2024-05-09T08:01:53.340 INFO:teuthology.orchestra.run.smithi007.stdout:[osd.1] 2024-05-09T08:01:53.340 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQDZgjxmOPr/ARAAwvkLBRXPBEc8bz2umxL8Ng== 2024-05-09T08:01:53.341 INFO:teuthology.orchestra.run.smithi007.stdout:[osd.2] 2024-05-09T08:01:53.341 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQDcgjxm9yfCGxAA3zd3H7QzkaAQr6UNef5d8Q== 2024-05-09T08:01:53.341 INFO:teuthology.orchestra.run.smithi007.stdout:[osd.3] 2024-05-09T08:01:53.341 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQDfgjxmIdfqJxAAwdUXoZ79VSpeiBcnROQDTQ== 2024-05-09T08:01:53.341 INFO:teuthology.orchestra.run.smithi007.stdout:[mgr.x] 2024-05-09T08:01:53.341 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQDSgjxmNeZBNBAAmYF7bq8EFfd6sy/sAPeb+A== 2024-05-09T08:01:53.341 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.b] 2024-05-09T08:01:53.341 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQDTgjxmkokuKhAAUHKXBd3x9I7/PEVJmk//Ow== 2024-05-09T08:01:53.341 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.d] 2024-05-09T08:01:53.341 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQDTgjxmLg2NMRAAqzWpDRGLN9uS+Yz+KxeWFw== 2024-05-09T08:01:53.341 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.f] 2024-05-09T08:01:53.341 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQDTgjxml3ObOBAAVHKCwZ2UIQoK8RzJLvGWQA== 2024-05-09T08:01:53.341 INFO:teuthology.orchestra.run.smithi007.stdout:[osd.4] 2024-05-09T08:01:53.342 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQDkgjxmircFDxAADZJ+1MSbZ9r9SFs8nv71OA== 2024-05-09T08:01:53.342 INFO:teuthology.orchestra.run.smithi007.stdout:[osd.5] 2024-05-09T08:01:53.342 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQDngjxmpMdmDRAAK4ekaf4t+I4T1qcnoBEAAw== 2024-05-09T08:01:53.342 INFO:teuthology.orchestra.run.smithi007.stdout:[osd.6] 2024-05-09T08:01:53.342 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQDqgjxm5NBcCxAAvv5k56kE19W4ACKxmX3P3Q== 2024-05-09T08:01:53.342 INFO:teuthology.orchestra.run.smithi007.stdout:[osd.7] 2024-05-09T08:01:53.342 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQDtgjxmjQ56CRAAEtxjr8+LyZH1t57WJ2tUrQ== 2024-05-09T08:01:53.342 INFO:teuthology.orchestra.run.smithi007.stdout:[client.0] 2024-05-09T08:01:53.342 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQDUgjxm5c1NBhAAhpgT9kneioN3K30M2lYFsQ== 2024-05-09T08:01:53.387 INFO:teuthology.orchestra.run.smithi001.stdout:[mgr.y] 2024-05-09T08:01:53.387 INFO:teuthology.orchestra.run.smithi001.stdout: key = AQDSgjxmEL2FLxAAgnbSy8rR3mKk5F8MPPe4QA== 2024-05-09T08:01:53.388 INFO:teuthology.orchestra.run.smithi001.stdout:[mds.a] 2024-05-09T08:01:53.388 INFO:teuthology.orchestra.run.smithi001.stdout: key = AQDTgjxmpTpbAhAA54EwSqyvKxO3d2TMTtyjTw== 2024-05-09T08:01:53.388 INFO:teuthology.orchestra.run.smithi001.stdout:[mds.c] 2024-05-09T08:01:53.388 INFO:teuthology.orchestra.run.smithi001.stdout: key = AQDTgjxmPKCGERAAhl3IIV89JS8PlR7rWe4UGg== 2024-05-09T08:01:53.388 INFO:teuthology.orchestra.run.smithi001.stdout:[mds.e] 2024-05-09T08:01:53.388 INFO:teuthology.orchestra.run.smithi001.stdout: key = AQDTgjxmD470IBAAp0XZbd21UhBuvzd0ZJLzzQ== 2024-05-09T08:01:53.388 INFO:teuthology.orchestra.run.smithi001.stdout:[osd.0] 2024-05-09T08:01:53.388 INFO:teuthology.orchestra.run.smithi001.stdout: key = AQDVgjxmdODxMRAAzIfbYEQiiHhK9bYJHAprew== 2024-05-09T08:01:53.388 INFO:teuthology.orchestra.run.smithi001.stdout:[osd.1] 2024-05-09T08:01:53.388 INFO:teuthology.orchestra.run.smithi001.stdout: key = AQDZgjxmOPr/ARAAwvkLBRXPBEc8bz2umxL8Ng== 2024-05-09T08:01:53.388 INFO:teuthology.orchestra.run.smithi001.stdout:[osd.2] 2024-05-09T08:01:53.388 INFO:teuthology.orchestra.run.smithi001.stdout: key = AQDcgjxm9yfCGxAA3zd3H7QzkaAQr6UNef5d8Q== 2024-05-09T08:01:53.388 INFO:teuthology.orchestra.run.smithi001.stdout:[osd.3] 2024-05-09T08:01:53.388 INFO:teuthology.orchestra.run.smithi001.stdout: key = AQDfgjxmIdfqJxAAwdUXoZ79VSpeiBcnROQDTQ== 2024-05-09T08:01:53.389 INFO:teuthology.orchestra.run.smithi001.stdout:[mgr.x] 2024-05-09T08:01:53.389 INFO:teuthology.orchestra.run.smithi001.stdout: key = AQDSgjxmNeZBNBAAmYF7bq8EFfd6sy/sAPeb+A== 2024-05-09T08:01:53.389 INFO:teuthology.orchestra.run.smithi001.stdout:[mds.b] 2024-05-09T08:01:53.389 INFO:teuthology.orchestra.run.smithi001.stdout: key = AQDTgjxmkokuKhAAUHKXBd3x9I7/PEVJmk//Ow== 2024-05-09T08:01:53.389 INFO:teuthology.orchestra.run.smithi001.stdout:[mds.d] 2024-05-09T08:01:53.389 INFO:teuthology.orchestra.run.smithi001.stdout: key = AQDTgjxmLg2NMRAAqzWpDRGLN9uS+Yz+KxeWFw== 2024-05-09T08:01:53.389 INFO:teuthology.orchestra.run.smithi001.stdout:[mds.f] 2024-05-09T08:01:53.389 INFO:teuthology.orchestra.run.smithi001.stdout: key = AQDTgjxml3ObOBAAVHKCwZ2UIQoK8RzJLvGWQA== 2024-05-09T08:01:53.389 INFO:teuthology.orchestra.run.smithi001.stdout:[osd.4] 2024-05-09T08:01:53.389 INFO:teuthology.orchestra.run.smithi001.stdout: key = AQDkgjxmircFDxAADZJ+1MSbZ9r9SFs8nv71OA== 2024-05-09T08:01:53.389 INFO:teuthology.orchestra.run.smithi001.stdout:[osd.5] 2024-05-09T08:01:53.389 INFO:teuthology.orchestra.run.smithi001.stdout: key = AQDngjxmpMdmDRAAK4ekaf4t+I4T1qcnoBEAAw== 2024-05-09T08:01:53.389 INFO:teuthology.orchestra.run.smithi001.stdout:[osd.6] 2024-05-09T08:01:53.390 INFO:teuthology.orchestra.run.smithi001.stdout: key = AQDqgjxm5NBcCxAAvv5k56kE19W4ACKxmX3P3Q== 2024-05-09T08:01:53.390 INFO:teuthology.orchestra.run.smithi001.stdout:[osd.7] 2024-05-09T08:01:53.390 INFO:teuthology.orchestra.run.smithi001.stdout: key = AQDtgjxmjQ56CRAAEtxjr8+LyZH1t57WJ2tUrQ== 2024-05-09T08:01:53.390 INFO:teuthology.orchestra.run.smithi001.stdout:[client.0] 2024-05-09T08:01:53.390 INFO:teuthology.orchestra.run.smithi001.stdout: key = AQDUgjxm5c1NBhAAhpgT9kneioN3K30M2lYFsQ== 2024-05-09T08:01:53.391 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-05-09T08:01:53.438 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-05-09T08:01:53.561 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-09T08:01:53.566 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-09T08:01:53.690 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-09T08:01:53.695 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-09T08:01:53.820 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.e --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-09T08:01:53.827 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.e --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-09T08:01:53.958 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-09T08:01:53.964 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-09T08:01:54.097 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-09T08:01:54.102 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-09T08:01:54.233 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-09T08:01:54.241 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-09T08:01:54.373 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-09T08:01:54.379 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-09T08:01:54.516 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-05-09T08:01:54.522 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-05-09T08:01:54.659 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-09T08:01:54.665 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-09T08:01:54.807 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-09T08:01:54.812 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-09T08:01:54.950 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.f --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-09T08:01:54.956 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.f --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-09T08:01:55.082 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-09T08:01:55.087 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-09T08:01:55.216 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-09T08:01:55.223 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-09T08:01:55.353 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-09T08:01:55.358 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-09T08:01:55.485 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-09T08:01:55.490 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-09T08:01:55.627 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-09T08:01:55.633 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-09T08:01:55.771 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-05-09T08:01:55.771 DEBUG:teuthology.orchestra.run.smithi001:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-05-09T08:01:55.862 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-05-09T08:01:56.018 DEBUG:teuthology.orchestra.run.smithi001:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-05-09T08:01:56.091 DEBUG:teuthology.orchestra.run.smithi007:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-05-09T08:01:56.117 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-05-09T08:01:56.214 DEBUG:teuthology.orchestra.run.smithi007:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-05-09T08:01:56.239 DEBUG:teuthology.orchestra.run.smithi007:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-05-09T08:01:56.304 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-05-09T08:01:56.399 DEBUG:teuthology.orchestra.run.smithi007:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-05-09T08:01:56.423 DEBUG:teuthology.orchestra.run.smithi001:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-05-09T08:01:56.431 DEBUG:teuthology.orchestra.run.smithi007:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-05-09T08:01:56.480 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-05-09T08:01:56.481 INFO:tasks.ceph.mon.a:Restarting daemon 2024-05-09T08:01:56.481 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-05-09T08:01:56.486 INFO:tasks.ceph.mon.a:Started 2024-05-09T08:01:56.486 INFO:tasks.ceph.mon.b:Restarting daemon 2024-05-09T08:01:56.486 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-05-09T08:01:56.521 INFO:tasks.ceph.mon.b:Started 2024-05-09T08:01:56.521 INFO:tasks.ceph.mon.c:Restarting daemon 2024-05-09T08:01:56.522 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-05-09T08:01:56.523 INFO:tasks.ceph.mon.c:Started 2024-05-09T08:01:56.524 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-05-09T08:01:56.524 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-05-09T08:01:56.524 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-05-09T08:01:56.531 INFO:tasks.ceph.mgr.y:Started 2024-05-09T08:01:56.531 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-05-09T08:01:56.531 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-05-09T08:01:56.533 INFO:tasks.ceph.mgr.x:Started 2024-05-09T08:01:56.533 INFO:tasks.ceph:Setting crush tunables to default 2024-05-09T08:01:56.533 DEBUG:teuthology.orchestra.run.smithi001:> sudo ceph --cluster ceph osd crush tunables default 2024-05-09T08:01:56.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:01:56.701+0000 7f71e5600640 1 Processor -- start 2024-05-09T08:01:56.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:01:56.701+0000 7f71e5600640 1 -- start start 2024-05-09T08:01:56.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:01:56.702+0000 7f71e5600640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f71e00cac40 0x7f71e006da40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:01:56.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:01:56.702+0000 7f71e5600640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f71e015a610 0x7f71e015aac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:01:56.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:01:56.702+0000 7f71e5600640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f71e00cb020 con 0x7f71e00cd2a0 2024-05-09T08:01:56.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:01:56.703+0000 7f71e5600640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f71e0154dd0 con 0x7f71e015a610 2024-05-09T08:01:56.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:01:56.703+0000 7f71e5600640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f71e00cce10 con 0x7f71e00cac40 2024-05-09T08:01:56.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:01:56.703+0000 7f71e4c00640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f71e015a610 0x7f71e015aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:01:56.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:01:56.703+0000 7f71dfe00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f71e00cac40 0x7f71e006da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:01:56.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:01:56.703+0000 7f71df400640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f71e00cd2a0 0x7f71e015ba80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:37266/0 (socket says 172.21.15.1:37266) 2024-05-09T08:01:56.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:01:56.703+0000 7f71df400640 1 -- 172.21.15.1:0/1962743724 learned_addr learned my addr 172.21.15.1:0/1962743724 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T08:01:56.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:01:56.703+0000 7f71dfe00640 1 -- 172.21.15.1:0/1962743724 >> v2:172.21.15.1:3300/0 conn(0x7f71e015a610 msgr2=0x7f71e015aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:01:56.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:01:56.703+0000 7f71dfe00640 1 --2- 172.21.15.1:0/1962743724 >> v2:172.21.15.1:3300/0 conn(0x7f71e015a610 0x7f71e015aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:01:56.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:01:56.703+0000 7f71dfe00640 1 -- 172.21.15.1:0/1962743724 >> v1:172.21.15.1:6789/0 conn(0x7f71e00cd2a0 legacy=0x7f71e015ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:01:56.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:01:56.703+0000 7f71dfe00640 1 -- 172.21.15.1:0/1962743724 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71e00cc550 con 0x7f71e00cac40 2024-05-09T08:01:56.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:01:56.704+0000 7f71dfe00640 1 --2- 172.21.15.1:0/1962743724 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71e00cac40 0x7f71e006da40 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f71e00d0c90 tx=0x7f71d002a1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=68cc4fd157e8b927 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:01:56.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:01:56.705+0000 7f71dea00640 1 -- 172.21.15.1:0/1962743724 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71d0037070 con 0x7f71e00cac40 2024-05-09T08:01:56.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:01:56.705+0000 7f71dea00640 1 -- 172.21.15.1:0/1962743724 <== mon.1 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71d0032440 con 0x7f71e00cac40 2024-05-09T08:01:56.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:01:56.705+0000 7f71dea00640 1 -- 172.21.15.1:0/1962743724 <== mon.1 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71d0032810 con 0x7f71e00cac40 2024-05-09T08:01:56.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:01:56.705+0000 7f71e5600640 1 -- 172.21.15.1:0/1962743724 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71e00cac40 msgr2=0x7f71e006da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:01:56.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:01:56.705+0000 7f71e5600640 1 --2- 172.21.15.1:0/1962743724 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71e00cac40 0x7f71e006da40 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f71e00d0c90 tx=0x7f71d002a1a0 comp rx=0 tx=0).stop 2024-05-09T08:01:56.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:01:56.705+0000 7f71e5600640 1 -- 172.21.15.1:0/1962743724 shutdown_connections 2024-05-09T08:01:56.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:01:56.705+0000 7f71e5600640 1 --2- 172.21.15.1:0/1962743724 >> v2:172.21.15.1:3300/0 conn(0x7f71e015a610 0x7f71e015aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:01:56.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:01:56.705+0000 7f71e5600640 1 --2- 172.21.15.1:0/1962743724 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71e00cac40 0x7f71e006da40 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:01:56.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:01:56.705+0000 7f71e5600640 1 -- 172.21.15.1:0/1962743724 >> 172.21.15.1:0/1962743724 conn(0x7f71e00670b0 msgr2=0x7f71e00674c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:01:56.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:01:56.706+0000 7f71e5600640 1 -- 172.21.15.1:0/1962743724 shutdown_connections 2024-05-09T08:01:56.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:01:56.706+0000 7f71e5600640 1 -- 172.21.15.1:0/1962743724 wait complete. 2024-05-09T08:01:56.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:01:56.708+0000 7f71e5600640 1 Processor -- start 2024-05-09T08:01:56.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:01:56.708+0000 7f71e5600640 1 -- start start 2024-05-09T08:01:56.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:01:56.709+0000 7f71e5600640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f71e00cac40 0x7f71e0151070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:01:56.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:01:56.709+0000 7f71e5600640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f71e015a610 0x7f71e0152340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:01:56.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:01:56.709+0000 7f71e5600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71e01f7b30 0x7f71e0156590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:01:56.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:01:56.709+0000 7f71e5600640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f71e00cc7c0 con 0x7f71e015a610 2024-05-09T08:01:56.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:01:56.709+0000 7f71e5600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f71e015b410 con 0x7f71e01f7b30 2024-05-09T08:01:56.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:01:56.709+0000 7f71e5600640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f71e00ccaa0 con 0x7f71e00cac40 2024-05-09T08:01:56.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:01:56.709+0000 7f71df400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f71e015a610 0x7f71e0152340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:01:56.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:01:56.709+0000 7f71df400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f71e015a610 0x7f71e0152340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:39612/0 (socket says 172.21.15.1:39612) 2024-05-09T08:01:56.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:01:56.709+0000 7f71df400640 1 -- 172.21.15.1:0/1342648743 learned_addr learned my addr 172.21.15.1:0/1342648743 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:01:56.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:01:56.709+0000 7f71dfe00640 1 --2- 172.21.15.1:0/1342648743 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f71e00cac40 0x7f71e0151070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:01:56.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:01:56.709+0000 7f71e4c00640 1 --2- 172.21.15.1:0/1342648743 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71e01f7b30 0x7f71e0156590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:01:56.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:01:56.710+0000 7f71dfe00640 1 -- 172.21.15.1:0/1342648743 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71e01f7b30 msgr2=0x7f71e0156590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:01:56.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:01:56.710+0000 7f71dfe00640 1 --2- 172.21.15.1:0/1342648743 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71e01f7b30 0x7f71e0156590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:01:56.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:01:56.710+0000 7f71dfe00640 1 -- 172.21.15.1:0/1342648743 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f71e015a610 msgr2=0x7f71e0152340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:01:56.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:01:56.710+0000 7f71dfe00640 1 --2- 172.21.15.1:0/1342648743 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f71e015a610 0x7f71e0152340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:01:56.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:01:56.710+0000 7f71dfe00640 1 -- 172.21.15.1:0/1342648743 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71e014f800 con 0x7f71e00cac40 2024-05-09T08:01:56.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:01:56.710+0000 7f71e4c00640 1 --2- 172.21.15.1:0/1342648743 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71e01f7b30 0x7f71e0156590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:01:56.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:01:56.710+0000 7f71df400640 1 --2- 172.21.15.1:0/1342648743 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f71e015a610 0x7f71e0152340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:01:56.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:01:56.711+0000 7f71dfe00640 1 --2- 172.21.15.1:0/1342648743 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f71e00cac40 0x7f71e0151070 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f71d0031000 tx=0x7f71d00046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:01:56.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:01:56.711+0000 7f71d7e00640 1 -- 172.21.15.1:0/1342648743 <== mon.2 v2:172.21.15.7:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71d0032e60 con 0x7f71e00cac40 2024-05-09T08:01:56.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:01:56.711+0000 7f71d7e00640 1 -- 172.21.15.1:0/1342648743 <== mon.2 v2:172.21.15.7:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71d0030770 con 0x7f71e00cac40 2024-05-09T08:01:56.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:01:56.711+0000 7f71d7e00640 1 -- 172.21.15.1:0/1342648743 <== mon.2 v2:172.21.15.7:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71d0037040 con 0x7f71e00cac40 2024-05-09T08:01:56.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:01:56.711+0000 7f71e5600640 1 -- 172.21.15.1:0/1342648743 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71e014fae0 con 0x7f71e00cac40 2024-05-09T08:01:56.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:01:56.711+0000 7f71e5600640 1 -- 172.21.15.1:0/1342648743 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f71e01500a0 con 0x7f71e00cac40 2024-05-09T08:01:56.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:01:56.711+0000 7f71d7e00640 1 -- 172.21.15.1:0/1342648743 <== mon.2 v2:172.21.15.7:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f71d0044020 con 0x7f71e00cac40 2024-05-09T08:01:56.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:01:56.711+0000 7f71d7e00640 1 -- 172.21.15.1:0/1342648743 <== mon.2 v2:172.21.15.7:3301/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f71d003dcb0 con 0x7f71e00cac40 2024-05-09T08:01:56.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:01:56.712+0000 7f71e5600640 1 -- 172.21.15.1:0/1342648743 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71e004f660 con 0x7f71e00cac40 2024-05-09T08:01:56.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:01:56.715+0000 7f71d7e00640 1 -- 172.21.15.1:0/1342648743 <== mon.2 v2:172.21.15.7:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f71d003d440 con 0x7f71e00cac40 2024-05-09T08:01:56.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:01:56.788+0000 7f71e5600640 1 -- 172.21.15.1:0/1342648743 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f71e006e0d0 con 0x7f71e00cac40 2024-05-09T08:01:56.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:01:56.792+0000 7f71d7e00640 1 -- 172.21.15.1:0/1342648743 <== mon.2 v2:172.21.15.7:3301/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) ==== 124+0+0 (secure 0 0 0) 0x7f71d003d620 con 0x7f71e00cac40 2024-05-09T08:01:56.809 INFO:teuthology.orchestra.run.smithi001.stderr:adjusted tunables profile to default 2024-05-09T08:01:56.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:01:56.794+0000 7f71e5600640 1 -- 172.21.15.1:0/1342648743 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f71e00cac40 msgr2=0x7f71e0151070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:01:56.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:01:56.794+0000 7f71e5600640 1 --2- 172.21.15.1:0/1342648743 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f71e00cac40 0x7f71e0151070 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f71d0031000 tx=0x7f71d00046c0 comp rx=0 tx=0).stop 2024-05-09T08:01:56.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:01:56.794+0000 7f71e5600640 1 -- 172.21.15.1:0/1342648743 shutdown_connections 2024-05-09T08:01:56.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:01:56.794+0000 7f71e5600640 1 --2- 172.21.15.1:0/1342648743 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71e01f7b30 0x7f71e0156590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:01:56.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:01:56.794+0000 7f71e5600640 1 --2- 172.21.15.1:0/1342648743 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f71e015a610 0x7f71e0152340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:01:56.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:01:56.794+0000 7f71e5600640 1 --2- 172.21.15.1:0/1342648743 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f71e00cac40 0x7f71e0151070 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:01:56.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:01:56.794+0000 7f71e5600640 1 -- 172.21.15.1:0/1342648743 >> 172.21.15.1:0/1342648743 conn(0x7f71e00670b0 msgr2=0x7f71e0153b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:01:56.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:01:56.794+0000 7f71e5600640 1 -- 172.21.15.1:0/1342648743 shutdown_connections 2024-05-09T08:01:56.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:01:56.794+0000 7f71e5600640 1 -- 172.21.15.1:0/1342648743 wait complete. 2024-05-09T08:01:56.829 INFO:tasks.ceph:check_enable_crimson: False 2024-05-09T08:01:56.829 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-05-09T08:01:56.829 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-05-09T08:01:56.829 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-05-09T08:01:56.850 INFO:tasks.ceph.mgr.y.smithi001.stderr:2024-05-09T08:01:56.834+0000 7fa06500d040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-09T08:01:56.899 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-05-09T08:01:56.899 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-05-09T08:01:56.922 INFO:tasks.ceph.mgr.x.smithi007.stderr:2024-05-09T08:01:56.905+0000 7f934988f040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-09T08:01:57.023 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-05-09T08:01:57.023 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-05-09T08:01:57.045 INFO:tasks.ceph.mgr.y.smithi001.stderr:2024-05-09T08:01:57.028+0000 7fa06500d040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-09T08:01:57.079 INFO:tasks.ceph.mgr.x.smithi007.stderr:2024-05-09T08:01:57.063+0000 7f934988f040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-09T08:01:57.095 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-05-09T08:01:57.095 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-05-09T08:01:57.114 INFO:tasks.ceph.mgr.y.smithi001.stderr:2024-05-09T08:01:57.098+0000 7fa06500d040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-09T08:01:57.135 INFO:tasks.ceph.mgr.x.smithi007.stderr:2024-05-09T08:01:57.119+0000 7f934988f040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-09T08:01:57.192 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-05-09T08:01:57.192 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-05-09T08:01:57.219 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-05-09T08:01:57.219 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-05-09T08:01:57.283 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-05-09T08:01:57.284 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-05-09T08:01:57.347 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-05-09T08:01:57.348 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-05-09T08:01:57.394 INFO:tasks.ceph.mgr.x.smithi007.stderr:2024-05-09T08:01:57.378+0000 7f934988f040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-09T08:01:57.413 DEBUG:teuthology.orchestra.run.smithi007:> sudo ceph --cluster ceph osd new c064454e-a9b5-4a24-9a77-17f0a4085952 0 2024-05-09T08:01:57.443 INFO:tasks.ceph.mgr.x.smithi007.stderr:2024-05-09T08:01:57.427+0000 7f934988f040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-09T08:01:57.449 INFO:tasks.ceph.mgr.y.smithi001.stderr:2024-05-09T08:01:57.433+0000 7fa06500d040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-09T08:01:57.507 INFO:tasks.ceph.mgr.y.smithi001.stderr:2024-05-09T08:01:57.490+0000 7fa06500d040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-09T08:01:57.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.527+0000 7fa6b3dc1640 1 Processor -- start 2024-05-09T08:01:57.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.527+0000 7fa6b3dc1640 1 -- start start 2024-05-09T08:01:57.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.528+0000 7fa6b3dc1640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fa6ac071dc0 0x7fa6ac0c2d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:01:57.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.528+0000 7fa6b3dc1640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa6ac158dc0 con 0x7fa6ac15c5d0 2024-05-09T08:01:57.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.528+0000 7fa6b3dc1640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa6ac1590a0 con 0x7fa6ac0719e0 2024-05-09T08:01:57.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.528+0000 7fa6b3dc1640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7fa6ac0b3360 con 0x7fa6ac071dc0 2024-05-09T08:01:57.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.528+0000 7fa6b135c640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fa6ac071dc0 0x7fa6ac0c2d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:01:57.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.528+0000 7fa6b1b5d640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fa6ac0719e0 0x7fa6ac158380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:42900/0 (socket says 172.21.15.7:42900) 2024-05-09T08:01:57.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.528+0000 7fa6b1b5d640 1 -- 172.21.15.7:0/3424913837 learned_addr learned my addr 172.21.15.7:0/3424913837 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-09T08:01:57.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.529+0000 7fa6b135c640 1 -- 172.21.15.7:0/3424913837 >> v1:172.21.15.7:6789/0 conn(0x7fa6ac0719e0 legacy=0x7fa6ac158380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:01:57.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.529+0000 7fa6b135c640 1 -- 172.21.15.7:0/3424913837 >> v1:172.21.15.1:6789/0 conn(0x7fa6ac15c5d0 legacy=0x7fa6ac152170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:01:57.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.529+0000 7fa6b0b5b640 1 -- 172.21.15.7:0/3424913837 <== mon.1 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 719430555 0 0) 0x7fa6ac1590a0 con 0x7fa6ac0719e0 2024-05-09T08:01:57.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.529+0000 7fa6b135c640 1 -- 172.21.15.7:0/3424913837 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6ac15baf0 con 0x7fa6ac071dc0 2024-05-09T08:01:57.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.529+0000 7fa6b135c640 1 --2- 172.21.15.7:0/3424913837 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fa6ac071dc0 0x7fa6ac0c2d60 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fa6a80097d0 tx=0x7fa6a802a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d1b2397150ed16e5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:01:57.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.529+0000 7fa6b0b5b640 1 -- 172.21.15.7:0/3424913837 <== mon.2 v2:172.21.15.7:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6a8037070 con 0x7fa6ac071dc0 2024-05-09T08:01:57.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.530+0000 7fa6b0b5b640 1 -- 172.21.15.7:0/3424913837 <== mon.2 v2:172.21.15.7:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6a8032440 con 0x7fa6ac071dc0 2024-05-09T08:01:57.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.530+0000 7fa6b0b5b640 1 -- 172.21.15.7:0/3424913837 <== mon.2 v2:172.21.15.7:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6a8032810 con 0x7fa6ac071dc0 2024-05-09T08:01:57.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.530+0000 7fa6b3dc1640 1 -- 172.21.15.7:0/3424913837 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fa6ac071dc0 msgr2=0x7fa6ac0c2d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:01:57.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.530+0000 7fa6b3dc1640 1 --2- 172.21.15.7:0/3424913837 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fa6ac071dc0 0x7fa6ac0c2d60 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fa6a80097d0 tx=0x7fa6a802a1c0 comp rx=0 tx=0).stop 2024-05-09T08:01:57.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.530+0000 7fa6b3dc1640 1 -- 172.21.15.7:0/3424913837 shutdown_connections 2024-05-09T08:01:57.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.530+0000 7fa6b3dc1640 1 --2- 172.21.15.7:0/3424913837 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fa6ac071dc0 0x7fa6ac0c2d60 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:01:57.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.530+0000 7fa6b3dc1640 1 -- 172.21.15.7:0/3424913837 >> 172.21.15.7:0/3424913837 conn(0x7fa6ac06b880 msgr2=0x7fa6ac06fb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:01:57.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.530+0000 7fa6b3dc1640 1 -- 172.21.15.7:0/3424913837 shutdown_connections 2024-05-09T08:01:57.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.530+0000 7fa6b3dc1640 1 -- 172.21.15.7:0/3424913837 wait complete. 2024-05-09T08:01:57.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.531+0000 7fa6b3dc1640 1 Processor -- start 2024-05-09T08:01:57.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.531+0000 7fa6b3dc1640 1 -- start start 2024-05-09T08:01:57.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.532+0000 7fa6b3dc1640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6ac0719e0 0x7fa6ac0cdf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:01:57.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.532+0000 7fa6b3dc1640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa6ac071dc0 0x7fa6ac1f15d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:01:57.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.532+0000 7fa6b3dc1640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fa6ac15c5d0 0x7fa6ac1f1b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:01:57.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.532+0000 7fa6b3dc1640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fa6ac15c0b0 con 0x7fa6ac071dc0 2024-05-09T08:01:57.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.532+0000 7fa6b3dc1640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa6ac158a90 con 0x7fa6ac0719e0 2024-05-09T08:01:57.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.532+0000 7fa6b3dc1640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7fa6ac152880 con 0x7fa6ac15c5d0 2024-05-09T08:01:57.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.532+0000 7fa6b1b5d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6ac0719e0 0x7fa6ac0cdf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:01:57.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.532+0000 7fa6b235e640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fa6ac15c5d0 0x7fa6ac1f1b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:01:57.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.532+0000 7fa6b1b5d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6ac0719e0 0x7fa6ac0cdf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60010/0 (socket says 172.21.15.7:60010) 2024-05-09T08:01:57.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.532+0000 7fa6b1b5d640 1 -- 172.21.15.7:0/492522473 learned_addr learned my addr 172.21.15.7:0/492522473 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-05-09T08:01:57.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.533+0000 7fa6b235e640 1 -- 172.21.15.7:0/492522473 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6ac0719e0 msgr2=0x7fa6ac0cdf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:01:57.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.533+0000 7fa6b235e640 1 --2- 172.21.15.7:0/492522473 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6ac0719e0 0x7fa6ac0cdf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:01:57.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.533+0000 7fa6b235e640 1 -- 172.21.15.7:0/492522473 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa6ac071dc0 msgr2=0x7fa6ac1f15d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:01:57.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.533+0000 7fa6b235e640 1 --2- 172.21.15.7:0/492522473 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa6ac071dc0 0x7fa6ac1f15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:01:57.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.533+0000 7fa6b235e640 1 -- 172.21.15.7:0/492522473 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6ac156880 con 0x7fa6ac15c5d0 2024-05-09T08:01:57.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.533+0000 7fa6b235e640 1 --2- 172.21.15.7:0/492522473 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fa6ac15c5d0 0x7fa6ac1f1b10 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa6a400be20 tx=0x7fa6a400bf10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:01:57.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.533+0000 7fa6a27fc640 1 -- 172.21.15.7:0/492522473 <== mon.2 v2:172.21.15.7:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6a4019070 con 0x7fa6ac15c5d0 2024-05-09T08:01:57.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.533+0000 7fa6a27fc640 1 -- 172.21.15.7:0/492522473 <== mon.2 v2:172.21.15.7:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6a4014440 con 0x7fa6ac15c5d0 2024-05-09T08:01:57.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.533+0000 7fa6a27fc640 1 -- 172.21.15.7:0/492522473 <== mon.2 v2:172.21.15.7:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6a4014830 con 0x7fa6ac15c5d0 2024-05-09T08:01:57.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.533+0000 7fa6b3dc1640 1 -- 172.21.15.7:0/492522473 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6ac156b20 con 0x7fa6ac15c5d0 2024-05-09T08:01:57.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.533+0000 7fa6b3dc1640 1 -- 172.21.15.7:0/492522473 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6ac157c90 con 0x7fa6ac15c5d0 2024-05-09T08:01:57.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.533+0000 7fa6a27fc640 1 -- 172.21.15.7:0/492522473 <== mon.2 v2:172.21.15.7:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fa6a401c070 con 0x7fa6ac15c5d0 2024-05-09T08:01:57.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.533+0000 7fa6a27fc640 1 -- 172.21.15.7:0/492522473 <== mon.2 v2:172.21.15.7:3301/0 5 ==== osd_map(2..2 src has 1..2) ==== 725+0+0 (secure 0 0 0) 0x7fa6a401eba0 con 0x7fa6ac15c5d0 2024-05-09T08:01:57.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.534+0000 7fa6b3dc1640 1 -- 172.21.15.7:0/492522473 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6ac157a90 con 0x7fa6ac15c5d0 2024-05-09T08:01:57.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.536+0000 7fa6a27fc640 1 -- 172.21.15.7:0/492522473 <== mon.2 v2:172.21.15.7:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fa6a4023020 con 0x7fa6ac15c5d0 2024-05-09T08:01:57.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.609+0000 7fa6b3dc1640 1 -- 172.21.15.7:0/492522473 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_command({"prefix": "osd new", "uuid": "c064454e-a9b5-4a24-9a77-17f0a4085952", "id": 0} v 0) -- 0x7fa6ac074da0 con 0x7fa6ac15c5d0 2024-05-09T08:01:57.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.614+0000 7fa6a27fc640 1 -- 172.21.15.7:0/492522473 <== mon.2 v2:172.21.15.7:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "c064454e-a9b5-4a24-9a77-17f0a4085952", "id": 0}]=0 v3) ==== 112+0+2 (secure 0 0 0) 0x7fa6ac074da0 con 0x7fa6ac15c5d0 2024-05-09T08:01:57.630 INFO:teuthology.orchestra.run.smithi007.stdout:0 2024-05-09T08:01:57.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.615+0000 7fa6b3dc1640 1 -- 172.21.15.7:0/492522473 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fa6ac15c5d0 msgr2=0x7fa6ac1f1b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:01:57.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.615+0000 7fa6b3dc1640 1 --2- 172.21.15.7:0/492522473 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fa6ac15c5d0 0x7fa6ac1f1b10 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa6a400be20 tx=0x7fa6a400bf10 comp rx=0 tx=0).stop 2024-05-09T08:01:57.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.615+0000 7fa6b3dc1640 1 -- 172.21.15.7:0/492522473 shutdown_connections 2024-05-09T08:01:57.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.615+0000 7fa6b3dc1640 1 --2- 172.21.15.7:0/492522473 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fa6ac15c5d0 0x7fa6ac1f1b10 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:01:57.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.615+0000 7fa6b3dc1640 1 --2- 172.21.15.7:0/492522473 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa6ac071dc0 0x7fa6ac1f15d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:01:57.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.615+0000 7fa6b3dc1640 1 --2- 172.21.15.7:0/492522473 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6ac0719e0 0x7fa6ac0cdf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:01:57.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.615+0000 7fa6b3dc1640 1 -- 172.21.15.7:0/492522473 >> 172.21.15.7:0/492522473 conn(0x7fa6ac06b880 msgr2=0x7fa6ac0c3430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:01:57.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.615+0000 7fa6b3dc1640 1 -- 172.21.15.7:0/492522473 shutdown_connections 2024-05-09T08:01:57.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.615+0000 7fa6b3dc1640 1 -- 172.21.15.7:0/492522473 wait complete. 2024-05-09T08:01:57.641 DEBUG:teuthology.orchestra.run.smithi007:> sudo ceph --cluster ceph osd new 1a9ea687-2ff6-407a-949b-0e7e1083af21 1 2024-05-09T08:01:57.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.725+0000 7f117e2fc640 1 Processor -- start 2024-05-09T08:01:57.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.725+0000 7f117e2fc640 1 -- start start 2024-05-09T08:01:57.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.726+0000 7f117e2fc640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f1178154a60 0x7f11780c4a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:01:57.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.726+0000 7f117e2fc640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f11780c8ef0 0x7f11780cdec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:01:57.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.726+0000 7f117e2fc640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f117815e0c0 con 0x7f11780ce400 2024-05-09T08:01:57.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.726+0000 7f117e2fc640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f117815ec40 con 0x7f11780c8ef0 2024-05-09T08:01:57.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.726+0000 7f117e2fc640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f11780ca340 con 0x7f1178154a60 2024-05-09T08:01:57.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.726+0000 7f1177fff640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f1178154a60 0x7f11780c4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:01:57.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.726+0000 7f117c899640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f11780ce400 0x7f11780ca7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:42908/0 (socket says 172.21.15.7:42908) 2024-05-09T08:01:57.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.726+0000 7f11777fe640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f11780c8ef0 0x7f11780cdec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:01:57.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.726+0000 7f117c899640 1 -- 172.21.15.7:0/1297698172 learned_addr learned my addr 172.21.15.7:0/1297698172 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-09T08:01:57.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.727+0000 7f1177fff640 1 -- 172.21.15.7:0/1297698172 >> v2:172.21.15.7:3300/0 conn(0x7f11780c8ef0 msgr2=0x7f11780cdec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:01:57.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.727+0000 7f1177fff640 1 --2- 172.21.15.7:0/1297698172 >> v2:172.21.15.7:3300/0 conn(0x7f11780c8ef0 0x7f11780cdec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:01:57.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.727+0000 7f1177fff640 1 -- 172.21.15.7:0/1297698172 >> v1:172.21.15.7:6789/0 conn(0x7f11780ce400 legacy=0x7f11780ca7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:01:57.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.727+0000 7f1177fff640 1 -- 172.21.15.7:0/1297698172 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f117815e3a0 con 0x7f1178154a60 2024-05-09T08:01:57.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.727+0000 7f11777fe640 1 --2- 172.21.15.7:0/1297698172 >> v2:172.21.15.7:3300/0 conn(0x7f11780c8ef0 0x7f11780cdec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:01:57.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.727+0000 7f1177fff640 1 --2- 172.21.15.7:0/1297698172 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f1178154a60 0x7f11780c4a20 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f11600097a0 tx=0x7f116002a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2dd9656cffbecad6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:01:57.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.728+0000 7f1176ffd640 1 -- 172.21.15.7:0/1297698172 <== mon.2 v2:172.21.15.7:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1160037070 con 0x7f1178154a60 2024-05-09T08:01:57.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.728+0000 7f1176ffd640 1 -- 172.21.15.7:0/1297698172 <== mon.2 v2:172.21.15.7:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1160032440 con 0x7f1178154a60 2024-05-09T08:01:57.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.728+0000 7f1176ffd640 1 -- 172.21.15.7:0/1297698172 <== mon.2 v2:172.21.15.7:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1160032810 con 0x7f1178154a60 2024-05-09T08:01:57.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.728+0000 7f117e2fc640 1 -- 172.21.15.7:0/1297698172 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f1178154a60 msgr2=0x7f11780c4a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:01:57.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.728+0000 7f117e2fc640 1 --2- 172.21.15.7:0/1297698172 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f1178154a60 0x7f11780c4a20 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f11600097a0 tx=0x7f116002a180 comp rx=0 tx=0).stop 2024-05-09T08:01:57.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.728+0000 7f117e2fc640 1 -- 172.21.15.7:0/1297698172 shutdown_connections 2024-05-09T08:01:57.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.728+0000 7f117e2fc640 1 --2- 172.21.15.7:0/1297698172 >> v2:172.21.15.7:3300/0 conn(0x7f11780c8ef0 0x7f11780cdec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:01:57.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.728+0000 7f117e2fc640 1 --2- 172.21.15.7:0/1297698172 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f1178154a60 0x7f11780c4a20 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:01:57.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.728+0000 7f117e2fc640 1 -- 172.21.15.7:0/1297698172 >> 172.21.15.7:0/1297698172 conn(0x7f1178066ed0 msgr2=0x7f11780672e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:01:57.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.728+0000 7f117e2fc640 1 -- 172.21.15.7:0/1297698172 shutdown_connections 2024-05-09T08:01:57.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.728+0000 7f117e2fc640 1 -- 172.21.15.7:0/1297698172 wait complete. 2024-05-09T08:01:57.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.730+0000 7f117e2fc640 1 Processor -- start 2024-05-09T08:01:57.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.730+0000 7f117e2fc640 1 -- start start 2024-05-09T08:01:57.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.730+0000 7f117e2fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f11780c8ef0 0x7f11780c5e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:01:57.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.730+0000 7f117e2fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f11780ce400 0x7f11781f36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:01:57.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.730+0000 7f117e2fc640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f1178154a60 0x7f11780c7900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:01:57.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.730+0000 7f117e2fc640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f11780c9580 con 0x7f11780ce400 2024-05-09T08:01:57.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.730+0000 7f117e2fc640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f117815e960 con 0x7f11780c8ef0 2024-05-09T08:01:57.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.730+0000 7f117e2fc640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f1178154e40 con 0x7f1178154a60 2024-05-09T08:01:57.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.731+0000 7f1177fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f11780c8ef0 0x7f11780c5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:01:57.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.731+0000 7f117c899640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f1178154a60 0x7f11780c7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:01:57.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.731+0000 7f1177fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f11780c8ef0 0x7f11780c5e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60034/0 (socket says 172.21.15.7:60034) 2024-05-09T08:01:57.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.731+0000 7f1177fff640 1 -- 172.21.15.7:0/3983056043 learned_addr learned my addr 172.21.15.7:0/3983056043 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-05-09T08:01:57.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.731+0000 7f1177fff640 1 -- 172.21.15.7:0/3983056043 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f1178154a60 msgr2=0x7f11780c7900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:01:57.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.731+0000 7f1177fff640 1 --2- 172.21.15.7:0/3983056043 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f1178154a60 0x7f11780c7900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:01:57.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.731+0000 7f1177fff640 1 -- 172.21.15.7:0/3983056043 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f11780ce400 msgr2=0x7f11781f36d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:01:57.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.731+0000 7f1177fff640 1 --2- 172.21.15.7:0/3983056043 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f11780ce400 0x7f11781f36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:01:57.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.731+0000 7f1177fff640 1 -- 172.21.15.7:0/3983056043 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11781f4380 con 0x7f11780c8ef0 2024-05-09T08:01:57.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.731+0000 7f117c899640 1 --2- 172.21.15.7:0/3983056043 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f1178154a60 0x7f11780c7900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:01:57.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.731+0000 7f1177fff640 1 --2- 172.21.15.7:0/3983056043 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f11780c8ef0 0x7f11780c5e10 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f1160000c00 tx=0x7f11600046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:01:57.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.731+0000 7f1174ff9640 1 -- 172.21.15.7:0/3983056043 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1160037040 con 0x7f11780c8ef0 2024-05-09T08:01:57.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.731+0000 7f1174ff9640 1 -- 172.21.15.7:0/3983056043 <== mon.1 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1160042070 con 0x7f11780c8ef0 2024-05-09T08:01:57.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.731+0000 7f1174ff9640 1 -- 172.21.15.7:0/3983056043 <== mon.1 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1160030920 con 0x7f11780c8ef0 2024-05-09T08:01:57.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.731+0000 7f117e2fc640 1 -- 172.21.15.7:0/3983056043 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11781f40a0 con 0x7f11780c8ef0 2024-05-09T08:01:57.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.731+0000 7f117e2fc640 1 -- 172.21.15.7:0/3983056043 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11780c6e20 con 0x7f11780c8ef0 2024-05-09T08:01:57.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.731+0000 7f1174ff9640 1 -- 172.21.15.7:0/3983056043 <== mon.1 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f1160030ac0 con 0x7f11780c8ef0 2024-05-09T08:01:57.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.732+0000 7f1174ff9640 1 -- 172.21.15.7:0/3983056043 <== mon.1 v2:172.21.15.7:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 840+0+0 (secure 0 0 0) 0x7f1160047bd0 con 0x7f11780c8ef0 2024-05-09T08:01:57.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.732+0000 7f117e2fc640 1 -- 172.21.15.7:0/3983056043 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11780c7540 con 0x7f11780c8ef0 2024-05-09T08:01:57.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.734+0000 7f1174ff9640 1 -- 172.21.15.7:0/3983056043 <== mon.1 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f1160047e20 con 0x7f11780c8ef0 2024-05-09T08:01:57.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.808+0000 7f117e2fc640 1 -- 172.21.15.7:0/3983056043 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd new", "uuid": "1a9ea687-2ff6-407a-949b-0e7e1083af21", "id": 1} v 0) -- 0x7f1178074d70 con 0x7f11780c8ef0 2024-05-09T08:01:57.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.813+0000 7f1174ff9640 1 -- 172.21.15.7:0/3983056043 <== mon.1 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "1a9ea687-2ff6-407a-949b-0e7e1083af21", "id": 1}]=0 v4) ==== 112+0+2 (secure 0 0 0) 0x7f1160047460 con 0x7f11780c8ef0 2024-05-09T08:01:57.830 INFO:teuthology.orchestra.run.smithi007.stdout:1 2024-05-09T08:01:57.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.814+0000 7f117e2fc640 1 -- 172.21.15.7:0/3983056043 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f11780c8ef0 msgr2=0x7f11780c5e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:01:57.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.814+0000 7f117e2fc640 1 --2- 172.21.15.7:0/3983056043 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f11780c8ef0 0x7f11780c5e10 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f1160000c00 tx=0x7f11600046c0 comp rx=0 tx=0).stop 2024-05-09T08:01:57.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.814+0000 7f117e2fc640 1 -- 172.21.15.7:0/3983056043 shutdown_connections 2024-05-09T08:01:57.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.814+0000 7f117e2fc640 1 --2- 172.21.15.7:0/3983056043 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f1178154a60 0x7f11780c7900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:01:57.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.814+0000 7f117e2fc640 1 --2- 172.21.15.7:0/3983056043 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f11780ce400 0x7f11781f36d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:01:57.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.814+0000 7f117e2fc640 1 --2- 172.21.15.7:0/3983056043 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f11780c8ef0 0x7f11780c5e10 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:01:57.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.814+0000 7f117e2fc640 1 -- 172.21.15.7:0/3983056043 >> 172.21.15.7:0/3983056043 conn(0x7f1178066ed0 msgr2=0x7f1178152720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:01:57.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.815+0000 7f117e2fc640 1 -- 172.21.15.7:0/3983056043 shutdown_connections 2024-05-09T08:01:57.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.815+0000 7f117e2fc640 1 -- 172.21.15.7:0/3983056043 wait complete. 2024-05-09T08:01:57.841 DEBUG:teuthology.orchestra.run.smithi007:> sudo ceph --cluster ceph osd new c88af08b-7c9a-4dd8-9640-44138239addb 2 2024-05-09T08:01:57.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.926+0000 7ff973abd640 1 Processor -- start 2024-05-09T08:01:57.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.926+0000 7ff973abd640 1 -- start start 2024-05-09T08:01:57.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.927+0000 7ff973abd640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ff96c06c9e0 0x7ff96c06ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:01:57.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.927+0000 7ff973abd640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff96c15e630 con 0x7ff96c06cdc0 2024-05-09T08:01:57.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.927+0000 7ff973abd640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff96c15e900 con 0x7ff96c0cc260 2024-05-09T08:01:57.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.927+0000 7ff973abd640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7ff96c0ca920 con 0x7ff96c06c9e0 2024-05-09T08:01:57.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.927+0000 7ff971859640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ff96c06c9e0 0x7ff96c06ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:01:57.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.927+0000 7ff971058640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7ff96c0cc260 0x7ff96c1545b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:42918/0 (socket says 172.21.15.7:42918) 2024-05-09T08:01:57.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.927+0000 7ff971058640 1 -- 172.21.15.7:0/2743217755 learned_addr learned my addr 172.21.15.7:0/2743217755 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-09T08:01:57.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.927+0000 7ff970857640 1 -- 172.21.15.7:0/2743217755 <== mon.1 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2167765975 0 0) 0x7ff96c15e900 con 0x7ff96c0cc260 2024-05-09T08:01:57.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.927+0000 7ff971859640 1 -- 172.21.15.7:0/2743217755 >> v1:172.21.15.7:6789/0 conn(0x7ff96c0cc260 legacy=0x7ff96c1545b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:01:57.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.927+0000 7ff971859640 1 -- 172.21.15.7:0/2743217755 >> v1:172.21.15.1:6789/0 conn(0x7ff96c06cdc0 legacy=0x7ff96c0cb090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:01:57.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.927+0000 7ff971859640 1 -- 172.21.15.7:0/2743217755 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff96c06d6a0 con 0x7ff96c06c9e0 2024-05-09T08:01:57.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.928+0000 7ff971859640 1 --2- 172.21.15.7:0/2743217755 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ff96c06c9e0 0x7ff96c06ddc0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7ff968009dc0 tx=0x7ff96802a5f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7976ea52f649f9d4 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:01:57.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.928+0000 7ff970857640 1 -- 172.21.15.7:0/2743217755 <== mon.2 v2:172.21.15.7:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff968007740 con 0x7ff96c06c9e0 2024-05-09T08:01:57.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.928+0000 7ff970857640 1 -- 172.21.15.7:0/2743217755 <== mon.2 v2:172.21.15.7:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9680078e0 con 0x7ff96c06c9e0 2024-05-09T08:01:57.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.928+0000 7ff970857640 1 -- 172.21.15.7:0/2743217755 <== mon.2 v2:172.21.15.7:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff968007cb0 con 0x7ff96c06c9e0 2024-05-09T08:01:57.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.928+0000 7ff973abd640 1 -- 172.21.15.7:0/2743217755 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ff96c06c9e0 msgr2=0x7ff96c06ddc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:01:57.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.928+0000 7ff973abd640 1 --2- 172.21.15.7:0/2743217755 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ff96c06c9e0 0x7ff96c06ddc0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7ff968009dc0 tx=0x7ff96802a5f0 comp rx=0 tx=0).stop 2024-05-09T08:01:57.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.928+0000 7ff973abd640 1 -- 172.21.15.7:0/2743217755 shutdown_connections 2024-05-09T08:01:57.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.928+0000 7ff973abd640 1 --2- 172.21.15.7:0/2743217755 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ff96c06c9e0 0x7ff96c06ddc0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:01:57.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.928+0000 7ff973abd640 1 -- 172.21.15.7:0/2743217755 >> 172.21.15.7:0/2743217755 conn(0x7ff96c0cffa0 msgr2=0x7ff96c0d03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:01:57.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.929+0000 7ff973abd640 1 -- 172.21.15.7:0/2743217755 shutdown_connections 2024-05-09T08:01:57.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.929+0000 7ff973abd640 1 -- 172.21.15.7:0/2743217755 wait complete. 2024-05-09T08:01:57.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.930+0000 7ff973abd640 1 Processor -- start 2024-05-09T08:01:57.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.930+0000 7ff973abd640 1 -- start start 2024-05-09T08:01:57.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.930+0000 7ff973abd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff96c06c9e0 0x7ff96c14acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:01:57.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.930+0000 7ff973abd640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ff96c06cdc0 0x7ff96c1574a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:01:57.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.930+0000 7ff973abd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff96c0cc260 0x7ff96c15a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:01:57.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.930+0000 7ff973abd640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ff96c0cab70 con 0x7ff96c06c9e0 2024-05-09T08:01:57.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.930+0000 7ff973abd640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff96c15ebe0 con 0x7ff96c0cc260 2024-05-09T08:01:57.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.930+0000 7ff973abd640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7ff96c154cc0 con 0x7ff96c06cdc0 2024-05-09T08:01:57.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.930+0000 7ff971058640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ff96c06cdc0 0x7ff96c1574a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:01:57.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.930+0000 7ff97205a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff96c0cc260 0x7ff96c15a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:01:57.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.930+0000 7ff97205a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff96c0cc260 0x7ff96c15a180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60040/0 (socket says 172.21.15.7:60040) 2024-05-09T08:01:57.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.930+0000 7ff971058640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ff96c06cdc0 0x7ff96c1574a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3301/0 says I am v2:172.21.15.7:39080/0 (socket says 172.21.15.7:39080) 2024-05-09T08:01:57.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.930+0000 7ff97205a640 1 -- 172.21.15.7:0/180987326 learned_addr learned my addr 172.21.15.7:0/180987326 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-05-09T08:01:57.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.931+0000 7ff97205a640 1 -- 172.21.15.7:0/180987326 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ff96c06cdc0 msgr2=0x7ff96c1574a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:01:57.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.931+0000 7ff97205a640 1 --2- 172.21.15.7:0/180987326 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ff96c06cdc0 0x7ff96c1574a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:01:57.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.931+0000 7ff97205a640 1 -- 172.21.15.7:0/180987326 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff96c06c9e0 msgr2=0x7ff96c14acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:01:57.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.931+0000 7ff97205a640 1 --2- 172.21.15.7:0/180987326 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff96c06c9e0 0x7ff96c14acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:01:57.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.931+0000 7ff97205a640 1 -- 172.21.15.7:0/180987326 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff96c14b230 con 0x7ff96c0cc260 2024-05-09T08:01:57.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.931+0000 7ff971058640 1 --2- 172.21.15.7:0/180987326 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ff96c06cdc0 0x7ff96c1574a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:01:57.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.931+0000 7ff97205a640 1 --2- 172.21.15.7:0/180987326 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff96c0cc260 0x7ff96c15a180 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7ff96400c410 tx=0x7ff96400c8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:01:57.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.931+0000 7ff9627fc640 1 -- 172.21.15.7:0/180987326 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff964017070 con 0x7ff96c0cc260 2024-05-09T08:01:57.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.931+0000 7ff9627fc640 1 -- 172.21.15.7:0/180987326 <== mon.1 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9640035f0 con 0x7ff96c0cc260 2024-05-09T08:01:57.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.931+0000 7ff973abd640 1 -- 172.21.15.7:0/180987326 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff96c156c00 con 0x7ff96c0cc260 2024-05-09T08:01:57.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.931+0000 7ff9627fc640 1 -- 172.21.15.7:0/180987326 <== mon.1 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff964003a20 con 0x7ff96c0cc260 2024-05-09T08:01:57.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.931+0000 7ff973abd640 1 -- 172.21.15.7:0/180987326 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff96c1571c0 con 0x7ff96c0cc260 2024-05-09T08:01:57.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.931+0000 7ff9627fc640 1 -- 172.21.15.7:0/180987326 <== mon.1 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7ff96401a070 con 0x7ff96c0cc260 2024-05-09T08:01:57.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.931+0000 7ff9627fc640 1 -- 172.21.15.7:0/180987326 <== mon.1 v2:172.21.15.7:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 955+0+0 (secure 0 0 0) 0x7ff964012820 con 0x7ff96c0cc260 2024-05-09T08:01:57.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.932+0000 7ff973abd640 1 -- 172.21.15.7:0/180987326 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff96c0ca5f0 con 0x7ff96c0cc260 2024-05-09T08:01:57.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:57.934+0000 7ff9627fc640 1 -- 172.21.15.7:0/180987326 <== mon.1 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7ff96c0ca5f0 con 0x7ff96c0cc260 2024-05-09T08:01:58.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.008+0000 7ff973abd640 1 -- 172.21.15.7:0/180987326 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd new", "uuid": "c88af08b-7c9a-4dd8-9640-44138239addb", "id": 2} v 0) -- 0x7ff96c074d70 con 0x7ff96c0cc260 2024-05-09T08:01:58.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.012+0000 7ff9627fc640 1 -- 172.21.15.7:0/180987326 <== mon.1 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "c88af08b-7c9a-4dd8-9640-44138239addb", "id": 2}]=0 v5) ==== 112+0+2 (secure 0 0 0) 0x7ff964021070 con 0x7ff96c0cc260 2024-05-09T08:01:58.029 INFO:teuthology.orchestra.run.smithi007.stdout:2 2024-05-09T08:01:58.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.013+0000 7ff973abd640 1 -- 172.21.15.7:0/180987326 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff96c0cc260 msgr2=0x7ff96c15a180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:01:58.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.013+0000 7ff973abd640 1 --2- 172.21.15.7:0/180987326 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff96c0cc260 0x7ff96c15a180 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7ff96400c410 tx=0x7ff96400c8d0 comp rx=0 tx=0).stop 2024-05-09T08:01:58.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.013+0000 7ff973abd640 1 -- 172.21.15.7:0/180987326 shutdown_connections 2024-05-09T08:01:58.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.013+0000 7ff973abd640 1 --2- 172.21.15.7:0/180987326 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff96c0cc260 0x7ff96c15a180 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:01:58.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.013+0000 7ff973abd640 1 --2- 172.21.15.7:0/180987326 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ff96c06cdc0 0x7ff96c1574a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:01:58.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.013+0000 7ff973abd640 1 --2- 172.21.15.7:0/180987326 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff96c06c9e0 0x7ff96c14acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:01:58.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.013+0000 7ff973abd640 1 -- 172.21.15.7:0/180987326 >> 172.21.15.7:0/180987326 conn(0x7ff96c0cffa0 msgr2=0x7ff96c0cc690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:01:58.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.013+0000 7ff973abd640 1 -- 172.21.15.7:0/180987326 shutdown_connections 2024-05-09T08:01:58.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.013+0000 7ff973abd640 1 -- 172.21.15.7:0/180987326 wait complete. 2024-05-09T08:01:58.040 DEBUG:teuthology.orchestra.run.smithi007:> sudo ceph --cluster ceph osd new c087b7e3-a41d-4333-9365-32e3bd8c8e6f 3 2024-05-09T08:01:58.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.124+0000 7f0ef701c640 1 Processor -- start 2024-05-09T08:01:58.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.124+0000 7f0ef701c640 1 -- start start 2024-05-09T08:01:58.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.124+0000 7f0ef701c640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f0ef00c3aa0 0x7f0ef00c91e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:01:58.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.124+0000 7f0ef701c640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f0ef015a5f0 0x7f0ef015e630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:01:58.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.124+0000 7f0ef701c640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0ef00c7e80 con 0x7f0ef01547e0 2024-05-09T08:01:58.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.124+0000 7f0ef701c640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f0ef015dbe0 con 0x7f0ef00c3aa0 2024-05-09T08:01:58.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.124+0000 7f0ef701c640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f0ef00c4000 con 0x7f0ef015a5f0 2024-05-09T08:01:58.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.125+0000 7f0ef4db8640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f0ef01547e0 0x7f0ef0070750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:42920/0 (socket says 172.21.15.7:42920) 2024-05-09T08:01:58.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.125+0000 7f0ef4db8640 1 -- 172.21.15.7:0/568629244 learned_addr learned my addr 172.21.15.7:0/568629244 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-09T08:01:58.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.125+0000 7f0ef55b9640 1 --2- 172.21.15.7:0/568629244 >> v2:172.21.15.7:3300/0 conn(0x7f0ef015a5f0 0x7f0ef015e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:01:58.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.125+0000 7f0eef7fe640 1 -- 172.21.15.7:0/568629244 <== mon.1 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2742152421 0 0) 0x7f0ef00c7e80 con 0x7f0ef01547e0 2024-05-09T08:01:58.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.125+0000 7f0eef7fe640 1 -- 172.21.15.7:0/568629244 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0ee0003520 con 0x7f0ef01547e0 2024-05-09T08:01:58.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.125+0000 7f0eeffff640 1 --2- 172.21.15.7:0/568629244 >> v2:172.21.15.1:3300/0 conn(0x7f0ef00c3aa0 0x7f0ef00c91e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:01:58.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.125+0000 7f0ef55b9640 1 -- 172.21.15.7:0/568629244 >> v2:172.21.15.1:3300/0 conn(0x7f0ef00c3aa0 msgr2=0x7f0ef00c91e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:01:58.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.125+0000 7f0ef55b9640 1 --2- 172.21.15.7:0/568629244 >> v2:172.21.15.1:3300/0 conn(0x7f0ef00c3aa0 0x7f0ef00c91e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:01:58.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.125+0000 7f0ef55b9640 1 -- 172.21.15.7:0/568629244 >> v1:172.21.15.7:6789/0 conn(0x7f0ef01547e0 legacy=0x7f0ef0070750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:01:58.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.125+0000 7f0ef55b9640 1 -- 172.21.15.7:0/568629244 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ef00c8160 con 0x7f0ef015a5f0 2024-05-09T08:01:58.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.125+0000 7f0eeffff640 1 --2- 172.21.15.7:0/568629244 >> v2:172.21.15.1:3300/0 conn(0x7f0ef00c3aa0 0x7f0ef00c91e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T08:01:58.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.126+0000 7f0ef55b9640 1 --2- 172.21.15.7:0/568629244 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ef015a5f0 0x7f0ef015e630 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f0ef00ccd60 tx=0x7f0ee802a5f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3c791ebc75be1951 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:01:58.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.126+0000 7f0eef7fe640 1 -- 172.21.15.7:0/568629244 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ee8007740 con 0x7f0ef015a5f0 2024-05-09T08:01:58.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.126+0000 7f0eef7fe640 1 -- 172.21.15.7:0/568629244 <== mon.1 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ee80078e0 con 0x7f0ef015a5f0 2024-05-09T08:01:58.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.126+0000 7f0eef7fe640 1 -- 172.21.15.7:0/568629244 <== mon.1 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ee8007cb0 con 0x7f0ef015a5f0 2024-05-09T08:01:58.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.126+0000 7f0ef701c640 1 -- 172.21.15.7:0/568629244 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ef015a5f0 msgr2=0x7f0ef015e630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:01:58.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.126+0000 7f0ef701c640 1 --2- 172.21.15.7:0/568629244 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ef015a5f0 0x7f0ef015e630 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f0ef00ccd60 tx=0x7f0ee802a5f0 comp rx=0 tx=0).stop 2024-05-09T08:01:58.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.126+0000 7f0ef701c640 1 -- 172.21.15.7:0/568629244 shutdown_connections 2024-05-09T08:01:58.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.126+0000 7f0ef701c640 1 --2- 172.21.15.7:0/568629244 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ef015a5f0 0x7f0ef015e630 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:01:58.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.126+0000 7f0ef701c640 1 --2- 172.21.15.7:0/568629244 >> v2:172.21.15.1:3300/0 conn(0x7f0ef00c3aa0 0x7f0ef00c91e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:01:58.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.126+0000 7f0ef701c640 1 -- 172.21.15.7:0/568629244 >> 172.21.15.7:0/568629244 conn(0x7f0ef0072b60 msgr2=0x7f0ef0072f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:01:58.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.127+0000 7f0ef701c640 1 -- 172.21.15.7:0/568629244 shutdown_connections 2024-05-09T08:01:58.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.127+0000 7f0ef701c640 1 -- 172.21.15.7:0/568629244 wait complete. 2024-05-09T08:01:58.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.128+0000 7f0ef701c640 1 Processor -- start 2024-05-09T08:01:58.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.128+0000 7f0ef701c640 1 -- start start 2024-05-09T08:01:58.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.129+0000 7f0ef701c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ef00c3aa0 0x7f0ef0155a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:01:58.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.129+0000 7f0ef701c640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f0ef01547e0 0x7f0ef014eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:01:58.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.129+0000 7f0ef701c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0ef015a5f0 0x7f0ef01f0570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:01:58.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.129+0000 7f0ef701c640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f0ef015e110 con 0x7f0ef015a5f0 2024-05-09T08:01:58.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.129+0000 7f0ef701c640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0ef015eb70 con 0x7f0ef00c3aa0 2024-05-09T08:01:58.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.129+0000 7f0ef701c640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f0ef0154bc0 con 0x7f0ef01547e0 2024-05-09T08:01:58.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.129+0000 7f0ef4db8640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ef00c3aa0 0x7f0ef0155a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:01:58.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.129+0000 7f0eeffff640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f0ef01547e0 0x7f0ef014eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:01:58.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.129+0000 7f0ef4db8640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ef00c3aa0 0x7f0ef0155a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60060/0 (socket says 172.21.15.7:60060) 2024-05-09T08:01:58.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.129+0000 7f0eeffff640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f0ef01547e0 0x7f0ef014eea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3301/0 says I am v2:172.21.15.7:39082/0 (socket says 172.21.15.7:39082) 2024-05-09T08:01:58.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.129+0000 7f0ef4db8640 1 -- 172.21.15.7:0/1167508820 learned_addr learned my addr 172.21.15.7:0/1167508820 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-05-09T08:01:58.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.129+0000 7f0ef55b9640 1 --2- 172.21.15.7:0/1167508820 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0ef015a5f0 0x7f0ef01f0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:01:58.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.129+0000 7f0eeffff640 1 -- 172.21.15.7:0/1167508820 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ef00c3aa0 msgr2=0x7f0ef0155a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:01:58.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.129+0000 7f0eeffff640 1 --2- 172.21.15.7:0/1167508820 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ef00c3aa0 0x7f0ef0155a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:01:58.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.129+0000 7f0eeffff640 1 -- 172.21.15.7:0/1167508820 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0ef015a5f0 msgr2=0x7f0ef01f0570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:01:58.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.129+0000 7f0eeffff640 1 --2- 172.21.15.7:0/1167508820 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0ef015a5f0 0x7f0ef01f0570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:01:58.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.129+0000 7f0eeffff640 1 -- 172.21.15.7:0/1167508820 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ef015b8c0 con 0x7f0ef01547e0 2024-05-09T08:01:58.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.129+0000 7f0ef4db8640 1 --2- 172.21.15.7:0/1167508820 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ef00c3aa0 0x7f0ef0155a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:01:58.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.129+0000 7f0eeffff640 1 --2- 172.21.15.7:0/1167508820 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f0ef01547e0 0x7f0ef014eea0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f0edc00c610 tx=0x7f0edc00cad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:01:58.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.129+0000 7f0eed7fa640 1 -- 172.21.15.7:0/1167508820 <== mon.2 v2:172.21.15.7:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0edc015070 con 0x7f0ef01547e0 2024-05-09T08:01:58.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.129+0000 7f0eed7fa640 1 -- 172.21.15.7:0/1167508820 <== mon.2 v2:172.21.15.7:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0edc011800 con 0x7f0ef01547e0 2024-05-09T08:01:58.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.129+0000 7f0eed7fa640 1 -- 172.21.15.7:0/1167508820 <== mon.2 v2:172.21.15.7:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0edc011bd0 con 0x7f0ef01547e0 2024-05-09T08:01:58.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.130+0000 7f0ef701c640 1 -- 172.21.15.7:0/1167508820 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ef015be90 con 0x7f0ef01547e0 2024-05-09T08:01:58.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.130+0000 7f0ef701c640 1 -- 172.21.15.7:0/1167508820 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ef015d910 con 0x7f0ef01547e0 2024-05-09T08:01:58.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.130+0000 7f0eed7fa640 1 -- 172.21.15.7:0/1167508820 <== mon.2 v2:172.21.15.7:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f0edc018070 con 0x7f0ef01547e0 2024-05-09T08:01:58.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.130+0000 7f0eed7fa640 1 -- 172.21.15.7:0/1167508820 <== mon.2 v2:172.21.15.7:3301/0 5 ==== osd_map(5..5 src has 1..5) ==== 1070+0+0 (secure 0 0 0) 0x7f0edc0187f0 con 0x7f0ef01547e0 2024-05-09T08:01:58.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.130+0000 7f0ef701c640 1 -- 172.21.15.7:0/1167508820 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ef015d6a0 con 0x7f0ef01547e0 2024-05-09T08:01:58.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.133+0000 7f0eed7fa640 1 -- 172.21.15.7:0/1167508820 <== mon.2 v2:172.21.15.7:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f0edc018360 con 0x7f0ef01547e0 2024-05-09T08:01:58.203 INFO:tasks.ceph.mgr.x.smithi007.stderr:2024-05-09T08:01:58.187+0000 7f934988f040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-09T08:01:58.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.206+0000 7f0ef701c640 1 -- 172.21.15.7:0/1167508820 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_command({"prefix": "osd new", "uuid": "c087b7e3-a41d-4333-9365-32e3bd8c8e6f", "id": 3} v 0) -- 0x7f0ef015c060 con 0x7f0ef01547e0 2024-05-09T08:01:58.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.211+0000 7f0eed7fa640 1 -- 172.21.15.7:0/1167508820 <== mon.2 v2:172.21.15.7:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "c087b7e3-a41d-4333-9365-32e3bd8c8e6f", "id": 3}]=0 v6) ==== 112+0+2 (secure 0 0 0) 0x7f0edc018b10 con 0x7f0ef01547e0 2024-05-09T08:01:58.227 INFO:teuthology.orchestra.run.smithi007.stdout:3 2024-05-09T08:01:58.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.212+0000 7f0ef701c640 1 -- 172.21.15.7:0/1167508820 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f0ef01547e0 msgr2=0x7f0ef014eea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:01:58.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.212+0000 7f0ef701c640 1 --2- 172.21.15.7:0/1167508820 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f0ef01547e0 0x7f0ef014eea0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f0edc00c610 tx=0x7f0edc00cad0 comp rx=0 tx=0).stop 2024-05-09T08:01:58.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.212+0000 7f0ef701c640 1 -- 172.21.15.7:0/1167508820 shutdown_connections 2024-05-09T08:01:58.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.212+0000 7f0ef701c640 1 --2- 172.21.15.7:0/1167508820 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0ef015a5f0 0x7f0ef01f0570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:01:58.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.212+0000 7f0ef701c640 1 --2- 172.21.15.7:0/1167508820 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f0ef01547e0 0x7f0ef014eea0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:01:58.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.212+0000 7f0ef701c640 1 --2- 172.21.15.7:0/1167508820 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ef00c3aa0 0x7f0ef0155a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:01:58.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.212+0000 7f0ef701c640 1 -- 172.21.15.7:0/1167508820 >> 172.21.15.7:0/1167508820 conn(0x7f0ef0072b60 msgr2=0x7f0ef006c8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:01:58.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.212+0000 7f0ef701c640 1 -- 172.21.15.7:0/1167508820 shutdown_connections 2024-05-09T08:01:58.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.212+0000 7f0ef701c640 1 -- 172.21.15.7:0/1167508820 wait complete. 2024-05-09T08:01:58.239 DEBUG:teuthology.orchestra.run.smithi007:> sudo ceph --cluster ceph osd new b4576b9c-871d-47a8-8d36-65af525d0200 4 2024-05-09T08:01:58.298 INFO:tasks.ceph.mgr.x.smithi007.stderr:2024-05-09T08:01:58.281+0000 7f934988f040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-09T08:01:58.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.322+0000 7fea3d977640 1 Processor -- start 2024-05-09T08:01:58.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.323+0000 7fea3d977640 1 -- start start 2024-05-09T08:01:58.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.323+0000 7fea3d977640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fea3806c900 0x7fea380c7280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:01:58.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.323+0000 7fea3d977640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fea3815b860 con 0x7fea3806cce0 2024-05-09T08:01:58.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.323+0000 7fea3d977640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fea380b5de0 con 0x7fea380c77c0 2024-05-09T08:01:58.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.323+0000 7fea3d977640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7fea380c7f30 con 0x7fea3806c900 2024-05-09T08:01:58.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.323+0000 7fea367fc640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fea380c77c0 0x7fea380c81f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:42936/0 (socket says 172.21.15.7:42936) 2024-05-09T08:01:58.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.323+0000 7fea367fc640 1 -- 172.21.15.7:0/3252097136 learned_addr learned my addr 172.21.15.7:0/3252097136 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-09T08:01:58.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.324+0000 7fea35ffb640 1 -- 172.21.15.7:0/3252097136 <== mon.1 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3535572781 0 0) 0x7fea380b5de0 con 0x7fea380c77c0 2024-05-09T08:01:58.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.324+0000 7fea35ffb640 1 -- 172.21.15.7:0/3252097136 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fea24003520 con 0x7fea380c77c0 2024-05-09T08:01:58.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.324+0000 7fea36ffd640 1 --2- 172.21.15.7:0/3252097136 >> v2:172.21.15.1:3300/0 conn(0x7fea3806c900 0x7fea380c7280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:01:58.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.324+0000 7fea35ffb640 1 -- 172.21.15.7:0/3252097136 <== mon.1 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1548590998 0 0) 0x7fea24003520 con 0x7fea380c77c0 2024-05-09T08:01:58.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.324+0000 7fea35ffb640 1 -- 172.21.15.7:0/3252097136 >> v2:172.21.15.1:3300/0 conn(0x7fea3806c900 msgr2=0x7fea380c7280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:01:58.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.324+0000 7fea35ffb640 1 --2- 172.21.15.7:0/3252097136 >> v2:172.21.15.1:3300/0 conn(0x7fea3806c900 0x7fea380c7280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:01:58.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.324+0000 7fea35ffb640 1 -- 172.21.15.7:0/3252097136 >> v1:172.21.15.1:6789/0 conn(0x7fea3806cce0 legacy=0x7fea381547c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:01:58.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.324+0000 7fea35ffb640 1 -- 172.21.15.7:0/3252097136 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea380c7c50 con 0x7fea380c77c0 2024-05-09T08:01:58.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.324+0000 7fea35ffb640 1 -- 172.21.15.7:0/3252097136 <== mon.1 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4179846320 0 0) 0x7fea28001670 con 0x7fea380c77c0 2024-05-09T08:01:58.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.324+0000 7fea35ffb640 1 -- 172.21.15.7:0/3252097136 >> v1:172.21.15.7:6789/0 conn(0x7fea380c77c0 legacy=0x7fea380c81f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:01:58.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.324+0000 7fea35ffb640 1 --2- 172.21.15.7:0/3252097136 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fea240039b0 0x7fea24003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:01:58.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.324+0000 7fea35ffb640 1 --2- 172.21.15.7:0/3252097136 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fea24004800 0x7fea24004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:01:58.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.324+0000 7fea35ffb640 1 --2- 172.21.15.7:0/3252097136 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea240052e0 0x7fea2400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:01:58.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.324+0000 7fea35ffb640 1 -- 172.21.15.7:0/3252097136 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fea380c7f30 con 0x7fea240039b0 2024-05-09T08:01:58.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.324+0000 7fea35ffb640 1 -- 172.21.15.7:0/3252097136 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fea2400c1e0 con 0x7fea240052e0 2024-05-09T08:01:58.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.324+0000 7fea35ffb640 1 -- 172.21.15.7:0/3252097136 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7fea2400c3e0 con 0x7fea24004800 2024-05-09T08:01:58.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.324+0000 7fea36ffd640 1 --2- 172.21.15.7:0/3252097136 >> v2:172.21.15.1:3300/0 conn(0x7fea3806c900 0x7fea380c7280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T08:01:58.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.324+0000 7fea367fc640 1 --2- 172.21.15.7:0/3252097136 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fea24004800 0x7fea24004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:01:58.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.324+0000 7fea377fe640 1 --2- 172.21.15.7:0/3252097136 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea240052e0 0x7fea2400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:01:58.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.325+0000 7fea367fc640 1 -- 172.21.15.7:0/3252097136 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea240052e0 msgr2=0x7fea2400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:01:58.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.325+0000 7fea367fc640 1 --2- 172.21.15.7:0/3252097136 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea240052e0 0x7fea2400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:01:58.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.325+0000 7fea367fc640 1 -- 172.21.15.7:0/3252097136 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fea240039b0 msgr2=0x7fea24003db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:01:58.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.325+0000 7fea367fc640 1 --2- 172.21.15.7:0/3252097136 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fea240039b0 0x7fea24003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:01:58.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.325+0000 7fea367fc640 1 -- 172.21.15.7:0/3252097136 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea24003520 con 0x7fea24004800 2024-05-09T08:01:58.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.325+0000 7fea377fe640 1 -- 172.21.15.7:0/3252097136 reap_dead start 2024-05-09T08:01:58.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.325+0000 7fea367fc640 1 --2- 172.21.15.7:0/3252097136 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fea24004800 0x7fea24004c80 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fea2800c210 tx=0x7fea24005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=928bd3e869dd9c92 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:01:58.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.325+0000 7fea35ffb640 1 -- 172.21.15.7:0/3252097136 <== mon.2 v2:172.21.15.7:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea28007d30 con 0x7fea24004800 2024-05-09T08:01:58.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.325+0000 7fea35ffb640 1 -- 172.21.15.7:0/3252097136 <== mon.2 v2:172.21.15.7:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea28005af0 con 0x7fea24004800 2024-05-09T08:01:58.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.325+0000 7fea35ffb640 1 -- 172.21.15.7:0/3252097136 <== mon.2 v2:172.21.15.7:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea28007070 con 0x7fea24004800 2024-05-09T08:01:58.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.326+0000 7fea3d977640 1 -- 172.21.15.7:0/3252097136 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fea24004800 msgr2=0x7fea24004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:01:58.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.326+0000 7fea3d977640 1 --2- 172.21.15.7:0/3252097136 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fea24004800 0x7fea24004c80 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fea2800c210 tx=0x7fea24005830 comp rx=0 tx=0).stop 2024-05-09T08:01:58.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.326+0000 7fea3d977640 1 -- 172.21.15.7:0/3252097136 shutdown_connections 2024-05-09T08:01:58.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.326+0000 7fea3d977640 1 --2- 172.21.15.7:0/3252097136 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fea24004800 0x7fea24004c80 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:01:58.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.326+0000 7fea3d977640 1 -- 172.21.15.7:0/3252097136 >> 172.21.15.7:0/3252097136 conn(0x7fea380cfe80 msgr2=0x7fea380d0290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:01:58.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.326+0000 7fea3d977640 1 -- 172.21.15.7:0/3252097136 shutdown_connections 2024-05-09T08:01:58.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.326+0000 7fea3d977640 1 -- 172.21.15.7:0/3252097136 wait complete. 2024-05-09T08:01:58.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.327+0000 7fea3d977640 1 Processor -- start 2024-05-09T08:01:58.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.327+0000 7fea3d977640 1 -- start start 2024-05-09T08:01:58.347 INFO:tasks.ceph.mgr.x.smithi007.stderr:2024-05-09T08:01:58.328+0000 7f934988f040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-09T08:01:58.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.328+0000 7fea3d977640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fea24004800 0x7fea3814eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:01:58.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.328+0000 7fea3d977640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fea381f3770 0x7fea38150250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:01:58.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.328+0000 7fea3d977640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea381f3bf0 0x7fea38150790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:01:58.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.328+0000 7fea3d977640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fea3815b060 con 0x7fea381f3770 2024-05-09T08:01:58.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.328+0000 7fea3d977640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fea3815ad80 con 0x7fea381f3bf0 2024-05-09T08:01:58.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.328+0000 7fea3d977640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7fea380cc000 con 0x7fea24004800 2024-05-09T08:01:58.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.328+0000 7fea36ffd640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fea24004800 0x7fea3814eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:01:58.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.328+0000 7fea36ffd640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fea24004800 0x7fea3814eab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3301/0 says I am v2:172.21.15.7:39100/0 (socket says 172.21.15.7:39100) 2024-05-09T08:01:58.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.328+0000 7fea36ffd640 1 -- 172.21.15.7:0/4129063359 learned_addr learned my addr 172.21.15.7:0/4129063359 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-05-09T08:01:58.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.328+0000 7fea377fe640 1 --2- 172.21.15.7:0/4129063359 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea381f3bf0 0x7fea38150790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:01:58.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.329+0000 7fea36ffd640 1 -- 172.21.15.7:0/4129063359 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea381f3bf0 msgr2=0x7fea38150790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:01:58.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.329+0000 7fea36ffd640 1 --2- 172.21.15.7:0/4129063359 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea381f3bf0 0x7fea38150790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:01:58.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.329+0000 7fea36ffd640 1 -- 172.21.15.7:0/4129063359 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fea381f3770 msgr2=0x7fea38150250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:01:58.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.329+0000 7fea36ffd640 1 --2- 172.21.15.7:0/4129063359 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fea381f3770 0x7fea38150250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:01:58.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.329+0000 7fea36ffd640 1 -- 172.21.15.7:0/4129063359 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea3814f500 con 0x7fea24004800 2024-05-09T08:01:58.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.329+0000 7fea36ffd640 1 --2- 172.21.15.7:0/4129063359 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fea24004800 0x7fea3814eab0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fea2000c820 tx=0x7fea2000cce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:01:58.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.329+0000 7fea2ffff640 1 -- 172.21.15.7:0/4129063359 <== mon.2 v2:172.21.15.7:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea20007a30 con 0x7fea24004800 2024-05-09T08:01:58.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.329+0000 7fea2ffff640 1 -- 172.21.15.7:0/4129063359 <== mon.2 v2:172.21.15.7:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea20007bd0 con 0x7fea24004800 2024-05-09T08:01:58.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.329+0000 7fea3d977640 1 -- 172.21.15.7:0/4129063359 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea380c68d0 con 0x7fea24004800 2024-05-09T08:01:58.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.329+0000 7fea2ffff640 1 -- 172.21.15.7:0/4129063359 <== mon.2 v2:172.21.15.7:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea200113f0 con 0x7fea24004800 2024-05-09T08:01:58.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.329+0000 7fea3d977640 1 -- 172.21.15.7:0/4129063359 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fea3814f770 con 0x7fea24004800 2024-05-09T08:01:58.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.329+0000 7fea2ffff640 1 -- 172.21.15.7:0/4129063359 <== mon.2 v2:172.21.15.7:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fea20011920 con 0x7fea24004800 2024-05-09T08:01:58.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.329+0000 7fea2ffff640 1 -- 172.21.15.7:0/4129063359 <== mon.2 v2:172.21.15.7:3301/0 5 ==== osd_map(6..6 src has 1..6) ==== 1185+0+0 (secure 0 0 0) 0x7fea20019cb0 con 0x7fea24004800 2024-05-09T08:01:58.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.330+0000 7fea3d977640 1 -- 172.21.15.7:0/4129063359 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea38074d70 con 0x7fea24004800 2024-05-09T08:01:58.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.332+0000 7fea2ffff640 1 -- 172.21.15.7:0/4129063359 <== mon.2 v2:172.21.15.7:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fea20015020 con 0x7fea24004800 2024-05-09T08:01:58.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.405+0000 7fea3d977640 1 -- 172.21.15.7:0/4129063359 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_command({"prefix": "osd new", "uuid": "b4576b9c-871d-47a8-8d36-65af525d0200", "id": 4} v 0) -- 0x7fea38158310 con 0x7fea24004800 2024-05-09T08:01:58.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.410+0000 7fea2ffff640 1 -- 172.21.15.7:0/4129063359 <== mon.2 v2:172.21.15.7:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "b4576b9c-871d-47a8-8d36-65af525d0200", "id": 4}]=0 v7) ==== 112+0+2 (secure 0 0 0) 0x7fea20011df0 con 0x7fea24004800 2024-05-09T08:01:58.427 INFO:teuthology.orchestra.run.smithi007.stdout:4 2024-05-09T08:01:58.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.411+0000 7fea3d977640 1 -- 172.21.15.7:0/4129063359 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fea24004800 msgr2=0x7fea3814eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:01:58.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.411+0000 7fea3d977640 1 --2- 172.21.15.7:0/4129063359 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fea24004800 0x7fea3814eab0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fea2000c820 tx=0x7fea2000cce0 comp rx=0 tx=0).stop 2024-05-09T08:01:58.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.411+0000 7fea3d977640 1 -- 172.21.15.7:0/4129063359 shutdown_connections 2024-05-09T08:01:58.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.411+0000 7fea3d977640 1 --2- 172.21.15.7:0/4129063359 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea381f3bf0 0x7fea38150790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:01:58.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.411+0000 7fea3d977640 1 --2- 172.21.15.7:0/4129063359 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fea381f3770 0x7fea38150250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:01:58.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.411+0000 7fea3d977640 1 --2- 172.21.15.7:0/4129063359 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fea24004800 0x7fea3814eab0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:01:58.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.411+0000 7fea3d977640 1 -- 172.21.15.7:0/4129063359 >> 172.21.15.7:0/4129063359 conn(0x7fea380cfe80 msgr2=0x7fea380ce590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:01:58.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.412+0000 7fea3d977640 1 -- 172.21.15.7:0/4129063359 shutdown_connections 2024-05-09T08:01:58.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.412+0000 7fea3d977640 1 -- 172.21.15.7:0/4129063359 wait complete. 2024-05-09T08:01:58.439 DEBUG:teuthology.orchestra.run.smithi007:> sudo ceph --cluster ceph osd new 902b3d94-72f3-4499-b21a-e32453ef5144 5 2024-05-09T08:01:58.519 INFO:tasks.ceph.mgr.y.smithi001.stderr:2024-05-09T08:01:58.502+0000 7fa06500d040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-09T08:01:58.523 INFO:tasks.ceph.mgr.x.smithi007.stderr:2024-05-09T08:01:58.506+0000 7f934988f040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-09T08:01:58.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.522+0000 7f1714729640 1 Processor -- start 2024-05-09T08:01:58.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.522+0000 7f1714729640 1 -- start start 2024-05-09T08:01:58.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.523+0000 7f1714729640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f170c1546f0 0x7f170c0702e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:01:58.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.523+0000 7f1714729640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f170c06b4a0 con 0x7f170c0cd130 2024-05-09T08:01:58.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.523+0000 7f1714729640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f170c06b780 con 0x7f170c06c540 2024-05-09T08:01:58.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.523+0000 7f1714729640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f170c06b250 con 0x7f170c1546f0 2024-05-09T08:01:58.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.523+0000 7f17124c5640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f170c1546f0 0x7f170c0702e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:01:58.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.523+0000 7f1712cc6640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f170c06c540 0x7f170c15e1d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:42948/0 (socket says 172.21.15.7:42948) 2024-05-09T08:01:58.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.523+0000 7f1712cc6640 1 -- 172.21.15.7:0/3370122345 learned_addr learned my addr 172.21.15.7:0/3370122345 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-09T08:01:58.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.523+0000 7f17124c5640 1 -- 172.21.15.7:0/3370122345 >> v1:172.21.15.7:6789/0 conn(0x7f170c06c540 legacy=0x7f170c15e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:01:58.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.523+0000 7f17124c5640 1 -- 172.21.15.7:0/3370122345 >> v1:172.21.15.1:6789/0 conn(0x7f170c0cd130 legacy=0x7f170c15dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:01:58.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.523+0000 7f17114c3640 1 -- 172.21.15.7:0/3370122345 <== mon.1 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4078753121 0 0) 0x7f170c06b780 con 0x7f170c06c540 2024-05-09T08:01:58.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.523+0000 7f17124c5640 1 -- 172.21.15.7:0/3370122345 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f170c06ba60 con 0x7f170c1546f0 2024-05-09T08:01:58.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.524+0000 7f17124c5640 1 --2- 172.21.15.7:0/3370122345 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f170c1546f0 0x7f170c0702e0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f170c06b3d0 tx=0x7f16fc02a1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=46608c283fa24a59 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:01:58.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.524+0000 7f17114c3640 1 -- 172.21.15.7:0/3370122345 <== mon.2 v2:172.21.15.7:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16fc037070 con 0x7f170c1546f0 2024-05-09T08:01:58.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.524+0000 7f17114c3640 1 -- 172.21.15.7:0/3370122345 <== mon.2 v2:172.21.15.7:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16fc032440 con 0x7f170c1546f0 2024-05-09T08:01:58.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.524+0000 7f17114c3640 1 -- 172.21.15.7:0/3370122345 <== mon.2 v2:172.21.15.7:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16fc032810 con 0x7f170c1546f0 2024-05-09T08:01:58.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.525+0000 7f1714729640 1 -- 172.21.15.7:0/3370122345 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f170c1546f0 msgr2=0x7f170c0702e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:01:58.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.525+0000 7f1714729640 1 --2- 172.21.15.7:0/3370122345 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f170c1546f0 0x7f170c0702e0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f170c06b3d0 tx=0x7f16fc02a1a0 comp rx=0 tx=0).stop 2024-05-09T08:01:58.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.525+0000 7f1714729640 1 -- 172.21.15.7:0/3370122345 shutdown_connections 2024-05-09T08:01:58.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.525+0000 7f1714729640 1 --2- 172.21.15.7:0/3370122345 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f170c1546f0 0x7f170c0702e0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:01:58.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.525+0000 7f1714729640 1 -- 172.21.15.7:0/3370122345 >> 172.21.15.7:0/3370122345 conn(0x7f170c0cc580 msgr2=0x7f170c0cc990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:01:58.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.525+0000 7f1714729640 1 -- 172.21.15.7:0/3370122345 shutdown_connections 2024-05-09T08:01:58.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.525+0000 7f1714729640 1 -- 172.21.15.7:0/3370122345 wait complete. 2024-05-09T08:01:58.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.526+0000 7f1714729640 1 Processor -- start 2024-05-09T08:01:58.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.526+0000 7f1714729640 1 -- start start 2024-05-09T08:01:58.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.527+0000 7f1714729640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f170c06c540 0x7f170c06f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:01:58.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.527+0000 7f1714729640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f170c0cd130 0x7f170c06fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:01:58.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.527+0000 7f1714729640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f170c1546f0 0x7f170c0ca380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:01:58.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.527+0000 7f1714729640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f170c154ad0 con 0x7f170c1546f0 2024-05-09T08:01:58.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.527+0000 7f1714729640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f170c06c0b0 con 0x7f170c0cd130 2024-05-09T08:01:58.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.527+0000 7f1714729640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f170c0c75e0 con 0x7f170c06c540 2024-05-09T08:01:58.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.527+0000 7f17124c5640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f170c06c540 0x7f170c06f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:01:58.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.527+0000 7f1711cc4640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f170c0cd130 0x7f170c06fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:01:58.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.527+0000 7f1711cc4640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f170c0cd130 0x7f170c06fc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60086/0 (socket says 172.21.15.7:60086) 2024-05-09T08:01:58.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.527+0000 7f17124c5640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f170c06c540 0x7f170c06f6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3301/0 says I am v2:172.21.15.7:39120/0 (socket says 172.21.15.7:39120) 2024-05-09T08:01:58.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.527+0000 7f1711cc4640 1 -- 172.21.15.7:0/904384807 learned_addr learned my addr 172.21.15.7:0/904384807 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-05-09T08:01:58.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.527+0000 7f17124c5640 1 -- 172.21.15.7:0/904384807 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f170c0cd130 msgr2=0x7f170c06fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:01:58.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.527+0000 7f17124c5640 1 --2- 172.21.15.7:0/904384807 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f170c0cd130 0x7f170c06fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:01:58.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.527+0000 7f17124c5640 1 -- 172.21.15.7:0/904384807 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f170c1546f0 msgr2=0x7f170c0ca380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:01:58.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.527+0000 7f17124c5640 1 --2- 172.21.15.7:0/904384807 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f170c1546f0 0x7f170c0ca380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:01:58.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.527+0000 7f17124c5640 1 -- 172.21.15.7:0/904384807 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f170c0cc000 con 0x7f170c06c540 2024-05-09T08:01:58.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.527+0000 7f1711cc4640 1 --2- 172.21.15.7:0/904384807 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f170c0cd130 0x7f170c06fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:01:58.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.528+0000 7f17124c5640 1 --2- 172.21.15.7:0/904384807 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f170c06c540 0x7f170c06f6c0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f170c0670a0 tx=0x7f16fc0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:01:58.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.528+0000 7f1702ffd640 1 -- 172.21.15.7:0/904384807 <== mon.2 v2:172.21.15.7:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16fc042020 con 0x7f170c06c540 2024-05-09T08:01:58.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.528+0000 7f1702ffd640 1 -- 172.21.15.7:0/904384807 <== mon.2 v2:172.21.15.7:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16fc0309b0 con 0x7f170c06c540 2024-05-09T08:01:58.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.528+0000 7f1714729640 1 -- 172.21.15.7:0/904384807 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f170c145860 con 0x7f170c06c540 2024-05-09T08:01:58.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.528+0000 7f1702ffd640 1 -- 172.21.15.7:0/904384807 <== mon.2 v2:172.21.15.7:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16fc037040 con 0x7f170c06c540 2024-05-09T08:01:58.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.528+0000 7f1714729640 1 -- 172.21.15.7:0/904384807 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f170c146e40 con 0x7f170c06c540 2024-05-09T08:01:58.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.528+0000 7f1702ffd640 1 -- 172.21.15.7:0/904384807 <== mon.2 v2:172.21.15.7:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f16fc045020 con 0x7f170c06c540 2024-05-09T08:01:58.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.528+0000 7f1702ffd640 1 -- 172.21.15.7:0/904384807 <== mon.2 v2:172.21.15.7:3301/0 5 ==== osd_map(7..7 src has 1..7) ==== 1300+0+0 (secure 0 0 0) 0x7f16fc03e9a0 con 0x7f170c06c540 2024-05-09T08:01:58.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.529+0000 7f1714729640 1 -- 172.21.15.7:0/904384807 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f170c146c70 con 0x7f170c06c540 2024-05-09T08:01:58.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.531+0000 7f1702ffd640 1 -- 172.21.15.7:0/904384807 <== mon.2 v2:172.21.15.7:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f16fc04c030 con 0x7f170c06c540 2024-05-09T08:01:58.581 INFO:tasks.ceph.mgr.x.smithi007.stderr:2024-05-09T08:01:58.564+0000 7f934988f040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-09T08:01:58.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.604+0000 7f1714729640 1 -- 172.21.15.7:0/904384807 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_command({"prefix": "osd new", "uuid": "902b3d94-72f3-4499-b21a-e32453ef5144", "id": 5} v 0) -- 0x7f170c145d80 con 0x7f170c06c540 2024-05-09T08:01:58.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.609+0000 7f1702ffd640 1 -- 172.21.15.7:0/904384807 <== mon.2 v2:172.21.15.7:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "902b3d94-72f3-4499-b21a-e32453ef5144", "id": 5}]=0 v8) ==== 112+0+2 (secure 0 0 0) 0x7f170c145d80 con 0x7f170c06c540 2024-05-09T08:01:58.626 INFO:teuthology.orchestra.run.smithi007.stdout:5 2024-05-09T08:01:58.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.610+0000 7f1714729640 1 -- 172.21.15.7:0/904384807 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f170c06c540 msgr2=0x7f170c06f6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:01:58.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.610+0000 7f1714729640 1 --2- 172.21.15.7:0/904384807 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f170c06c540 0x7f170c06f6c0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f170c0670a0 tx=0x7f16fc0046c0 comp rx=0 tx=0).stop 2024-05-09T08:01:58.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.610+0000 7f1714729640 1 -- 172.21.15.7:0/904384807 shutdown_connections 2024-05-09T08:01:58.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.610+0000 7f1714729640 1 --2- 172.21.15.7:0/904384807 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f170c0cd130 0x7f170c06fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:01:58.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.610+0000 7f1714729640 1 --2- 172.21.15.7:0/904384807 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f170c1546f0 0x7f170c0ca380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:01:58.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.610+0000 7f1714729640 1 --2- 172.21.15.7:0/904384807 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f170c06c540 0x7f170c06f6c0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:01:58.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.610+0000 7f1714729640 1 -- 172.21.15.7:0/904384807 >> 172.21.15.7:0/904384807 conn(0x7f170c0cc580 msgr2=0x7f170c0c8440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:01:58.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.610+0000 7f1714729640 1 -- 172.21.15.7:0/904384807 shutdown_connections 2024-05-09T08:01:58.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.610+0000 7f1714729640 1 -- 172.21.15.7:0/904384807 wait complete. 2024-05-09T08:01:58.628 INFO:tasks.ceph.mgr.y.smithi001.stderr:2024-05-09T08:01:58.611+0000 7fa06500d040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-09T08:01:58.636 INFO:tasks.ceph.mgr.x.smithi007.stderr:2024-05-09T08:01:58.620+0000 7f934988f040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-09T08:01:58.637 DEBUG:teuthology.orchestra.run.smithi007:> sudo ceph --cluster ceph osd new ead1d771-1728-4f85-adb4-26be21cedff3 6 2024-05-09T08:01:58.681 INFO:tasks.ceph.mgr.y.smithi001.stderr:2024-05-09T08:01:58.664+0000 7fa06500d040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-09T08:01:58.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.722+0000 7f5b13012640 1 Processor -- start 2024-05-09T08:01:58.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.722+0000 7f5b13012640 1 -- start start 2024-05-09T08:01:58.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.723+0000 7f5b13012640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f5b0c0719e0 0x7f5b0c0c2d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:01:58.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.723+0000 7f5b13012640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f5b0c15c5d0 0x7f5b0c152170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:01:58.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.723+0000 7f5b13012640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b0c158dc0 con 0x7f5b0c071dc0 2024-05-09T08:01:58.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.723+0000 7f5b13012640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f5b0c0b3360 con 0x7f5b0c15c5d0 2024-05-09T08:01:58.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.723+0000 7f5b13012640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f5b0c0c32a0 con 0x7f5b0c0719e0 2024-05-09T08:01:58.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.723+0000 7f5b10dae640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f5b0c0719e0 0x7f5b0c0c2d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:01:58.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.723+0000 7f5b10dae640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f5b0c0719e0 0x7f5b0c0c2d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3301/0 says I am v2:172.21.15.7:39122/0 (socket says 172.21.15.7:39122) 2024-05-09T08:01:58.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.723+0000 7f5b115af640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f5b0c15c5d0 0x7f5b0c152170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:01:58.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.723+0000 7f5b10dae640 1 -- 172.21.15.7:0/2502251068 learned_addr learned my addr 172.21.15.7:0/2502251068 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-05-09T08:01:58.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.723+0000 7f5b10dae640 1 -- 172.21.15.7:0/2502251068 >> v2:172.21.15.7:3300/0 conn(0x7f5b0c15c5d0 msgr2=0x7f5b0c152170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:01:58.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.723+0000 7f5b10dae640 1 --2- 172.21.15.7:0/2502251068 >> v2:172.21.15.7:3300/0 conn(0x7f5b0c15c5d0 0x7f5b0c152170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:01:58.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.723+0000 7f5b10dae640 1 -- 172.21.15.7:0/2502251068 >> v1:172.21.15.7:6789/0 conn(0x7f5b0c071dc0 legacy=0x7f5b0c158380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:01:58.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.723+0000 7f5b10dae640 1 -- 172.21.15.7:0/2502251068 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b0c1590a0 con 0x7f5b0c0719e0 2024-05-09T08:01:58.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.723+0000 7f5b115af640 1 --2- 172.21.15.7:0/2502251068 >> v2:172.21.15.7:3300/0 conn(0x7f5b0c15c5d0 0x7f5b0c152170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:01:58.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.724+0000 7f5b10dae640 1 --2- 172.21.15.7:0/2502251068 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f5b0c0719e0 0x7f5b0c0c2d60 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f5b00009870 tx=0x7f5b0002a250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3e71505c08e15397 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:01:58.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.724+0000 7f5b0b7fe640 1 -- 172.21.15.7:0/2502251068 <== mon.2 v2:172.21.15.7:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b00037070 con 0x7f5b0c0719e0 2024-05-09T08:01:58.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.724+0000 7f5b0b7fe640 1 -- 172.21.15.7:0/2502251068 <== mon.2 v2:172.21.15.7:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b00032440 con 0x7f5b0c0719e0 2024-05-09T08:01:58.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.724+0000 7f5b0b7fe640 1 -- 172.21.15.7:0/2502251068 <== mon.2 v2:172.21.15.7:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b00032810 con 0x7f5b0c0719e0 2024-05-09T08:01:58.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.724+0000 7f5b13012640 1 -- 172.21.15.7:0/2502251068 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f5b0c0719e0 msgr2=0x7f5b0c0c2d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:01:58.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.724+0000 7f5b13012640 1 --2- 172.21.15.7:0/2502251068 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f5b0c0719e0 0x7f5b0c0c2d60 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f5b00009870 tx=0x7f5b0002a250 comp rx=0 tx=0).stop 2024-05-09T08:01:58.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.725+0000 7f5b13012640 1 -- 172.21.15.7:0/2502251068 shutdown_connections 2024-05-09T08:01:58.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.725+0000 7f5b13012640 1 --2- 172.21.15.7:0/2502251068 >> v2:172.21.15.7:3300/0 conn(0x7f5b0c15c5d0 0x7f5b0c152170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:01:58.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.725+0000 7f5b13012640 1 --2- 172.21.15.7:0/2502251068 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f5b0c0719e0 0x7f5b0c0c2d60 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:01:58.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.725+0000 7f5b13012640 1 -- 172.21.15.7:0/2502251068 >> 172.21.15.7:0/2502251068 conn(0x7f5b0c06b880 msgr2=0x7f5b0c06fb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:01:58.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.725+0000 7f5b13012640 1 -- 172.21.15.7:0/2502251068 shutdown_connections 2024-05-09T08:01:58.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.725+0000 7f5b13012640 1 -- 172.21.15.7:0/2502251068 wait complete. 2024-05-09T08:01:58.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.726+0000 7f5b13012640 1 Processor -- start 2024-05-09T08:01:58.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.726+0000 7f5b13012640 1 -- start start 2024-05-09T08:01:58.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.727+0000 7f5b13012640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5b0c0719e0 0x7f5b0c1527d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:01:58.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.727+0000 7f5b13012640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b0c071dc0 0x7f5b0c1ee640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:01:58.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.727+0000 7f5b13012640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f5b0c15c5d0 0x7f5b0c1e4120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:01:58.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.727+0000 7f5b13012640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f5b0c15c0b0 con 0x7f5b0c0719e0 2024-05-09T08:01:58.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.727+0000 7f5b13012640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5b0c15bdd0 con 0x7f5b0c071dc0 2024-05-09T08:01:58.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.727+0000 7f5b13012640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f5b0c158a90 con 0x7f5b0c15c5d0 2024-05-09T08:01:58.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.727+0000 7f5b115af640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f5b0c15c5d0 0x7f5b0c1e4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:01:58.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.727+0000 7f5b0bfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b0c071dc0 0x7f5b0c1ee640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:01:58.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.727+0000 7f5b115af640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f5b0c15c5d0 0x7f5b0c1e4120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3301/0 says I am v2:172.21.15.7:39130/0 (socket says 172.21.15.7:39130) 2024-05-09T08:01:58.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.727+0000 7f5b115af640 1 -- 172.21.15.7:0/391250125 learned_addr learned my addr 172.21.15.7:0/391250125 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-05-09T08:01:58.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.727+0000 7f5b115af640 1 -- 172.21.15.7:0/391250125 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b0c071dc0 msgr2=0x7f5b0c1ee640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:01:58.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.727+0000 7f5b115af640 1 --2- 172.21.15.7:0/391250125 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b0c071dc0 0x7f5b0c1ee640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:01:58.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.727+0000 7f5b115af640 1 -- 172.21.15.7:0/391250125 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5b0c0719e0 msgr2=0x7f5b0c1527d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:01:58.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.727+0000 7f5b115af640 1 --2- 172.21.15.7:0/391250125 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5b0c0719e0 0x7f5b0c1527d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:01:58.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.727+0000 7f5b115af640 1 -- 172.21.15.7:0/391250125 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b0c154510 con 0x7f5b0c15c5d0 2024-05-09T08:01:58.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.727+0000 7f5b0bfff640 1 --2- 172.21.15.7:0/391250125 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b0c071dc0 0x7f5b0c1ee640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:01:58.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.727+0000 7f5b115af640 1 --2- 172.21.15.7:0/391250125 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f5b0c15c5d0 0x7f5b0c1e4120 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f5b04009e60 tx=0x7f5b0400c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:01:58.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.727+0000 7f5b097fa640 1 -- 172.21.15.7:0/391250125 <== mon.2 v2:172.21.15.7:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b04018030 con 0x7f5b0c15c5d0 2024-05-09T08:01:58.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.727+0000 7f5b097fa640 1 -- 172.21.15.7:0/391250125 <== mon.2 v2:172.21.15.7:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b0400cdf0 con 0x7f5b0c15c5d0 2024-05-09T08:01:58.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.727+0000 7f5b097fa640 1 -- 172.21.15.7:0/391250125 <== mon.2 v2:172.21.15.7:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b04003bb0 con 0x7f5b0c15c5d0 2024-05-09T08:01:58.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.727+0000 7f5b13012640 1 -- 172.21.15.7:0/391250125 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b0c1edb80 con 0x7f5b0c15c5d0 2024-05-09T08:01:58.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.727+0000 7f5b13012640 1 -- 172.21.15.7:0/391250125 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b0c154ab0 con 0x7f5b0c15c5d0 2024-05-09T08:01:58.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.728+0000 7f5b097fa640 1 -- 172.21.15.7:0/391250125 <== mon.2 v2:172.21.15.7:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f5b04005d10 con 0x7f5b0c15c5d0 2024-05-09T08:01:58.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.728+0000 7f5b097fa640 1 -- 172.21.15.7:0/391250125 <== mon.2 v2:172.21.15.7:3301/0 5 ==== osd_map(8..8 src has 1..8) ==== 1415+0+0 (secure 0 0 0) 0x7f5b0401da50 con 0x7f5b0c15c5d0 2024-05-09T08:01:58.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.729+0000 7f5b13012640 1 -- 172.21.15.7:0/391250125 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b0c074d70 con 0x7f5b0c15c5d0 2024-05-09T08:01:58.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.731+0000 7f5b097fa640 1 -- 172.21.15.7:0/391250125 <== mon.2 v2:172.21.15.7:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f5b04022070 con 0x7f5b0c15c5d0 2024-05-09T08:01:58.760 INFO:tasks.ceph.mgr.x.smithi007.stderr:2024-05-09T08:01:58.743+0000 7f934988f040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-09T08:01:58.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.805+0000 7f5b13012640 1 -- 172.21.15.7:0/391250125 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_command({"prefix": "osd new", "uuid": "ead1d771-1728-4f85-adb4-26be21cedff3", "id": 6} v 0) -- 0x7f5b0c0c93b0 con 0x7f5b0c15c5d0 2024-05-09T08:01:58.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.810+0000 7f5b097fa640 1 -- 172.21.15.7:0/391250125 <== mon.2 v2:172.21.15.7:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "ead1d771-1728-4f85-adb4-26be21cedff3", "id": 6}]=0 v9) ==== 112+0+2 (secure 0 0 0) 0x7f5b04010040 con 0x7f5b0c15c5d0 2024-05-09T08:01:58.827 INFO:teuthology.orchestra.run.smithi007.stdout:6 2024-05-09T08:01:58.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.811+0000 7f5b13012640 1 -- 172.21.15.7:0/391250125 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f5b0c15c5d0 msgr2=0x7f5b0c1e4120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:01:58.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.811+0000 7f5b13012640 1 --2- 172.21.15.7:0/391250125 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f5b0c15c5d0 0x7f5b0c1e4120 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f5b04009e60 tx=0x7f5b0400c490 comp rx=0 tx=0).stop 2024-05-09T08:01:58.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.811+0000 7f5b13012640 1 -- 172.21.15.7:0/391250125 shutdown_connections 2024-05-09T08:01:58.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.811+0000 7f5b13012640 1 --2- 172.21.15.7:0/391250125 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b0c071dc0 0x7f5b0c1ee640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:01:58.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.811+0000 7f5b13012640 1 --2- 172.21.15.7:0/391250125 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f5b0c15c5d0 0x7f5b0c1e4120 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:01:58.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.811+0000 7f5b13012640 1 --2- 172.21.15.7:0/391250125 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5b0c0719e0 0x7f5b0c1527d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:01:58.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.811+0000 7f5b13012640 1 -- 172.21.15.7:0/391250125 >> 172.21.15.7:0/391250125 conn(0x7f5b0c06b880 msgr2=0x7f5b0c0cca10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:01:58.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.811+0000 7f5b13012640 1 -- 172.21.15.7:0/391250125 shutdown_connections 2024-05-09T08:01:58.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.811+0000 7f5b13012640 1 -- 172.21.15.7:0/391250125 wait complete. 2024-05-09T08:01:58.838 DEBUG:teuthology.orchestra.run.smithi007:> sudo ceph --cluster ceph osd new 3ab5a201-9394-48a8-8918-e95197893669 7 2024-05-09T08:01:58.883 INFO:tasks.ceph.mgr.y.smithi001.stderr:2024-05-09T08:01:58.867+0000 7fa06500d040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-09T08:01:58.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.921+0000 7f171b65d640 1 Processor -- start 2024-05-09T08:01:58.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.921+0000 7f171b65d640 1 -- start start 2024-05-09T08:01:58.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.922+0000 7f171b65d640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f1714154bc0 0x7f171406d490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:01:58.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.922+0000 7f171b65d640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f171415e0e0 con 0x7f17141547e0 2024-05-09T08:01:58.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.922+0000 7f171b65d640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f171415e380 con 0x7f171415dbc0 2024-05-09T08:01:58.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.922+0000 7f171b65d640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f17140ca9f0 con 0x7f1714154bc0 2024-05-09T08:01:58.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.922+0000 7f1719bfa640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f171415dbc0 0x7f17140cae80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:42960/0 (socket says 172.21.15.7:42960) 2024-05-09T08:01:58.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.922+0000 7f1719bfa640 1 -- 172.21.15.7:0/105790550 learned_addr learned my addr 172.21.15.7:0/105790550 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-09T08:01:58.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.923+0000 7f170bfff640 1 -- 172.21.15.7:0/105790550 <== mon.1 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4053119125 0 0) 0x7f171415e380 con 0x7f171415dbc0 2024-05-09T08:01:58.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.923+0000 7f170bfff640 1 -- 172.21.15.7:0/105790550 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1700003520 con 0x7f171415dbc0 2024-05-09T08:01:58.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.923+0000 7f1718bf8640 1 --2- 172.21.15.7:0/105790550 >> v2:172.21.15.1:3300/0 conn(0x7f1714154bc0 0x7f171406d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:01:58.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.923+0000 7f170bfff640 1 -- 172.21.15.7:0/105790550 <== mon.1 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2827370561 0 0) 0x7f1700003520 con 0x7f171415dbc0 2024-05-09T08:01:58.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.923+0000 7f170bfff640 1 -- 172.21.15.7:0/105790550 >> v2:172.21.15.1:3300/0 conn(0x7f1714154bc0 msgr2=0x7f171406d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:01:58.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.923+0000 7f170bfff640 1 --2- 172.21.15.7:0/105790550 >> v2:172.21.15.1:3300/0 conn(0x7f1714154bc0 0x7f171406d490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:01:58.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.923+0000 7f170bfff640 1 -- 172.21.15.7:0/105790550 >> v1:172.21.15.1:6789/0 conn(0x7f17141547e0 legacy=0x7f17140c4cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:01:58.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.923+0000 7f170bfff640 1 -- 172.21.15.7:0/105790550 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f171415e660 con 0x7f171415dbc0 2024-05-09T08:01:58.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.923+0000 7f170bfff640 1 -- 172.21.15.7:0/105790550 <== mon.1 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4179846320 0 0) 0x7f170c001a10 con 0x7f171415dbc0 2024-05-09T08:01:58.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.923+0000 7f170bfff640 1 -- 172.21.15.7:0/105790550 >> v1:172.21.15.7:6789/0 conn(0x7f171415dbc0 legacy=0x7f17140cae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:01:58.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.923+0000 7f170bfff640 1 --2- 172.21.15.7:0/105790550 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1700003a00 0x7f1700003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:01:58.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.923+0000 7f170bfff640 1 --2- 172.21.15.7:0/105790550 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1700004800 0x7f1700004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:01:58.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.923+0000 7f170bfff640 1 --2- 172.21.15.7:0/105790550 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f1700005210 0x7f170000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:01:58.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.923+0000 7f170bfff640 1 -- 172.21.15.7:0/105790550 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f17140ca9f0 con 0x7f1700004800 2024-05-09T08:01:58.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.923+0000 7f170bfff640 1 -- 172.21.15.7:0/105790550 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f170000c120 con 0x7f1700003a00 2024-05-09T08:01:58.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.923+0000 7f170bfff640 1 -- 172.21.15.7:0/105790550 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f170000c320 con 0x7f1700005210 2024-05-09T08:01:58.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.923+0000 7f17193f9640 1 --2- 172.21.15.7:0/105790550 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1700003a00 0x7f1700003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:01:58.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.923+0000 7f1719bfa640 1 --2- 172.21.15.7:0/105790550 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f1700005210 0x7f170000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:01:58.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.923+0000 7f17193f9640 1 -- 172.21.15.7:0/105790550 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f1700005210 msgr2=0x7f170000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:01:58.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.923+0000 7f17193f9640 1 --2- 172.21.15.7:0/105790550 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f1700005210 0x7f170000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:01:58.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.923+0000 7f1718bf8640 1 --2- 172.21.15.7:0/105790550 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1700004800 0x7f1700004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:01:58.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.923+0000 7f17193f9640 1 -- 172.21.15.7:0/105790550 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1700004800 msgr2=0x7f1700004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:01:58.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.923+0000 7f17193f9640 1 --2- 172.21.15.7:0/105790550 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1700004800 0x7f1700004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:01:58.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.923+0000 7f1719bfa640 1 -- 172.21.15.7:0/105790550 reap_dead start 2024-05-09T08:01:58.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.923+0000 7f17193f9640 1 -- 172.21.15.7:0/105790550 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1700003520 con 0x7f1700003a00 2024-05-09T08:01:58.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.924+0000 7f17193f9640 1 --2- 172.21.15.7:0/105790550 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1700003a00 0x7f1700003e00 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f1704001650 tx=0x7f1700005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2498348bf6e37d4e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:01:58.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.924+0000 7f170bfff640 1 -- 172.21.15.7:0/105790550 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1704001220 con 0x7f1700003a00 2024-05-09T08:01:58.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.924+0000 7f170bfff640 1 -- 172.21.15.7:0/105790550 <== mon.1 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f170402e070 con 0x7f1700003a00 2024-05-09T08:01:58.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.924+0000 7f170bfff640 1 -- 172.21.15.7:0/105790550 <== mon.1 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f170402a7c0 con 0x7f1700003a00 2024-05-09T08:01:58.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.925+0000 7f171b65d640 1 -- 172.21.15.7:0/105790550 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1700003a00 msgr2=0x7f1700003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:01:58.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.925+0000 7f171b65d640 1 --2- 172.21.15.7:0/105790550 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1700003a00 0x7f1700003e00 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f1704001650 tx=0x7f1700005830 comp rx=0 tx=0).stop 2024-05-09T08:01:58.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.925+0000 7f171b65d640 1 -- 172.21.15.7:0/105790550 shutdown_connections 2024-05-09T08:01:58.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.925+0000 7f171b65d640 1 --2- 172.21.15.7:0/105790550 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1700003a00 0x7f1700003e00 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:01:58.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.925+0000 7f171b65d640 1 -- 172.21.15.7:0/105790550 >> 172.21.15.7:0/105790550 conn(0x7f17140cfe40 msgr2=0x7f17140d0250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:01:58.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.925+0000 7f171b65d640 1 -- 172.21.15.7:0/105790550 shutdown_connections 2024-05-09T08:01:58.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.925+0000 7f171b65d640 1 -- 172.21.15.7:0/105790550 wait complete. 2024-05-09T08:01:58.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.926+0000 7f171b65d640 1 Processor -- start 2024-05-09T08:01:58.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.926+0000 7f171b65d640 1 -- start start 2024-05-09T08:01:58.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.927+0000 7f171b65d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1700003a00 0x7f17141f4560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:01:58.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.927+0000 7f171b65d640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f1714148490 0x7f17141511a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:01:58.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.927+0000 7f171b65d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17141488d0 0x7f17141516e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:01:58.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.927+0000 7f171b65d640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f17140cc420 con 0x7f1700003a00 2024-05-09T08:01:58.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.927+0000 7f171b65d640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f17140cc2a0 con 0x7f17141488d0 2024-05-09T08:01:58.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.927+0000 7f171b65d640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f17140c4cc0 con 0x7f1714148490 2024-05-09T08:01:58.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.927+0000 7f1718bf8640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f1714148490 0x7f17141511a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:01:58.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.927+0000 7f1719bfa640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17141488d0 0x7f17141516e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:01:58.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.927+0000 7f1719bfa640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17141488d0 0x7f17141516e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60110/0 (socket says 172.21.15.7:60110) 2024-05-09T08:01:58.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.927+0000 7f1718bf8640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f1714148490 0x7f17141511a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3301/0 says I am v2:172.21.15.7:39142/0 (socket says 172.21.15.7:39142) 2024-05-09T08:01:58.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.927+0000 7f1719bfa640 1 -- 172.21.15.7:0/181049715 learned_addr learned my addr 172.21.15.7:0/181049715 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-05-09T08:01:58.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.927+0000 7f1718bf8640 1 -- 172.21.15.7:0/181049715 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17141488d0 msgr2=0x7f17141516e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:01:58.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.927+0000 7f1718bf8640 1 --2- 172.21.15.7:0/181049715 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17141488d0 0x7f17141516e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:01:58.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.927+0000 7f1718bf8640 1 -- 172.21.15.7:0/181049715 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1700003a00 msgr2=0x7f17141f4560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:01:58.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.927+0000 7f17193f9640 1 --2- 172.21.15.7:0/181049715 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1700003a00 0x7f17141f4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:01:58.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.928+0000 7f1718bf8640 1 --2- 172.21.15.7:0/181049715 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1700003a00 0x7f17141f4560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:01:58.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.928+0000 7f1718bf8640 1 -- 172.21.15.7:0/181049715 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f171415b2e0 con 0x7f1714148490 2024-05-09T08:01:58.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.928+0000 7f1719bfa640 1 --2- 172.21.15.7:0/181049715 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17141488d0 0x7f17141516e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:01:58.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.928+0000 7f1718bf8640 1 --2- 172.21.15.7:0/181049715 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f1714148490 0x7f17141511a0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f171406a6c0 tx=0x7f171000e8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:01:58.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.928+0000 7f1709ffb640 1 -- 172.21.15.7:0/181049715 <== mon.2 v2:172.21.15.7:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f171000f040 con 0x7f1714148490 2024-05-09T08:01:58.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.928+0000 7f1709ffb640 1 -- 172.21.15.7:0/181049715 <== mon.2 v2:172.21.15.7:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1710004230 con 0x7f1714148490 2024-05-09T08:01:58.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.928+0000 7f171b65d640 1 -- 172.21.15.7:0/181049715 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1714147ba0 con 0x7f1714148490 2024-05-09T08:01:58.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.928+0000 7f1709ffb640 1 -- 172.21.15.7:0/181049715 <== mon.2 v2:172.21.15.7:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1710003610 con 0x7f1714148490 2024-05-09T08:01:58.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.928+0000 7f1709ffb640 1 -- 172.21.15.7:0/181049715 <== mon.2 v2:172.21.15.7:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f1710003b40 con 0x7f1714148490 2024-05-09T08:01:58.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.928+0000 7f171b65d640 1 -- 172.21.15.7:0/181049715 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f171415a000 con 0x7f1714148490 2024-05-09T08:01:58.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.928+0000 7f1709ffb640 1 -- 172.21.15.7:0/181049715 <== mon.2 v2:172.21.15.7:3301/0 5 ==== osd_map(9..9 src has 1..9) ==== 1530+0+0 (secure 0 0 0) 0x7f1710013b10 con 0x7f1714148490 2024-05-09T08:01:58.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.929+0000 7f171b65d640 1 -- 172.21.15.7:0/181049715 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f171415e380 con 0x7f1714148490 2024-05-09T08:01:58.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:58.931+0000 7f1709ffb640 1 -- 172.21.15.7:0/181049715 <== mon.2 v2:172.21.15.7:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f171415e380 con 0x7f1714148490 2024-05-09T08:01:58.953 INFO:tasks.ceph.mgr.y.smithi001.stderr:2024-05-09T08:01:58.936+0000 7fa06500d040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-09T08:01:59.017 INFO:tasks.ceph.mgr.y.smithi001.stderr:2024-05-09T08:01:59.000+0000 7fa06500d040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-09T08:01:59.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:59.006+0000 7f171b65d640 1 -- 172.21.15.7:0/181049715 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_command({"prefix": "osd new", "uuid": "3ab5a201-9394-48a8-8918-e95197893669", "id": 7} v 0) -- 0x7f17140b38b0 con 0x7f1714148490 2024-05-09T08:01:59.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:59.010+0000 7f1709ffb640 1 -- 172.21.15.7:0/181049715 <== mon.2 v2:172.21.15.7:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "3ab5a201-9394-48a8-8918-e95197893669", "id": 7}]=0 v10) ==== 112+0+2 (secure 0 0 0) 0x7f17140b38b0 con 0x7f1714148490 2024-05-09T08:01:59.027 INFO:teuthology.orchestra.run.smithi007.stdout:7 2024-05-09T08:01:59.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:59.012+0000 7f171b65d640 1 -- 172.21.15.7:0/181049715 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f1714148490 msgr2=0x7f17141511a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:01:59.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:59.012+0000 7f171b65d640 1 --2- 172.21.15.7:0/181049715 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f1714148490 0x7f17141511a0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f171406a6c0 tx=0x7f171000e8c0 comp rx=0 tx=0).stop 2024-05-09T08:01:59.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:59.012+0000 7f171b65d640 1 -- 172.21.15.7:0/181049715 shutdown_connections 2024-05-09T08:01:59.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:59.012+0000 7f171b65d640 1 --2- 172.21.15.7:0/181049715 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17141488d0 0x7f17141516e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:01:59.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:59.012+0000 7f171b65d640 1 --2- 172.21.15.7:0/181049715 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f1714148490 0x7f17141511a0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:01:59.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:59.012+0000 7f171b65d640 1 --2- 172.21.15.7:0/181049715 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1700003a00 0x7f17141f4560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:01:59.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:59.012+0000 7f171b65d640 1 -- 172.21.15.7:0/181049715 >> 172.21.15.7:0/181049715 conn(0x7f17140cfe40 msgr2=0x7f17140c5600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:01:59.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:59.012+0000 7f171b65d640 1 -- 172.21.15.7:0/181049715 shutdown_connections 2024-05-09T08:01:59.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-09T08:01:59.012+0000 7f171b65d640 1 -- 172.21.15.7:0/181049715 wait complete. 2024-05-09T08:01:59.039 INFO:tasks.ceph.osd.0:Restarting daemon 2024-05-09T08:01:59.039 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-05-09T08:01:59.046 INFO:tasks.ceph.osd.0:Started 2024-05-09T08:01:59.046 INFO:tasks.ceph.osd.1:Restarting daemon 2024-05-09T08:01:59.046 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-05-09T08:01:59.053 INFO:tasks.ceph.osd.1:Started 2024-05-09T08:01:59.053 INFO:tasks.ceph.osd.2:Restarting daemon 2024-05-09T08:01:59.054 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-05-09T08:01:59.062 INFO:tasks.ceph.osd.2:Started 2024-05-09T08:01:59.062 INFO:tasks.ceph.osd.3:Restarting daemon 2024-05-09T08:01:59.063 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-05-09T08:01:59.068 INFO:tasks.ceph.osd.3:Started 2024-05-09T08:01:59.068 INFO:tasks.ceph.osd.4:Restarting daemon 2024-05-09T08:01:59.068 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-05-09T08:01:59.070 INFO:tasks.ceph.osd.4:Started 2024-05-09T08:01:59.070 INFO:tasks.ceph.osd.5:Restarting daemon 2024-05-09T08:01:59.071 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-05-09T08:01:59.073 INFO:tasks.ceph.osd.5:Started 2024-05-09T08:01:59.073 INFO:tasks.ceph.osd.6:Restarting daemon 2024-05-09T08:01:59.073 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-05-09T08:01:59.075 INFO:tasks.ceph.osd.6:Started 2024-05-09T08:01:59.075 INFO:tasks.ceph.osd.7:Restarting daemon 2024-05-09T08:01:59.075 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-05-09T08:01:59.077 INFO:tasks.ceph.osd.7:Started 2024-05-09T08:01:59.078 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T08:01:59.188 INFO:tasks.ceph.mgr.x.smithi007.stderr:2024-05-09T08:01:59.172+0000 7f934988f040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-09T08:01:59.210 INFO:tasks.ceph.mgr.y.smithi001.stderr:2024-05-09T08:01:59.193+0000 7fa06500d040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-09T08:01:59.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:01:59.279+0000 7fd4ba600640 1 Processor -- start 2024-05-09T08:01:59.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:01:59.279+0000 7fd4ba600640 1 -- start start 2024-05-09T08:01:59.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:01:59.281+0000 7fd4ba600640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fd4b40c6ea0 0x7fd4b4072900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:01:59.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:01:59.281+0000 7fd4ba600640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fd4b406ece0 0x7fd4b4159010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:01:59.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:01:59.281+0000 7fd4ba600640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd4b4073200 con 0x7fd4b406dca0 2024-05-09T08:01:59.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:01:59.281+0000 7fd4ba600640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7fd4b4072e40 con 0x7fd4b40c6ea0 2024-05-09T08:01:59.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:01:59.281+0000 7fd4ba600640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7fd4b40737c0 con 0x7fd4b406ece0 2024-05-09T08:01:59.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:01:59.281+0000 7fd4b3400640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7fd4b406dca0 0x7fd4b406e080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:37304/0 (socket says 172.21.15.1:37304) 2024-05-09T08:01:59.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:01:59.281+0000 7fd4b3400640 1 -- 172.21.15.1:0/772754656 learned_addr learned my addr 172.21.15.1:0/772754656 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T08:01:59.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:01:59.281+0000 7fd4b3e00640 1 --2- 172.21.15.1:0/772754656 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fd4b406ece0 0x7fd4b4159010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:01:59.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:01:59.281+0000 7fd4b2a00640 1 --2- 172.21.15.1:0/772754656 >> v2:172.21.15.1:3300/0 conn(0x7fd4b40c6ea0 0x7fd4b4072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:01:59.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:01:59.281+0000 7fd4b3e00640 1 -- 172.21.15.1:0/772754656 >> v2:172.21.15.1:3300/0 conn(0x7fd4b40c6ea0 msgr2=0x7fd4b4072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:01:59.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:01:59.281+0000 7fd4b3e00640 1 --2- 172.21.15.1:0/772754656 >> v2:172.21.15.1:3300/0 conn(0x7fd4b40c6ea0 0x7fd4b4072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:01:59.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:01:59.281+0000 7fd4b3e00640 1 -- 172.21.15.1:0/772754656 >> v1:172.21.15.1:6789/0 conn(0x7fd4b406dca0 legacy=0x7fd4b406e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:01:59.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:01:59.281+0000 7fd4b3e00640 1 -- 172.21.15.1:0/772754656 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4b40734e0 con 0x7fd4b406ece0 2024-05-09T08:01:59.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:01:59.281+0000 7fd4b2000640 1 -- 172.21.15.1:0/772754656 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3961802129 0 0) 0x7fd4b4073200 con 0x7fd4b406dca0 2024-05-09T08:01:59.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:01:59.282+0000 7fd4b2a00640 1 --2- 172.21.15.1:0/772754656 >> v2:172.21.15.1:3300/0 conn(0x7fd4b40c6ea0 0x7fd4b4072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T08:01:59.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:01:59.283+0000 7fd4b3e00640 1 --2- 172.21.15.1:0/772754656 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fd4b406ece0 0x7fd4b4159010 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fd4ac009d90 tx=0x7fd4ac02a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d3fd9ef78fac03fb server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:01:59.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:01:59.283+0000 7fd4b2000640 1 -- 172.21.15.1:0/772754656 <== mon.2 v2:172.21.15.7:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4ac007740 con 0x7fd4b406ece0 2024-05-09T08:01:59.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:01:59.283+0000 7fd4b2000640 1 -- 172.21.15.1:0/772754656 <== mon.2 v2:172.21.15.7:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4ac0078e0 con 0x7fd4b406ece0 2024-05-09T08:01:59.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:01:59.283+0000 7fd4b2000640 1 -- 172.21.15.1:0/772754656 <== mon.2 v2:172.21.15.7:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4ac007cb0 con 0x7fd4b406ece0 2024-05-09T08:01:59.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:01:59.283+0000 7fd4ba600640 1 -- 172.21.15.1:0/772754656 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fd4b406ece0 msgr2=0x7fd4b4159010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:01:59.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:01:59.283+0000 7fd4ba600640 1 --2- 172.21.15.1:0/772754656 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fd4b406ece0 0x7fd4b4159010 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fd4ac009d90 tx=0x7fd4ac02a660 comp rx=0 tx=0).stop 2024-05-09T08:01:59.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:01:59.284+0000 7fd4ba600640 1 -- 172.21.15.1:0/772754656 shutdown_connections 2024-05-09T08:01:59.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:01:59.284+0000 7fd4ba600640 1 --2- 172.21.15.1:0/772754656 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fd4b406ece0 0x7fd4b4159010 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:01:59.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:01:59.284+0000 7fd4ba600640 1 --2- 172.21.15.1:0/772754656 >> v2:172.21.15.1:3300/0 conn(0x7fd4b40c6ea0 0x7fd4b4072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:01:59.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:01:59.284+0000 7fd4ba600640 1 -- 172.21.15.1:0/772754656 >> 172.21.15.1:0/772754656 conn(0x7fd4b40cc070 msgr2=0x7fd4b40cc480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:01:59.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:01:59.284+0000 7fd4ba600640 1 -- 172.21.15.1:0/772754656 shutdown_connections 2024-05-09T08:01:59.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:01:59.284+0000 7fd4ba600640 1 -- 172.21.15.1:0/772754656 wait complete. 2024-05-09T08:01:59.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:01:59.285+0000 7fd4ba600640 1 Processor -- start 2024-05-09T08:01:59.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:01:59.286+0000 7fd4ba600640 1 -- start start 2024-05-09T08:01:59.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:01:59.286+0000 7fd4ba600640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd4b406dca0 0x7fd4b414e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:01:59.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:01:59.286+0000 7fd4ba600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4b406ece0 0x7fd4b41ef7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:01:59.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:01:59.286+0000 7fd4ba600640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fd4b40c6ea0 0x7fd4b40cb210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:01:59.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:01:59.286+0000 7fd4ba600640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fd4b4072470 con 0x7fd4b406dca0 2024-05-09T08:01:59.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:01:59.286+0000 7fd4ba600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd4b4159550 con 0x7fd4b406ece0 2024-05-09T08:01:59.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:01:59.286+0000 7fd4ba600640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7fd4b4072100 con 0x7fd4b40c6ea0 2024-05-09T08:01:59.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:01:59.287+0000 7fd4b3e00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fd4b40c6ea0 0x7fd4b40cb210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:01:59.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:01:59.287+0000 7fd4b2a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4b406ece0 0x7fd4b41ef7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:01:59.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:01:59.287+0000 7fd4b3400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd4b406dca0 0x7fd4b414e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:01:59.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:01:59.287+0000 7fd4b3e00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fd4b40c6ea0 0x7fd4b40cb210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3301/0 says I am v2:172.21.15.1:54178/0 (socket says 172.21.15.1:54178) 2024-05-09T08:01:59.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:01:59.287+0000 7fd4b3e00640 1 -- 172.21.15.1:0/99998556 learned_addr learned my addr 172.21.15.1:0/99998556 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:01:59.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:01:59.287+0000 7fd4b3e00640 1 -- 172.21.15.1:0/99998556 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4b406ece0 msgr2=0x7fd4b41ef7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:01:59.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:01:59.287+0000 7fd4b3e00640 1 --2- 172.21.15.1:0/99998556 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4b406ece0 0x7fd4b41ef7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:01:59.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:01:59.287+0000 7fd4b3e00640 1 -- 172.21.15.1:0/99998556 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd4b406dca0 msgr2=0x7fd4b414e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:01:59.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:01:59.287+0000 7fd4b3e00640 1 --2- 172.21.15.1:0/99998556 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd4b406dca0 0x7fd4b414e010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:01:59.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:01:59.287+0000 7fd4b3e00640 1 -- 172.21.15.1:0/99998556 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4b41eee40 con 0x7fd4b40c6ea0 2024-05-09T08:01:59.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:01:59.287+0000 7fd4b2a00640 1 --2- 172.21.15.1:0/99998556 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4b406ece0 0x7fd4b41ef7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:01:59.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:01:59.287+0000 7fd4b3400640 1 --2- 172.21.15.1:0/99998556 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd4b406dca0 0x7fd4b414e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:01:59.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:01:59.288+0000 7fd4b3e00640 1 --2- 172.21.15.1:0/99998556 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fd4b40c6ea0 0x7fd4b40cb210 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fd4ac009eb0 tx=0x7fd4ac004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:01:59.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:01:59.288+0000 7fd4ab400640 1 -- 172.21.15.1:0/99998556 <== mon.2 v2:172.21.15.7:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4ac040070 con 0x7fd4b40c6ea0 2024-05-09T08:01:59.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:01:59.288+0000 7fd4ab400640 1 -- 172.21.15.1:0/99998556 <== mon.2 v2:172.21.15.7:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4ac031c90 con 0x7fd4b40c6ea0 2024-05-09T08:01:59.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:01:59.288+0000 7fd4ab400640 1 -- 172.21.15.1:0/99998556 <== mon.2 v2:172.21.15.7:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4ac045680 con 0x7fd4b40c6ea0 2024-05-09T08:01:59.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:01:59.288+0000 7fd4ba600640 1 -- 172.21.15.1:0/99998556 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4b41ef120 con 0x7fd4b40c6ea0 2024-05-09T08:01:59.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:01:59.288+0000 7fd4ba600640 1 -- 172.21.15.1:0/99998556 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4b414d810 con 0x7fd4b40c6ea0 2024-05-09T08:01:59.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:01:59.288+0000 7fd4ab400640 1 -- 172.21.15.1:0/99998556 <== mon.2 v2:172.21.15.7:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fd4ac039040 con 0x7fd4b40c6ea0 2024-05-09T08:01:59.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:01:59.288+0000 7fd4ab400640 1 -- 172.21.15.1:0/99998556 <== mon.2 v2:172.21.15.7:3301/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7fd4ac03cb40 con 0x7fd4b40c6ea0 2024-05-09T08:01:59.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:01:59.289+0000 7fd4ba600640 1 -- 172.21.15.1:0/99998556 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4b4073200 con 0x7fd4b40c6ea0 2024-05-09T08:01:59.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:01:59.292+0000 7fd4ab400640 1 -- 172.21.15.1:0/99998556 <== mon.2 v2:172.21.15.7:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fd4b4073200 con 0x7fd4b40c6ea0 2024-05-09T08:01:59.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:01:59.356+0000 7fd4ba600640 1 -- 172.21.15.1:0/99998556 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd4b4074d70 con 0x7fd4b40c6ea0 2024-05-09T08:01:59.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:01:59.357+0000 7fd4ab400640 1 -- 172.21.15.1:0/99998556 <== mon.2 v2:172.21.15.7:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7fd4ac007840 con 0x7fd4b40c6ea0 2024-05-09T08:01:59.374 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T08:01:59.374 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":10,"fsid":"54329ec8-3270-4043-9a82-c6cd7420aa7f","created":"2024-05-09T08:01:56.666970+0000","modified":"2024-05-09T08:01:59.008225+0000","last_up_change":"0.000000","last_in_change":"2024-05-09T08:01:59.008225+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"c064454e-a9b5-4a24-9a77-17f0a4085952","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"1a9ea687-2ff6-407a-949b-0e7e1083af21","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"c88af08b-7c9a-4dd8-9640-44138239addb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"c087b7e3-a41d-4333-9365-32e3bd8c8e6f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"b4576b9c-871d-47a8-8d36-65af525d0200","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"902b3d94-72f3-4499-b21a-e32453ef5144","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"ead1d771-1728-4f85-adb4-26be21cedff3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"3ab5a201-9394-48a8-8918-e95197893669","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T08:01:59.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:01:59.358+0000 7fd4ba600640 1 -- 172.21.15.1:0/99998556 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fd4b40c6ea0 msgr2=0x7fd4b40cb210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:01:59.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:01:59.358+0000 7fd4ba600640 1 --2- 172.21.15.1:0/99998556 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fd4b40c6ea0 0x7fd4b40cb210 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fd4ac009eb0 tx=0x7fd4ac004a40 comp rx=0 tx=0).stop 2024-05-09T08:01:59.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:01:59.359+0000 7fd4ba600640 1 -- 172.21.15.1:0/99998556 shutdown_connections 2024-05-09T08:01:59.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:01:59.359+0000 7fd4ba600640 1 --2- 172.21.15.1:0/99998556 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4b406ece0 0x7fd4b41ef7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:01:59.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:01:59.359+0000 7fd4ba600640 1 --2- 172.21.15.1:0/99998556 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fd4b40c6ea0 0x7fd4b40cb210 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:01:59.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:01:59.359+0000 7fd4ba600640 1 --2- 172.21.15.1:0/99998556 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd4b406dca0 0x7fd4b414e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:01:59.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:01:59.359+0000 7fd4ba600640 1 -- 172.21.15.1:0/99998556 >> 172.21.15.1:0/99998556 conn(0x7fd4b40cc070 msgr2=0x7fd4b414cb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:01:59.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:01:59.359+0000 7fd4ba600640 1 -- 172.21.15.1:0/99998556 shutdown_connections 2024-05-09T08:01:59.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:01:59.359+0000 7fd4ba600640 1 -- 172.21.15.1:0/99998556 wait complete. 2024-05-09T08:01:59.391 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-05-09T08:01:59.392 INFO:tasks.ceph:Waiting for OSDs to come up 2024-05-09T08:01:59.463 INFO:tasks.ceph.mgr.x.smithi007.stderr:2024-05-09T08:01:59.447+0000 7f934988f040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-09T08:01:59.518 INFO:tasks.ceph.mgr.x.smithi007.stderr:2024-05-09T08:01:59.501+0000 7f934988f040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-09T08:01:59.640 INFO:tasks.ceph.mgr.x.smithi007.stderr:2024-05-09T08:01:59.623+0000 7f934988f040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-09T08:01:59.668 INFO:tasks.ceph.osd.4.smithi007.stderr:2024-05-09T08:01:59.652+0000 7fef7a60f640 -1 Falling back to public interface 2024-05-09T08:01:59.680 INFO:tasks.ceph.osd.5.smithi007.stderr:2024-05-09T08:01:59.663+0000 7f1f2dd23640 -1 Falling back to public interface 2024-05-09T08:01:59.691 INFO:tasks.ceph.osd.6.smithi007.stderr:2024-05-09T08:01:59.675+0000 7fd8ecb88640 -1 Falling back to public interface 2024-05-09T08:01:59.698 INFO:tasks.ceph.mgr.y.smithi001.stderr:2024-05-09T08:01:59.681+0000 7fa06500d040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-09T08:01:59.700 INFO:tasks.ceph.osd.7.smithi007.stderr:2024-05-09T08:01:59.684+0000 7fe7a3420640 -1 Falling back to public interface 2024-05-09T08:01:59.735 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-05-09T08:01:59.718+0000 7fe0e83bc640 -1 Falling back to public interface 2024-05-09T08:01:59.758 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-05-09T08:01:59.741+0000 7f39d4216640 -1 Falling back to public interface 2024-05-09T08:01:59.773 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-05-09T08:01:59.757+0000 7ff335486640 -1 Falling back to public interface 2024-05-09T08:01:59.791 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-05-09T08:01:59.774+0000 7efe1e016640 -1 Falling back to public interface 2024-05-09T08:01:59.938 INFO:tasks.ceph.mgr.x.smithi007.stderr:2024-05-09T08:01:59.922+0000 7f934988f040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-09T08:01:59.988 INFO:tasks.ceph.mgr.x.smithi007.stderr:2024-05-09T08:01:59.972+0000 7f934988f040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-09T08:02:00.014 INFO:tasks.ceph.mgr.y.smithi001.stderr:2024-05-09T08:01:59.997+0000 7fa06500d040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-09T08:02:00.078 INFO:tasks.ceph.mgr.y.smithi001.stderr:2024-05-09T08:02:00.061+0000 7fa06500d040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-09T08:02:00.152 INFO:tasks.ceph.mgr.x.smithi007.stderr:2024-05-09T08:02:00.135+0000 7f934988f040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-09T08:02:00.198 DEBUG:teuthology.orchestra.run.smithi001:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-05-09T08:02:00.216 INFO:tasks.ceph.mgr.y.smithi001.stderr:2024-05-09T08:02:00.200+0000 7fa06500d040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-09T08:02:00.297 INFO:tasks.ceph.mgr.x.smithi007.stderr:2024-05-09T08:02:00.280+0000 7f934988f040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-09T08:02:00.349 INFO:tasks.ceph.mgr.x.smithi007.stderr:2024-05-09T08:02:00.332+0000 7f934988f040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-09T08:02:00.364 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T08:02:00.348+0000 7f15aac00640 1 Processor -- start 2024-05-09T08:02:00.364 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T08:02:00.348+0000 7f15aac00640 1 -- start start 2024-05-09T08:02:00.366 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T08:02:00.349+0000 7f15aac00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f15a4158fe0 0x7f15a4163ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:00.366 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T08:02:00.349+0000 7f15aac00640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f15a40c6bc0 con 0x7f15a40c7530 2024-05-09T08:02:00.366 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T08:02:00.349+0000 7f15aac00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f15a40c6ea0 con 0x7f15a41529a0 2024-05-09T08:02:00.366 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T08:02:00.349+0000 7f15aac00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f15a4071540 con 0x7f15a4158fe0 2024-05-09T08:02:00.366 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T08:02:00.349+0000 7f15a3e00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f15a41529a0 0x7f15a4071d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.1:38374/0 (socket says 172.21.15.1:38374) 2024-05-09T08:02:00.366 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T08:02:00.349+0000 7f15a3e00640 1 -- 172.21.15.1:0/3310159846 learned_addr learned my addr 172.21.15.1:0/3310159846 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T08:02:00.366 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T08:02:00.349+0000 7f15a8c00640 1 --2- 172.21.15.1:0/3310159846 >> v2:172.21.15.7:3300/0 conn(0x7f15a4158fe0 0x7f15a4163ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:00.367 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T08:02:00.350+0000 7f15a3400640 1 -- 172.21.15.1:0/3310159846 <== mon.1 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3866834639 0 0) 0x7f15a40c6ea0 con 0x7f15a41529a0 2024-05-09T08:02:00.368 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T08:02:00.350+0000 7f15a3400640 1 -- 172.21.15.1:0/3310159846 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1588003520 con 0x7f15a41529a0 2024-05-09T08:02:00.368 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T08:02:00.350+0000 7f15a3400640 1 -- 172.21.15.1:0/3310159846 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1476170407 0 0) 0x7f15a40c6bc0 con 0x7f15a40c7530 2024-05-09T08:02:00.368 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T08:02:00.350+0000 7f15a8c00640 1 -- 172.21.15.1:0/3310159846 >> v1:172.21.15.7:6789/0 conn(0x7f15a41529a0 legacy=0x7f15a4071d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:00.368 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T08:02:00.350+0000 7f15a8c00640 1 -- 172.21.15.1:0/3310159846 >> v1:172.21.15.1:6789/0 conn(0x7f15a40c7530 legacy=0x7f15a40cbb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:00.368 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T08:02:00.350+0000 7f15a8c00640 1 -- 172.21.15.1:0/3310159846 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15a4071820 con 0x7f15a4158fe0 2024-05-09T08:02:00.368 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T08:02:00.351+0000 7f15a8c00640 1 --2- 172.21.15.1:0/3310159846 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15a4158fe0 0x7f15a4163ef0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f159c0097d0 tx=0x7f159c02a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c86f9e380e3dc246 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:00.368 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T08:02:00.351+0000 7f15a3400640 1 -- 172.21.15.1:0/3310159846 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f159c037070 con 0x7f15a4158fe0 2024-05-09T08:02:00.368 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T08:02:00.351+0000 7f15a3400640 1 -- 172.21.15.1:0/3310159846 <== mon.1 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f159c032440 con 0x7f15a4158fe0 2024-05-09T08:02:00.368 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T08:02:00.351+0000 7f15a3400640 1 -- 172.21.15.1:0/3310159846 <== mon.1 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f159c032810 con 0x7f15a4158fe0 2024-05-09T08:02:00.369 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T08:02:00.352+0000 7f15aac00640 1 -- 172.21.15.1:0/3310159846 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15a4158fe0 msgr2=0x7f15a4163ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:00.369 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T08:02:00.352+0000 7f15aac00640 1 --2- 172.21.15.1:0/3310159846 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15a4158fe0 0x7f15a4163ef0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f159c0097d0 tx=0x7f159c02a270 comp rx=0 tx=0).stop 2024-05-09T08:02:00.369 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T08:02:00.352+0000 7f15aac00640 1 -- 172.21.15.1:0/3310159846 shutdown_connections 2024-05-09T08:02:00.369 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T08:02:00.352+0000 7f15aac00640 1 --2- 172.21.15.1:0/3310159846 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15a4158fe0 0x7f15a4163ef0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:00.369 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T08:02:00.352+0000 7f15aac00640 1 -- 172.21.15.1:0/3310159846 >> 172.21.15.1:0/3310159846 conn(0x7f15a4066f10 msgr2=0x7f15a40672f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:02:00.369 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T08:02:00.352+0000 7f15aac00640 1 -- 172.21.15.1:0/3310159846 shutdown_connections 2024-05-09T08:02:00.369 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T08:02:00.352+0000 7f15aac00640 1 -- 172.21.15.1:0/3310159846 wait complete. 2024-05-09T08:02:00.370 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T08:02:00.353+0000 7f15aac00640 1 Processor -- start 2024-05-09T08:02:00.370 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T08:02:00.354+0000 7f15aac00640 1 -- start start 2024-05-09T08:02:00.372 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T08:02:00.354+0000 7f15aac00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f15a40c7530 0x7f15a41e43e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:00.372 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T08:02:00.354+0000 7f15aac00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15a41529a0 0x7f15a414bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:00.372 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T08:02:00.354+0000 7f15aac00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f15a4158fe0 0x7f15a41e29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:00.372 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T08:02:00.354+0000 7f15aac00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f15a4071170 con 0x7f15a4158fe0 2024-05-09T08:02:00.372 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T08:02:00.354+0000 7f15aac00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f15a4070ff0 con 0x7f15a41529a0 2024-05-09T08:02:00.372 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T08:02:00.354+0000 7f15aac00640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f15a40cc220 con 0x7f15a40c7530 2024-05-09T08:02:00.372 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T08:02:00.355+0000 7f15a3e00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f15a40c7530 0x7f15a41e43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:00.372 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T08:02:00.355+0000 7f15a3e00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f15a40c7530 0x7f15a41e43e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3301/0 says I am v2:172.21.15.1:54190/0 (socket says 172.21.15.1:54190) 2024-05-09T08:02:00.372 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T08:02:00.355+0000 7f15a3e00640 1 -- 172.21.15.1:0/4003352796 learned_addr learned my addr 172.21.15.1:0/4003352796 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:02:00.372 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T08:02:00.355+0000 7f159b400640 1 --2- 172.21.15.1:0/4003352796 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15a41529a0 0x7f15a414bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:00.373 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T08:02:00.355+0000 7f15a8c00640 1 --2- 172.21.15.1:0/4003352796 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f15a4158fe0 0x7f15a41e29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:00.373 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T08:02:00.355+0000 7f15a3e00640 1 -- 172.21.15.1:0/4003352796 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15a41529a0 msgr2=0x7f15a414bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:00.373 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T08:02:00.355+0000 7f15a3e00640 1 --2- 172.21.15.1:0/4003352796 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15a41529a0 0x7f15a414bc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:00.373 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T08:02:00.355+0000 7f15a3e00640 1 -- 172.21.15.1:0/4003352796 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f15a4158fe0 msgr2=0x7f15a41e29e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:00.373 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T08:02:00.355+0000 7f15a3e00640 1 --2- 172.21.15.1:0/4003352796 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f15a4158fe0 0x7f15a41e29e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:00.373 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T08:02:00.355+0000 7f15a3e00640 1 -- 172.21.15.1:0/4003352796 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15a40ca7f0 con 0x7f15a40c7530 2024-05-09T08:02:00.373 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T08:02:00.355+0000 7f159b400640 1 --2- 172.21.15.1:0/4003352796 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15a41529a0 0x7f15a414bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:02:00.373 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T08:02:00.356+0000 7f15a3e00640 1 --2- 172.21.15.1:0/4003352796 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f15a40c7530 0x7f15a41e43e0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f159400c410 tx=0x7f159400c8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:00.374 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T08:02:00.356+0000 7f15a0c00640 1 -- 172.21.15.1:0/4003352796 <== mon.2 v2:172.21.15.7:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1594017070 con 0x7f15a40c7530 2024-05-09T08:02:00.374 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T08:02:00.356+0000 7f15a0c00640 1 -- 172.21.15.1:0/4003352796 <== mon.2 v2:172.21.15.7:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1594005ce0 con 0x7f15a40c7530 2024-05-09T08:02:00.374 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T08:02:00.356+0000 7f15aac00640 1 -- 172.21.15.1:0/4003352796 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15a40c16e0 con 0x7f15a40c7530 2024-05-09T08:02:00.374 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T08:02:00.356+0000 7f15a0c00640 1 -- 172.21.15.1:0/4003352796 <== mon.2 v2:172.21.15.7:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15940072f0 con 0x7f15a40c7530 2024-05-09T08:02:00.374 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T08:02:00.356+0000 7f15aac00640 1 -- 172.21.15.1:0/4003352796 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f15a40cacb0 con 0x7f15a40c7530 2024-05-09T08:02:00.374 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T08:02:00.356+0000 7f15a0c00640 1 -- 172.21.15.1:0/4003352796 <== mon.2 v2:172.21.15.7:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f159401a070 con 0x7f15a40c7530 2024-05-09T08:02:00.375 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T08:02:00.356+0000 7f15a0c00640 1 -- 172.21.15.1:0/4003352796 <== mon.2 v2:172.21.15.7:3301/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f1594007490 con 0x7f15a40c7530 2024-05-09T08:02:00.375 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T08:02:00.357+0000 7f15aac00640 1 -- 172.21.15.1:0/4003352796 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15a404f5f0 con 0x7f15a40c7530 2024-05-09T08:02:00.376 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T08:02:00.360+0000 7f15a0c00640 1 -- 172.21.15.1:0/4003352796 <== mon.2 v2:172.21.15.7:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f1594021080 con 0x7f15a40c7530 2024-05-09T08:02:00.423 INFO:tasks.ceph.mgr.x.smithi007.stderr:2024-05-09T08:02:00.407+0000 7f934988f040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-09T08:02:00.441 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T08:02:00.425+0000 7f15aac00640 1 -- 172.21.15.1:0/4003352796 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f15a414fb40 con 0x7f15a40c7530 2024-05-09T08:02:00.442 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T08:02:00.426+0000 7f15a0c00640 1 -- 172.21.15.1:0/4003352796 <== mon.2 v2:172.21.15.7:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f15a40ca7f0 con 0x7f15a40c7530 2024-05-09T08:02:00.443 INFO:teuthology.misc.health.smithi001.stdout: 2024-05-09T08:02:00.443 INFO:teuthology.misc.health.smithi001.stdout:{"epoch":10,"fsid":"54329ec8-3270-4043-9a82-c6cd7420aa7f","created":"2024-05-09T08:01:56.666970+0000","modified":"2024-05-09T08:01:59.008225+0000","last_up_change":"0.000000","last_in_change":"2024-05-09T08:01:59.008225+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"c064454e-a9b5-4a24-9a77-17f0a4085952","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"1a9ea687-2ff6-407a-949b-0e7e1083af21","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"c88af08b-7c9a-4dd8-9640-44138239addb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"c087b7e3-a41d-4333-9365-32e3bd8c8e6f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"b4576b9c-871d-47a8-8d36-65af525d0200","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"902b3d94-72f3-4499-b21a-e32453ef5144","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"ead1d771-1728-4f85-adb4-26be21cedff3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"3ab5a201-9394-48a8-8918-e95197893669","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T08:02:00.444 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T08:02:00.427+0000 7f15aac00640 1 -- 172.21.15.1:0/4003352796 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f15a40c7530 msgr2=0x7f15a41e43e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:00.444 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T08:02:00.427+0000 7f15aac00640 1 --2- 172.21.15.1:0/4003352796 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f15a40c7530 0x7f15a41e43e0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f159400c410 tx=0x7f159400c8d0 comp rx=0 tx=0).stop 2024-05-09T08:02:00.444 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T08:02:00.427+0000 7f15aac00640 1 -- 172.21.15.1:0/4003352796 shutdown_connections 2024-05-09T08:02:00.444 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T08:02:00.427+0000 7f15aac00640 1 --2- 172.21.15.1:0/4003352796 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15a41529a0 0x7f15a414bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:00.444 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T08:02:00.427+0000 7f15aac00640 1 --2- 172.21.15.1:0/4003352796 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f15a4158fe0 0x7f15a41e29e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:00.444 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T08:02:00.427+0000 7f15aac00640 1 --2- 172.21.15.1:0/4003352796 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f15a40c7530 0x7f15a41e43e0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:00.444 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T08:02:00.427+0000 7f15aac00640 1 -- 172.21.15.1:0/4003352796 >> 172.21.15.1:0/4003352796 conn(0x7f15a4066f10 msgr2=0x7f15a4156c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:02:00.444 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T08:02:00.427+0000 7f15aac00640 1 -- 172.21.15.1:0/4003352796 shutdown_connections 2024-05-09T08:02:00.444 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T08:02:00.427+0000 7f15aac00640 1 -- 172.21.15.1:0/4003352796 wait complete. 2024-05-09T08:02:00.456 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-05-09T08:02:00.471 INFO:tasks.ceph.mgr.x.smithi007.stderr:2024-05-09T08:02:00.454+0000 7f934988f040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-09T08:02:00.577 INFO:tasks.ceph.mgr.y.smithi001.stderr:2024-05-09T08:02:00.561+0000 7fa06500d040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-09T08:02:00.637 INFO:tasks.ceph.mgr.y.smithi001.stderr:2024-05-09T08:02:00.620+0000 7fa06500d040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-09T08:02:00.828 INFO:tasks.ceph.mgr.y.smithi001.stderr:2024-05-09T08:02:00.812+0000 7fa06500d040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-09T08:02:00.974 INFO:tasks.ceph.mgr.x.smithi007.stderr:2024-05-09T08:02:00.958+0000 7f934988f040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-09T08:02:01.003 INFO:tasks.ceph.mgr.y.smithi001.stderr:2024-05-09T08:02:00.986+0000 7fa06500d040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-09T08:02:01.064 INFO:tasks.ceph.mgr.y.smithi001.stderr:2024-05-09T08:02:01.047+0000 7fa06500d040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-09T08:02:01.096 INFO:tasks.ceph.mgr.x.smithi007.stderr:2024-05-09T08:02:01.080+0000 7f934988f040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-09T08:02:01.155 INFO:tasks.ceph.mgr.y.smithi001.stderr:2024-05-09T08:02:01.138+0000 7fa06500d040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-09T08:02:01.211 INFO:tasks.ceph.mgr.y.smithi001.stderr:2024-05-09T08:02:01.194+0000 7fa06500d040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-09T08:02:01.775 INFO:tasks.ceph.mgr.y.smithi001.stderr:2024-05-09T08:02:01.759+0000 7fa06500d040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-09T08:02:01.912 INFO:tasks.ceph.mgr.y.smithi001.stderr:2024-05-09T08:02:01.895+0000 7fa06500d040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-09T08:02:02.118 INFO:tasks.ceph.mgr.x.smithi007.stderr:2024-05-09T08:02:02.102+0000 7f92eac8f640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-05-09T08:02:02.119 INFO:tasks.ceph.mgr.x.smithi007.stderr:2024-05-09T08:02:02.102+0000 7f92eac8f640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-05-09T08:02:03.109 INFO:tasks.ceph.osd.4.smithi007.stderr:2024-05-09T08:02:03.093+0000 7fef7a60f640 -1 osd.4 0 log_to_monitors true 2024-05-09T08:02:03.158 INFO:tasks.ceph.osd.5.smithi007.stderr:2024-05-09T08:02:03.141+0000 7f1f2dd23640 -1 osd.5 0 log_to_monitors true 2024-05-09T08:02:03.186 INFO:tasks.ceph.osd.7.smithi007.stderr:2024-05-09T08:02:03.169+0000 7fe7a3420640 -1 osd.7 0 log_to_monitors true 2024-05-09T08:02:03.203 INFO:tasks.ceph.osd.6.smithi007.stderr:2024-05-09T08:02:03.187+0000 7fd8ecb88640 -1 osd.6 0 log_to_monitors true 2024-05-09T08:02:03.408 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-05-09T08:02:03.391+0000 7efe1e016640 -1 osd.3 0 log_to_monitors true 2024-05-09T08:02:03.419 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-05-09T08:02:03.403+0000 7ff335486640 -1 osd.1 0 log_to_monitors true 2024-05-09T08:02:03.578 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-05-09T08:02:03.562+0000 7fe0e83bc640 -1 osd.0 0 log_to_monitors true 2024-05-09T08:02:03.641 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-05-09T08:02:03.625+0000 7f39d4216640 -1 osd.2 0 log_to_monitors true 2024-05-09T08:02:06.158 INFO:tasks.ceph.osd.5.smithi007.stderr:2024-05-09T08:02:06.141+0000 7f1f29cb9640 -1 osd.5 0 waiting for initial osdmap 2024-05-09T08:02:06.173 INFO:tasks.ceph.osd.5.smithi007.stderr:2024-05-09T08:02:06.157+0000 7f1f21ad9640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T08:02:06.189 INFO:tasks.ceph.osd.7.smithi007.stderr:2024-05-09T08:02:06.173+0000 7fe79ed17640 -1 osd.7 0 waiting for initial osdmap 2024-05-09T08:02:06.190 INFO:tasks.ceph.osd.6.smithi007.stderr:2024-05-09T08:02:06.174+0000 7fd8e8b1e640 -1 osd.6 0 waiting for initial osdmap 2024-05-09T08:02:06.194 INFO:tasks.ceph.osd.4.smithi007.stderr:2024-05-09T08:02:06.177+0000 7fef75f98640 -1 osd.4 0 waiting for initial osdmap 2024-05-09T08:02:06.203 INFO:tasks.ceph.osd.4.smithi007.stderr:2024-05-09T08:02:06.187+0000 7fef6ddb8640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T08:02:06.203 INFO:tasks.ceph.osd.6.smithi007.stderr:2024-05-09T08:02:06.187+0000 7fd8e093e640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T08:02:06.204 INFO:tasks.ceph.osd.7.smithi007.stderr:2024-05-09T08:02:06.187+0000 7fe796b37640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T08:02:06.477 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-05-09T08:02:06.461+0000 7fe0e3c00640 -1 osd.0 0 waiting for initial osdmap 2024-05-09T08:02:06.483 INFO:tasks.ceph.osd.0.smithi001.stderr:2024-05-09T08:02:06.467+0000 7fe0d9200640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T08:02:06.501 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-05-09T08:02:06.485+0000 7ff330400640 -1 osd.1 0 waiting for initial osdmap 2024-05-09T08:02:06.503 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-05-09T08:02:06.486+0000 7f39cea00640 -1 osd.2 0 waiting for initial osdmap 2024-05-09T08:02:06.507 INFO:tasks.ceph.osd.1.smithi001.stderr:2024-05-09T08:02:06.490+0000 7ff326400640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T08:02:06.510 INFO:tasks.ceph.osd.2.smithi001.stderr:2024-05-09T08:02:06.494+0000 7f39c4a00640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T08:02:06.524 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-05-09T08:02:06.507+0000 7efe18800640 -1 osd.3 0 waiting for initial osdmap 2024-05-09T08:02:06.533 INFO:tasks.ceph.osd.3.smithi001.stderr:2024-05-09T08:02:06.516+0000 7efe0e800640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T08:02:07.264 DEBUG:teuthology.orchestra.run.smithi001:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-05-09T08:02:07.452 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T08:02:07.435+0000 7fa125c00640 1 Processor -- start 2024-05-09T08:02:07.452 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T08:02:07.435+0000 7fa125c00640 1 -- start start 2024-05-09T08:02:07.454 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T08:02:07.437+0000 7fa125c00640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fa120152310 0x7fa1200c4e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:07.454 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T08:02:07.437+0000 7fa125c00640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa120158d00 con 0x7fa1201526f0 2024-05-09T08:02:07.454 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T08:02:07.437+0000 7fa125c00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa120158fe0 con 0x7fa120067920 2024-05-09T08:02:07.454 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T08:02:07.437+0000 7fa125c00640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7fa1200c40e0 con 0x7fa120152310 2024-05-09T08:02:07.454 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T08:02:07.437+0000 7fa11fe00640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fa120152310 0x7fa1200c4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:07.454 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T08:02:07.437+0000 7fa11fe00640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fa120152310 0x7fa1200c4e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:47392/0 (socket says 172.21.15.1:47392) 2024-05-09T08:02:07.455 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T08:02:07.437+0000 7fa11fe00640 1 -- 172.21.15.1:0/3657160249 learned_addr learned my addr 172.21.15.1:0/3657160249 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:02:07.455 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T08:02:07.438+0000 7fa11fe00640 1 -- 172.21.15.1:0/3657160249 >> v1:172.21.15.7:6789/0 conn(0x7fa120067920 legacy=0x7fa1201581e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:07.456 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T08:02:07.438+0000 7fa11fe00640 1 -- 172.21.15.1:0/3657160249 >> v1:172.21.15.1:6789/0 conn(0x7fa1201526f0 legacy=0x7fa1200c94e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:07.456 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T08:02:07.438+0000 7fa11fe00640 1 -- 172.21.15.1:0/3657160249 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1200c4330 con 0x7fa120152310 2024-05-09T08:02:07.456 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T08:02:07.438+0000 7fa11f400640 1 -- 172.21.15.1:0/3657160249 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1518657017 0 0) 0x7fa120158d00 con 0x7fa1201526f0 2024-05-09T08:02:07.456 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T08:02:07.439+0000 7fa11fe00640 1 --2- 172.21.15.1:0/3657160249 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa120152310 0x7fa1200c4e10 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fa110009870 tx=0x7fa11002a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3d83f1e04d25485 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:07.457 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T08:02:07.439+0000 7fa11f400640 1 -- 172.21.15.1:0/3657160249 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa110037070 con 0x7fa120152310 2024-05-09T08:02:07.457 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T08:02:07.439+0000 7fa11f400640 1 -- 172.21.15.1:0/3657160249 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa11002ae50 con 0x7fa120152310 2024-05-09T08:02:07.457 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T08:02:07.439+0000 7fa11f400640 1 -- 172.21.15.1:0/3657160249 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa110032860 con 0x7fa120152310 2024-05-09T08:02:07.457 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T08:02:07.440+0000 7fa125c00640 1 -- 172.21.15.1:0/3657160249 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa120152310 msgr2=0x7fa1200c4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:07.457 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T08:02:07.440+0000 7fa125c00640 1 --2- 172.21.15.1:0/3657160249 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa120152310 0x7fa1200c4e10 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fa110009870 tx=0x7fa11002a300 comp rx=0 tx=0).stop 2024-05-09T08:02:07.457 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T08:02:07.440+0000 7fa125c00640 1 -- 172.21.15.1:0/3657160249 shutdown_connections 2024-05-09T08:02:07.457 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T08:02:07.440+0000 7fa125c00640 1 --2- 172.21.15.1:0/3657160249 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa120152310 0x7fa1200c4e10 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:07.457 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T08:02:07.440+0000 7fa125c00640 1 -- 172.21.15.1:0/3657160249 >> 172.21.15.1:0/3657160249 conn(0x7fa12006d490 msgr2=0x7fa12006d870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:02:07.457 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T08:02:07.440+0000 7fa125c00640 1 -- 172.21.15.1:0/3657160249 shutdown_connections 2024-05-09T08:02:07.457 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T08:02:07.440+0000 7fa125c00640 1 -- 172.21.15.1:0/3657160249 wait complete. 2024-05-09T08:02:07.458 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T08:02:07.442+0000 7fa125c00640 1 Processor -- start 2024-05-09T08:02:07.458 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T08:02:07.442+0000 7fa125c00640 1 -- start start 2024-05-09T08:02:07.461 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T08:02:07.443+0000 7fa125c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa120067920 0x7fa1201471f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:07.461 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T08:02:07.443+0000 7fa125c00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fa120152310 0x7fa1200cc4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:07.461 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T08:02:07.443+0000 7fa125c00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa1201526f0 0x7fa1201f3de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:07.461 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T08:02:07.443+0000 7fa125c00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fa1201588f0 con 0x7fa1201526f0 2024-05-09T08:02:07.461 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T08:02:07.443+0000 7fa125c00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa120067d50 con 0x7fa120067920 2024-05-09T08:02:07.461 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T08:02:07.443+0000 7fa125c00640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7fa120158a70 con 0x7fa120152310 2024-05-09T08:02:07.461 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T08:02:07.443+0000 7fa11fe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa120067920 0x7fa1201471f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:07.461 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T08:02:07.443+0000 7fa125200640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa1201526f0 0x7fa1201f3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:07.462 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T08:02:07.443+0000 7fa117e00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fa120152310 0x7fa1200cc4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:07.462 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T08:02:07.443+0000 7fa11fe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa120067920 0x7fa1201471f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.1:36858/0 (socket says 172.21.15.1:36858) 2024-05-09T08:02:07.462 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T08:02:07.443+0000 7fa11fe00640 1 -- 172.21.15.1:0/4145573545 learned_addr learned my addr 172.21.15.1:0/4145573545 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:02:07.462 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T08:02:07.443+0000 7fa11fe00640 1 -- 172.21.15.1:0/4145573545 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fa120152310 msgr2=0x7fa1200cc4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:07.462 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T08:02:07.443+0000 7fa11fe00640 1 --2- 172.21.15.1:0/4145573545 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fa120152310 0x7fa1200cc4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:07.462 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T08:02:07.443+0000 7fa11fe00640 1 -- 172.21.15.1:0/4145573545 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa1201526f0 msgr2=0x7fa1201f3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:07.462 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T08:02:07.443+0000 7fa11fe00640 1 --2- 172.21.15.1:0/4145573545 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa1201526f0 0x7fa1201f3de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:07.462 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T08:02:07.444+0000 7fa11fe00640 1 -- 172.21.15.1:0/4145573545 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa120146330 con 0x7fa120067920 2024-05-09T08:02:07.462 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T08:02:07.444+0000 7fa117e00640 1 --2- 172.21.15.1:0/4145573545 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fa120152310 0x7fa1200cc4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:02:07.462 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T08:02:07.444+0000 7fa125200640 1 --2- 172.21.15.1:0/4145573545 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa1201526f0 0x7fa1201f3de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:02:07.462 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T08:02:07.444+0000 7fa11fe00640 1 --2- 172.21.15.1:0/4145573545 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa120067920 0x7fa1201471f0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa110009870 tx=0x7fa1100046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:07.462 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T08:02:07.444+0000 7fa11cc00640 1 -- 172.21.15.1:0/4145573545 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa110041070 con 0x7fa120067920 2024-05-09T08:02:07.462 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T08:02:07.444+0000 7fa11cc00640 1 -- 172.21.15.1:0/4145573545 <== mon.1 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa11003c870 con 0x7fa120067920 2024-05-09T08:02:07.463 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T08:02:07.444+0000 7fa11cc00640 1 -- 172.21.15.1:0/4145573545 <== mon.1 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa110037040 con 0x7fa120067920 2024-05-09T08:02:07.463 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T08:02:07.444+0000 7fa125c00640 1 -- 172.21.15.1:0/4145573545 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa120159ab0 con 0x7fa120067920 2024-05-09T08:02:07.463 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T08:02:07.444+0000 7fa125c00640 1 -- 172.21.15.1:0/4145573545 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1201595d0 con 0x7fa120067920 2024-05-09T08:02:07.463 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T08:02:07.445+0000 7fa125c00640 1 -- 172.21.15.1:0/4145573545 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa120075390 con 0x7fa120067920 2024-05-09T08:02:07.463 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T08:02:07.446+0000 7fa11cc00640 1 -- 172.21.15.1:0/4145573545 <== mon.1 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 4) ==== 90442+0+0 (secure 0 0 0) 0x7fa110044070 con 0x7fa120067920 2024-05-09T08:02:07.464 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T08:02:07.447+0000 7fa11cc00640 1 --2- 172.21.15.1:0/4145573545 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fa0fc06bda0 0x7fa0fc06e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:07.464 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T08:02:07.447+0000 7fa11cc00640 1 -- 172.21.15.1:0/4145573545 <== mon.1 v2:172.21.15.7:3300/0 5 ==== osd_map(13..13 src has 1..13) ==== 4471+0+0 (secure 0 0 0) 0x7fa1100ab5e0 con 0x7fa120067920 2024-05-09T08:02:07.464 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T08:02:07.447+0000 7fa117e00640 1 --2- 172.21.15.1:0/4145573545 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fa0fc06bda0 0x7fa0fc06e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:07.464 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T08:02:07.448+0000 7fa117e00640 1 --2- 172.21.15.1:0/4145573545 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fa0fc06bda0 0x7fa0fc06e260 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fa120072ad0 tx=0x7fa108009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:07.469 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T08:02:07.452+0000 7fa11cc00640 1 -- 172.21.15.1:0/4145573545 <== mon.1 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fa120075390 con 0x7fa120067920 2024-05-09T08:02:07.664 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T08:02:07.647+0000 7fa125c00640 1 -- 172.21.15.1:0/4145573545 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa1200674f0 con 0x7fa120067920 2024-05-09T08:02:07.665 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T08:02:07.648+0000 7fa11cc00640 1 -- 172.21.15.1:0/4145573545 <== mon.1 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v13) ==== 74+0+10777 (secure 0 0 0) 0x7fa1200674f0 con 0x7fa120067920 2024-05-09T08:02:07.666 INFO:teuthology.misc.health.smithi001.stdout: 2024-05-09T08:02:07.666 INFO:teuthology.misc.health.smithi001.stdout:{"epoch":13,"fsid":"54329ec8-3270-4043-9a82-c6cd7420aa7f","created":"2024-05-09T08:01:56.666970+0000","modified":"2024-05-09T08:02:07.146082+0000","last_up_change":"2024-05-09T08:02:07.146082+0000","last_in_change":"2024-05-09T08:01:59.008225+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"c064454e-a9b5-4a24-9a77-17f0a4085952","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6800","nonce":1513349547},{"type":"v1","addr":"172.21.15.1:6801","nonce":1513349547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":1513349547},{"type":"v1","addr":"172.21.15.1:6803","nonce":1513349547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":1513349547},{"type":"v1","addr":"172.21.15.1:6807","nonce":1513349547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":1513349547},{"type":"v1","addr":"172.21.15.1:6805","nonce":1513349547}]},"public_addr":"172.21.15.1:6801/1513349547","cluster_addr":"172.21.15.1:6803/1513349547","heartbeat_back_addr":"172.21.15.1:6807/1513349547","heartbeat_front_addr":"172.21.15.1:6805/1513349547","state":["exists","up"]},{"osd":1,"uuid":"1a9ea687-2ff6-407a-949b-0e7e1083af21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":2942255259},{"type":"v1","addr":"172.21.15.1:6817","nonce":2942255259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":2942255259},{"type":"v1","addr":"172.21.15.1:6819","nonce":2942255259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":2942255259},{"type":"v1","addr":"172.21.15.1:6823","nonce":2942255259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6820","nonce":2942255259},{"type":"v1","addr":"172.21.15.1:6821","nonce":2942255259}]},"public_addr":"172.21.15.1:6817/2942255259","cluster_addr":"172.21.15.1:6819/2942255259","heartbeat_back_addr":"172.21.15.1:6823/2942255259","heartbeat_front_addr":"172.21.15.1:6821/2942255259","state":["exists","up"]},{"osd":2,"uuid":"c88af08b-7c9a-4dd8-9640-44138239addb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":2925031811},{"type":"v1","addr":"172.21.15.1:6809","nonce":2925031811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":2925031811},{"type":"v1","addr":"172.21.15.1:6811","nonce":2925031811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":2925031811},{"type":"v1","addr":"172.21.15.1:6815","nonce":2925031811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":2925031811},{"type":"v1","addr":"172.21.15.1:6813","nonce":2925031811}]},"public_addr":"172.21.15.1:6809/2925031811","cluster_addr":"172.21.15.1:6811/2925031811","heartbeat_back_addr":"172.21.15.1:6815/2925031811","heartbeat_front_addr":"172.21.15.1:6813/2925031811","state":["exists","up"]},{"osd":3,"uuid":"c087b7e3-a41d-4333-9365-32e3bd8c8e6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6824","nonce":2497776654},{"type":"v1","addr":"172.21.15.1:6825","nonce":2497776654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":2497776654},{"type":"v1","addr":"172.21.15.1:6827","nonce":2497776654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":2497776654},{"type":"v1","addr":"172.21.15.1:6831","nonce":2497776654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6828","nonce":2497776654},{"type":"v1","addr":"172.21.15.1:6829","nonce":2497776654}]},"public_addr":"172.21.15.1:6825/2497776654","cluster_addr":"172.21.15.1:6827/2497776654","heartbeat_back_addr":"172.21.15.1:6831/2497776654","heartbeat_front_addr":"172.21.15.1:6829/2497776654","state":["exists","up"]},{"osd":4,"uuid":"b4576b9c-871d-47a8-8d36-65af525d0200","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":2957026670},{"type":"v1","addr":"172.21.15.7:6801","nonce":2957026670}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2957026670},{"type":"v1","addr":"172.21.15.7:6803","nonce":2957026670}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":2957026670},{"type":"v1","addr":"172.21.15.7:6807","nonce":2957026670}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2957026670},{"type":"v1","addr":"172.21.15.7:6805","nonce":2957026670}]},"public_addr":"172.21.15.7:6801/2957026670","cluster_addr":"172.21.15.7:6803/2957026670","heartbeat_back_addr":"172.21.15.7:6807/2957026670","heartbeat_front_addr":"172.21.15.7:6805/2957026670","state":["exists","up"]},{"osd":5,"uuid":"902b3d94-72f3-4499-b21a-e32453ef5144","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1670295242},{"type":"v1","addr":"172.21.15.7:6809","nonce":1670295242}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1670295242},{"type":"v1","addr":"172.21.15.7:6811","nonce":1670295242}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1670295242},{"type":"v1","addr":"172.21.15.7:6815","nonce":1670295242}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1670295242},{"type":"v1","addr":"172.21.15.7:6813","nonce":1670295242}]},"public_addr":"172.21.15.7:6809/1670295242","cluster_addr":"172.21.15.7:6811/1670295242","heartbeat_back_addr":"172.21.15.7:6815/1670295242","heartbeat_front_addr":"172.21.15.7:6813/1670295242","state":["exists","up"]},{"osd":6,"uuid":"ead1d771-1728-4f85-adb4-26be21cedff3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":2852086880},{"type":"v1","addr":"172.21.15.7:6817","nonce":2852086880}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":2852086880},{"type":"v1","addr":"172.21.15.7:6819","nonce":2852086880}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":2852086880},{"type":"v1","addr":"172.21.15.7:6823","nonce":2852086880}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":2852086880},{"type":"v1","addr":"172.21.15.7:6821","nonce":2852086880}]},"public_addr":"172.21.15.7:6817/2852086880","cluster_addr":"172.21.15.7:6819/2852086880","heartbeat_back_addr":"172.21.15.7:6823/2852086880","heartbeat_front_addr":"172.21.15.7:6821/2852086880","state":["exists","up"]},{"osd":7,"uuid":"3ab5a201-9394-48a8-8918-e95197893669","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":1442884138},{"type":"v1","addr":"172.21.15.7:6825","nonce":1442884138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1442884138},{"type":"v1","addr":"172.21.15.7:6827","nonce":1442884138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1442884138},{"type":"v1","addr":"172.21.15.7:6831","nonce":1442884138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1442884138},{"type":"v1","addr":"172.21.15.7:6829","nonce":1442884138}]},"public_addr":"172.21.15.7:6825/1442884138","cluster_addr":"172.21.15.7:6827/1442884138","heartbeat_back_addr":"172.21.15.7:6831/1442884138","heartbeat_front_addr":"172.21.15.7:6829/1442884138","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T08:02:07.670 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T08:02:07.651+0000 7fa125c00640 1 -- 172.21.15.1:0/4145573545 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fa0fc06bda0 msgr2=0x7fa0fc06e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:07.670 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T08:02:07.651+0000 7fa125c00640 1 --2- 172.21.15.1:0/4145573545 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fa0fc06bda0 0x7fa0fc06e260 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fa120072ad0 tx=0x7fa108009040 comp rx=0 tx=0).stop 2024-05-09T08:02:07.670 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T08:02:07.651+0000 7fa125c00640 1 -- 172.21.15.1:0/4145573545 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa120067920 msgr2=0x7fa1201471f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:07.670 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T08:02:07.651+0000 7fa125c00640 1 --2- 172.21.15.1:0/4145573545 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa120067920 0x7fa1201471f0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa110009870 tx=0x7fa1100046c0 comp rx=0 tx=0).stop 2024-05-09T08:02:07.670 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T08:02:07.651+0000 7fa125c00640 1 -- 172.21.15.1:0/4145573545 shutdown_connections 2024-05-09T08:02:07.670 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T08:02:07.651+0000 7fa125c00640 1 --2- 172.21.15.1:0/4145573545 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fa0fc06bda0 0x7fa0fc06e260 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:07.670 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T08:02:07.651+0000 7fa125c00640 1 --2- 172.21.15.1:0/4145573545 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa1201526f0 0x7fa1201f3de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:07.670 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T08:02:07.651+0000 7fa125c00640 1 --2- 172.21.15.1:0/4145573545 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fa120152310 0x7fa1200cc4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:07.670 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T08:02:07.651+0000 7fa125c00640 1 --2- 172.21.15.1:0/4145573545 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa120067920 0x7fa1201471f0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:07.670 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T08:02:07.651+0000 7fa125c00640 1 -- 172.21.15.1:0/4145573545 >> 172.21.15.1:0/4145573545 conn(0x7fa12006d490 msgr2=0x7fa120156920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:02:07.670 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T08:02:07.652+0000 7fa125c00640 1 -- 172.21.15.1:0/4145573545 shutdown_connections 2024-05-09T08:02:07.671 INFO:teuthology.misc.health.smithi001.stderr:2024-05-09T08:02:07.652+0000 7fa125c00640 1 -- 172.21.15.1:0/4145573545 wait complete. 2024-05-09T08:02:07.682 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-05-09T08:02:07.683 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-05-09T08:02:07.683 INFO:tasks.ceph.mds.a:Restarting daemon 2024-05-09T08:02:07.683 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-05-09T08:02:07.690 INFO:tasks.ceph.mds.a:Started 2024-05-09T08:02:07.690 INFO:tasks.ceph.mds.c:Restarting daemon 2024-05-09T08:02:07.691 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-05-09T08:02:07.697 INFO:tasks.ceph.mds.c:Started 2024-05-09T08:02:07.698 INFO:tasks.ceph.mds.e:Restarting daemon 2024-05-09T08:02:07.698 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i e 2024-05-09T08:02:07.704 INFO:tasks.ceph.mds.e:Started 2024-05-09T08:02:07.704 INFO:tasks.ceph.mds.b:Restarting daemon 2024-05-09T08:02:07.705 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-05-09T08:02:07.707 INFO:tasks.ceph.mds.b:Started 2024-05-09T08:02:07.708 INFO:tasks.ceph.mds.d:Restarting daemon 2024-05-09T08:02:07.708 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-05-09T08:02:07.710 INFO:tasks.ceph.mds.d:Started 2024-05-09T08:02:07.710 INFO:tasks.ceph.mds.f:Restarting daemon 2024-05-09T08:02:07.710 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i f 2024-05-09T08:02:07.712 INFO:tasks.ceph.mds.f:Started 2024-05-09T08:02:07.712 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-05-09T08:02:07.713 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T08:02:07.799 INFO:tasks.ceph.mds.d.smithi007.stdout:starting mds.d at 2024-05-09T08:02:07.799 INFO:tasks.ceph.mds.b.smithi007.stdout:starting mds.b at 2024-05-09T08:02:07.800 INFO:tasks.ceph.mds.f.smithi007.stdout:starting mds.f at 2024-05-09T08:02:07.842 INFO:tasks.ceph.mds.a.smithi001.stdout:starting mds.a at 2024-05-09T08:02:07.844 INFO:tasks.ceph.mds.c.smithi001.stdout:starting mds.c at 2024-05-09T08:02:07.846 INFO:tasks.ceph.mds.e.smithi001.stdout:starting mds.e at 2024-05-09T08:02:07.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:07.873+0000 7feb7f200640 1 Processor -- start 2024-05-09T08:02:07.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:07.873+0000 7feb7f200640 1 -- start start 2024-05-09T08:02:07.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:07.874+0000 7feb7f200640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7feb780cd040 0x7feb7815ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:07.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:07.874+0000 7feb7f200640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7feb7806d350 0x7feb7806da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:07.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:07.874+0000 7feb7f200640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feb7815b550 con 0x7feb7815a5e0 2024-05-09T08:02:07.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:07.874+0000 7feb7f200640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7feb7806cec0 con 0x7feb7806d350 2024-05-09T08:02:07.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:07.874+0000 7feb7f200640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7feb7806d7b0 con 0x7feb780cd040 2024-05-09T08:02:07.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:07.874+0000 7feb7d400640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7feb7806d350 0x7feb7806da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:07.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:07.874+0000 7feb7de00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7feb780cd040 0x7feb7815ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:07.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:07.874+0000 7feb7d400640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7feb7806d350 0x7feb7806da60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:47440/0 (socket says 172.21.15.1:47440) 2024-05-09T08:02:07.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:07.874+0000 7feb7d400640 1 -- 172.21.15.1:0/2128461442 learned_addr learned my addr 172.21.15.1:0/2128461442 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:02:07.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:07.875+0000 7feb7de00640 1 -- 172.21.15.1:0/2128461442 >> v2:172.21.15.1:3300/0 conn(0x7feb7806d350 msgr2=0x7feb7806da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:02:07.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:07.875+0000 7feb7de00640 1 --2- 172.21.15.1:0/2128461442 >> v2:172.21.15.1:3300/0 conn(0x7feb7806d350 0x7feb7806da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:07.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:07.875+0000 7feb7de00640 1 -- 172.21.15.1:0/2128461442 >> v1:172.21.15.7:6789/0 conn(0x7feb7815a5e0 legacy=0x7feb780ca5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:07.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:07.875+0000 7feb7de00640 1 -- 172.21.15.1:0/2128461442 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb7806dfa0 con 0x7feb780cd040 2024-05-09T08:02:07.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:07.875+0000 7feb7ca00640 1 -- 172.21.15.1:0/2128461442 <== mon.1 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3991934531 0 0) 0x7feb7815b550 con 0x7feb7815a5e0 2024-05-09T08:02:07.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:07.876+0000 7feb7de00640 1 --2- 172.21.15.1:0/2128461442 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7feb780cd040 0x7feb7815ba50 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7feb6c0097a0 tx=0x7feb6c02a060 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8473e34316cd4dc6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:07.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:07.876+0000 7feb7ca00640 1 -- 172.21.15.1:0/2128461442 <== mon.2 v2:172.21.15.7:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb6c037070 con 0x7feb780cd040 2024-05-09T08:02:07.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:07.876+0000 7feb7ca00640 1 -- 172.21.15.1:0/2128461442 <== mon.2 v2:172.21.15.7:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb6c02adb0 con 0x7feb780cd040 2024-05-09T08:02:07.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:07.876+0000 7feb7ca00640 1 -- 172.21.15.1:0/2128461442 <== mon.2 v2:172.21.15.7:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb6c032660 con 0x7feb780cd040 2024-05-09T08:02:07.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:07.876+0000 7feb7f200640 1 -- 172.21.15.1:0/2128461442 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7feb780cd040 msgr2=0x7feb7815ba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:07.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:07.876+0000 7feb7f200640 1 --2- 172.21.15.1:0/2128461442 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7feb780cd040 0x7feb7815ba50 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7feb6c0097a0 tx=0x7feb6c02a060 comp rx=0 tx=0).stop 2024-05-09T08:02:07.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:07.877+0000 7feb7f200640 1 -- 172.21.15.1:0/2128461442 shutdown_connections 2024-05-09T08:02:07.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:07.877+0000 7feb7f200640 1 --2- 172.21.15.1:0/2128461442 >> v2:172.21.15.1:3300/0 conn(0x7feb7806d350 0x7feb7806da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:07.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:07.877+0000 7feb7f200640 1 --2- 172.21.15.1:0/2128461442 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7feb780cd040 0x7feb7815ba50 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:07.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:07.877+0000 7feb7f200640 1 -- 172.21.15.1:0/2128461442 >> 172.21.15.1:0/2128461442 conn(0x7feb780b1780 msgr2=0x7feb780b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:02:07.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:07.877+0000 7feb7f200640 1 -- 172.21.15.1:0/2128461442 shutdown_connections 2024-05-09T08:02:07.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:07.877+0000 7feb7f200640 1 -- 172.21.15.1:0/2128461442 wait complete. 2024-05-09T08:02:07.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:07.878+0000 7feb7f200640 1 Processor -- start 2024-05-09T08:02:07.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:07.878+0000 7feb7f200640 1 -- start start 2024-05-09T08:02:07.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:07.879+0000 7feb7f200640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7feb7806d350 0x7feb780c8230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:07.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:07.879+0000 7feb7f200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feb780cd040 0x7feb781f36a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:07.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:07.879+0000 7feb7f200640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7feb7815a5e0 0x7feb781f3be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:07.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:07.879+0000 7feb7f200640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7feb7815b270 con 0x7feb7806d350 2024-05-09T08:02:07.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:07.879+0000 7feb7f200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7feb7806c8b0 con 0x7feb780cd040 2024-05-09T08:02:07.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:07.879+0000 7feb7f200640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7feb7806cb50 con 0x7feb7815a5e0 2024-05-09T08:02:07.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:07.879+0000 7feb7de00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7feb7806d350 0x7feb780c8230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:07.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:07.879+0000 7feb7de00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7feb7806d350 0x7feb780c8230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:47444/0 (socket says 172.21.15.1:47444) 2024-05-09T08:02:07.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:07.879+0000 7feb7de00640 1 -- 172.21.15.1:0/2287047923 learned_addr learned my addr 172.21.15.1:0/2287047923 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:02:07.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:07.880+0000 7feb7d400640 1 --2- 172.21.15.1:0/2287047923 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feb780cd040 0x7feb781f36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:07.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:07.880+0000 7feb7e800640 1 --2- 172.21.15.1:0/2287047923 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7feb7815a5e0 0x7feb781f3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:07.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:07.880+0000 7feb7de00640 1 -- 172.21.15.1:0/2287047923 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7feb7815a5e0 msgr2=0x7feb781f3be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:07.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:07.880+0000 7feb7de00640 1 --2- 172.21.15.1:0/2287047923 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7feb7815a5e0 0x7feb781f3be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:07.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:07.880+0000 7feb7de00640 1 -- 172.21.15.1:0/2287047923 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feb780cd040 msgr2=0x7feb781f36a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:07.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:07.880+0000 7feb7de00640 1 --2- 172.21.15.1:0/2287047923 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feb780cd040 0x7feb781f36a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:07.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:07.880+0000 7feb7de00640 1 -- 172.21.15.1:0/2287047923 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb780d06a0 con 0x7feb7806d350 2024-05-09T08:02:07.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:07.880+0000 7feb7de00640 1 --2- 172.21.15.1:0/2287047923 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7feb7806d350 0x7feb780c8230 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7feb6c035040 tx=0x7feb6c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:07.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:07.880+0000 7feb72000640 1 -- 172.21.15.1:0/2287047923 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb6c037040 con 0x7feb7806d350 2024-05-09T08:02:07.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:07.880+0000 7feb72000640 1 -- 172.21.15.1:0/2287047923 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb6c004030 con 0x7feb7806d350 2024-05-09T08:02:07.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:07.881+0000 7feb7f200640 1 -- 172.21.15.1:0/2287047923 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb78155080 con 0x7feb7806d350 2024-05-09T08:02:07.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:07.881+0000 7feb72000640 1 -- 172.21.15.1:0/2287047923 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb6c004420 con 0x7feb7806d350 2024-05-09T08:02:07.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:07.881+0000 7feb7f200640 1 -- 172.21.15.1:0/2287047923 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb78155640 con 0x7feb7806d350 2024-05-09T08:02:07.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:07.882+0000 7feb7f200640 1 -- 172.21.15.1:0/2287047923 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb7804f660 con 0x7feb7806d350 2024-05-09T08:02:07.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:07.883+0000 7feb72000640 1 -- 172.21.15.1:0/2287047923 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 4) ==== 90442+0+0 (secure 0 0 0) 0x7feb6c003610 con 0x7feb7806d350 2024-05-09T08:02:07.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:07.883+0000 7feb72000640 1 --2- 172.21.15.1:0/2287047923 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7feb5c06bdd0 0x7feb5c06e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:07.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:07.883+0000 7feb72000640 1 -- 172.21.15.1:0/2287047923 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(13..13 src has 1..13) ==== 4471+0+0 (secure 0 0 0) 0x7feb6c0ab470 con 0x7feb7806d350 2024-05-09T08:02:07.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:07.884+0000 7feb7d400640 1 --2- 172.21.15.1:0/2287047923 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7feb5c06bdd0 0x7feb5c06e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:07.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:07.884+0000 7feb7d400640 1 --2- 172.21.15.1:0/2287047923 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7feb5c06bdd0 0x7feb5c06e290 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7feb781591c0 tx=0x7feb68009240 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:07.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:07.889+0000 7feb72000640 1 -- 172.21.15.1:0/2287047923 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7feb6c079ce0 con 0x7feb7806d350 2024-05-09T08:02:08.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:08.082+0000 7feb7f200640 1 -- 172.21.15.1:0/2287047923 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7feb780cb260 con 0x7feb7806d350 2024-05-09T08:02:08.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:08.084+0000 7feb72000640 1 -- 172.21.15.1:0/2287047923 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v13) ==== 74+0+10777 (secure 0 0 0) 0x7feb6c07dfe0 con 0x7feb7806d350 2024-05-09T08:02:08.101 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T08:02:08.101 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":13,"fsid":"54329ec8-3270-4043-9a82-c6cd7420aa7f","created":"2024-05-09T08:01:56.666970+0000","modified":"2024-05-09T08:02:07.146082+0000","last_up_change":"2024-05-09T08:02:07.146082+0000","last_in_change":"2024-05-09T08:01:59.008225+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"c064454e-a9b5-4a24-9a77-17f0a4085952","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6800","nonce":1513349547},{"type":"v1","addr":"172.21.15.1:6801","nonce":1513349547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":1513349547},{"type":"v1","addr":"172.21.15.1:6803","nonce":1513349547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":1513349547},{"type":"v1","addr":"172.21.15.1:6807","nonce":1513349547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":1513349547},{"type":"v1","addr":"172.21.15.1:6805","nonce":1513349547}]},"public_addr":"172.21.15.1:6801/1513349547","cluster_addr":"172.21.15.1:6803/1513349547","heartbeat_back_addr":"172.21.15.1:6807/1513349547","heartbeat_front_addr":"172.21.15.1:6805/1513349547","state":["exists","up"]},{"osd":1,"uuid":"1a9ea687-2ff6-407a-949b-0e7e1083af21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":2942255259},{"type":"v1","addr":"172.21.15.1:6817","nonce":2942255259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":2942255259},{"type":"v1","addr":"172.21.15.1:6819","nonce":2942255259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":2942255259},{"type":"v1","addr":"172.21.15.1:6823","nonce":2942255259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6820","nonce":2942255259},{"type":"v1","addr":"172.21.15.1:6821","nonce":2942255259}]},"public_addr":"172.21.15.1:6817/2942255259","cluster_addr":"172.21.15.1:6819/2942255259","heartbeat_back_addr":"172.21.15.1:6823/2942255259","heartbeat_front_addr":"172.21.15.1:6821/2942255259","state":["exists","up"]},{"osd":2,"uuid":"c88af08b-7c9a-4dd8-9640-44138239addb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":2925031811},{"type":"v1","addr":"172.21.15.1:6809","nonce":2925031811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":2925031811},{"type":"v1","addr":"172.21.15.1:6811","nonce":2925031811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":2925031811},{"type":"v1","addr":"172.21.15.1:6815","nonce":2925031811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":2925031811},{"type":"v1","addr":"172.21.15.1:6813","nonce":2925031811}]},"public_addr":"172.21.15.1:6809/2925031811","cluster_addr":"172.21.15.1:6811/2925031811","heartbeat_back_addr":"172.21.15.1:6815/2925031811","heartbeat_front_addr":"172.21.15.1:6813/2925031811","state":["exists","up"]},{"osd":3,"uuid":"c087b7e3-a41d-4333-9365-32e3bd8c8e6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6824","nonce":2497776654},{"type":"v1","addr":"172.21.15.1:6825","nonce":2497776654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":2497776654},{"type":"v1","addr":"172.21.15.1:6827","nonce":2497776654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":2497776654},{"type":"v1","addr":"172.21.15.1:6831","nonce":2497776654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6828","nonce":2497776654},{"type":"v1","addr":"172.21.15.1:6829","nonce":2497776654}]},"public_addr":"172.21.15.1:6825/2497776654","cluster_addr":"172.21.15.1:6827/2497776654","heartbeat_back_addr":"172.21.15.1:6831/2497776654","heartbeat_front_addr":"172.21.15.1:6829/2497776654","state":["exists","up"]},{"osd":4,"uuid":"b4576b9c-871d-47a8-8d36-65af525d0200","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":2957026670},{"type":"v1","addr":"172.21.15.7:6801","nonce":2957026670}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2957026670},{"type":"v1","addr":"172.21.15.7:6803","nonce":2957026670}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":2957026670},{"type":"v1","addr":"172.21.15.7:6807","nonce":2957026670}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2957026670},{"type":"v1","addr":"172.21.15.7:6805","nonce":2957026670}]},"public_addr":"172.21.15.7:6801/2957026670","cluster_addr":"172.21.15.7:6803/2957026670","heartbeat_back_addr":"172.21.15.7:6807/2957026670","heartbeat_front_addr":"172.21.15.7:6805/2957026670","state":["exists","up"]},{"osd":5,"uuid":"902b3d94-72f3-4499-b21a-e32453ef5144","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1670295242},{"type":"v1","addr":"172.21.15.7:6809","nonce":1670295242}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1670295242},{"type":"v1","addr":"172.21.15.7:6811","nonce":1670295242}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1670295242},{"type":"v1","addr":"172.21.15.7:6815","nonce":1670295242}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1670295242},{"type":"v1","addr":"172.21.15.7:6813","nonce":1670295242}]},"public_addr":"172.21.15.7:6809/1670295242","cluster_addr":"172.21.15.7:6811/1670295242","heartbeat_back_addr":"172.21.15.7:6815/1670295242","heartbeat_front_addr":"172.21.15.7:6813/1670295242","state":["exists","up"]},{"osd":6,"uuid":"ead1d771-1728-4f85-adb4-26be21cedff3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":2852086880},{"type":"v1","addr":"172.21.15.7:6817","nonce":2852086880}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":2852086880},{"type":"v1","addr":"172.21.15.7:6819","nonce":2852086880}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":2852086880},{"type":"v1","addr":"172.21.15.7:6823","nonce":2852086880}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":2852086880},{"type":"v1","addr":"172.21.15.7:6821","nonce":2852086880}]},"public_addr":"172.21.15.7:6817/2852086880","cluster_addr":"172.21.15.7:6819/2852086880","heartbeat_back_addr":"172.21.15.7:6823/2852086880","heartbeat_front_addr":"172.21.15.7:6821/2852086880","state":["exists","up"]},{"osd":7,"uuid":"3ab5a201-9394-48a8-8918-e95197893669","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":1442884138},{"type":"v1","addr":"172.21.15.7:6825","nonce":1442884138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1442884138},{"type":"v1","addr":"172.21.15.7:6827","nonce":1442884138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1442884138},{"type":"v1","addr":"172.21.15.7:6831","nonce":1442884138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1442884138},{"type":"v1","addr":"172.21.15.7:6829","nonce":1442884138}]},"public_addr":"172.21.15.7:6825/1442884138","cluster_addr":"172.21.15.7:6827/1442884138","heartbeat_back_addr":"172.21.15.7:6831/1442884138","heartbeat_front_addr":"172.21.15.7:6829/1442884138","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T08:02:08.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:08.087+0000 7feb7f200640 1 -- 172.21.15.1:0/2287047923 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7feb5c06bdd0 msgr2=0x7feb5c06e290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:08.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:08.087+0000 7feb7f200640 1 --2- 172.21.15.1:0/2287047923 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7feb5c06bdd0 0x7feb5c06e290 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7feb781591c0 tx=0x7feb68009240 comp rx=0 tx=0).stop 2024-05-09T08:02:08.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:08.087+0000 7feb7f200640 1 -- 172.21.15.1:0/2287047923 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7feb7806d350 msgr2=0x7feb780c8230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:08.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:08.087+0000 7feb7f200640 1 --2- 172.21.15.1:0/2287047923 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7feb7806d350 0x7feb780c8230 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7feb6c035040 tx=0x7feb6c0046c0 comp rx=0 tx=0).stop 2024-05-09T08:02:08.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:08.087+0000 7feb7f200640 1 -- 172.21.15.1:0/2287047923 shutdown_connections 2024-05-09T08:02:08.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:08.087+0000 7feb7f200640 1 --2- 172.21.15.1:0/2287047923 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7feb5c06bdd0 0x7feb5c06e290 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:08.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:08.087+0000 7feb7f200640 1 --2- 172.21.15.1:0/2287047923 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feb780cd040 0x7feb781f36a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:08.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:08.087+0000 7feb7f200640 1 --2- 172.21.15.1:0/2287047923 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7feb7815a5e0 0x7feb781f3be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:08.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:08.087+0000 7feb7f200640 1 --2- 172.21.15.1:0/2287047923 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7feb7806d350 0x7feb780c8230 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:08.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:08.087+0000 7feb7f200640 1 -- 172.21.15.1:0/2287047923 >> 172.21.15.1:0/2287047923 conn(0x7feb780b1780 msgr2=0x7feb78159740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:02:08.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:08.087+0000 7feb7f200640 1 -- 172.21.15.1:0/2287047923 shutdown_connections 2024-05-09T08:02:08.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:08.087+0000 7feb7f200640 1 -- 172.21.15.1:0/2287047923 wait complete. 2024-05-09T08:02:08.121 INFO:tasks.cephfs.filesystem.ceph_manager:[] 2024-05-09T08:02:08.121 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T08:02:08.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:08.315+0000 7f931f200640 1 Processor -- start 2024-05-09T08:02:08.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:08.316+0000 7f931f200640 1 -- start start 2024-05-09T08:02:08.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:08.317+0000 7f931f200640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f93200c8180 0x7f93200cc730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:08.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:08.317+0000 7f931f200640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f932015c790 con 0x7f93200c7660 2024-05-09T08:02:08.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:08.317+0000 7f931f200640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f93200c6a30 con 0x7f9320152500 2024-05-09T08:02:08.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:08.317+0000 7f931f200640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f93200c5e50 con 0x7f93200c8180 2024-05-09T08:02:08.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:08.317+0000 7f9315400640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f93200c8180 0x7f93200cc730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:08.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:08.318+0000 7f9315400640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f93200c8180 0x7f93200cc730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:47446/0 (socket says 172.21.15.1:47446) 2024-05-09T08:02:08.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:08.318+0000 7f9315400640 1 -- 172.21.15.1:0/793377256 learned_addr learned my addr 172.21.15.1:0/793377256 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:02:08.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:08.318+0000 7f9315400640 1 -- 172.21.15.1:0/793377256 >> v1:172.21.15.7:6789/0 conn(0x7f9320152500 legacy=0x7f932015c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:08.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:08.318+0000 7f931d400640 1 -- 172.21.15.1:0/793377256 <== mon.1 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1269486013 0 0) 0x7f93200c6a30 con 0x7f9320152500 2024-05-09T08:02:08.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:08.318+0000 7f9315400640 1 -- 172.21.15.1:0/793377256 >> v1:172.21.15.1:6789/0 conn(0x7f93200c7660 legacy=0x7f9320162bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:08.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:08.318+0000 7f9315400640 1 -- 172.21.15.1:0/793377256 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93200c6cf0 con 0x7f93200c8180 2024-05-09T08:02:08.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:08.319+0000 7f9315400640 1 --2- 172.21.15.1:0/793377256 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f93200c8180 0x7f93200cc730 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f9308009870 tx=0x7f930802a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d19766550f247b0c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:08.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:08.319+0000 7f931d400640 1 -- 172.21.15.1:0/793377256 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9308037070 con 0x7f93200c8180 2024-05-09T08:02:08.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:08.319+0000 7f931d400640 1 -- 172.21.15.1:0/793377256 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f930802ae50 con 0x7f93200c8180 2024-05-09T08:02:08.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:08.319+0000 7f931d400640 1 -- 172.21.15.1:0/793377256 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9308032860 con 0x7f93200c8180 2024-05-09T08:02:08.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:08.320+0000 7f931f200640 1 -- 172.21.15.1:0/793377256 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f93200c8180 msgr2=0x7f93200cc730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:08.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:08.320+0000 7f931f200640 1 --2- 172.21.15.1:0/793377256 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f93200c8180 0x7f93200cc730 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f9308009870 tx=0x7f930802a300 comp rx=0 tx=0).stop 2024-05-09T08:02:08.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:08.320+0000 7f931f200640 1 -- 172.21.15.1:0/793377256 shutdown_connections 2024-05-09T08:02:08.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:08.320+0000 7f931f200640 1 --2- 172.21.15.1:0/793377256 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f93200c8180 0x7f93200cc730 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:08.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:08.320+0000 7f931f200640 1 -- 172.21.15.1:0/793377256 >> 172.21.15.1:0/793377256 conn(0x7f93200cbc10 msgr2=0x7f93200cc020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:02:08.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:08.320+0000 7f931f200640 1 -- 172.21.15.1:0/793377256 shutdown_connections 2024-05-09T08:02:08.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:08.320+0000 7f931f200640 1 -- 172.21.15.1:0/793377256 wait complete. 2024-05-09T08:02:08.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:08.322+0000 7f931f200640 1 Processor -- start 2024-05-09T08:02:08.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:08.322+0000 7f931f200640 1 -- start start 2024-05-09T08:02:08.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:08.322+0000 7f931f200640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f93200c7660 0x7f93201e34b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:08.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:08.322+0000 7f931f200640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f93200c8180 0x7f93201e40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:08.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:08.322+0000 7f931f200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9320152500 0x7f93201efbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:08.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:08.323+0000 7f931f200640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f93200c72b0 con 0x7f93200c7660 2024-05-09T08:02:08.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:08.323+0000 7f931f200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f93200c22d0 con 0x7f9320152500 2024-05-09T08:02:08.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:08.323+0000 7f931f200640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f932015b880 con 0x7f93200c8180 2024-05-09T08:02:08.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:08.323+0000 7f931de00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f93200c7660 0x7f93201e34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:08.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:08.323+0000 7f931de00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f93200c7660 0x7f93201e34b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:47450/0 (socket says 172.21.15.1:47450) 2024-05-09T08:02:08.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:08.323+0000 7f931de00640 1 -- 172.21.15.1:0/1783340903 learned_addr learned my addr 172.21.15.1:0/1783340903 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:02:08.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:08.323+0000 7f9315400640 1 --2- 172.21.15.1:0/1783340903 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f93200c8180 0x7f93201e40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:08.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:08.323+0000 7f931e800640 1 --2- 172.21.15.1:0/1783340903 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9320152500 0x7f93201efbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:08.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:08.323+0000 7f931de00640 1 -- 172.21.15.1:0/1783340903 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f93200c8180 msgr2=0x7f93201e40e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:08.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:08.323+0000 7f931de00640 1 --2- 172.21.15.1:0/1783340903 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f93200c8180 0x7f93201e40e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:08.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:08.323+0000 7f931de00640 1 -- 172.21.15.1:0/1783340903 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9320152500 msgr2=0x7f93201efbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:08.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:08.323+0000 7f931de00640 1 --2- 172.21.15.1:0/1783340903 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9320152500 0x7f93201efbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:08.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:08.323+0000 7f931de00640 1 -- 172.21.15.1:0/1783340903 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f932015a030 con 0x7f93200c7660 2024-05-09T08:02:08.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:08.323+0000 7f9315400640 1 --2- 172.21.15.1:0/1783340903 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f93200c8180 0x7f93201e40e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T08:02:08.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:08.323+0000 7f931de00640 1 --2- 172.21.15.1:0/1783340903 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f93200c7660 0x7f93201e34b0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f931000c3f0 tx=0x7f931000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:08.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:08.324+0000 7f9316a00640 1 -- 172.21.15.1:0/1783340903 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9310017070 con 0x7f93200c7660 2024-05-09T08:02:08.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:08.324+0000 7f9316a00640 1 -- 172.21.15.1:0/1783340903 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93100035f0 con 0x7f93200c7660 2024-05-09T08:02:08.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:08.324+0000 7f9316a00640 1 -- 172.21.15.1:0/1783340903 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93100039e0 con 0x7f93200c7660 2024-05-09T08:02:08.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:08.324+0000 7f931f200640 1 -- 172.21.15.1:0/1783340903 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f932015a310 con 0x7f93200c7660 2024-05-09T08:02:08.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:08.324+0000 7f931f200640 1 -- 172.21.15.1:0/1783340903 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93200c52c0 con 0x7f93200c7660 2024-05-09T08:02:08.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:08.325+0000 7f931f200640 1 -- 172.21.15.1:0/1783340903 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f932004f660 con 0x7f93200c7660 2024-05-09T08:02:08.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:08.326+0000 7f9316a00640 1 -- 172.21.15.1:0/1783340903 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 4) ==== 90442+0+0 (secure 0 0 0) 0x7f931001a070 con 0x7f93200c7660 2024-05-09T08:02:08.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:08.327+0000 7f9316a00640 1 --2- 172.21.15.1:0/1783340903 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f92fc06bda0 0x7f92fc06e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:08.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:08.327+0000 7f9316a00640 1 -- 172.21.15.1:0/1783340903 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 4471+0+0 (secure 0 0 0) 0x7f9310089d00 con 0x7f93200c7660 2024-05-09T08:02:08.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:08.327+0000 7f9315400640 1 --2- 172.21.15.1:0/1783340903 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f92fc06bda0 0x7f92fc06e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:08.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:08.328+0000 7f9315400640 1 --2- 172.21.15.1:0/1783340903 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f92fc06bda0 0x7f92fc06e260 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f9308034000 tx=0x7f930802e000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:08.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:08.331+0000 7f9316a00640 1 -- 172.21.15.1:0/1783340903 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f9310058bc0 con 0x7f93200c7660 2024-05-09T08:02:08.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:08.564+0000 7f931f200640 1 -- 172.21.15.1:0/1783340903 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f93200c2a40 con 0x7f93200c7660 2024-05-09T08:02:08.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:08.565+0000 7f9316a00640 1 -- 172.21.15.1:0/1783340903 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1389 (secure 0 0 0) 0x7f93100584c0 con 0x7f93200c7660 2024-05-09T08:02:08.582 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T08:02:08.582 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4220,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/1157980939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":1157980939},{"type":"v1","addr":"172.21.15.7:6837","nonce":1157980939}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-05-09T08:02:08.583 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 2 2024-05-09T08:02:08.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:08.568+0000 7f931f200640 1 -- 172.21.15.1:0/1783340903 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f92fc06bda0 msgr2=0x7f92fc06e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:08.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:08.568+0000 7f931f200640 1 --2- 172.21.15.1:0/1783340903 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f92fc06bda0 0x7f92fc06e260 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f9308034000 tx=0x7f930802e000 comp rx=0 tx=0).stop 2024-05-09T08:02:08.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:08.568+0000 7f931f200640 1 -- 172.21.15.1:0/1783340903 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f93200c7660 msgr2=0x7f93201e34b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:08.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:08.568+0000 7f931f200640 1 --2- 172.21.15.1:0/1783340903 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f93200c7660 0x7f93201e34b0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f931000c3f0 tx=0x7f931000c8b0 comp rx=0 tx=0).stop 2024-05-09T08:02:08.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:08.568+0000 7f931f200640 1 -- 172.21.15.1:0/1783340903 shutdown_connections 2024-05-09T08:02:08.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:08.568+0000 7f931f200640 1 --2- 172.21.15.1:0/1783340903 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f92fc06bda0 0x7f92fc06e260 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:08.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:08.568+0000 7f931f200640 1 --2- 172.21.15.1:0/1783340903 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9320152500 0x7f93201efbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:08.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:08.568+0000 7f931f200640 1 --2- 172.21.15.1:0/1783340903 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f93200c8180 0x7f93201e40e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:08.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:08.568+0000 7f931f200640 1 --2- 172.21.15.1:0/1783340903 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f93200c7660 0x7f93201e34b0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:08.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:08.568+0000 7f931f200640 1 -- 172.21.15.1:0/1783340903 >> 172.21.15.1:0/1783340903 conn(0x7f93200cbc10 msgr2=0x7f93200c8f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:02:08.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:08.568+0000 7f931f200640 1 -- 172.21.15.1:0/1783340903 shutdown_connections 2024-05-09T08:02:08.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:08.568+0000 7f931f200640 1 -- 172.21.15.1:0/1783340903 wait complete. 2024-05-09T08:02:10.602 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T08:02:10.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:10.809+0000 7f9bc7200640 1 Processor -- start 2024-05-09T08:02:10.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:10.809+0000 7f9bc7200640 1 -- start start 2024-05-09T08:02:10.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:10.811+0000 7f9bc7200640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f9bc0152b20 0x7f9bc0072900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:10.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:10.811+0000 7f9bc7200640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9bc0073200 con 0x7f9bc006dca0 2024-05-09T08:02:10.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:10.811+0000 7f9bc7200640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9bc00734e0 con 0x7f9bc00c6ea0 2024-05-09T08:02:10.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:10.811+0000 7f9bc7200640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f9bc0072e40 con 0x7f9bc0152b20 2024-05-09T08:02:10.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:10.811+0000 7f9bb7e00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f9bc00c6ea0 0x7f9bc0152410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.1:56556/0 (socket says 172.21.15.1:56556) 2024-05-09T08:02:10.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:10.811+0000 7f9bb7e00640 1 -- 172.21.15.1:0/594706000 learned_addr learned my addr 172.21.15.1:0/594706000 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T08:02:10.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:10.811+0000 7f9bc5200640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f9bc0152b20 0x7f9bc0072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:10.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:10.812+0000 7f9bc5200640 1 -- 172.21.15.1:0/594706000 >> v1:172.21.15.7:6789/0 conn(0x7f9bc00c6ea0 legacy=0x7f9bc0152410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:10.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:10.812+0000 7f9bc5200640 1 -- 172.21.15.1:0/594706000 >> v1:172.21.15.1:6789/0 conn(0x7f9bc006dca0 legacy=0x7f9bc006e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:10.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:10.812+0000 7f9bc5200640 1 -- 172.21.15.1:0/594706000 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bc0071e20 con 0x7f9bc0152b20 2024-05-09T08:02:10.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:10.814+0000 7f9bc5200640 1 --2- 172.21.15.1:0/594706000 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9bc0152b20 0x7f9bc0072900 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f9bb8009d90 tx=0x7f9bb802a780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c9c509e327076412 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:10.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:10.814+0000 7f9bbf400640 1 -- 172.21.15.1:0/594706000 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bb8037070 con 0x7f9bc0152b20 2024-05-09T08:02:10.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:10.814+0000 7f9bbf400640 1 -- 172.21.15.1:0/594706000 <== mon.1 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9bb8007740 con 0x7f9bc0152b20 2024-05-09T08:02:10.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:10.814+0000 7f9bbf400640 1 -- 172.21.15.1:0/594706000 <== mon.1 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bb8007b10 con 0x7f9bc0152b20 2024-05-09T08:02:10.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:10.815+0000 7f9bc7200640 1 -- 172.21.15.1:0/594706000 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9bc0152b20 msgr2=0x7f9bc0072900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:10.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:10.815+0000 7f9bc7200640 1 --2- 172.21.15.1:0/594706000 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9bc0152b20 0x7f9bc0072900 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f9bb8009d90 tx=0x7f9bb802a780 comp rx=0 tx=0).stop 2024-05-09T08:02:10.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:10.815+0000 7f9bc7200640 1 -- 172.21.15.1:0/594706000 shutdown_connections 2024-05-09T08:02:10.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:10.815+0000 7f9bc7200640 1 --2- 172.21.15.1:0/594706000 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9bc0152b20 0x7f9bc0072900 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:10.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:10.815+0000 7f9bc7200640 1 -- 172.21.15.1:0/594706000 >> 172.21.15.1:0/594706000 conn(0x7f9bc00cc070 msgr2=0x7f9bc00cc480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:02:10.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:10.815+0000 7f9bc7200640 1 -- 172.21.15.1:0/594706000 shutdown_connections 2024-05-09T08:02:10.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:10.815+0000 7f9bc7200640 1 -- 172.21.15.1:0/594706000 wait complete. 2024-05-09T08:02:10.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:10.818+0000 7f9bc7200640 1 Processor -- start 2024-05-09T08:02:10.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:10.818+0000 7f9bc7200640 1 -- start start 2024-05-09T08:02:10.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:10.820+0000 7f9bc7200640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9bc006dca0 0x7f9bc00cabf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:10.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:10.820+0000 7f9bc7200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9bc00c6ea0 0x7f9bc00cb130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:10.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:10.820+0000 7f9bc7200640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f9bc0152b20 0x7f9bc0144dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:10.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:10.820+0000 7f9bc7200640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f9bc0072470 con 0x7f9bc006dca0 2024-05-09T08:02:10.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:10.820+0000 7f9bc7200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9bc006ece0 con 0x7f9bc00c6ea0 2024-05-09T08:02:10.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:10.820+0000 7f9bc7200640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f9bc006ee60 con 0x7f9bc0152b20 2024-05-09T08:02:10.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:10.820+0000 7f9bb7e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9bc00c6ea0 0x7f9bc00cb130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:10.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:10.820+0000 7f9bc5200640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f9bc0152b20 0x7f9bc0144dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:10.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:10.820+0000 7f9bbfe00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9bc006dca0 0x7f9bc00cabf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:10.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:10.820+0000 7f9bc5200640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f9bc0152b20 0x7f9bc0144dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3301/0 says I am v2:172.21.15.1:41894/0 (socket says 172.21.15.1:41894) 2024-05-09T08:02:10.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:10.820+0000 7f9bb7e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9bc00c6ea0 0x7f9bc00cb130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.1:36938/0 (socket says 172.21.15.1:36938) 2024-05-09T08:02:10.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:10.820+0000 7f9bc5200640 1 -- 172.21.15.1:0/3118322740 learned_addr learned my addr 172.21.15.1:0/3118322740 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:02:10.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:10.821+0000 7f9bc5200640 1 -- 172.21.15.1:0/3118322740 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9bc00c6ea0 msgr2=0x7f9bc00cb130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:10.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:10.821+0000 7f9bc5200640 1 --2- 172.21.15.1:0/3118322740 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9bc00c6ea0 0x7f9bc00cb130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:10.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:10.821+0000 7f9bc5200640 1 -- 172.21.15.1:0/3118322740 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9bc006dca0 msgr2=0x7f9bc00cabf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:10.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:10.821+0000 7f9bc5200640 1 --2- 172.21.15.1:0/3118322740 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9bc006dca0 0x7f9bc00cabf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:10.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:10.821+0000 7f9bc5200640 1 -- 172.21.15.1:0/3118322740 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bc00c45e0 con 0x7f9bc0152b20 2024-05-09T08:02:10.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:10.821+0000 7f9bb7e00640 1 --2- 172.21.15.1:0/3118322740 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9bc00c6ea0 0x7f9bc00cb130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:02:10.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:10.821+0000 7f9bbfe00640 1 --2- 172.21.15.1:0/3118322740 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9bc006dca0 0x7f9bc00cabf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:02:10.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:10.821+0000 7f9bc5200640 1 --2- 172.21.15.1:0/3118322740 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f9bc0152b20 0x7f9bc0144dd0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f9bb8009d90 tx=0x7f9bb8004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:10.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:10.821+0000 7f9bbcc00640 1 -- 172.21.15.1:0/3118322740 <== mon.2 v2:172.21.15.7:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bb8042020 con 0x7f9bc0152b20 2024-05-09T08:02:10.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:10.821+0000 7f9bbcc00640 1 -- 172.21.15.1:0/3118322740 <== mon.2 v2:172.21.15.7:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9bb8032990 con 0x7f9bc0152b20 2024-05-09T08:02:10.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:10.821+0000 7f9bbcc00640 1 -- 172.21.15.1:0/3118322740 <== mon.2 v2:172.21.15.7:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bb8037040 con 0x7f9bc0152b20 2024-05-09T08:02:10.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:10.821+0000 7f9bc7200640 1 -- 172.21.15.1:0/3118322740 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9bc01e6600 con 0x7f9bc0152b20 2024-05-09T08:02:10.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:10.822+0000 7f9bc7200640 1 -- 172.21.15.1:0/3118322740 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f9bc00cbc00 con 0x7f9bc0152b20 2024-05-09T08:02:10.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:10.823+0000 7f9bc7200640 1 -- 172.21.15.1:0/3118322740 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9bc00711a0 con 0x7f9bc0152b20 2024-05-09T08:02:10.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:10.824+0000 7f9bbcc00640 1 -- 172.21.15.1:0/3118322740 <== mon.2 v2:172.21.15.7:3301/0 4 ==== mgrmap(e 4) ==== 90442+0+0 (secure 0 0 0) 0x7f9bb8045020 con 0x7f9bc0152b20 2024-05-09T08:02:10.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:10.824+0000 7f9bbcc00640 1 --2- 172.21.15.1:0/3118322740 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f9b9c06bdd0 0x7f9b9c06e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:10.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:10.825+0000 7f9bbcc00640 1 -- 172.21.15.1:0/3118322740 <== mon.2 v2:172.21.15.7:3301/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f9bb80abb60 con 0x7f9bc0152b20 2024-05-09T08:02:10.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:10.825+0000 7f9bbfe00640 1 --2- 172.21.15.1:0/3118322740 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f9b9c06bdd0 0x7f9b9c06e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:10.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:10.825+0000 7f9bbfe00640 1 --2- 172.21.15.1:0/3118322740 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f9b9c06bdd0 0x7f9b9c06e290 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f9bb0001b70 tx=0x7f9bb0009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:10.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:10.830+0000 7f9bbcc00640 1 -- 172.21.15.1:0/3118322740 <== mon.2 v2:172.21.15.7:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f9bc00711a0 con 0x7f9bc0152b20 2024-05-09T08:02:11.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:11.071+0000 7f9bc7200640 1 -- 172.21.15.1:0/3118322740 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9bc01e67d0 con 0x7f9bc0152b20 2024-05-09T08:02:11.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:11.072+0000 7f9bbcc00640 1 -- 172.21.15.1:0/3118322740 <== mon.2 v2:172.21.15.7:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+5491 (secure 0 0 0) 0x7f9bc01e67d0 con 0x7f9bc0152b20 2024-05-09T08:02:11.089 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T08:02:11.089 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4220,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/1157980939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":1157980939},{"type":"v1","addr":"172.21.15.7:6837","nonce":1157980939}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4235,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6833/3269798795","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6832","nonce":3269798795},{"type":"v1","addr":"172.21.15.1:6833","nonce":3269798795}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4241,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/725115499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6835","nonce":725115499},{"type":"v1","addr":"172.21.15.1:6837","nonce":725115499}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4276,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6838/1827115274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1827115274},{"type":"v1","addr":"172.21.15.7:6838","nonce":1827115274}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/1032767352","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":1032767352},{"type":"v1","addr":"172.21.15.7:6839","nonce":1032767352}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4294,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6836/1509134635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":1509134635},{"type":"v1","addr":"172.21.15.1:6836","nonce":1509134635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-05-09T08:02:11.089 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 3 2024-05-09T08:02:11.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:11.076+0000 7f9bc7200640 1 -- 172.21.15.1:0/3118322740 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f9b9c06bdd0 msgr2=0x7f9b9c06e290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:11.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:11.076+0000 7f9bc7200640 1 --2- 172.21.15.1:0/3118322740 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f9b9c06bdd0 0x7f9b9c06e290 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f9bb0001b70 tx=0x7f9bb0009040 comp rx=0 tx=0).stop 2024-05-09T08:02:11.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:11.076+0000 7f9bc7200640 1 -- 172.21.15.1:0/3118322740 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f9bc0152b20 msgr2=0x7f9bc0144dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:11.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:11.076+0000 7f9bc7200640 1 --2- 172.21.15.1:0/3118322740 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f9bc0152b20 0x7f9bc0144dd0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f9bb8009d90 tx=0x7f9bb8004a40 comp rx=0 tx=0).stop 2024-05-09T08:02:11.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:11.076+0000 7f9bc7200640 1 -- 172.21.15.1:0/3118322740 shutdown_connections 2024-05-09T08:02:11.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:11.076+0000 7f9bc7200640 1 --2- 172.21.15.1:0/3118322740 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f9b9c06bdd0 0x7f9b9c06e290 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:11.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:11.076+0000 7f9bc7200640 1 --2- 172.21.15.1:0/3118322740 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9bc00c6ea0 0x7f9bc00cb130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:11.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:11.076+0000 7f9bc7200640 1 --2- 172.21.15.1:0/3118322740 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f9bc0152b20 0x7f9bc0144dd0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:11.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:11.076+0000 7f9bc7200640 1 --2- 172.21.15.1:0/3118322740 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9bc006dca0 0x7f9bc00cabf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:11.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:11.076+0000 7f9bc7200640 1 -- 172.21.15.1:0/3118322740 >> 172.21.15.1:0/3118322740 conn(0x7f9bc00cc070 msgr2=0x7f9bc014d980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:02:11.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:11.076+0000 7f9bc7200640 1 -- 172.21.15.1:0/3118322740 shutdown_connections 2024-05-09T08:02:11.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:11.076+0000 7f9bc7200640 1 -- 172.21.15.1:0/3118322740 wait complete. 2024-05-09T08:02:11.113 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T08:02:11.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:11.308+0000 7fc002400640 1 Processor -- start 2024-05-09T08:02:11.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:11.308+0000 7fc002400640 1 -- start start 2024-05-09T08:02:11.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:11.310+0000 7fc002400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fbffc0c8180 0x7fbffc0cc730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:11.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:11.310+0000 7fc002400640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fbffc0c7660 0x7fbffc0c7ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:11.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:11.310+0000 7fc002400640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbffc15c790 con 0x7fbffc152500 2024-05-09T08:02:11.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:11.310+0000 7fc002400640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7fbffc0c5e50 con 0x7fbffc0c7660 2024-05-09T08:02:11.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:11.310+0000 7fc002400640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fbffc0c22d0 con 0x7fbffc0c8180 2024-05-09T08:02:11.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:11.310+0000 7fc001a00640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fbffc0c7660 0x7fbffc0c7ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:11.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:11.310+0000 7fbffbe00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fbffc0c8180 0x7fbffc0cc730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:11.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:11.310+0000 7fc001000640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fbffc152500 0x7fbffc15c080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.1:56572/0 (socket says 172.21.15.1:56572) 2024-05-09T08:02:11.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:11.310+0000 7fbffbe00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fbffc0c8180 0x7fbffc0cc730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.1:36954/0 (socket says 172.21.15.1:36954) 2024-05-09T08:02:11.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:11.310+0000 7fc001000640 1 -- 172.21.15.1:0/1704094841 learned_addr learned my addr 172.21.15.1:0/1704094841 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T08:02:11.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:11.311+0000 7fbffbe00640 1 -- 172.21.15.1:0/1704094841 >> v2:172.21.15.1:3300/0 conn(0x7fbffc0c7660 msgr2=0x7fbffc0c7ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:02:11.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:11.311+0000 7fbffbe00640 1 --2- 172.21.15.1:0/1704094841 >> v2:172.21.15.1:3300/0 conn(0x7fbffc0c7660 0x7fbffc0c7ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:11.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:11.311+0000 7fbffbe00640 1 -- 172.21.15.1:0/1704094841 >> v1:172.21.15.7:6789/0 conn(0x7fbffc152500 legacy=0x7fbffc15c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:11.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:11.311+0000 7fbffb400640 1 -- 172.21.15.1:0/1704094841 <== mon.1 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3810438371 0 0) 0x7fbff0001110 con 0x7fbffc152500 2024-05-09T08:02:11.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:11.311+0000 7fbffbe00640 1 -- 172.21.15.1:0/1704094841 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbffc0c6a30 con 0x7fbffc0c8180 2024-05-09T08:02:11.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:11.312+0000 7fbffbe00640 1 --2- 172.21.15.1:0/1704094841 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbffc0c8180 0x7fbffc0cc730 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fbffc155f40 tx=0x7fbfe802a1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=58643a8356eb95c7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:11.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:11.312+0000 7fbffb400640 1 -- 172.21.15.1:0/1704094841 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfe8037070 con 0x7fbffc0c8180 2024-05-09T08:02:11.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:11.312+0000 7fbffb400640 1 -- 172.21.15.1:0/1704094841 <== mon.1 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbfe8032440 con 0x7fbffc0c8180 2024-05-09T08:02:11.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:11.312+0000 7fbffb400640 1 -- 172.21.15.1:0/1704094841 <== mon.1 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfe8032810 con 0x7fbffc0c8180 2024-05-09T08:02:11.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:11.313+0000 7fc002400640 1 -- 172.21.15.1:0/1704094841 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbffc0c8180 msgr2=0x7fbffc0cc730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:11.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:11.313+0000 7fc002400640 1 --2- 172.21.15.1:0/1704094841 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbffc0c8180 0x7fbffc0cc730 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fbffc155f40 tx=0x7fbfe802a1a0 comp rx=0 tx=0).stop 2024-05-09T08:02:11.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:11.313+0000 7fc002400640 1 -- 172.21.15.1:0/1704094841 shutdown_connections 2024-05-09T08:02:11.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:11.313+0000 7fc002400640 1 --2- 172.21.15.1:0/1704094841 >> v2:172.21.15.1:3300/0 conn(0x7fbffc0c7660 0x7fbffc0c7ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:11.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:11.313+0000 7fc002400640 1 --2- 172.21.15.1:0/1704094841 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbffc0c8180 0x7fbffc0cc730 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:11.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:11.313+0000 7fc002400640 1 -- 172.21.15.1:0/1704094841 >> 172.21.15.1:0/1704094841 conn(0x7fbffc0cbc10 msgr2=0x7fbffc0cc020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:02:11.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:11.313+0000 7fc002400640 1 -- 172.21.15.1:0/1704094841 shutdown_connections 2024-05-09T08:02:11.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:11.313+0000 7fc002400640 1 -- 172.21.15.1:0/1704094841 wait complete. 2024-05-09T08:02:11.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:11.314+0000 7fc002400640 1 Processor -- start 2024-05-09T08:02:11.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:11.315+0000 7fc002400640 1 -- start start 2024-05-09T08:02:11.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:11.315+0000 7fc002400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbffc0c7660 0x7fbffc1e4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:11.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:11.316+0000 7fc002400640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fbffc0c8180 0x7fbffc1f0760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:11.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:11.316+0000 7fc002400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbffc152500 0x7fbffc1f0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:11.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:11.316+0000 7fc002400640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fbffc0c72b0 con 0x7fbffc0c7660 2024-05-09T08:02:11.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:11.316+0000 7fc002400640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fbffc0c6fd0 con 0x7fbffc152500 2024-05-09T08:02:11.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:11.316+0000 7fc002400640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7fbffc15b880 con 0x7fbffc0c8180 2024-05-09T08:02:11.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:11.316+0000 7fc001000640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbffc0c7660 0x7fbffc1e4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:11.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:11.316+0000 7fc001000640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbffc0c7660 0x7fbffc1e4e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:47472/0 (socket says 172.21.15.1:47472) 2024-05-09T08:02:11.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:11.316+0000 7fc001000640 1 -- 172.21.15.1:0/1206520172 learned_addr learned my addr 172.21.15.1:0/1206520172 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:02:11.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:11.316+0000 7fbffbe00640 1 --2- 172.21.15.1:0/1206520172 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fbffc0c8180 0x7fbffc1f0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:11.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:11.316+0000 7fc001a00640 1 --2- 172.21.15.1:0/1206520172 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbffc152500 0x7fbffc1f0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:11.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:11.316+0000 7fc001000640 1 -- 172.21.15.1:0/1206520172 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fbffc0c8180 msgr2=0x7fbffc1f0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:11.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:11.316+0000 7fc001000640 1 --2- 172.21.15.1:0/1206520172 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fbffc0c8180 0x7fbffc1f0760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:11.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:11.316+0000 7fc001000640 1 -- 172.21.15.1:0/1206520172 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbffc152500 msgr2=0x7fbffc1f0ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:11.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:11.316+0000 7fc001000640 1 --2- 172.21.15.1:0/1206520172 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbffc152500 0x7fbffc1f0ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:11.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:11.316+0000 7fc001000640 1 -- 172.21.15.1:0/1206520172 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbffc157b80 con 0x7fbffc0c7660 2024-05-09T08:02:11.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:11.316+0000 7fc001a00640 1 --2- 172.21.15.1:0/1206520172 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbffc152500 0x7fbffc1f0ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:02:11.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:11.317+0000 7fc001000640 1 --2- 172.21.15.1:0/1206520172 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbffc0c7660 0x7fbffc1e4e50 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fbff000c460 tx=0x7fbff000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:11.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:11.317+0000 7fbff8c00640 1 -- 172.21.15.1:0/1206520172 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbff0018070 con 0x7fbffc0c7660 2024-05-09T08:02:11.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:11.317+0000 7fbff8c00640 1 -- 172.21.15.1:0/1206520172 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbff0007500 con 0x7fbffc0c7660 2024-05-09T08:02:11.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:11.317+0000 7fbff8c00640 1 -- 172.21.15.1:0/1206520172 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbff0004a20 con 0x7fbffc0c7660 2024-05-09T08:02:11.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:11.317+0000 7fc002400640 1 -- 172.21.15.1:0/1206520172 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbffc14ecc0 con 0x7fbffc0c7660 2024-05-09T08:02:11.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:11.317+0000 7fc002400640 1 -- 172.21.15.1:0/1206520172 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbffc1575c0 con 0x7fbffc0c7660 2024-05-09T08:02:11.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:11.318+0000 7fc002400640 1 -- 172.21.15.1:0/1206520172 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbffc04f660 con 0x7fbffc0c7660 2024-05-09T08:02:11.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:11.319+0000 7fbff8c00640 1 -- 172.21.15.1:0/1206520172 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 4) ==== 90442+0+0 (secure 0 0 0) 0x7fbff0004bc0 con 0x7fbffc0c7660 2024-05-09T08:02:11.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:11.321+0000 7fbff8c00640 1 --2- 172.21.15.1:0/1206520172 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fbfdc06bdd0 0x7fbfdc06e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:11.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:11.321+0000 7fbff8c00640 1 -- 172.21.15.1:0/1206520172 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7fbff0089cc0 con 0x7fbffc0c7660 2024-05-09T08:02:11.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:11.321+0000 7fbffbe00640 1 --2- 172.21.15.1:0/1206520172 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fbfdc06bdd0 0x7fbfdc06e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:11.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:11.322+0000 7fbffbe00640 1 --2- 172.21.15.1:0/1206520172 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fbfdc06bdd0 0x7fbfdc06e290 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fbfe8034000 tx=0x7fbfe802e000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:11.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:11.326+0000 7fbff8c00640 1 -- 172.21.15.1:0/1206520172 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fbffc15c790 con 0x7fbffc0c7660 2024-05-09T08:02:11.547 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:11.530+0000 7fc002400640 1 -- 172.21.15.1:0/1206520172 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbffc0c2a40 con 0x7fbffc0c7660 2024-05-09T08:02:11.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:11.532+0000 7fbff8c00640 1 -- 172.21.15.1:0/1206520172 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) ==== 74+0+12765 (secure 0 0 0) 0x7fbff00583b0 con 0x7fbffc0c7660 2024-05-09T08:02:11.549 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T08:02:11.549 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":16,"fsid":"54329ec8-3270-4043-9a82-c6cd7420aa7f","created":"2024-05-09T08:01:56.666970+0000","modified":"2024-05-09T08:02:10.810413+0000","last_up_change":"2024-05-09T08:02:07.146082+0000","last_in_change":"2024-05-09T08:01:59.008225+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T08:02:09.165511+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c064454e-a9b5-4a24-9a77-17f0a4085952","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6800","nonce":1513349547},{"type":"v1","addr":"172.21.15.1:6801","nonce":1513349547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":1513349547},{"type":"v1","addr":"172.21.15.1:6803","nonce":1513349547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":1513349547},{"type":"v1","addr":"172.21.15.1:6807","nonce":1513349547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":1513349547},{"type":"v1","addr":"172.21.15.1:6805","nonce":1513349547}]},"public_addr":"172.21.15.1:6801/1513349547","cluster_addr":"172.21.15.1:6803/1513349547","heartbeat_back_addr":"172.21.15.1:6807/1513349547","heartbeat_front_addr":"172.21.15.1:6805/1513349547","state":["exists","up"]},{"osd":1,"uuid":"1a9ea687-2ff6-407a-949b-0e7e1083af21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":2942255259},{"type":"v1","addr":"172.21.15.1:6817","nonce":2942255259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":2942255259},{"type":"v1","addr":"172.21.15.1:6819","nonce":2942255259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":2942255259},{"type":"v1","addr":"172.21.15.1:6823","nonce":2942255259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6820","nonce":2942255259},{"type":"v1","addr":"172.21.15.1:6821","nonce":2942255259}]},"public_addr":"172.21.15.1:6817/2942255259","cluster_addr":"172.21.15.1:6819/2942255259","heartbeat_back_addr":"172.21.15.1:6823/2942255259","heartbeat_front_addr":"172.21.15.1:6821/2942255259","state":["exists","up"]},{"osd":2,"uuid":"c88af08b-7c9a-4dd8-9640-44138239addb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":2925031811},{"type":"v1","addr":"172.21.15.1:6809","nonce":2925031811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":2925031811},{"type":"v1","addr":"172.21.15.1:6811","nonce":2925031811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":2925031811},{"type":"v1","addr":"172.21.15.1:6815","nonce":2925031811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":2925031811},{"type":"v1","addr":"172.21.15.1:6813","nonce":2925031811}]},"public_addr":"172.21.15.1:6809/2925031811","cluster_addr":"172.21.15.1:6811/2925031811","heartbeat_back_addr":"172.21.15.1:6815/2925031811","heartbeat_front_addr":"172.21.15.1:6813/2925031811","state":["exists","up"]},{"osd":3,"uuid":"c087b7e3-a41d-4333-9365-32e3bd8c8e6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6824","nonce":2497776654},{"type":"v1","addr":"172.21.15.1:6825","nonce":2497776654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":2497776654},{"type":"v1","addr":"172.21.15.1:6827","nonce":2497776654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":2497776654},{"type":"v1","addr":"172.21.15.1:6831","nonce":2497776654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6828","nonce":2497776654},{"type":"v1","addr":"172.21.15.1:6829","nonce":2497776654}]},"public_addr":"172.21.15.1:6825/2497776654","cluster_addr":"172.21.15.1:6827/2497776654","heartbeat_back_addr":"172.21.15.1:6831/2497776654","heartbeat_front_addr":"172.21.15.1:6829/2497776654","state":["exists","up"]},{"osd":4,"uuid":"b4576b9c-871d-47a8-8d36-65af525d0200","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":2957026670},{"type":"v1","addr":"172.21.15.7:6801","nonce":2957026670}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2957026670},{"type":"v1","addr":"172.21.15.7:6803","nonce":2957026670}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":2957026670},{"type":"v1","addr":"172.21.15.7:6807","nonce":2957026670}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2957026670},{"type":"v1","addr":"172.21.15.7:6805","nonce":2957026670}]},"public_addr":"172.21.15.7:6801/2957026670","cluster_addr":"172.21.15.7:6803/2957026670","heartbeat_back_addr":"172.21.15.7:6807/2957026670","heartbeat_front_addr":"172.21.15.7:6805/2957026670","state":["exists","up"]},{"osd":5,"uuid":"902b3d94-72f3-4499-b21a-e32453ef5144","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1670295242},{"type":"v1","addr":"172.21.15.7:6809","nonce":1670295242}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1670295242},{"type":"v1","addr":"172.21.15.7:6811","nonce":1670295242}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1670295242},{"type":"v1","addr":"172.21.15.7:6815","nonce":1670295242}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1670295242},{"type":"v1","addr":"172.21.15.7:6813","nonce":1670295242}]},"public_addr":"172.21.15.7:6809/1670295242","cluster_addr":"172.21.15.7:6811/1670295242","heartbeat_back_addr":"172.21.15.7:6815/1670295242","heartbeat_front_addr":"172.21.15.7:6813/1670295242","state":["exists","up"]},{"osd":6,"uuid":"ead1d771-1728-4f85-adb4-26be21cedff3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":2852086880},{"type":"v1","addr":"172.21.15.7:6817","nonce":2852086880}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":2852086880},{"type":"v1","addr":"172.21.15.7:6819","nonce":2852086880}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":2852086880},{"type":"v1","addr":"172.21.15.7:6823","nonce":2852086880}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":2852086880},{"type":"v1","addr":"172.21.15.7:6821","nonce":2852086880}]},"public_addr":"172.21.15.7:6817/2852086880","cluster_addr":"172.21.15.7:6819/2852086880","heartbeat_back_addr":"172.21.15.7:6823/2852086880","heartbeat_front_addr":"172.21.15.7:6821/2852086880","state":["exists","up"]},{"osd":7,"uuid":"3ab5a201-9394-48a8-8918-e95197893669","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":1442884138},{"type":"v1","addr":"172.21.15.7:6825","nonce":1442884138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1442884138},{"type":"v1","addr":"172.21.15.7:6827","nonce":1442884138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1442884138},{"type":"v1","addr":"172.21.15.7:6831","nonce":1442884138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1442884138},{"type":"v1","addr":"172.21.15.7:6829","nonce":1442884138}]},"public_addr":"172.21.15.7:6825/1442884138","cluster_addr":"172.21.15.7:6827/1442884138","heartbeat_back_addr":"172.21.15.7:6831/1442884138","heartbeat_front_addr":"172.21.15.7:6829/1442884138","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:02:04.615675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:02:04.385974+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:02:04.619217+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:02:04.401681+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:02:04.097081+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:02:04.129981+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:02:04.225327+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:02:04.193351+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T08:02:11.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:11.534+0000 7fc002400640 1 -- 172.21.15.1:0/1206520172 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fbfdc06bdd0 msgr2=0x7fbfdc06e290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:11.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:11.534+0000 7fc002400640 1 --2- 172.21.15.1:0/1206520172 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fbfdc06bdd0 0x7fbfdc06e290 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fbfe8034000 tx=0x7fbfe802e000 comp rx=0 tx=0).stop 2024-05-09T08:02:11.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:11.535+0000 7fc002400640 1 -- 172.21.15.1:0/1206520172 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbffc0c7660 msgr2=0x7fbffc1e4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:11.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:11.535+0000 7fc002400640 1 --2- 172.21.15.1:0/1206520172 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbffc0c7660 0x7fbffc1e4e50 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fbff000c460 tx=0x7fbff000c920 comp rx=0 tx=0).stop 2024-05-09T08:02:11.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:11.535+0000 7fc002400640 1 -- 172.21.15.1:0/1206520172 shutdown_connections 2024-05-09T08:02:11.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:11.535+0000 7fc002400640 1 --2- 172.21.15.1:0/1206520172 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fbfdc06bdd0 0x7fbfdc06e290 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:11.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:11.535+0000 7fc002400640 1 --2- 172.21.15.1:0/1206520172 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbffc152500 0x7fbffc1f0ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:11.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:11.535+0000 7fc002400640 1 --2- 172.21.15.1:0/1206520172 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fbffc0c8180 0x7fbffc1f0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:11.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:11.535+0000 7fc002400640 1 --2- 172.21.15.1:0/1206520172 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbffc0c7660 0x7fbffc1e4e50 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:11.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:11.535+0000 7fc002400640 1 -- 172.21.15.1:0/1206520172 >> 172.21.15.1:0/1206520172 conn(0x7fbffc0cbc10 msgr2=0x7fbffc0ca240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:02:11.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:11.535+0000 7fc002400640 1 -- 172.21.15.1:0/1206520172 shutdown_connections 2024-05-09T08:02:11.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:11.535+0000 7fc002400640 1 -- 172.21.15.1:0/1206520172 wait complete. 2024-05-09T08:02:11.571 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T08:02:09.165511+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-09T08:02:11.572 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-09T08:02:11.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:11.779+0000 7f66caa00640 1 Processor -- start 2024-05-09T08:02:11.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:11.779+0000 7f66caa00640 1 -- start start 2024-05-09T08:02:11.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:11.781+0000 7f66caa00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f66c4154a60 0x7f66c406d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:11.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:11.781+0000 7f66caa00640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f66c40cc4c0 0x7f66c406c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:11.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:11.781+0000 7f66caa00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f66c406ccb0 0x7f66c40b4210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:11.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:11.781+0000 7f66caa00640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f66c406dff0 con 0x7f66c40cc4c0 2024-05-09T08:02:11.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:11.781+0000 7f66caa00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f66c415eb60 con 0x7f66c4154a60 2024-05-09T08:02:11.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:11.781+0000 7f66caa00640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f66c415a830 con 0x7f66c406ccb0 2024-05-09T08:02:11.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:11.781+0000 7f66c3e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f66c4154a60 0x7f66c406d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:11.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:11.781+0000 7f66c3400640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f66c40cc4c0 0x7f66c406c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:11.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:11.781+0000 7f66c3e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f66c4154a60 0x7f66c406d7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.1:36976/0 (socket says 172.21.15.1:36976) 2024-05-09T08:02:11.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:11.781+0000 7f66c3e00640 1 -- 172.21.15.1:0/2059200438 learned_addr learned my addr 172.21.15.1:0/2059200438 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:02:11.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:11.781+0000 7f66c8a00640 1 --2- 172.21.15.1:0/2059200438 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f66c406ccb0 0x7f66c40b4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:11.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:11.781+0000 7f66c3e00640 1 -- 172.21.15.1:0/2059200438 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f66c406ccb0 msgr2=0x7f66c40b4210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:02:11.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:11.781+0000 7f66c3e00640 1 --2- 172.21.15.1:0/2059200438 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f66c406ccb0 0x7f66c40b4210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:11.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:11.781+0000 7f66c3e00640 1 -- 172.21.15.1:0/2059200438 >> v2:172.21.15.1:3300/0 conn(0x7f66c40cc4c0 msgr2=0x7f66c406c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:02:11.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:11.781+0000 7f66c3e00640 1 --2- 172.21.15.1:0/2059200438 >> v2:172.21.15.1:3300/0 conn(0x7f66c40cc4c0 0x7f66c406c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:11.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:11.781+0000 7f66c3e00640 1 -- 172.21.15.1:0/2059200438 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66c415aad0 con 0x7f66c4154a60 2024-05-09T08:02:11.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:11.781+0000 7f66c8a00640 1 --2- 172.21.15.1:0/2059200438 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f66c406ccb0 0x7f66c40b4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:02:11.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:11.781+0000 7f66c3400640 1 --2- 172.21.15.1:0/2059200438 >> v2:172.21.15.1:3300/0 conn(0x7f66c40cc4c0 0x7f66c406c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:02:11.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:11.783+0000 7f66c3e00640 1 --2- 172.21.15.1:0/2059200438 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66c4154a60 0x7f66c406d7e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f66c408d5e0 tx=0x7f66b402a140 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3e7dd01171d7dcf5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:11.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:11.783+0000 7f66c2a00640 1 -- 172.21.15.1:0/2059200438 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66b4037070 con 0x7f66c4154a60 2024-05-09T08:02:11.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:11.783+0000 7f66c2a00640 1 -- 172.21.15.1:0/2059200438 <== mon.1 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66b4032440 con 0x7f66c4154a60 2024-05-09T08:02:11.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:11.783+0000 7f66c2a00640 1 -- 172.21.15.1:0/2059200438 <== mon.1 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66b4032810 con 0x7f66c4154a60 2024-05-09T08:02:11.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:11.784+0000 7f66caa00640 1 -- 172.21.15.1:0/2059200438 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66c4154a60 msgr2=0x7f66c406d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:11.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:11.784+0000 7f66caa00640 1 --2- 172.21.15.1:0/2059200438 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66c4154a60 0x7f66c406d7e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f66c408d5e0 tx=0x7f66b402a140 comp rx=0 tx=0).stop 2024-05-09T08:02:11.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:11.784+0000 7f66caa00640 1 -- 172.21.15.1:0/2059200438 shutdown_connections 2024-05-09T08:02:11.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:11.784+0000 7f66caa00640 1 --2- 172.21.15.1:0/2059200438 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f66c406ccb0 0x7f66c40b4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:11.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:11.784+0000 7f66caa00640 1 --2- 172.21.15.1:0/2059200438 >> v2:172.21.15.1:3300/0 conn(0x7f66c40cc4c0 0x7f66c406c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:11.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:11.784+0000 7f66caa00640 1 --2- 172.21.15.1:0/2059200438 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66c4154a60 0x7f66c406d7e0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:11.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:11.784+0000 7f66caa00640 1 -- 172.21.15.1:0/2059200438 >> 172.21.15.1:0/2059200438 conn(0x7f66c40cffa0 msgr2=0x7f66c40d03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:02:11.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:11.784+0000 7f66caa00640 1 -- 172.21.15.1:0/2059200438 shutdown_connections 2024-05-09T08:02:11.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:11.784+0000 7f66caa00640 1 -- 172.21.15.1:0/2059200438 wait complete. 2024-05-09T08:02:11.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:11.786+0000 7f66caa00640 1 Processor -- start 2024-05-09T08:02:11.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:11.786+0000 7f66caa00640 1 -- start start 2024-05-09T08:02:11.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:11.787+0000 7f66caa00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66c406ccb0 0x7f66c41f0180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:11.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:11.787+0000 7f66caa00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f66c40cc4c0 0x7f66c4153280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:11.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:11.787+0000 7f66caa00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f66c4154a60 0x7f66c414da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:11.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:11.787+0000 7f66caa00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f66c415b370 con 0x7f66c4154a60 2024-05-09T08:02:11.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:11.787+0000 7f66caa00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f66c415b090 con 0x7f66c406ccb0 2024-05-09T08:02:11.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:11.787+0000 7f66caa00640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f66c415e7b0 con 0x7f66c40cc4c0 2024-05-09T08:02:11.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:11.787+0000 7f66c3e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66c406ccb0 0x7f66c41f0180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:11.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:11.787+0000 7f66c3400640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f66c40cc4c0 0x7f66c4153280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:11.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:11.787+0000 7f66c3400640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f66c40cc4c0 0x7f66c4153280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3301/0 says I am v2:172.21.15.1:41916/0 (socket says 172.21.15.1:41916) 2024-05-09T08:02:11.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:11.787+0000 7f66c3e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66c406ccb0 0x7f66c41f0180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.1:36990/0 (socket says 172.21.15.1:36990) 2024-05-09T08:02:11.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:11.787+0000 7f66c3400640 1 -- 172.21.15.1:0/3952806662 learned_addr learned my addr 172.21.15.1:0/3952806662 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:02:11.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:11.787+0000 7f66c8a00640 1 --2- 172.21.15.1:0/3952806662 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f66c4154a60 0x7f66c414da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:11.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:11.788+0000 7f66c3400640 1 -- 172.21.15.1:0/3952806662 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66c406ccb0 msgr2=0x7f66c41f0180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:11.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:11.788+0000 7f66c3400640 1 --2- 172.21.15.1:0/3952806662 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66c406ccb0 0x7f66c41f0180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:11.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:11.788+0000 7f66c3400640 1 -- 172.21.15.1:0/3952806662 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f66c4154a60 msgr2=0x7f66c414da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:11.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:11.788+0000 7f66c3400640 1 --2- 172.21.15.1:0/3952806662 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f66c4154a60 0x7f66c414da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:11.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:11.788+0000 7f66c3400640 1 -- 172.21.15.1:0/3952806662 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66c414e810 con 0x7f66c40cc4c0 2024-05-09T08:02:11.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:11.788+0000 7f66c3e00640 1 --2- 172.21.15.1:0/3952806662 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66c406ccb0 0x7f66c41f0180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:02:11.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:11.788+0000 7f66c3400640 1 --2- 172.21.15.1:0/3952806662 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f66c40cc4c0 0x7f66c4153280 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f66b000ebe0 tx=0x7f66b0009e20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:11.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:11.788+0000 7f66bbe00640 1 -- 172.21.15.1:0/3952806662 <== mon.2 v2:172.21.15.7:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66b0017070 con 0x7f66c40cc4c0 2024-05-09T08:02:11.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:11.788+0000 7f66bbe00640 1 -- 172.21.15.1:0/3952806662 <== mon.2 v2:172.21.15.7:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66b000ca00 con 0x7f66c40cc4c0 2024-05-09T08:02:11.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:11.788+0000 7f66bbe00640 1 -- 172.21.15.1:0/3952806662 <== mon.2 v2:172.21.15.7:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66b000cdd0 con 0x7f66c40cc4c0 2024-05-09T08:02:11.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:11.788+0000 7f66caa00640 1 -- 172.21.15.1:0/3952806662 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66c414ead0 con 0x7f66c40cc4c0 2024-05-09T08:02:11.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:11.789+0000 7f66caa00640 1 -- 172.21.15.1:0/3952806662 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f66c4150c60 con 0x7f66c40cc4c0 2024-05-09T08:02:11.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:11.790+0000 7f66caa00640 1 -- 172.21.15.1:0/3952806662 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66c40cc000 con 0x7f66c40cc4c0 2024-05-09T08:02:11.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:11.791+0000 7f66bbe00640 1 -- 172.21.15.1:0/3952806662 <== mon.2 v2:172.21.15.7:3301/0 4 ==== mgrmap(e 4) ==== 90442+0+0 (secure 0 0 0) 0x7f66b001a070 con 0x7f66c40cc4c0 2024-05-09T08:02:11.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:11.791+0000 7f66bbe00640 1 --2- 172.21.15.1:0/3952806662 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f66a406bda0 0x7f66a406e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:11.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:11.792+0000 7f66bbe00640 1 -- 172.21.15.1:0/3952806662 <== mon.2 v2:172.21.15.7:3301/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f66b008ac20 con 0x7f66c40cc4c0 2024-05-09T08:02:11.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:11.792+0000 7f66c3e00640 1 --2- 172.21.15.1:0/3952806662 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f66a406bda0 0x7f66a406e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:11.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:11.792+0000 7f66c3e00640 1 --2- 172.21.15.1:0/3952806662 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f66a406bda0 0x7f66a406e260 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f66b4034000 tx=0x7f66b402e000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:11.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:11.797+0000 7f66bbe00640 1 -- 172.21.15.1:0/3952806662 <== mon.2 v2:172.21.15.7:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f66c40cc000 con 0x7f66c40cc4c0 2024-05-09T08:02:12.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:12.007+0000 7f66caa00640 1 -- 172.21.15.1:0/3952806662 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f66c4074d70 con 0x7f66c40cc4c0 2024-05-09T08:02:12.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:12.008+0000 7f66bbe00640 1 -- 172.21.15.1:0/3952806662 <== mon.2 v2:172.21.15.7:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v17) ==== 93+0+10 (secure 0 0 0) 0x7f66b0059310 con 0x7f66c40cc4c0 2024-05-09T08:02:12.024 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 1 2024-05-09T08:02:12.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:12.010+0000 7f66caa00640 1 -- 172.21.15.1:0/3952806662 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f66a406bda0 msgr2=0x7f66a406e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:12.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:12.010+0000 7f66caa00640 1 --2- 172.21.15.1:0/3952806662 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f66a406bda0 0x7f66a406e260 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f66b4034000 tx=0x7f66b402e000 comp rx=0 tx=0).stop 2024-05-09T08:02:12.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:12.010+0000 7f66caa00640 1 -- 172.21.15.1:0/3952806662 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f66c40cc4c0 msgr2=0x7f66c4153280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:12.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:12.010+0000 7f66caa00640 1 --2- 172.21.15.1:0/3952806662 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f66c40cc4c0 0x7f66c4153280 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f66b000ebe0 tx=0x7f66b0009e20 comp rx=0 tx=0).stop 2024-05-09T08:02:12.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:12.011+0000 7f66caa00640 1 -- 172.21.15.1:0/3952806662 shutdown_connections 2024-05-09T08:02:12.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:12.011+0000 7f66caa00640 1 --2- 172.21.15.1:0/3952806662 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f66a406bda0 0x7f66a406e260 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:12.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:12.011+0000 7f66caa00640 1 --2- 172.21.15.1:0/3952806662 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f66c4154a60 0x7f66c414da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:12.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:12.011+0000 7f66caa00640 1 --2- 172.21.15.1:0/3952806662 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f66c40cc4c0 0x7f66c4153280 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:12.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:12.011+0000 7f66caa00640 1 --2- 172.21.15.1:0/3952806662 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66c406ccb0 0x7f66c41f0180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:12.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:12.011+0000 7f66caa00640 1 -- 172.21.15.1:0/3952806662 >> 172.21.15.1:0/3952806662 conn(0x7f66c40cffa0 msgr2=0x7f66c40cde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:02:12.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:12.011+0000 7f66caa00640 1 -- 172.21.15.1:0/3952806662 shutdown_connections 2024-05-09T08:02:12.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:12.011+0000 7f66caa00640 1 -- 172.21.15.1:0/3952806662 wait complete. 2024-05-09T08:02:12.045 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-09T08:02:12.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:12.214+0000 7f4ce0200640 1 Processor -- start 2024-05-09T08:02:12.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:12.214+0000 7f4ce0200640 1 -- start start 2024-05-09T08:02:12.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:12.216+0000 7f4ce0200640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f4cd815a710 0x7f4cd80b4430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:12.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:12.216+0000 7f4ce0200640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4cd815adc0 con 0x7f4cd80cb070 2024-05-09T08:02:12.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:12.216+0000 7f4ce0200640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4cd815b0a0 con 0x7f4cd8154970 2024-05-09T08:02:12.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:12.216+0000 7f4ce0200640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f4cd80cbab0 con 0x7f4cd815a710 2024-05-09T08:02:12.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:12.216+0000 7f4cdce00640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f4cd80cb070 0x7f4cd80cfe40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:52168/0 (socket says 172.21.15.1:52168) 2024-05-09T08:02:12.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:12.216+0000 7f4cdce00640 1 -- 172.21.15.1:0/2668125638 learned_addr learned my addr 172.21.15.1:0/2668125638 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T08:02:12.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:12.216+0000 7f4cdd800640 1 --1- 172.21.15.1:0/2668125638 >> v1:172.21.15.7:6789/0 conn(0x7f4cd8154970 0x7f4cd815e6c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.1:56584/0 (socket says 172.21.15.1:56584) 2024-05-09T08:02:12.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:12.216+0000 7f4cde200640 1 --2- 172.21.15.1:0/2668125638 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f4cd815a710 0x7f4cd80b4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:12.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:12.216+0000 7f4cd7e00640 1 -- 172.21.15.1:0/2668125638 <== mon.1 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2388109683 0 0) 0x7f4cd815b0a0 con 0x7f4cd8154970 2024-05-09T08:02:12.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:12.216+0000 7f4cde200640 1 -- 172.21.15.1:0/2668125638 >> v1:172.21.15.7:6789/0 conn(0x7f4cd8154970 legacy=0x7f4cd815e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:12.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:12.217+0000 7f4cde200640 1 -- 172.21.15.1:0/2668125638 >> v1:172.21.15.1:6789/0 conn(0x7f4cd80cb070 legacy=0x7f4cd80cfe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:12.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:12.217+0000 7f4cde200640 1 -- 172.21.15.1:0/2668125638 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cd815b380 con 0x7f4cd815a710 2024-05-09T08:02:12.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:12.218+0000 7f4cde200640 1 --2- 172.21.15.1:0/2668125638 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f4cd815a710 0x7f4cd80b4430 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f4cd00097d0 tx=0x7f4cd002a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a651c0646917c258 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:12.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:12.218+0000 7f4cd7e00640 1 -- 172.21.15.1:0/2668125638 <== mon.2 v2:172.21.15.7:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cd0037070 con 0x7f4cd815a710 2024-05-09T08:02:12.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:12.218+0000 7f4cd7e00640 1 -- 172.21.15.1:0/2668125638 <== mon.2 v2:172.21.15.7:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4cd002ad90 con 0x7f4cd815a710 2024-05-09T08:02:12.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:12.218+0000 7f4cd7e00640 1 -- 172.21.15.1:0/2668125638 <== mon.2 v2:172.21.15.7:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cd0032660 con 0x7f4cd815a710 2024-05-09T08:02:12.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:12.218+0000 7f4ce0200640 1 -- 172.21.15.1:0/2668125638 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f4cd815a710 msgr2=0x7f4cd80b4430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:12.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:12.218+0000 7f4ce0200640 1 --2- 172.21.15.1:0/2668125638 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f4cd815a710 0x7f4cd80b4430 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f4cd00097d0 tx=0x7f4cd002a100 comp rx=0 tx=0).stop 2024-05-09T08:02:12.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:12.219+0000 7f4ce0200640 1 -- 172.21.15.1:0/2668125638 shutdown_connections 2024-05-09T08:02:12.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:12.219+0000 7f4ce0200640 1 --2- 172.21.15.1:0/2668125638 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f4cd815a710 0x7f4cd80b4430 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:12.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:12.219+0000 7f4ce0200640 1 -- 172.21.15.1:0/2668125638 >> 172.21.15.1:0/2668125638 conn(0x7f4cd808bff0 msgr2=0x7f4cd808c400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:02:12.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:12.219+0000 7f4ce0200640 1 -- 172.21.15.1:0/2668125638 shutdown_connections 2024-05-09T08:02:12.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:12.219+0000 7f4ce0200640 1 -- 172.21.15.1:0/2668125638 wait complete. 2024-05-09T08:02:12.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:12.221+0000 7f4ce0200640 1 Processor -- start 2024-05-09T08:02:12.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:12.221+0000 7f4ce0200640 1 -- start start 2024-05-09T08:02:12.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:12.222+0000 7f4ce0200640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4cd8154970 0x7f4cd8148240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:12.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:12.222+0000 7f4ce0200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4cd815a710 0x7f4cd8148780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:12.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:12.222+0000 7f4ce0200640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f4cd815b8d0 0x7f4cd815ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:12.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:12.222+0000 7f4ce0200640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f4cd815ab60 con 0x7f4cd8154970 2024-05-09T08:02:12.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:12.222+0000 7f4ce0200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f4cd80b3fa0 con 0x7f4cd815a710 2024-05-09T08:02:12.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:12.222+0000 7f4ce0200640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f4cd80b3c30 con 0x7f4cd815b8d0 2024-05-09T08:02:12.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:12.222+0000 7f4cde200640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f4cd815b8d0 0x7f4cd815ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:12.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:12.222+0000 7f4cdd800640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4cd8154970 0x7f4cd8148240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:12.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:12.222+0000 7f4cdce00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4cd815a710 0x7f4cd8148780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:12.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:12.222+0000 7f4cde200640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f4cd815b8d0 0x7f4cd815ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3301/0 says I am v2:172.21.15.1:41932/0 (socket says 172.21.15.1:41932) 2024-05-09T08:02:12.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:12.222+0000 7f4cde200640 1 -- 172.21.15.1:0/720260221 learned_addr learned my addr 172.21.15.1:0/720260221 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:02:12.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:12.223+0000 7f4cde200640 1 -- 172.21.15.1:0/720260221 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4cd815a710 msgr2=0x7f4cd8148780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:12.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:12.223+0000 7f4cde200640 1 --2- 172.21.15.1:0/720260221 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4cd815a710 0x7f4cd8148780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:12.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:12.223+0000 7f4cde200640 1 -- 172.21.15.1:0/720260221 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4cd8154970 msgr2=0x7f4cd8148240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:12.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:12.223+0000 7f4cde200640 1 --2- 172.21.15.1:0/720260221 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4cd8154970 0x7f4cd8148240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:12.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:12.223+0000 7f4cde200640 1 -- 172.21.15.1:0/720260221 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cd81e4fe0 con 0x7f4cd815b8d0 2024-05-09T08:02:12.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:12.223+0000 7f4cde200640 1 --2- 172.21.15.1:0/720260221 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f4cd815b8d0 0x7f4cd815ccf0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f4cd0009720 tx=0x7f4cd00046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:12.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:12.223+0000 7f4cd5600640 1 -- 172.21.15.1:0/720260221 <== mon.2 v2:172.21.15.7:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cd0041070 con 0x7f4cd815b8d0 2024-05-09T08:02:12.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:12.224+0000 7f4cd5600640 1 -- 172.21.15.1:0/720260221 <== mon.2 v2:172.21.15.7:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4cd0032e10 con 0x7f4cd815b8d0 2024-05-09T08:02:12.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:12.224+0000 7f4ce0200640 1 -- 172.21.15.1:0/720260221 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4cd81e52a0 con 0x7f4cd815b8d0 2024-05-09T08:02:12.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:12.224+0000 7f4cd5600640 1 -- 172.21.15.1:0/720260221 <== mon.2 v2:172.21.15.7:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cd0037040 con 0x7f4cd815b8d0 2024-05-09T08:02:12.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:12.224+0000 7f4ce0200640 1 -- 172.21.15.1:0/720260221 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f4cd81e3f10 con 0x7f4cd815b8d0 2024-05-09T08:02:12.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:12.225+0000 7f4ce0200640 1 -- 172.21.15.1:0/720260221 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4cd8074d70 con 0x7f4cd815b8d0 2024-05-09T08:02:12.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:12.226+0000 7f4cd5600640 1 -- 172.21.15.1:0/720260221 <== mon.2 v2:172.21.15.7:3301/0 4 ==== mgrmap(e 4) ==== 90442+0+0 (secure 0 0 0) 0x7f4cd0044070 con 0x7f4cd815b8d0 2024-05-09T08:02:12.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:12.227+0000 7f4cd5600640 1 --2- 172.21.15.1:0/720260221 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f4cb806bda0 0x7f4cb806e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:12.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:12.227+0000 7f4cd5600640 1 -- 172.21.15.1:0/720260221 <== mon.2 v2:172.21.15.7:3301/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7f4cd00ab080 con 0x7f4cd815b8d0 2024-05-09T08:02:12.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:12.227+0000 7f4cdd800640 1 --2- 172.21.15.1:0/720260221 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f4cb806bda0 0x7f4cb806e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:12.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:12.228+0000 7f4cdd800640 1 --2- 172.21.15.1:0/720260221 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f4cb806bda0 0x7f4cb806e260 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f4ccc001920 tx=0x7f4ccc0018b0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:12.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:12.231+0000 7f4cd5600640 1 -- 172.21.15.1:0/720260221 <== mon.2 v2:172.21.15.7:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f4cd0079770 con 0x7f4cd815b8d0 2024-05-09T08:02:12.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:12.439+0000 7f4ce0200640 1 -- 172.21.15.1:0/720260221 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f4cd80ccfe0 con 0x7f4cd815b8d0 2024-05-09T08:02:12.456 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T08:02:12.457 INFO:teuthology.orchestra.run.smithi001.stdout:[ 2024-05-09T08:02:12.457 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:02:12.457 INFO:teuthology.orchestra.run.smithi001.stdout: "poolnum": 1, 2024-05-09T08:02:12.457 INFO:teuthology.orchestra.run.smithi001.stdout: "poolname": ".mgr" 2024-05-09T08:02:12.457 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:02:12.457 INFO:teuthology.orchestra.run.smithi001.stdout:] 2024-05-09T08:02:12.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:12.440+0000 7f4cd5600640 1 -- 172.21.15.1:0/720260221 <== mon.2 v2:172.21.15.7:3301/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v17) ==== 84+0+65 (secure 0 0 0) 0x7f4cd007da70 con 0x7f4cd815b8d0 2024-05-09T08:02:12.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:12.442+0000 7f4ce0200640 1 -- 172.21.15.1:0/720260221 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f4cb806bda0 msgr2=0x7f4cb806e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:12.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:12.442+0000 7f4ce0200640 1 --2- 172.21.15.1:0/720260221 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f4cb806bda0 0x7f4cb806e260 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f4ccc001920 tx=0x7f4ccc0018b0 comp rx=0 tx=0).stop 2024-05-09T08:02:12.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:12.442+0000 7f4ce0200640 1 -- 172.21.15.1:0/720260221 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f4cd815b8d0 msgr2=0x7f4cd815ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:12.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:12.442+0000 7f4ce0200640 1 --2- 172.21.15.1:0/720260221 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f4cd815b8d0 0x7f4cd815ccf0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f4cd0009720 tx=0x7f4cd00046c0 comp rx=0 tx=0).stop 2024-05-09T08:02:12.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:12.442+0000 7f4ce0200640 1 -- 172.21.15.1:0/720260221 shutdown_connections 2024-05-09T08:02:12.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:12.442+0000 7f4ce0200640 1 --2- 172.21.15.1:0/720260221 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f4cb806bda0 0x7f4cb806e260 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:12.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:12.442+0000 7f4ce0200640 1 --2- 172.21.15.1:0/720260221 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4cd815a710 0x7f4cd8148780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:12.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:12.442+0000 7f4ce0200640 1 --2- 172.21.15.1:0/720260221 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f4cd815b8d0 0x7f4cd815ccf0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:12.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:12.442+0000 7f4ce0200640 1 --2- 172.21.15.1:0/720260221 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4cd8154970 0x7f4cd8148240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:12.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:12.442+0000 7f4ce0200640 1 -- 172.21.15.1:0/720260221 >> 172.21.15.1:0/720260221 conn(0x7f4cd808bff0 msgr2=0x7f4cd80cc550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:02:12.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:12.443+0000 7f4ce0200640 1 -- 172.21.15.1:0/720260221 shutdown_connections 2024-05-09T08:02:12.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:12.443+0000 7f4ce0200640 1 -- 172.21.15.1:0/720260221 wait complete. 2024-05-09T08:02:12.477 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-09T08:02:12.477 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-09T08:02:12.667 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:12.650+0000 7feabf200640 1 Processor -- start 2024-05-09T08:02:12.667 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:12.650+0000 7feabf200640 1 -- start start 2024-05-09T08:02:12.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:12.651+0000 7feabf200640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7feac00c54f0 0x7feac006bc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:12.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:12.651+0000 7feabf200640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7feac01528e0 0x7feac0072280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:12.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:12.651+0000 7feabf200640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feac01587e0 con 0x7feac00c58d0 2024-05-09T08:02:12.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:12.651+0000 7feabf200640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7feac0159360 con 0x7feac00c54f0 2024-05-09T08:02:12.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:12.651+0000 7feabf200640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7feac0071df0 con 0x7feac01528e0 2024-05-09T08:02:12.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:12.652+0000 7feabe800640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7feac01528e0 0x7feac0072280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:12.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:12.652+0000 7feabd400640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7feac00c58d0 0x7feac01521d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:52176/0 (socket says 172.21.15.1:52176) 2024-05-09T08:02:12.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:12.652+0000 7feabd400640 1 -- 172.21.15.1:0/1380955058 learned_addr learned my addr 172.21.15.1:0/1380955058 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T08:02:12.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:12.652+0000 7feabde00640 1 --2- 172.21.15.1:0/1380955058 >> v2:172.21.15.1:3300/0 conn(0x7feac00c54f0 0x7feac006bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:12.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:12.652+0000 7feabe800640 1 -- 172.21.15.1:0/1380955058 >> v2:172.21.15.1:3300/0 conn(0x7feac00c54f0 msgr2=0x7feac006bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:02:12.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:12.652+0000 7feabe800640 1 --2- 172.21.15.1:0/1380955058 >> v2:172.21.15.1:3300/0 conn(0x7feac00c54f0 0x7feac006bc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:12.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:12.652+0000 7feabe800640 1 -- 172.21.15.1:0/1380955058 >> v1:172.21.15.1:6789/0 conn(0x7feac00c58d0 legacy=0x7feac01521d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:12.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:12.652+0000 7feabe800640 1 -- 172.21.15.1:0/1380955058 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feac0158ac0 con 0x7feac01528e0 2024-05-09T08:02:12.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:12.654+0000 7feabe800640 1 --2- 172.21.15.1:0/1380955058 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feac01528e0 0x7feac0072280 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7feac006b610 tx=0x7feab802a710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5d2f2faa9509ae9d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:12.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:12.654+0000 7feabca00640 1 -- 172.21.15.1:0/1380955058 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feab8007740 con 0x7feac01528e0 2024-05-09T08:02:12.671 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:12.654+0000 7feabca00640 1 -- 172.21.15.1:0/1380955058 <== mon.1 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feab80078e0 con 0x7feac01528e0 2024-05-09T08:02:12.671 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:12.654+0000 7feabca00640 1 -- 172.21.15.1:0/1380955058 <== mon.1 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feab8007cb0 con 0x7feac01528e0 2024-05-09T08:02:12.671 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:12.654+0000 7feabf200640 1 -- 172.21.15.1:0/1380955058 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feac01528e0 msgr2=0x7feac0072280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:12.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:12.654+0000 7feabf200640 1 --2- 172.21.15.1:0/1380955058 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feac01528e0 0x7feac0072280 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7feac006b610 tx=0x7feab802a710 comp rx=0 tx=0).stop 2024-05-09T08:02:12.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:12.655+0000 7feabf200640 1 -- 172.21.15.1:0/1380955058 shutdown_connections 2024-05-09T08:02:12.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:12.655+0000 7feabf200640 1 --2- 172.21.15.1:0/1380955058 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feac01528e0 0x7feac0072280 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:12.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:12.655+0000 7feabf200640 1 --2- 172.21.15.1:0/1380955058 >> v2:172.21.15.1:3300/0 conn(0x7feac00c54f0 0x7feac006bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:12.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:12.655+0000 7feabf200640 1 -- 172.21.15.1:0/1380955058 >> 172.21.15.1:0/1380955058 conn(0x7feac006e080 msgr2=0x7feac006f510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:02:12.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:12.655+0000 7feabf200640 1 -- 172.21.15.1:0/1380955058 shutdown_connections 2024-05-09T08:02:12.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:12.655+0000 7feabf200640 1 -- 172.21.15.1:0/1380955058 wait complete. 2024-05-09T08:02:12.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:12.656+0000 7feabf200640 1 Processor -- start 2024-05-09T08:02:12.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:12.657+0000 7feabf200640 1 -- start start 2024-05-09T08:02:12.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:12.657+0000 7feabf200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feac00c54f0 0x7feac00c4330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:12.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:12.657+0000 7feabf200640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7feac00c58d0 0x7feac01531a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:12.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:12.657+0000 7feabf200640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7feac01528e0 0x7feac01536e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:12.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:12.657+0000 7feabf200640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7feac01583a0 con 0x7feac01528e0 2024-05-09T08:02:12.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:12.657+0000 7feabf200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7feac0159080 con 0x7feac00c54f0 2024-05-09T08:02:12.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:12.657+0000 7feabf200640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7feac0158520 con 0x7feac00c58d0 2024-05-09T08:02:12.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:12.658+0000 7feabde00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feac00c54f0 0x7feac00c4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:12.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:12.658+0000 7feabd400640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7feac00c58d0 0x7feac01531a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:12.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:12.658+0000 7feabde00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feac00c54f0 0x7feac00c4330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.1:37014/0 (socket says 172.21.15.1:37014) 2024-05-09T08:02:12.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:12.658+0000 7feabe800640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7feac01528e0 0x7feac01536e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:12.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:12.658+0000 7feabde00640 1 -- 172.21.15.1:0/55276571 learned_addr learned my addr 172.21.15.1:0/55276571 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:02:12.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:12.658+0000 7feabde00640 1 -- 172.21.15.1:0/55276571 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7feac00c58d0 msgr2=0x7feac01531a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:12.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:12.658+0000 7feabde00640 1 --2- 172.21.15.1:0/55276571 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7feac00c58d0 0x7feac01531a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:12.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:12.658+0000 7feabde00640 1 -- 172.21.15.1:0/55276571 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7feac01528e0 msgr2=0x7feac01536e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:12.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:12.658+0000 7feabde00640 1 --2- 172.21.15.1:0/55276571 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7feac01528e0 0x7feac01536e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:12.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:12.658+0000 7feabde00640 1 -- 172.21.15.1:0/55276571 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feac01544b0 con 0x7feac00c54f0 2024-05-09T08:02:12.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:12.658+0000 7feabd400640 1 --2- 172.21.15.1:0/55276571 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7feac00c58d0 0x7feac01531a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:02:12.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:12.658+0000 7feabde00640 1 --2- 172.21.15.1:0/55276571 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feac00c54f0 0x7feac00c4330 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7feab000e9e0 tx=0x7feab000eea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:12.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:12.659+0000 7feab6000640 1 -- 172.21.15.1:0/55276571 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feab0017070 con 0x7feac00c54f0 2024-05-09T08:02:12.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:12.659+0000 7feab6000640 1 -- 172.21.15.1:0/55276571 <== mon.1 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feab000c9d0 con 0x7feac00c54f0 2024-05-09T08:02:12.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:12.659+0000 7feab6000640 1 -- 172.21.15.1:0/55276571 <== mon.1 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feab000cdc0 con 0x7feac00c54f0 2024-05-09T08:02:12.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:12.659+0000 7feabf200640 1 -- 172.21.15.1:0/55276571 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feac0154790 con 0x7feac00c54f0 2024-05-09T08:02:12.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:12.659+0000 7feabf200640 1 -- 172.21.15.1:0/55276571 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feac0154d50 con 0x7feac00c54f0 2024-05-09T08:02:12.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:12.660+0000 7feabf200640 1 -- 172.21.15.1:0/55276571 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feac0074d70 con 0x7feac00c54f0 2024-05-09T08:02:12.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:12.661+0000 7feab6000640 1 -- 172.21.15.1:0/55276571 <== mon.1 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 4) ==== 90442+0+0 (secure 0 0 0) 0x7feab001a070 con 0x7feac00c54f0 2024-05-09T08:02:12.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:12.662+0000 7feab6000640 1 --2- 172.21.15.1:0/55276571 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7feaa006bda0 0x7feaa006e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:12.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:12.662+0000 7feab6000640 1 -- 172.21.15.1:0/55276571 <== mon.1 v2:172.21.15.7:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7feab008adb0 con 0x7feac00c54f0 2024-05-09T08:02:12.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:12.662+0000 7feabd400640 1 --2- 172.21.15.1:0/55276571 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7feaa006bda0 0x7feaa006e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:12.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:12.663+0000 7feabd400640 1 --2- 172.21.15.1:0/55276571 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7feaa006bda0 0x7feaa006e260 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7feac00b08d0 tx=0x7feaac007e80 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:12.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:12.667+0000 7feab6000640 1 -- 172.21.15.1:0/55276571 <== mon.1 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7feab00594a0 con 0x7feac00c54f0 2024-05-09T08:02:12.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:12.868+0000 7feabf200640 1 -- 172.21.15.1:0/55276571 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7feac0157f20 con 0x7feac00c54f0 2024-05-09T08:02:13.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:13.856+0000 7feab6000640 1 -- 172.21.15.1:0/55276571 <== mon.1 v2:172.21.15.7:3300/0 7 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7feab0055b10 con 0x7feac00c54f0 2024-05-09T08:02:13.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:13.857+0000 7feab6000640 1 -- 172.21.15.1:0/55276571 <== mon.1 v2:172.21.15.7:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v18) ==== 138+0+0 (secure 0 0 0) 0x7feab0051e60 con 0x7feac00c54f0 2024-05-09T08:02:13.878 INFO:teuthology.orchestra.run.smithi001.stderr:pool 'cephfs_metadata' created 2024-05-09T08:02:13.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:13.865+0000 7feabf200640 1 -- 172.21.15.1:0/55276571 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7feaa006bda0 msgr2=0x7feaa006e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:13.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:13.865+0000 7feabf200640 1 --2- 172.21.15.1:0/55276571 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7feaa006bda0 0x7feaa006e260 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7feac00b08d0 tx=0x7feaac007e80 comp rx=0 tx=0).stop 2024-05-09T08:02:13.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:13.865+0000 7feabf200640 1 -- 172.21.15.1:0/55276571 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feac00c54f0 msgr2=0x7feac00c4330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:13.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:13.865+0000 7feabf200640 1 --2- 172.21.15.1:0/55276571 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feac00c54f0 0x7feac00c4330 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7feab000e9e0 tx=0x7feab000eea0 comp rx=0 tx=0).stop 2024-05-09T08:02:13.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:13.868+0000 7feabf200640 1 -- 172.21.15.1:0/55276571 shutdown_connections 2024-05-09T08:02:13.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:13.868+0000 7feabf200640 1 --2- 172.21.15.1:0/55276571 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7feaa006bda0 0x7feaa006e260 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:13.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:13.868+0000 7feabf200640 1 --2- 172.21.15.1:0/55276571 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7feac01528e0 0x7feac01536e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:13.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:13.868+0000 7feabf200640 1 --2- 172.21.15.1:0/55276571 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7feac00c58d0 0x7feac01531a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:13.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:13.868+0000 7feabf200640 1 --2- 172.21.15.1:0/55276571 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feac00c54f0 0x7feac00c4330 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:13.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:13.868+0000 7feabf200640 1 -- 172.21.15.1:0/55276571 >> 172.21.15.1:0/55276571 conn(0x7feac006e080 msgr2=0x7feac00b12f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:02:13.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:13.868+0000 7feabf200640 1 -- 172.21.15.1:0/55276571 shutdown_connections 2024-05-09T08:02:13.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:13.870+0000 7feabf200640 1 -- 172.21.15.1:0/55276571 wait complete. 2024-05-09T08:02:13.906 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-05-09T08:02:14.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:14.089+0000 7fc045e00640 1 Processor -- start 2024-05-09T08:02:14.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:14.089+0000 7fc045e00640 1 -- start start 2024-05-09T08:02:14.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:14.091+0000 7fc045e00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fc04006f970 0x7fc0400c52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:14.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:14.091+0000 7fc045e00640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc0400c3d80 con 0x7fc04006f590 2024-05-09T08:02:14.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:14.091+0000 7fc045e00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc0400c5ed0 con 0x7fc040156fd0 2024-05-09T08:02:14.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:14.091+0000 7fc045e00640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7fc0400c38f0 con 0x7fc04006f970 2024-05-09T08:02:14.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:14.091+0000 7fc03fe00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fc04006f970 0x7fc0400c52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:14.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:14.091+0000 7fc03fe00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fc04006f970 0x7fc0400c52b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3301/0 says I am v2:172.21.15.1:42000/0 (socket says 172.21.15.1:42000) 2024-05-09T08:02:14.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:14.091+0000 7fc03fe00640 1 -- 172.21.15.1:0/2664503488 learned_addr learned my addr 172.21.15.1:0/2664503488 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:02:14.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:14.091+0000 7fc045400640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fc040156fd0 0x7fc040150d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.1:59858/0 (socket says 172.21.15.1:59858) 2024-05-09T08:02:14.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:14.091+0000 7fc044a00640 1 --1- 172.21.15.1:0/2664503488 >> v1:172.21.15.1:6789/0 conn(0x7fc04006f590 0x7fc040073100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:40226/0 (socket says 172.21.15.1:40226) 2024-05-09T08:02:14.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:14.092+0000 7fc03fe00640 1 -- 172.21.15.1:0/2664503488 >> v1:172.21.15.7:6789/0 conn(0x7fc040156fd0 legacy=0x7fc040150d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:14.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:14.092+0000 7fc03f400640 1 -- 172.21.15.1:0/2664503488 <== mon.1 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1933332934 0 0) 0x7fc0400c5ed0 con 0x7fc040156fd0 2024-05-09T08:02:14.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:14.092+0000 7fc03fe00640 1 -- 172.21.15.1:0/2664503488 >> v1:172.21.15.1:6789/0 conn(0x7fc04006f590 legacy=0x7fc040073100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:14.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:14.092+0000 7fc03fe00640 1 -- 172.21.15.1:0/2664503488 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0400c3000 con 0x7fc04006f970 2024-05-09T08:02:14.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:14.093+0000 7fc03fe00640 1 --2- 172.21.15.1:0/2664503488 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fc04006f970 0x7fc0400c52b0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fc02c0098a0 tx=0x7fc02c02a140 comp rx=0 tx=0).ready entity=mon.2 client_cookie=59ba3386b8325ca9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:14.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:14.093+0000 7fc03f400640 1 -- 172.21.15.1:0/2664503488 <== mon.2 v2:172.21.15.7:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc02c037070 con 0x7fc04006f970 2024-05-09T08:02:14.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:14.093+0000 7fc03f400640 1 -- 172.21.15.1:0/2664503488 <== mon.2 v2:172.21.15.7:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc02c02add0 con 0x7fc04006f970 2024-05-09T08:02:14.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:14.093+0000 7fc03f400640 1 -- 172.21.15.1:0/2664503488 <== mon.2 v2:172.21.15.7:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc02c032660 con 0x7fc04006f970 2024-05-09T08:02:14.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:14.094+0000 7fc045e00640 1 -- 172.21.15.1:0/2664503488 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fc04006f970 msgr2=0x7fc0400c52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:14.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:14.094+0000 7fc045e00640 1 --2- 172.21.15.1:0/2664503488 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fc04006f970 0x7fc0400c52b0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fc02c0098a0 tx=0x7fc02c02a140 comp rx=0 tx=0).stop 2024-05-09T08:02:14.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:14.094+0000 7fc045e00640 1 -- 172.21.15.1:0/2664503488 shutdown_connections 2024-05-09T08:02:14.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:14.094+0000 7fc045e00640 1 --2- 172.21.15.1:0/2664503488 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fc04006f970 0x7fc0400c52b0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:14.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:14.094+0000 7fc045e00640 1 -- 172.21.15.1:0/2664503488 >> 172.21.15.1:0/2664503488 conn(0x7fc0400cb9d0 msgr2=0x7fc0400cbde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:02:14.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:14.094+0000 7fc045e00640 1 -- 172.21.15.1:0/2664503488 shutdown_connections 2024-05-09T08:02:14.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:14.094+0000 7fc045e00640 1 -- 172.21.15.1:0/2664503488 wait complete. 2024-05-09T08:02:14.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:14.096+0000 7fc045e00640 1 Processor -- start 2024-05-09T08:02:14.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:14.096+0000 7fc045e00640 1 -- start start 2024-05-09T08:02:14.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:14.097+0000 7fc045e00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fc04006f590 0x7fc0401f0350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:14.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:14.097+0000 7fc045e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc04006f970 0x7fc04014e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:14.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:14.097+0000 7fc045e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc040156fd0 0x7fc0401e14c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:14.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:14.097+0000 7fc045e00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fc0400c3580 con 0x7fc04006f970 2024-05-09T08:02:14.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:14.097+0000 7fc045e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc0400c32a0 con 0x7fc040156fd0 2024-05-09T08:02:14.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:14.097+0000 7fc045e00640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7fc0400c57f0 con 0x7fc04006f590 2024-05-09T08:02:14.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:14.097+0000 7fc03fe00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc04006f970 0x7fc04014e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:14.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:14.097+0000 7fc03fe00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc04006f970 0x7fc04014e7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43302/0 (socket says 172.21.15.1:43302) 2024-05-09T08:02:14.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:14.097+0000 7fc03fe00640 1 -- 172.21.15.1:0/2686056646 learned_addr learned my addr 172.21.15.1:0/2686056646 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:02:14.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:14.097+0000 7fc044a00640 1 --2- 172.21.15.1:0/2686056646 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fc04006f590 0x7fc0401f0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:14.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:14.097+0000 7fc045400640 1 --2- 172.21.15.1:0/2686056646 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc040156fd0 0x7fc0401e14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:14.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:14.098+0000 7fc03fe00640 1 -- 172.21.15.1:0/2686056646 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fc04006f590 msgr2=0x7fc0401f0350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:14.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:14.098+0000 7fc03fe00640 1 --2- 172.21.15.1:0/2686056646 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fc04006f590 0x7fc0401f0350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:14.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:14.098+0000 7fc03fe00640 1 -- 172.21.15.1:0/2686056646 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc040156fd0 msgr2=0x7fc0401e14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:14.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:14.098+0000 7fc03fe00640 1 --2- 172.21.15.1:0/2686056646 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc040156fd0 0x7fc0401e14c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:14.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:14.098+0000 7fc03fe00640 1 -- 172.21.15.1:0/2686056646 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0401f0a70 con 0x7fc04006f970 2024-05-09T08:02:14.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:14.098+0000 7fc03fe00640 1 --2- 172.21.15.1:0/2686056646 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc04006f970 0x7fc04014e7b0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fc02c0097f0 tx=0x7fc02c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:14.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:14.099+0000 7fc03cc00640 1 -- 172.21.15.1:0/2686056646 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc02c037040 con 0x7fc04006f970 2024-05-09T08:02:14.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:14.099+0000 7fc03cc00640 1 -- 172.21.15.1:0/2686056646 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc02c004240 con 0x7fc04006f970 2024-05-09T08:02:14.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:14.099+0000 7fc045e00640 1 -- 172.21.15.1:0/2686056646 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0401f0d50 con 0x7fc04006f970 2024-05-09T08:02:14.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:14.099+0000 7fc03cc00640 1 -- 172.21.15.1:0/2686056646 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc02c003610 con 0x7fc04006f970 2024-05-09T08:02:14.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:14.099+0000 7fc045e00640 1 -- 172.21.15.1:0/2686056646 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc040155550 con 0x7fc04006f970 2024-05-09T08:02:14.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:14.100+0000 7fc045e00640 1 -- 172.21.15.1:0/2686056646 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc04014db10 con 0x7fc04006f970 2024-05-09T08:02:14.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:14.101+0000 7fc03cc00640 1 -- 172.21.15.1:0/2686056646 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc02c03a070 con 0x7fc04006f970 2024-05-09T08:02:14.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:14.102+0000 7fc03cc00640 1 --2- 172.21.15.1:0/2686056646 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fc02006be30 0x7fc02006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:14.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:14.102+0000 7fc03cc00640 1 -- 172.21.15.1:0/2686056646 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 5227+0+0 (secure 0 0 0) 0x7fc02c0ab170 con 0x7fc04006f970 2024-05-09T08:02:14.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:14.102+0000 7fc044a00640 1 --2- 172.21.15.1:0/2686056646 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fc02006be30 0x7fc02006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:14.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:14.103+0000 7fc044a00640 1 --2- 172.21.15.1:0/2686056646 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fc02006be30 0x7fc02006e2f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc034000f60 tx=0x7fc034007e20 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:14.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:14.108+0000 7fc03cc00640 1 -- 172.21.15.1:0/2686056646 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fc02c04a030 con 0x7fc04006f970 2024-05-09T08:02:14.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:14.311+0000 7fc045e00640 1 -- 172.21.15.1:0/2686056646 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) -- 0x7fc04004f6d0 con 0x7fc04006f970 2024-05-09T08:02:14.863 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:14.847+0000 7fc03cc00640 1 -- 172.21.15.1:0/2686056646 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v19) ==== 171+0+0 (secure 0 0 0) 0x7fc02c041030 con 0x7fc04006f970 2024-05-09T08:02:14.863 INFO:teuthology.orchestra.run.smithi001.stderr:pool 'cephfs_data' created 2024-05-09T08:02:14.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:14.852+0000 7fc045e00640 1 -- 172.21.15.1:0/2686056646 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fc02006be30 msgr2=0x7fc02006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:14.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:14.852+0000 7fc045e00640 1 --2- 172.21.15.1:0/2686056646 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fc02006be30 0x7fc02006e2f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc034000f60 tx=0x7fc034007e20 comp rx=0 tx=0).stop 2024-05-09T08:02:14.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:14.852+0000 7fc045e00640 1 -- 172.21.15.1:0/2686056646 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc04006f970 msgr2=0x7fc04014e7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:14.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:14.852+0000 7fc045e00640 1 --2- 172.21.15.1:0/2686056646 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc04006f970 0x7fc04014e7b0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fc02c0097f0 tx=0x7fc02c0046c0 comp rx=0 tx=0).stop 2024-05-09T08:02:14.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:14.852+0000 7fc045e00640 1 -- 172.21.15.1:0/2686056646 shutdown_connections 2024-05-09T08:02:14.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:14.852+0000 7fc045e00640 1 --2- 172.21.15.1:0/2686056646 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fc02006be30 0x7fc02006e2f0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:14.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:14.852+0000 7fc045e00640 1 --2- 172.21.15.1:0/2686056646 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc040156fd0 0x7fc0401e14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:14.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:14.852+0000 7fc045e00640 1 --2- 172.21.15.1:0/2686056646 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc04006f970 0x7fc04014e7b0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:14.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:14.852+0000 7fc045e00640 1 --2- 172.21.15.1:0/2686056646 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fc04006f590 0x7fc0401f0350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:14.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:14.852+0000 7fc045e00640 1 -- 172.21.15.1:0/2686056646 >> 172.21.15.1:0/2686056646 conn(0x7fc0400cb9d0 msgr2=0x7fc040143620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:02:14.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:14.855+0000 7fc045e00640 1 -- 172.21.15.1:0/2686056646 shutdown_connections 2024-05-09T08:02:14.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:14.855+0000 7fc045e00640 1 -- 172.21.15.1:0/2686056646 wait complete. 2024-05-09T08:02:14.907 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-09T08:02:15.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:15.071+0000 7f7765600640 1 Processor -- start 2024-05-09T08:02:15.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:15.071+0000 7f7765600640 1 -- start start 2024-05-09T08:02:15.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:15.072+0000 7f7765600640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f776006c900 0x7f77600c7280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:15.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:15.072+0000 7f7765600640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f77600c77c0 0x7f77600c81f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:15.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:15.072+0000 7f7765600640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f776015b860 con 0x7f77600c8730 2024-05-09T08:02:15.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:15.072+0000 7f7765600640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f77600c7f30 con 0x7f776006c900 2024-05-09T08:02:15.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:15.072+0000 7f7765600640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f77600b5de0 con 0x7f77600c77c0 2024-05-09T08:02:15.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:15.073+0000 7f775e000640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f77600c77c0 0x7f77600c81f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:15.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:15.073+0000 7f775ea00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f776006c900 0x7f77600c7280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:15.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:15.073+0000 7f775e000640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f77600c77c0 0x7f77600c81f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3301/0 says I am v2:172.21.15.1:42020/0 (socket says 172.21.15.1:42020) 2024-05-09T08:02:15.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:15.073+0000 7f775ea00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f776006c900 0x7f77600c7280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.1:41408/0 (socket says 172.21.15.1:41408) 2024-05-09T08:02:15.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:15.073+0000 7f775f400640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f77600c8730 0x7f77601547c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.1:59860/0 (socket says 172.21.15.1:59860) 2024-05-09T08:02:15.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:15.073+0000 7f775e000640 1 -- 172.21.15.1:0/691268066 learned_addr learned my addr 172.21.15.1:0/691268066 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:02:15.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:15.073+0000 7f775ea00640 1 -- 172.21.15.1:0/691268066 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f77600c77c0 msgr2=0x7f77600c81f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:02:15.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:15.073+0000 7f775ea00640 1 --2- 172.21.15.1:0/691268066 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f77600c77c0 0x7f77600c81f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:15.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:15.073+0000 7f775ea00640 1 -- 172.21.15.1:0/691268066 >> v1:172.21.15.7:6789/0 conn(0x7f77600c8730 legacy=0x7f77601547c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:15.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:15.073+0000 7f775ea00640 1 -- 172.21.15.1:0/691268066 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77600c7c50 con 0x7f776006c900 2024-05-09T08:02:15.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:15.073+0000 7f775e000640 1 --2- 172.21.15.1:0/691268066 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f77600c77c0 0x7f77600c81f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:02:15.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:15.073+0000 7f775d600640 1 -- 172.21.15.1:0/691268066 <== mon.1 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3273756821 0 0) 0x7f776015b860 con 0x7f77600c8730 2024-05-09T08:02:15.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:15.074+0000 7f775ea00640 1 --2- 172.21.15.1:0/691268066 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f776006c900 0x7f77600c7280 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f77500097a0 tx=0x7f775002a1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=20f1eed95328bad1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:15.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:15.075+0000 7f775d600640 1 -- 172.21.15.1:0/691268066 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7750037070 con 0x7f776006c900 2024-05-09T08:02:15.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:15.075+0000 7f775d600640 1 -- 172.21.15.1:0/691268066 <== mon.1 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7750032440 con 0x7f776006c900 2024-05-09T08:02:15.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:15.075+0000 7f775d600640 1 -- 172.21.15.1:0/691268066 <== mon.1 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7750032810 con 0x7f776006c900 2024-05-09T08:02:15.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:15.075+0000 7f7765600640 1 -- 172.21.15.1:0/691268066 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f776006c900 msgr2=0x7f77600c7280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:15.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:15.075+0000 7f7765600640 1 --2- 172.21.15.1:0/691268066 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f776006c900 0x7f77600c7280 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f77500097a0 tx=0x7f775002a1a0 comp rx=0 tx=0).stop 2024-05-09T08:02:15.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:15.075+0000 7f7765600640 1 -- 172.21.15.1:0/691268066 shutdown_connections 2024-05-09T08:02:15.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:15.075+0000 7f7765600640 1 --2- 172.21.15.1:0/691268066 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f77600c77c0 0x7f77600c81f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:15.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:15.075+0000 7f7765600640 1 --2- 172.21.15.1:0/691268066 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f776006c900 0x7f77600c7280 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:15.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:15.075+0000 7f7765600640 1 -- 172.21.15.1:0/691268066 >> 172.21.15.1:0/691268066 conn(0x7f77600cfe80 msgr2=0x7f77600d0290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:02:15.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:15.076+0000 7f7765600640 1 -- 172.21.15.1:0/691268066 shutdown_connections 2024-05-09T08:02:15.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:15.076+0000 7f7765600640 1 -- 172.21.15.1:0/691268066 wait complete. 2024-05-09T08:02:15.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:15.077+0000 7f7765600640 1 Processor -- start 2024-05-09T08:02:15.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:15.078+0000 7f7765600640 1 -- start start 2024-05-09T08:02:15.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:15.078+0000 7f7765600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f776006c900 0x7f77601f3e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:15.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:15.078+0000 7f7765600640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f77600c8730 0x7f77600ca970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:15.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:15.079+0000 7f7765600640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f77601eff90 0x7f77601597c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:15.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:15.079+0000 7f7765600640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f776015b060 con 0x7f77601eff90 2024-05-09T08:02:15.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:15.079+0000 7f7765600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f776015ad80 con 0x7f776006c900 2024-05-09T08:02:15.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:15.079+0000 7f7765600640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f7760154ed0 con 0x7f77600c8730 2024-05-09T08:02:15.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:15.079+0000 7f775e000640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f77600c8730 0x7f77600ca970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:15.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:15.079+0000 7f775ea00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f776006c900 0x7f77601f3e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:15.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:15.079+0000 7f775e000640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f77600c8730 0x7f77600ca970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3301/0 says I am v2:172.21.15.1:42026/0 (socket says 172.21.15.1:42026) 2024-05-09T08:02:15.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:15.079+0000 7f775e000640 1 -- 172.21.15.1:0/4097611359 learned_addr learned my addr 172.21.15.1:0/4097611359 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:02:15.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:15.079+0000 7f775f400640 1 --2- 172.21.15.1:0/4097611359 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f77601eff90 0x7f77601597c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:15.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:15.079+0000 7f775ea00640 1 -- 172.21.15.1:0/4097611359 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f77600c8730 msgr2=0x7f77600ca970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:15.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:15.079+0000 7f775ea00640 1 --2- 172.21.15.1:0/4097611359 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f77600c8730 0x7f77600ca970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:15.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:15.079+0000 7f775ea00640 1 -- 172.21.15.1:0/4097611359 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f77601eff90 msgr2=0x7f77601597c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:15.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:15.079+0000 7f775ea00640 1 --2- 172.21.15.1:0/4097611359 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f77601eff90 0x7f77601597c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:15.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:15.079+0000 7f775ea00640 1 -- 172.21.15.1:0/4097611359 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77600c42a0 con 0x7f776006c900 2024-05-09T08:02:15.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:15.079+0000 7f775e000640 1 --2- 172.21.15.1:0/4097611359 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f77600c8730 0x7f77600ca970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:02:15.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:15.080+0000 7f775ea00640 1 --2- 172.21.15.1:0/4097611359 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f776006c900 0x7f77601f3e30 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f7750031000 tx=0x7f77500046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:15.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:15.080+0000 7f7756a00640 1 -- 172.21.15.1:0/4097611359 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7750041020 con 0x7f776006c900 2024-05-09T08:02:15.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:15.080+0000 7f7756a00640 1 -- 172.21.15.1:0/4097611359 <== mon.1 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7750030720 con 0x7f776006c900 2024-05-09T08:02:15.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:15.080+0000 7f7756a00640 1 -- 172.21.15.1:0/4097611359 <== mon.1 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7750037040 con 0x7f776006c900 2024-05-09T08:02:15.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:15.080+0000 7f7765600640 1 -- 172.21.15.1:0/4097611359 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77600c4580 con 0x7f776006c900 2024-05-09T08:02:15.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:15.080+0000 7f7765600640 1 -- 172.21.15.1:0/4097611359 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77601f3630 con 0x7f776006c900 2024-05-09T08:02:15.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:15.082+0000 7f7765600640 1 -- 172.21.15.1:0/4097611359 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77600cdcd0 con 0x7f776006c900 2024-05-09T08:02:15.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:15.082+0000 7f7756a00640 1 -- 172.21.15.1:0/4097611359 <== mon.1 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7750044020 con 0x7f776006c900 2024-05-09T08:02:15.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:15.083+0000 7f7756a00640 1 --2- 172.21.15.1:0/4097611359 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f773c06be30 0x7f773c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:15.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:15.083+0000 7f7756a00640 1 -- 172.21.15.1:0/4097611359 <== mon.1 v2:172.21.15.7:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5609+0+0 (secure 0 0 0) 0x7f77500ab4f0 con 0x7f776006c900 2024-05-09T08:02:15.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:15.084+0000 7f775e000640 1 --2- 172.21.15.1:0/4097611359 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f773c06be30 0x7f773c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:15.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:15.084+0000 7f775e000640 1 --2- 172.21.15.1:0/4097611359 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f773c06be30 0x7f773c06e2f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f7748009c90 tx=0x7f774800b040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:15.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:15.088+0000 7f7756a00640 1 -- 172.21.15.1:0/4097611359 <== mon.1 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f77600cdcd0 con 0x7f776006c900 2024-05-09T08:02:15.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:15.330+0000 7f7765600640 1 -- 172.21.15.1:0/4097611359 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f776006ce10 con 0x7f776006c900 2024-05-09T08:02:15.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:15.898+0000 7f7756a00640 1 -- 172.21.15.1:0/4097611359 <== mon.1 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) ==== 172+0+0 (secure 0 0 0) 0x7f775004b030 con 0x7f776006c900 2024-05-09T08:02:15.919 INFO:teuthology.orchestra.run.smithi001.stderr:new fs with metadata pool 2 and data pool 3 2024-05-09T08:02:15.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:15.905+0000 7f7765600640 1 -- 172.21.15.1:0/4097611359 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f773c06be30 msgr2=0x7f773c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:15.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:15.905+0000 7f7765600640 1 --2- 172.21.15.1:0/4097611359 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f773c06be30 0x7f773c06e2f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f7748009c90 tx=0x7f774800b040 comp rx=0 tx=0).stop 2024-05-09T08:02:15.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:15.905+0000 7f7765600640 1 -- 172.21.15.1:0/4097611359 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f776006c900 msgr2=0x7f77601f3e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:15.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:15.905+0000 7f7765600640 1 --2- 172.21.15.1:0/4097611359 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f776006c900 0x7f77601f3e30 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f7750031000 tx=0x7f77500046c0 comp rx=0 tx=0).stop 2024-05-09T08:02:15.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:15.906+0000 7f7765600640 1 -- 172.21.15.1:0/4097611359 shutdown_connections 2024-05-09T08:02:15.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:15.906+0000 7f7765600640 1 --2- 172.21.15.1:0/4097611359 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f773c06be30 0x7f773c06e2f0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:15.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:15.906+0000 7f7765600640 1 --2- 172.21.15.1:0/4097611359 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f77601eff90 0x7f77601597c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:15.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:15.906+0000 7f7765600640 1 --2- 172.21.15.1:0/4097611359 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f77600c8730 0x7f77600ca970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:15.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:15.906+0000 7f7765600640 1 --2- 172.21.15.1:0/4097611359 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f776006c900 0x7f77601f3e30 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:15.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:15.906+0000 7f7765600640 1 -- 172.21.15.1:0/4097611359 >> 172.21.15.1:0/4097611359 conn(0x7f77600cfe80 msgr2=0x7f77600d0260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:02:15.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:15.907+0000 7f7765600640 1 -- 172.21.15.1:0/4097611359 shutdown_connections 2024-05-09T08:02:15.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:15.907+0000 7f7765600640 1 -- 172.21.15.1:0/4097611359 wait complete. 2024-05-09T08:02:15.947 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-05-09T08:02:15.947 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-05-09T08:02:16.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:16.110+0000 7f8fe2600640 1 Processor -- start 2024-05-09T08:02:16.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:16.111+0000 7f8fe2600640 1 -- start start 2024-05-09T08:02:16.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:16.112+0000 7f8fe2600640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f8fdc152550 0x7f8fdc0734e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:16.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:16.112+0000 7f8fe2600640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8fdc15c6f0 con 0x7f8fdc0723d0 2024-05-09T08:02:16.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:16.112+0000 7f8fe2600640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8fdc158f20 con 0x7f8fdc072830 2024-05-09T08:02:16.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:16.112+0000 7f8fe2600640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f8fdc06ed90 con 0x7f8fdc152550 2024-05-09T08:02:16.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:16.113+0000 7f8fdb400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f8fdc152550 0x7f8fdc0734e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:16.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:16.113+0000 7f8fdbe00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f8fdc072830 0x7f8fdc15bfe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.1:59864/0 (socket says 172.21.15.1:59864) 2024-05-09T08:02:16.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:16.113+0000 7f8fdbe00640 1 -- 172.21.15.1:0/3133586974 learned_addr learned my addr 172.21.15.1:0/3133586974 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T08:02:16.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:16.113+0000 7f8fdb400640 1 -- 172.21.15.1:0/3133586974 >> v1:172.21.15.7:6789/0 conn(0x7f8fdc072830 legacy=0x7f8fdc15bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:16.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:16.113+0000 7f8fdb400640 1 -- 172.21.15.1:0/3133586974 >> v1:172.21.15.1:6789/0 conn(0x7f8fdc0723d0 legacy=0x7f8fdc15b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:16.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:16.113+0000 7f8fdb400640 1 -- 172.21.15.1:0/3133586974 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fdc06eab0 con 0x7f8fdc152550 2024-05-09T08:02:16.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:16.114+0000 7f8fdb400640 1 --2- 172.21.15.1:0/3133586974 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8fdc152550 0x7f8fdc0734e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f8fcc009d90 tx=0x7f8fcc02a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2e52a08dac0b8437 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:16.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:16.114+0000 7f8fda000640 1 -- 172.21.15.1:0/3133586974 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fcc007740 con 0x7f8fdc152550 2024-05-09T08:02:16.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:16.114+0000 7f8fda000640 1 -- 172.21.15.1:0/3133586974 <== mon.1 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8fcc0078e0 con 0x7f8fdc152550 2024-05-09T08:02:16.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:16.114+0000 7f8fda000640 1 -- 172.21.15.1:0/3133586974 <== mon.1 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fcc007cb0 con 0x7f8fdc152550 2024-05-09T08:02:16.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:16.115+0000 7f8fe2600640 1 -- 172.21.15.1:0/3133586974 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8fdc152550 msgr2=0x7f8fdc0734e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:16.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:16.115+0000 7f8fe2600640 1 --2- 172.21.15.1:0/3133586974 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8fdc152550 0x7f8fdc0734e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f8fcc009d90 tx=0x7f8fcc02a660 comp rx=0 tx=0).stop 2024-05-09T08:02:16.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:16.115+0000 7f8fe2600640 1 -- 172.21.15.1:0/3133586974 shutdown_connections 2024-05-09T08:02:16.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:16.115+0000 7f8fe2600640 1 --2- 172.21.15.1:0/3133586974 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8fdc152550 0x7f8fdc0734e0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:16.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:16.115+0000 7f8fe2600640 1 -- 172.21.15.1:0/3133586974 >> 172.21.15.1:0/3133586974 conn(0x7f8fdc0c89b0 msgr2=0x7f8fdc0c8dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:02:16.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:16.115+0000 7f8fe2600640 1 -- 172.21.15.1:0/3133586974 shutdown_connections 2024-05-09T08:02:16.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:16.115+0000 7f8fe2600640 1 -- 172.21.15.1:0/3133586974 wait complete. 2024-05-09T08:02:16.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:16.117+0000 7f8fe2600640 1 Processor -- start 2024-05-09T08:02:16.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:16.117+0000 7f8fe2600640 1 -- start start 2024-05-09T08:02:16.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:16.118+0000 7f8fe2600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8fdc0723d0 0x7f8fdc1572b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:16.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:16.118+0000 7f8fe2600640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f8fdc072830 0x7f8fdc0c9cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:16.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:16.118+0000 7f8fe2600640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8fdc152550 0x7f8fdc13f520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:16.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:16.118+0000 7f8fe2600640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f8fdc06e860 con 0x7f8fdc152550 2024-05-09T08:02:16.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:16.118+0000 7f8fe2600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f8fdc071bd0 con 0x7f8fdc0723d0 2024-05-09T08:02:16.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:16.118+0000 7f8fe2600640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f8fdc071eb0 con 0x7f8fdc072830 2024-05-09T08:02:16.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:16.118+0000 7f8fdbe00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8fdc152550 0x7f8fdc13f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:16.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:16.118+0000 7f8fdb400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8fdc0723d0 0x7f8fdc1572b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:16.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:16.118+0000 7f8fdaa00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f8fdc072830 0x7f8fdc0c9cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:16.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:16.118+0000 7f8fdb400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8fdc0723d0 0x7f8fdc1572b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.1:41442/0 (socket says 172.21.15.1:41442) 2024-05-09T08:02:16.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:16.118+0000 7f8fdbe00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8fdc152550 0x7f8fdc13f520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43314/0 (socket says 172.21.15.1:43314) 2024-05-09T08:02:16.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:16.118+0000 7f8fdaa00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f8fdc072830 0x7f8fdc0c9cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3301/0 says I am v2:172.21.15.1:42034/0 (socket says 172.21.15.1:42034) 2024-05-09T08:02:16.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:16.118+0000 7f8fdb400640 1 -- 172.21.15.1:0/3396883788 learned_addr learned my addr 172.21.15.1:0/3396883788 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:02:16.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:16.119+0000 7f8fdbe00640 1 -- 172.21.15.1:0/3396883788 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f8fdc072830 msgr2=0x7f8fdc0c9cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:16.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:16.119+0000 7f8fdbe00640 1 --2- 172.21.15.1:0/3396883788 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f8fdc072830 0x7f8fdc0c9cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:16.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:16.119+0000 7f8fdbe00640 1 -- 172.21.15.1:0/3396883788 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8fdc0723d0 msgr2=0x7f8fdc1572b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:16.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:16.119+0000 7f8fdbe00640 1 --2- 172.21.15.1:0/3396883788 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8fdc0723d0 0x7f8fdc1572b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:16.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:16.119+0000 7f8fdbe00640 1 -- 172.21.15.1:0/3396883788 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fdc1f3710 con 0x7f8fdc152550 2024-05-09T08:02:16.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:16.119+0000 7f8fdbe00640 1 --2- 172.21.15.1:0/3396883788 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8fdc152550 0x7f8fdc13f520 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f8fd400c3f0 tx=0x7f8fd400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:16.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:16.119+0000 7f8fd3400640 1 -- 172.21.15.1:0/3396883788 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fd4017070 con 0x7f8fdc152550 2024-05-09T08:02:16.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:16.119+0000 7f8fd3400640 1 -- 172.21.15.1:0/3396883788 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8fd4005ce0 con 0x7f8fdc152550 2024-05-09T08:02:16.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:16.119+0000 7f8fd3400640 1 -- 172.21.15.1:0/3396883788 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fd40072d0 con 0x7f8fdc152550 2024-05-09T08:02:16.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:16.119+0000 7f8fe2600640 1 -- 172.21.15.1:0/3396883788 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fdc15b1d0 con 0x7f8fdc152550 2024-05-09T08:02:16.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:16.119+0000 7f8fe2600640 1 -- 172.21.15.1:0/3396883788 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fdc1f39b0 con 0x7f8fdc152550 2024-05-09T08:02:16.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:16.120+0000 7f8fe2600640 1 -- 172.21.15.1:0/3396883788 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8fdc068520 con 0x7f8fdc152550 2024-05-09T08:02:16.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:16.121+0000 7f8fd3400640 1 -- 172.21.15.1:0/3396883788 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8fd401a070 con 0x7f8fdc152550 2024-05-09T08:02:16.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:16.122+0000 7f8fd3400640 1 --2- 172.21.15.1:0/3396883788 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f8fbc06be30 0x7f8fbc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:16.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:16.122+0000 7f8fd3400640 1 -- 172.21.15.1:0/3396883788 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f8fd408a0d0 con 0x7f8fdc152550 2024-05-09T08:02:16.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:16.123+0000 7f8fdb400640 1 --2- 172.21.15.1:0/3396883788 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f8fbc06be30 0x7f8fbc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:16.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:16.123+0000 7f8fdb400640 1 --2- 172.21.15.1:0/3396883788 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f8fbc06be30 0x7f8fbc06e2f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f8fcc009bd0 tx=0x7f8fcc02f000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:16.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:16.127+0000 7f8fd3400640 1 -- 172.21.15.1:0/3396883788 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f8fd4058300 con 0x7f8fdc152550 2024-05-09T08:02:16.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:16.338+0000 7f8fe2600640 1 -- 172.21.15.1:0/3396883788 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) -- 0x7f8fdc0c5530 con 0x7f8fdc152550 2024-05-09T08:02:16.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:16.867+0000 7f8fd3400640 1 -- 172.21.15.1:0/3396883788 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v21) ==== 157+0+0 (secure 0 0 0) 0x7f8fd405c600 con 0x7f8fdc152550 2024-05-09T08:02:16.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:16.872+0000 7f8fe2600640 1 -- 172.21.15.1:0/3396883788 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f8fbc06be30 msgr2=0x7f8fbc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:16.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:16.872+0000 7f8fe2600640 1 --2- 172.21.15.1:0/3396883788 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f8fbc06be30 0x7f8fbc06e2f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f8fcc009bd0 tx=0x7f8fcc02f000 comp rx=0 tx=0).stop 2024-05-09T08:02:16.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:16.872+0000 7f8fe2600640 1 -- 172.21.15.1:0/3396883788 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8fdc152550 msgr2=0x7f8fdc13f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:16.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:16.872+0000 7f8fe2600640 1 --2- 172.21.15.1:0/3396883788 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8fdc152550 0x7f8fdc13f520 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f8fd400c3f0 tx=0x7f8fd400c8b0 comp rx=0 tx=0).stop 2024-05-09T08:02:16.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:16.872+0000 7f8fe2600640 1 -- 172.21.15.1:0/3396883788 shutdown_connections 2024-05-09T08:02:16.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:16.872+0000 7f8fe2600640 1 --2- 172.21.15.1:0/3396883788 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f8fbc06be30 0x7f8fbc06e2f0 secure :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f8fcc009bd0 tx=0x7f8fcc02f000 comp rx=0 tx=0).stop 2024-05-09T08:02:16.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:16.872+0000 7f8fe2600640 1 --2- 172.21.15.1:0/3396883788 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8fdc152550 0x7f8fdc13f520 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:16.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:16.872+0000 7f8fe2600640 1 --2- 172.21.15.1:0/3396883788 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f8fdc072830 0x7f8fdc0c9cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:16.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:16.872+0000 7f8fe2600640 1 --2- 172.21.15.1:0/3396883788 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8fdc0723d0 0x7f8fdc1572b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:16.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:16.872+0000 7f8fe2600640 1 -- 172.21.15.1:0/3396883788 >> 172.21.15.1:0/3396883788 conn(0x7f8fdc0c89b0 msgr2=0x7f8fdc0c5d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:02:16.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:16.873+0000 7f8fe2600640 1 -- 172.21.15.1:0/3396883788 shutdown_connections 2024-05-09T08:02:16.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:16.873+0000 7f8fe2600640 1 -- 172.21.15.1:0/3396883788 wait complete. 2024-05-09T08:02:16.910 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-05-09T08:02:17.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:17.106+0000 7f6b03200640 1 Processor -- start 2024-05-09T08:02:17.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:17.107+0000 7f6b03200640 1 -- start start 2024-05-09T08:02:17.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:17.108+0000 7f6b03200640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f6b041570e0 0x7f6b040c8460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:17.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:17.108+0000 7f6b03200640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6b040c3480 con 0x7f6b040c5ce0 2024-05-09T08:02:17.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:17.108+0000 7f6b03200640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6b0415a870 con 0x7f6b04151290 2024-05-09T08:02:17.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:17.108+0000 7f6b03200640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f6b040c3720 con 0x7f6b041570e0 2024-05-09T08:02:17.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:17.108+0000 7f6afbe00640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f6b040c5ce0 0x7f6b0415b030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:40246/0 (socket says 172.21.15.1:40246) 2024-05-09T08:02:17.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:17.108+0000 7f6afbe00640 1 -- 172.21.15.1:0/1544714934 learned_addr learned my addr 172.21.15.1:0/1544714934 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T08:02:17.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:17.108+0000 7f6b02800640 1 --2- 172.21.15.1:0/1544714934 >> v2:172.21.15.1:3300/0 conn(0x7f6b041570e0 0x7f6b040c8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:17.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:17.109+0000 7f6b01400640 1 -- 172.21.15.1:0/1544714934 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2663122616 0 0) 0x7f6b040c3480 con 0x7f6b040c5ce0 2024-05-09T08:02:17.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:17.109+0000 7f6b01400640 1 -- 172.21.15.1:0/1544714934 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6ae8005730 con 0x7f6b040c5ce0 2024-05-09T08:02:17.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:17.109+0000 7f6b01400640 1 -- 172.21.15.1:0/1544714934 <== mon.1 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1949337521 0 0) 0x7f6b0415a870 con 0x7f6b04151290 2024-05-09T08:02:17.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:17.109+0000 7f6b02800640 1 -- 172.21.15.1:0/1544714934 >> v1:172.21.15.7:6789/0 conn(0x7f6b04151290 legacy=0x7f6b04072ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:17.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:17.109+0000 7f6b02800640 1 -- 172.21.15.1:0/1544714934 >> v1:172.21.15.1:6789/0 conn(0x7f6b040c5ce0 legacy=0x7f6b0415b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:17.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:17.109+0000 7f6b02800640 1 -- 172.21.15.1:0/1544714934 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b040c3cc0 con 0x7f6b041570e0 2024-05-09T08:02:17.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:17.110+0000 7f6b02800640 1 --2- 172.21.15.1:0/1544714934 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6b041570e0 0x7f6b040c8460 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f6afc009d90 tx=0x7f6afc02a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9859034f166ec9a6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:17.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:17.110+0000 7f6b01400640 1 -- 172.21.15.1:0/1544714934 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6afc007740 con 0x7f6b041570e0 2024-05-09T08:02:17.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:17.110+0000 7f6b01400640 1 -- 172.21.15.1:0/1544714934 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6afc0078e0 con 0x7f6b041570e0 2024-05-09T08:02:17.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:17.110+0000 7f6b01400640 1 -- 172.21.15.1:0/1544714934 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6afc007cb0 con 0x7f6b041570e0 2024-05-09T08:02:17.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:17.111+0000 7f6b03200640 1 -- 172.21.15.1:0/1544714934 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6b041570e0 msgr2=0x7f6b040c8460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:17.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:17.111+0000 7f6b03200640 1 --2- 172.21.15.1:0/1544714934 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6b041570e0 0x7f6b040c8460 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f6afc009d90 tx=0x7f6afc02a6f0 comp rx=0 tx=0).stop 2024-05-09T08:02:17.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:17.111+0000 7f6b03200640 1 -- 172.21.15.1:0/1544714934 shutdown_connections 2024-05-09T08:02:17.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:17.111+0000 7f6b03200640 1 --2- 172.21.15.1:0/1544714934 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6b041570e0 0x7f6b040c8460 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:17.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:17.111+0000 7f6b03200640 1 -- 172.21.15.1:0/1544714934 >> 172.21.15.1:0/1544714934 conn(0x7f6b0406b250 msgr2=0x7f6b0406b630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:02:17.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:17.111+0000 7f6b03200640 1 -- 172.21.15.1:0/1544714934 shutdown_connections 2024-05-09T08:02:17.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:17.111+0000 7f6b03200640 1 -- 172.21.15.1:0/1544714934 wait complete. 2024-05-09T08:02:17.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:17.113+0000 7f6b03200640 1 Processor -- start 2024-05-09T08:02:17.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:17.113+0000 7f6b03200640 1 -- start start 2024-05-09T08:02:17.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:17.114+0000 7f6b03200640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6b040c5ce0 0x7f6b040cc110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:17.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:17.114+0000 7f6b03200640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f6b04151290 0x7f6b04159690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:17.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:17.114+0000 7f6b03200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6b041570e0 0x7f6b041f1550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:17.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:17.114+0000 7f6b03200640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f6b04151670 con 0x7f6b040c5ce0 2024-05-09T08:02:17.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:17.114+0000 7f6b03200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f6b0415ab10 con 0x7f6b041570e0 2024-05-09T08:02:17.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:17.114+0000 7f6b03200640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f6b041517f0 con 0x7f6b04151290 2024-05-09T08:02:17.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:17.114+0000 7f6b01e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6b040c5ce0 0x7f6b040cc110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:17.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:17.115+0000 7f6b01e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6b040c5ce0 0x7f6b040cc110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43342/0 (socket says 172.21.15.1:43342) 2024-05-09T08:02:17.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:17.115+0000 7f6b01e00640 1 -- 172.21.15.1:0/1249148935 learned_addr learned my addr 172.21.15.1:0/1249148935 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:02:17.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:17.115+0000 7f6afbe00640 1 --2- 172.21.15.1:0/1249148935 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f6b04151290 0x7f6b04159690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:17.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:17.115+0000 7f6b02800640 1 --2- 172.21.15.1:0/1249148935 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6b041570e0 0x7f6b041f1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:17.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:17.115+0000 7f6b01e00640 1 -- 172.21.15.1:0/1249148935 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f6b04151290 msgr2=0x7f6b04159690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:17.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:17.115+0000 7f6b01e00640 1 --2- 172.21.15.1:0/1249148935 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f6b04151290 0x7f6b04159690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:17.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:17.115+0000 7f6b01e00640 1 -- 172.21.15.1:0/1249148935 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6b041570e0 msgr2=0x7f6b041f1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:17.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:17.115+0000 7f6b01e00640 1 --2- 172.21.15.1:0/1249148935 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6b041570e0 0x7f6b041f1550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:17.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:17.115+0000 7f6b01e00640 1 -- 172.21.15.1:0/1249148935 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b041f4190 con 0x7f6b040c5ce0 2024-05-09T08:02:17.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:17.115+0000 7f6b01e00640 1 --2- 172.21.15.1:0/1249148935 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6b040c5ce0 0x7f6b040cc110 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f6af400c4e0 tx=0x7f6af400c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:17.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:17.116+0000 7f6afa000640 1 -- 172.21.15.1:0/1249148935 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6af4017070 con 0x7f6b040c5ce0 2024-05-09T08:02:17.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:17.116+0000 7f6afa000640 1 -- 172.21.15.1:0/1249148935 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6af40035f0 con 0x7f6b040c5ce0 2024-05-09T08:02:17.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:17.116+0000 7f6afa000640 1 -- 172.21.15.1:0/1249148935 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6af40039e0 con 0x7f6b040c5ce0 2024-05-09T08:02:17.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:17.116+0000 7f6b03200640 1 -- 172.21.15.1:0/1249148935 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b041f4430 con 0x7f6b040c5ce0 2024-05-09T08:02:17.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:17.116+0000 7f6b03200640 1 -- 172.21.15.1:0/1249148935 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b041f0710 con 0x7f6b040c5ce0 2024-05-09T08:02:17.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:17.117+0000 7f6b03200640 1 -- 172.21.15.1:0/1249148935 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b0404f6d0 con 0x7f6b040c5ce0 2024-05-09T08:02:17.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:17.118+0000 7f6afa000640 1 -- 172.21.15.1:0/1249148935 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6af401a070 con 0x7f6b040c5ce0 2024-05-09T08:02:17.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:17.119+0000 7f6afa000640 1 --2- 172.21.15.1:0/1249148935 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f6ae006be30 0x7f6ae006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:17.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:17.119+0000 7f6afa000640 1 -- 172.21.15.1:0/1249148935 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 6007+0+0 (secure 0 0 0) 0x7f6af408a220 con 0x7f6b040c5ce0 2024-05-09T08:02:17.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:17.119+0000 7f6afbe00640 1 --2- 172.21.15.1:0/1249148935 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f6ae006be30 0x7f6ae006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:17.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:17.120+0000 7f6afbe00640 1 --2- 172.21.15.1:0/1249148935 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f6ae006be30 0x7f6ae006e2f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f6b04155350 tx=0x7f6aec000bc0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:17.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:17.124+0000 7f6afa000640 1 -- 172.21.15.1:0/1249148935 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f6af4058450 con 0x7f6b040c5ce0 2024-05-09T08:02:17.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:17.325+0000 7f6b03200640 1 -- 172.21.15.1:0/1249148935 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f6b0406b840 con 0x7f6b040c5ce0 2024-05-09T08:02:18.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:18.912+0000 7f6afa000640 1 -- 172.21.15.1:0/1249148935 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v23) ==== 228+0+0 (secure 0 0 0) 0x7f6af405c750 con 0x7f6b040c5ce0 2024-05-09T08:02:18.930 INFO:teuthology.orchestra.run.smithi001.stderr:pool 'cephfs_data_ec' created 2024-05-09T08:02:18.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:18.917+0000 7f6b03200640 1 -- 172.21.15.1:0/1249148935 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f6ae006be30 msgr2=0x7f6ae006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:18.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:18.917+0000 7f6b03200640 1 --2- 172.21.15.1:0/1249148935 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f6ae006be30 0x7f6ae006e2f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f6b04155350 tx=0x7f6aec000bc0 comp rx=0 tx=0).stop 2024-05-09T08:02:18.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:18.917+0000 7f6b03200640 1 -- 172.21.15.1:0/1249148935 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6b040c5ce0 msgr2=0x7f6b040cc110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:18.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:18.917+0000 7f6b03200640 1 --2- 172.21.15.1:0/1249148935 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6b040c5ce0 0x7f6b040cc110 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f6af400c4e0 tx=0x7f6af400c9a0 comp rx=0 tx=0).stop 2024-05-09T08:02:18.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:18.918+0000 7f6b03200640 1 -- 172.21.15.1:0/1249148935 shutdown_connections 2024-05-09T08:02:18.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:18.918+0000 7f6b03200640 1 --2- 172.21.15.1:0/1249148935 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f6ae006be30 0x7f6ae006e2f0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:18.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:18.919+0000 7f6b03200640 1 --2- 172.21.15.1:0/1249148935 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6b041570e0 0x7f6b041f1550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:18.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:18.919+0000 7f6b03200640 1 --2- 172.21.15.1:0/1249148935 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f6b04151290 0x7f6b04159690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:18.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:18.919+0000 7f6b03200640 1 --2- 172.21.15.1:0/1249148935 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6b040c5ce0 0x7f6b040cc110 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:18.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:18.919+0000 7f6b03200640 1 -- 172.21.15.1:0/1249148935 >> 172.21.15.1:0/1249148935 conn(0x7f6b0406b250 msgr2=0x7f6b0406f130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:02:18.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:18.921+0000 7f6b03200640 1 -- 172.21.15.1:0/1249148935 shutdown_connections 2024-05-09T08:02:18.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:18.921+0000 7f6b03200640 1 -- 172.21.15.1:0/1249148935 wait complete. 2024-05-09T08:02:18.965 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-05-09T08:02:19.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:19.139+0000 7f8400a00640 1 Processor -- start 2024-05-09T08:02:19.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:19.139+0000 7f8400a00640 1 -- start start 2024-05-09T08:02:19.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:19.140+0000 7f8400a00640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f83fc06f3b0 0x7f83fc073140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:19.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:19.140+0000 7f8400a00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f83fc06f790 0x7f83fc071980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:19.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:19.140+0000 7f8400a00640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f83fc157190 con 0x7f83fc0c12b0 2024-05-09T08:02:19.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:19.140+0000 7f8400a00640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f83fc0b1ec0 con 0x7f83fc06f3b0 2024-05-09T08:02:19.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:19.140+0000 7f8400a00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f83fc0737f0 con 0x7f83fc06f790 2024-05-09T08:02:19.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:19.141+0000 7f83f9e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f83fc06f790 0x7f83fc071980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:19.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:19.141+0000 7f83f9e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f83fc06f790 0x7f83fc071980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.1:41454/0 (socket says 172.21.15.1:41454) 2024-05-09T08:02:19.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:19.141+0000 7f83f9e00640 1 -- 172.21.15.1:0/475736461 learned_addr learned my addr 172.21.15.1:0/475736461 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:02:19.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:19.141+0000 7f83fb200640 1 --1- 172.21.15.1:0/475736461 >> v1:172.21.15.1:6789/0 conn(0x7f83fc0c12b0 0x7f83fc150e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:40250/0 (socket says 172.21.15.1:40250) 2024-05-09T08:02:19.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:19.141+0000 7f83fa800640 1 --2- 172.21.15.1:0/475736461 >> v2:172.21.15.1:3300/0 conn(0x7f83fc06f3b0 0x7f83fc073140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:19.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:19.141+0000 7f83f9e00640 1 -- 172.21.15.1:0/475736461 >> v2:172.21.15.1:3300/0 conn(0x7f83fc06f3b0 msgr2=0x7f83fc073140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:02:19.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:19.141+0000 7f83f9e00640 1 --2- 172.21.15.1:0/475736461 >> v2:172.21.15.1:3300/0 conn(0x7f83fc06f3b0 0x7f83fc073140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:19.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:19.141+0000 7f83f9e00640 1 -- 172.21.15.1:0/475736461 >> v1:172.21.15.1:6789/0 conn(0x7f83fc0c12b0 legacy=0x7f83fc150e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:19.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:19.141+0000 7f83f9e00640 1 -- 172.21.15.1:0/475736461 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83fc072430 con 0x7f83fc06f790 2024-05-09T08:02:19.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:19.142+0000 7f83f9e00640 1 --2- 172.21.15.1:0/475736461 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f83fc06f790 0x7f83fc071980 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f83fc0ccd70 tx=0x7f83e402a7c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=848a0504f07955f1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:19.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:19.142+0000 7f83f9400640 1 -- 172.21.15.1:0/475736461 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83e4036070 con 0x7f83fc06f790 2024-05-09T08:02:19.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:19.143+0000 7f83f9400640 1 -- 172.21.15.1:0/475736461 <== mon.1 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83e4031600 con 0x7f83fc06f790 2024-05-09T08:02:19.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:19.143+0000 7f83f9400640 1 -- 172.21.15.1:0/475736461 <== mon.1 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83e40319f0 con 0x7f83fc06f790 2024-05-09T08:02:19.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:19.143+0000 7f8400a00640 1 -- 172.21.15.1:0/475736461 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f83fc06f790 msgr2=0x7f83fc071980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:19.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:19.143+0000 7f8400a00640 1 --2- 172.21.15.1:0/475736461 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f83fc06f790 0x7f83fc071980 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f83fc0ccd70 tx=0x7f83e402a7c0 comp rx=0 tx=0).stop 2024-05-09T08:02:19.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:19.143+0000 7f8400a00640 1 -- 172.21.15.1:0/475736461 shutdown_connections 2024-05-09T08:02:19.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:19.143+0000 7f8400a00640 1 --2- 172.21.15.1:0/475736461 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f83fc06f790 0x7f83fc071980 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:19.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:19.143+0000 7f8400a00640 1 --2- 172.21.15.1:0/475736461 >> v2:172.21.15.1:3300/0 conn(0x7f83fc06f3b0 0x7f83fc073140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:19.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:19.143+0000 7f8400a00640 1 -- 172.21.15.1:0/475736461 >> 172.21.15.1:0/475736461 conn(0x7f83fc0cbe20 msgr2=0x7f83fc0cc200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:02:19.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:19.143+0000 7f8400a00640 1 -- 172.21.15.1:0/475736461 shutdown_connections 2024-05-09T08:02:19.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:19.143+0000 7f8400a00640 1 -- 172.21.15.1:0/475736461 wait complete. 2024-05-09T08:02:19.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:19.145+0000 7f8400a00640 1 Processor -- start 2024-05-09T08:02:19.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:19.145+0000 7f8400a00640 1 -- start start 2024-05-09T08:02:19.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:19.146+0000 7f8400a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f83fc06f3b0 0x7f83fc151ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:19.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:19.146+0000 7f8400a00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f83fc06f790 0x7f83fc1e1890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:19.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:19.146+0000 7f8400a00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f83fc0c12b0 0x7f83fc1efa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:19.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:19.146+0000 7f8400a00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f83fc072cd0 con 0x7f83fc0c12b0 2024-05-09T08:02:19.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:19.146+0000 7f8400a00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f83fc0729f0 con 0x7f83fc06f3b0 2024-05-09T08:02:19.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:19.146+0000 7f8400a00640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f83fc071ec0 con 0x7f83fc06f790 2024-05-09T08:02:19.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:19.146+0000 7f83fb200640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f83fc0c12b0 0x7f83fc1efa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:19.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:19.147+0000 7f83f9e00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f83fc06f790 0x7f83fc1e1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:19.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:19.147+0000 7f83fb200640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f83fc0c12b0 0x7f83fc1efa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43362/0 (socket says 172.21.15.1:43362) 2024-05-09T08:02:19.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:19.147+0000 7f83fb200640 1 -- 172.21.15.1:0/2969152165 learned_addr learned my addr 172.21.15.1:0/2969152165 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:02:19.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:19.147+0000 7f83fa800640 1 --2- 172.21.15.1:0/2969152165 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f83fc06f3b0 0x7f83fc151ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:19.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:19.147+0000 7f83fa800640 1 -- 172.21.15.1:0/2969152165 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f83fc06f790 msgr2=0x7f83fc1e1890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:19.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:19.147+0000 7f83fa800640 1 --2- 172.21.15.1:0/2969152165 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f83fc06f790 0x7f83fc1e1890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:19.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:19.147+0000 7f83fa800640 1 -- 172.21.15.1:0/2969152165 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f83fc0c12b0 msgr2=0x7f83fc1efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:19.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:19.147+0000 7f83fa800640 1 --2- 172.21.15.1:0/2969152165 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f83fc0c12b0 0x7f83fc1efa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:19.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:19.147+0000 7f83fa800640 1 -- 172.21.15.1:0/2969152165 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83fc152500 con 0x7f83fc06f3b0 2024-05-09T08:02:19.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:19.147+0000 7f83fb200640 1 --2- 172.21.15.1:0/2969152165 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f83fc0c12b0 0x7f83fc1efa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:02:19.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:19.147+0000 7f83fa800640 1 --2- 172.21.15.1:0/2969152165 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f83fc06f3b0 0x7f83fc151ec0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f83ec00e880 tx=0x7f83ec00ed40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:19.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:19.148+0000 7f83f2a00640 1 -- 172.21.15.1:0/2969152165 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83ec017070 con 0x7f83fc06f3b0 2024-05-09T08:02:19.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:19.148+0000 7f83f2a00640 1 -- 172.21.15.1:0/2969152165 <== mon.1 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83ec00c940 con 0x7f83fc06f3b0 2024-05-09T08:02:19.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:19.148+0000 7f83f2a00640 1 -- 172.21.15.1:0/2969152165 <== mon.1 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83ec00cd30 con 0x7f83fc06f3b0 2024-05-09T08:02:19.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:19.148+0000 7f8400a00640 1 -- 172.21.15.1:0/2969152165 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83fc1f3120 con 0x7f83fc06f3b0 2024-05-09T08:02:19.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:19.148+0000 7f8400a00640 1 -- 172.21.15.1:0/2969152165 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83fc1f36e0 con 0x7f83fc06f3b0 2024-05-09T08:02:19.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:19.149+0000 7f8400a00640 1 -- 172.21.15.1:0/2969152165 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83fc074d70 con 0x7f83fc06f3b0 2024-05-09T08:02:19.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:19.150+0000 7f83f2a00640 1 -- 172.21.15.1:0/2969152165 <== mon.1 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f83ec01a070 con 0x7f83fc06f3b0 2024-05-09T08:02:19.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:19.151+0000 7f83f2a00640 1 --2- 172.21.15.1:0/2969152165 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f83d806be30 0x7f83d806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:19.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:19.151+0000 7f83f2a00640 1 -- 172.21.15.1:0/2969152165 <== mon.1 v2:172.21.15.7:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 6500+0+0 (secure 0 0 0) 0x7f83ec08b550 con 0x7f83fc06f3b0 2024-05-09T08:02:19.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:19.151+0000 7f83f9e00640 1 --2- 172.21.15.1:0/2969152165 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f83d806be30 0x7f83d806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:19.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:19.152+0000 7f83f9e00640 1 --2- 172.21.15.1:0/2969152165 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f83d806be30 0x7f83d806e2f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f83e4033000 tx=0x7f83e4009000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:19.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:19.156+0000 7f83f2a00640 1 -- 172.21.15.1:0/2969152165 <== mon.1 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f83ec059460 con 0x7f83fc06f3b0 2024-05-09T08:02:19.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:19.356+0000 7f8400a00640 1 -- 172.21.15.1:0/2969152165 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) -- 0x7f83fc0703e0 con 0x7f83fc06f3b0 2024-05-09T08:02:19.685 DEBUG:teuthology.orchestra.run.smithi001:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T08:02:19.693 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T08:02:19.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:19.918+0000 7f83f2a00640 1 -- 172.21.15.1:0/2969152165 <== mon.1 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v24) ==== 169+0+0 (secure 0 0 0) 0x7f83ec05d760 con 0x7f83fc06f3b0 2024-05-09T08:02:19.936 INFO:teuthology.orchestra.run.smithi001.stderr:set pool 4 allow_ec_overwrites to true 2024-05-09T08:02:19.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:19.927+0000 7f8400a00640 1 -- 172.21.15.1:0/2969152165 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f83d806be30 msgr2=0x7f83d806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:19.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:19.927+0000 7f8400a00640 1 --2- 172.21.15.1:0/2969152165 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f83d806be30 0x7f83d806e2f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f83e4033000 tx=0x7f83e4009000 comp rx=0 tx=0).stop 2024-05-09T08:02:19.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:19.927+0000 7f8400a00640 1 -- 172.21.15.1:0/2969152165 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f83fc06f3b0 msgr2=0x7f83fc151ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:19.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:19.927+0000 7f8400a00640 1 --2- 172.21.15.1:0/2969152165 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f83fc06f3b0 0x7f83fc151ec0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f83ec00e880 tx=0x7f83ec00ed40 comp rx=0 tx=0).stop 2024-05-09T08:02:19.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:19.931+0000 7f8400a00640 1 -- 172.21.15.1:0/2969152165 shutdown_connections 2024-05-09T08:02:19.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:19.931+0000 7f8400a00640 1 --2- 172.21.15.1:0/2969152165 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f83d806be30 0x7f83d806e2f0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:19.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:19.931+0000 7f8400a00640 1 --2- 172.21.15.1:0/2969152165 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f83fc0c12b0 0x7f83fc1efa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:19.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:19.931+0000 7f8400a00640 1 --2- 172.21.15.1:0/2969152165 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f83fc06f790 0x7f83fc1e1890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:19.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:19.931+0000 7f8400a00640 1 --2- 172.21.15.1:0/2969152165 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f83fc06f3b0 0x7f83fc151ec0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:19.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:19.931+0000 7f8400a00640 1 -- 172.21.15.1:0/2969152165 >> 172.21.15.1:0/2969152165 conn(0x7f83fc0cbe20 msgr2=0x7f83fc155200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:02:19.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:19.931+0000 7f8400a00640 1 -- 172.21.15.1:0/2969152165 shutdown_connections 2024-05-09T08:02:19.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:19.934+0000 7f8400a00640 1 -- 172.21.15.1:0/2969152165 wait complete. 2024-05-09T08:02:19.983 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2024-05-09T08:02:20.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:20.138+0000 7f6feec00640 1 Processor -- start 2024-05-09T08:02:20.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:20.139+0000 7f6feec00640 1 -- start start 2024-05-09T08:02:20.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:20.140+0000 7f6feec00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f6fe81513a0 0x7f6fe80c3070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:20.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:20.140+0000 7f6feec00640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f6fe80c7b10 0x7f6fe80c7fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:20.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:20.140+0000 7f6feec00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6fe80c5350 con 0x7f6fe80c5df0 2024-05-09T08:02:20.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:20.140+0000 7f6feec00640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f6fe80c3670 con 0x7f6fe80c7b10 2024-05-09T08:02:20.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:20.140+0000 7f6feec00640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f6fe80c3950 con 0x7f6fe81513a0 2024-05-09T08:02:20.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:20.140+0000 7f6fee200640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f6fe80c7b10 0x7f6fe80c7fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:20.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:20.140+0000 7f6fee200640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f6fe80c7b10 0x7f6fe80c7fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43376/0 (socket says 172.21.15.1:43376) 2024-05-09T08:02:20.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:20.140+0000 7f6fee200640 1 -- 172.21.15.1:0/3242505643 learned_addr learned my addr 172.21.15.1:0/3242505643 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:02:20.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:20.140+0000 7f6fed800640 1 --2- 172.21.15.1:0/3242505643 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f6fe81513a0 0x7f6fe80c3070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:20.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:20.141+0000 7f6fee200640 1 -- 172.21.15.1:0/3242505643 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f6fe81513a0 msgr2=0x7f6fe80c3070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:02:20.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:20.141+0000 7f6fee200640 1 --2- 172.21.15.1:0/3242505643 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f6fe81513a0 0x7f6fe80c3070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:20.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:20.141+0000 7f6fee200640 1 -- 172.21.15.1:0/3242505643 >> v1:172.21.15.7:6789/0 conn(0x7f6fe80c5df0 legacy=0x7f6fe8157240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:20.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:20.141+0000 7f6fe7e00640 1 -- 172.21.15.1:0/3242505643 <== mon.1 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 78208573 0 0) 0x7f6fe80c5350 con 0x7f6fe80c5df0 2024-05-09T08:02:20.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:20.141+0000 7f6fee200640 1 -- 172.21.15.1:0/3242505643 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fe80c55f0 con 0x7f6fe80c7b10 2024-05-09T08:02:20.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:20.142+0000 7f6fee200640 1 --2- 172.21.15.1:0/3242505643 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6fe80c7b10 0x7f6fe80c7fc0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f6fe0009e80 tx=0x7f6fe002a750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88d806059988e600 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:20.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:20.142+0000 7f6fe7e00640 1 -- 172.21.15.1:0/3242505643 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fe0036070 con 0x7f6fe80c7b10 2024-05-09T08:02:20.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:20.142+0000 7f6fe7e00640 1 -- 172.21.15.1:0/3242505643 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6fe0031600 con 0x7f6fe80c7b10 2024-05-09T08:02:20.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:20.142+0000 7f6fe7e00640 1 -- 172.21.15.1:0/3242505643 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fe00319d0 con 0x7f6fe80c7b10 2024-05-09T08:02:20.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:20.142+0000 7f6feec00640 1 -- 172.21.15.1:0/3242505643 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6fe80c7b10 msgr2=0x7f6fe80c7fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:20.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:20.142+0000 7f6feec00640 1 --2- 172.21.15.1:0/3242505643 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6fe80c7b10 0x7f6fe80c7fc0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f6fe0009e80 tx=0x7f6fe002a750 comp rx=0 tx=0).stop 2024-05-09T08:02:20.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:20.143+0000 7f6feec00640 1 -- 172.21.15.1:0/3242505643 shutdown_connections 2024-05-09T08:02:20.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:20.143+0000 7f6feec00640 1 --2- 172.21.15.1:0/3242505643 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6fe80c7b10 0x7f6fe80c7fc0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:20.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:20.143+0000 7f6feec00640 1 --2- 172.21.15.1:0/3242505643 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f6fe81513a0 0x7f6fe80c3070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:20.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:20.143+0000 7f6feec00640 1 -- 172.21.15.1:0/3242505643 >> 172.21.15.1:0/3242505643 conn(0x7f6fe806b250 msgr2=0x7f6fe806b630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:02:20.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:20.143+0000 7f6feec00640 1 -- 172.21.15.1:0/3242505643 shutdown_connections 2024-05-09T08:02:20.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:20.143+0000 7f6feec00640 1 -- 172.21.15.1:0/3242505643 wait complete. 2024-05-09T08:02:20.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:20.144+0000 7f6feec00640 1 Processor -- start 2024-05-09T08:02:20.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:20.144+0000 7f6feec00640 1 -- start start 2024-05-09T08:02:20.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:20.145+0000 7f6feec00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6fe80c5df0 0x7f6fe8153f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:20.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:20.145+0000 7f6feec00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6fe80c7b10 0x7f6fe806f980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:20.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:20.145+0000 7f6feec00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f6fe81513a0 0x7f6fe81e2310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:20.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:20.145+0000 7f6feec00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f6fe8151900 con 0x7f6fe80c7b10 2024-05-09T08:02:20.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:20.145+0000 7f6feec00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f6fe8151780 con 0x7f6fe80c5df0 2024-05-09T08:02:20.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:20.145+0000 7f6feec00640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f6fe80c8600 con 0x7f6fe81513a0 2024-05-09T08:02:20.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:20.146+0000 7f6fed800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6fe80c5df0 0x7f6fe8153f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:20.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:20.146+0000 7f6fee200640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f6fe81513a0 0x7f6fe81e2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:20.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:20.146+0000 7f6fee200640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f6fe81513a0 0x7f6fe81e2310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3301/0 says I am v2:172.21.15.1:42086/0 (socket says 172.21.15.1:42086) 2024-05-09T08:02:20.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:20.146+0000 7f6fed800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6fe80c5df0 0x7f6fe8153f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.1:41472/0 (socket says 172.21.15.1:41472) 2024-05-09T08:02:20.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:20.146+0000 7f6fee200640 1 -- 172.21.15.1:0/3884778873 learned_addr learned my addr 172.21.15.1:0/3884778873 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:02:20.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:20.146+0000 7f6fece00640 1 --2- 172.21.15.1:0/3884778873 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6fe80c7b10 0x7f6fe806f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:20.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:20.146+0000 7f6fed800640 1 -- 172.21.15.1:0/3884778873 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f6fe81513a0 msgr2=0x7f6fe81e2310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:20.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:20.146+0000 7f6fed800640 1 --2- 172.21.15.1:0/3884778873 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f6fe81513a0 0x7f6fe81e2310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:20.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:20.146+0000 7f6fed800640 1 -- 172.21.15.1:0/3884778873 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6fe80c7b10 msgr2=0x7f6fe806f980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:20.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:20.146+0000 7f6fed800640 1 --2- 172.21.15.1:0/3884778873 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6fe80c7b10 0x7f6fe806f980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:20.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:20.146+0000 7f6fed800640 1 -- 172.21.15.1:0/3884778873 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fe8152d20 con 0x7f6fe80c5df0 2024-05-09T08:02:20.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:20.146+0000 7f6fee200640 1 --2- 172.21.15.1:0/3884778873 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f6fe81513a0 0x7f6fe81e2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:02:20.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:20.146+0000 7f6fed800640 1 --2- 172.21.15.1:0/3884778873 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6fe80c5df0 0x7f6fe8153f30 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6fdc00e870 tx=0x7f6fdc00ed30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:20.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:20.147+0000 7f6fe5600640 1 -- 172.21.15.1:0/3884778873 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fdc017070 con 0x7f6fe80c5df0 2024-05-09T08:02:20.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:20.147+0000 7f6fe5600640 1 -- 172.21.15.1:0/3884778873 <== mon.1 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6fdc00ee60 con 0x7f6fe80c5df0 2024-05-09T08:02:20.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:20.147+0000 7f6fe5600640 1 -- 172.21.15.1:0/3884778873 <== mon.1 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fdc00cb60 con 0x7f6fe80c5df0 2024-05-09T08:02:20.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:20.147+0000 7f6feec00640 1 -- 172.21.15.1:0/3884778873 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fe8153000 con 0x7f6fe80c5df0 2024-05-09T08:02:20.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:20.147+0000 7f6feec00640 1 -- 172.21.15.1:0/3884778873 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fe81e1830 con 0x7f6fe80c5df0 2024-05-09T08:02:20.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:20.148+0000 7f6feec00640 1 -- 172.21.15.1:0/3884778873 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6fe804f660 con 0x7f6fe80c5df0 2024-05-09T08:02:20.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:20.149+0000 7f6fe5600640 1 -- 172.21.15.1:0/3884778873 <== mon.1 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6fdc01a070 con 0x7f6fe80c5df0 2024-05-09T08:02:20.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:20.150+0000 7f6fe5600640 1 --2- 172.21.15.1:0/3884778873 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f6fcc06be60 0x7f6fcc06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:20.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:20.150+0000 7f6fe5600640 1 -- 172.21.15.1:0/3884778873 <== mon.1 v2:172.21.15.7:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6500+0+0 (secure 0 0 0) 0x7f6fdc08b270 con 0x7f6fe80c5df0 2024-05-09T08:02:20.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:20.150+0000 7f6fece00640 1 --2- 172.21.15.1:0/3884778873 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f6fcc06be60 0x7f6fcc06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:20.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:20.151+0000 7f6fece00640 1 --2- 172.21.15.1:0/3884778873 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f6fcc06be60 0x7f6fcc06e320 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f6fe8157250 tx=0x7f6fd8009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:20.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:20.155+0000 7f6fe5600640 1 -- 172.21.15.1:0/3884778873 <== mon.1 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f6fdc059210 con 0x7f6fe80c5df0 2024-05-09T08:02:20.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:20.400+0000 7f6feec00640 1 -- 172.21.15.1:0/3884778873 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) -- 0x7f6fe8154e90 con 0x7f6fe80c5df0 2024-05-09T08:02:20.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:20.936+0000 7f6fe5600640 1 -- 172.21.15.1:0/3884778873 <== mon.1 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v5) ==== 137+0+0 (secure 0 0 0) 0x7f6fdc05d510 con 0x7f6fe80c5df0 2024-05-09T08:02:20.953 INFO:teuthology.orchestra.run.smithi001.stderr:added data pool 4 to fsmap 2024-05-09T08:02:20.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:20.940+0000 7f6feec00640 1 -- 172.21.15.1:0/3884778873 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f6fcc06be60 msgr2=0x7f6fcc06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:20.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:20.940+0000 7f6feec00640 1 --2- 172.21.15.1:0/3884778873 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f6fcc06be60 0x7f6fcc06e320 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f6fe8157250 tx=0x7f6fd8009040 comp rx=0 tx=0).stop 2024-05-09T08:02:20.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:20.940+0000 7f6feec00640 1 -- 172.21.15.1:0/3884778873 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6fe80c5df0 msgr2=0x7f6fe8153f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:20.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:20.940+0000 7f6feec00640 1 --2- 172.21.15.1:0/3884778873 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6fe80c5df0 0x7f6fe8153f30 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6fdc00e870 tx=0x7f6fdc00ed30 comp rx=0 tx=0).stop 2024-05-09T08:02:20.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:20.940+0000 7f6feec00640 1 -- 172.21.15.1:0/3884778873 shutdown_connections 2024-05-09T08:02:20.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:20.940+0000 7f6feec00640 1 --2- 172.21.15.1:0/3884778873 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f6fcc06be60 0x7f6fcc06e320 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:20.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:20.940+0000 7f6feec00640 1 --2- 172.21.15.1:0/3884778873 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f6fe81513a0 0x7f6fe81e2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:20.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:20.940+0000 7f6feec00640 1 --2- 172.21.15.1:0/3884778873 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6fe80c7b10 0x7f6fe806f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:20.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:20.940+0000 7f6feec00640 1 --2- 172.21.15.1:0/3884778873 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6fe80c5df0 0x7f6fe8153f30 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:20.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:20.940+0000 7f6feec00640 1 -- 172.21.15.1:0/3884778873 >> 172.21.15.1:0/3884778873 conn(0x7f6fe806b250 msgr2=0x7f6fe814fd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:02:20.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:20.940+0000 7f6feec00640 1 -- 172.21.15.1:0/3884778873 shutdown_connections 2024-05-09T08:02:20.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:20.940+0000 7f6feec00640 1 -- 172.21.15.1:0/3884778873 wait complete. 2024-05-09T08:02:20.976 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T08:02:21.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:21.142+0000 7fba92400640 1 Processor -- start 2024-05-09T08:02:21.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:21.143+0000 7fba92400640 1 -- start start 2024-05-09T08:02:21.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:21.144+0000 7fba92400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fba8c06dca0 0x7fba8c072900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:21.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:21.144+0000 7fba92400640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fba8c06ece0 0x7fba8c159010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:21.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:21.144+0000 7fba92400640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fba8c073200 con 0x7fba8c0c6ea0 2024-05-09T08:02:21.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:21.144+0000 7fba92400640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7fba8c072e40 con 0x7fba8c06ece0 2024-05-09T08:02:21.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:21.144+0000 7fba92400640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fba8c0737c0 con 0x7fba8c06dca0 2024-05-09T08:02:21.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:21.145+0000 7fba8be00640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fba8c06ece0 0x7fba8c159010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:21.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:21.145+0000 7fba82a00640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7fba8c0c6ea0 0x7fba8c06e080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:40266/0 (socket says 172.21.15.1:40266) 2024-05-09T08:02:21.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:21.145+0000 7fba8b400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fba8c06dca0 0x7fba8c072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:21.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:21.145+0000 7fba82a00640 1 -- 172.21.15.1:0/3714267711 learned_addr learned my addr 172.21.15.1:0/3714267711 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T08:02:21.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:21.145+0000 7fba8be00640 1 -- 172.21.15.1:0/3714267711 >> v2:172.21.15.7:3300/0 conn(0x7fba8c06dca0 msgr2=0x7fba8c072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:02:21.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:21.145+0000 7fba8be00640 1 --2- 172.21.15.1:0/3714267711 >> v2:172.21.15.7:3300/0 conn(0x7fba8c06dca0 0x7fba8c072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:21.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:21.145+0000 7fba8be00640 1 -- 172.21.15.1:0/3714267711 >> v1:172.21.15.1:6789/0 conn(0x7fba8c0c6ea0 legacy=0x7fba8c06e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:21.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:21.145+0000 7fba8be00640 1 -- 172.21.15.1:0/3714267711 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba8c0734e0 con 0x7fba8c06ece0 2024-05-09T08:02:21.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:21.146+0000 7fba8be00640 1 --2- 172.21.15.1:0/3714267711 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fba8c06ece0 0x7fba8c159010 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fba8c0cccb0 tx=0x7fba8402a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41d07fa8379928ab server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:21.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:21.146+0000 7fba8aa00640 1 -- 172.21.15.1:0/3714267711 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba84007740 con 0x7fba8c06ece0 2024-05-09T08:02:21.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:21.147+0000 7fba8aa00640 1 -- 172.21.15.1:0/3714267711 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba840078e0 con 0x7fba8c06ece0 2024-05-09T08:02:21.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:21.147+0000 7fba8aa00640 1 -- 172.21.15.1:0/3714267711 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba84007cb0 con 0x7fba8c06ece0 2024-05-09T08:02:21.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:21.147+0000 7fba92400640 1 -- 172.21.15.1:0/3714267711 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fba8c06ece0 msgr2=0x7fba8c159010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:21.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:21.147+0000 7fba92400640 1 --2- 172.21.15.1:0/3714267711 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fba8c06ece0 0x7fba8c159010 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fba8c0cccb0 tx=0x7fba8402a710 comp rx=0 tx=0).stop 2024-05-09T08:02:21.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:21.147+0000 7fba92400640 1 -- 172.21.15.1:0/3714267711 shutdown_connections 2024-05-09T08:02:21.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:21.147+0000 7fba92400640 1 --2- 172.21.15.1:0/3714267711 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fba8c06ece0 0x7fba8c159010 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:21.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:21.147+0000 7fba92400640 1 --2- 172.21.15.1:0/3714267711 >> v2:172.21.15.7:3300/0 conn(0x7fba8c06dca0 0x7fba8c072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:21.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:21.147+0000 7fba92400640 1 -- 172.21.15.1:0/3714267711 >> 172.21.15.1:0/3714267711 conn(0x7fba8c0cc070 msgr2=0x7fba8c0cc480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:02:21.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:21.147+0000 7fba92400640 1 -- 172.21.15.1:0/3714267711 shutdown_connections 2024-05-09T08:02:21.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:21.147+0000 7fba92400640 1 -- 172.21.15.1:0/3714267711 wait complete. 2024-05-09T08:02:21.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:21.149+0000 7fba92400640 1 Processor -- start 2024-05-09T08:02:21.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:21.149+0000 7fba92400640 1 -- start start 2024-05-09T08:02:21.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:21.150+0000 7fba92400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fba8c06dca0 0x7fba8c0c2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:21.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:21.150+0000 7fba92400640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fba8c06ece0 0x7fba8c0c58d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:21.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:21.150+0000 7fba92400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fba8c0c6ea0 0x7fba8c147560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:21.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:21.150+0000 7fba92400640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fba8c072470 con 0x7fba8c06dca0 2024-05-09T08:02:21.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:21.150+0000 7fba92400640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fba8c159550 con 0x7fba8c0c6ea0 2024-05-09T08:02:21.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:21.150+0000 7fba92400640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7fba8c072100 con 0x7fba8c06ece0 2024-05-09T08:02:21.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:21.150+0000 7fba8b400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fba8c06dca0 0x7fba8c0c2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:21.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:21.150+0000 7fba8b400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fba8c06dca0 0x7fba8c0c2bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43404/0 (socket says 172.21.15.1:43404) 2024-05-09T08:02:21.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:21.150+0000 7fba8b400640 1 -- 172.21.15.1:0/3004497998 learned_addr learned my addr 172.21.15.1:0/3004497998 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:02:21.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:21.151+0000 7fba82a00640 1 --2- 172.21.15.1:0/3004497998 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fba8c06ece0 0x7fba8c0c58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:21.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:21.151+0000 7fba8be00640 1 --2- 172.21.15.1:0/3004497998 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fba8c0c6ea0 0x7fba8c147560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:21.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:21.151+0000 7fba8b400640 1 -- 172.21.15.1:0/3004497998 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fba8c06ece0 msgr2=0x7fba8c0c58d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:21.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:21.151+0000 7fba8b400640 1 --2- 172.21.15.1:0/3004497998 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fba8c06ece0 0x7fba8c0c58d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:21.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:21.151+0000 7fba8b400640 1 -- 172.21.15.1:0/3004497998 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fba8c0c6ea0 msgr2=0x7fba8c147560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:21.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:21.151+0000 7fba8b400640 1 --2- 172.21.15.1:0/3004497998 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fba8c0c6ea0 0x7fba8c147560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:21.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:21.151+0000 7fba8b400640 1 -- 172.21.15.1:0/3004497998 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba8c148cf0 con 0x7fba8c06dca0 2024-05-09T08:02:21.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:21.151+0000 7fba8b400640 1 --2- 172.21.15.1:0/3004497998 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fba8c06dca0 0x7fba8c0c2bd0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fba7c00e9e0 tx=0x7fba7c00eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:21.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:21.151+0000 7fba83e00640 1 -- 172.21.15.1:0/3004497998 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba7c017070 con 0x7fba8c06dca0 2024-05-09T08:02:21.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:21.151+0000 7fba83e00640 1 -- 172.21.15.1:0/3004497998 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba7c00c9d0 con 0x7fba8c06dca0 2024-05-09T08:02:21.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:21.152+0000 7fba92400640 1 -- 172.21.15.1:0/3004497998 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba8c148730 con 0x7fba8c06dca0 2024-05-09T08:02:21.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:21.152+0000 7fba83e00640 1 -- 172.21.15.1:0/3004497998 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba7c00cdc0 con 0x7fba8c06dca0 2024-05-09T08:02:21.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:21.152+0000 7fba92400640 1 -- 172.21.15.1:0/3004497998 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba8c0c23d0 con 0x7fba8c06dca0 2024-05-09T08:02:21.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:21.153+0000 7fba92400640 1 -- 172.21.15.1:0/3004497998 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba8c074d70 con 0x7fba8c06dca0 2024-05-09T08:02:21.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:21.154+0000 7fba83e00640 1 -- 172.21.15.1:0/3004497998 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fba7c01a070 con 0x7fba8c06dca0 2024-05-09T08:02:21.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:21.155+0000 7fba83e00640 1 --2- 172.21.15.1:0/3004497998 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fba6806be30 0x7fba6806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:21.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:21.155+0000 7fba83e00640 1 -- 172.21.15.1:0/3004497998 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fba7c08b090 con 0x7fba8c06dca0 2024-05-09T08:02:21.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:21.155+0000 7fba82a00640 1 --2- 172.21.15.1:0/3004497998 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fba6806be30 0x7fba6806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:21.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:21.156+0000 7fba82a00640 1 --2- 172.21.15.1:0/3004497998 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fba6806be30 0x7fba6806e2f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fba8c1590c0 tx=0x7fba74007e80 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:21.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:21.160+0000 7fba83e00640 1 -- 172.21.15.1:0/3004497998 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fba7c059140 con 0x7fba8c06dca0 2024-05-09T08:02:21.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:21.398+0000 7fba92400640 1 -- 172.21.15.1:0/3004497998 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fba8c0ccd20 con 0x7fba8c06dca0 2024-05-09T08:02:21.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:21.399+0000 7fba83e00640 1 -- 172.21.15.1:0/3004497998 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+6799 (secure 0 0 0) 0x7fba7c05d440 con 0x7fba8c06dca0 2024-05-09T08:02:21.416 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T08:02:21.416 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":5,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4220,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/1157980939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":1157980939},{"type":"v1","addr":"172.21.15.7:6837","nonce":1157980939}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4235,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6833/3269798795","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6832","nonce":3269798795},{"type":"v1","addr":"172.21.15.1:6833","nonce":3269798795}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4241,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/725115499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6835","nonce":725115499},{"type":"v1","addr":"172.21.15.1:6837","nonce":725115499}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4276,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6838/1827115274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1827115274},{"type":"v1","addr":"172.21.15.7:6838","nonce":1827115274}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/1032767352","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":1032767352},{"type":"v1","addr":"172.21.15.7:6839","nonce":1032767352}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":5,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-09T08:02:15.332696+0000","modified":"2024-05-09T08:02:20.403902+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4294},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4294":{"gid":4294,"name":"c","rank":0,"incarnation":4,"state":"up:creating","state_seq":1,"addr":"172.21.15.1:6836/1509134635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":1509134635},{"type":"v1","addr":"172.21.15.1:6836","nonce":1509134635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-05-09T08:02:21.416 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 5 2024-05-09T08:02:21.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:21.401+0000 7fba92400640 1 -- 172.21.15.1:0/3004497998 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fba6806be30 msgr2=0x7fba6806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:21.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:21.401+0000 7fba92400640 1 --2- 172.21.15.1:0/3004497998 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fba6806be30 0x7fba6806e2f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fba8c1590c0 tx=0x7fba74007e80 comp rx=0 tx=0).stop 2024-05-09T08:02:21.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:21.401+0000 7fba92400640 1 -- 172.21.15.1:0/3004497998 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fba8c06dca0 msgr2=0x7fba8c0c2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:21.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:21.401+0000 7fba92400640 1 --2- 172.21.15.1:0/3004497998 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fba8c06dca0 0x7fba8c0c2bd0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fba7c00e9e0 tx=0x7fba7c00eea0 comp rx=0 tx=0).stop 2024-05-09T08:02:21.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:21.402+0000 7fba92400640 1 -- 172.21.15.1:0/3004497998 shutdown_connections 2024-05-09T08:02:21.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:21.402+0000 7fba92400640 1 --2- 172.21.15.1:0/3004497998 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fba6806be30 0x7fba6806e2f0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:21.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:21.402+0000 7fba92400640 1 --2- 172.21.15.1:0/3004497998 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fba8c0c6ea0 0x7fba8c147560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:21.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:21.402+0000 7fba92400640 1 --2- 172.21.15.1:0/3004497998 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fba8c06ece0 0x7fba8c0c58d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:21.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:21.402+0000 7fba92400640 1 --2- 172.21.15.1:0/3004497998 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fba8c06dca0 0x7fba8c0c2bd0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:21.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:21.402+0000 7fba92400640 1 -- 172.21.15.1:0/3004497998 >> 172.21.15.1:0/3004497998 conn(0x7fba8c0cc070 msgr2=0x7fba8c0c7fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:02:21.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:21.402+0000 7fba92400640 1 -- 172.21.15.1:0/3004497998 shutdown_connections 2024-05-09T08:02:21.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:21.402+0000 7fba92400640 1 -- 172.21.15.1:0/3004497998 wait complete. 2024-05-09T08:02:21.437 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T08:02:21.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:21.626+0000 7fe177200640 1 Processor -- start 2024-05-09T08:02:21.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:21.627+0000 7fe177200640 1 -- start start 2024-05-09T08:02:21.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:21.628+0000 7fe177200640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe178154790 0x7fe17815a440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:21.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:21.628+0000 7fe177200640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fe1780c7dd0 0x7fe1780c44b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:21.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:21.629+0000 7fe177200640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe1780c6a50 con 0x7fe17806b250 2024-05-09T08:02:21.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:21.629+0000 7fe177200640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fe1780c8240 con 0x7fe178154790 2024-05-09T08:02:21.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:21.629+0000 7fe177200640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7fe178154b70 con 0x7fe1780c7dd0 2024-05-09T08:02:21.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:21.629+0000 7fe175e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe178154790 0x7fe17815a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:21.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:21.629+0000 7fe176800640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fe17806b250 0x7fe17806b680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.1:59884/0 (socket says 172.21.15.1:59884) 2024-05-09T08:02:21.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:21.629+0000 7fe176800640 1 -- 172.21.15.1:0/2332918349 learned_addr learned my addr 172.21.15.1:0/2332918349 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T08:02:21.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:21.629+0000 7fe175400640 1 --2- 172.21.15.1:0/2332918349 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fe1780c7dd0 0x7fe1780c44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:21.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:21.629+0000 7fe175e00640 1 -- 172.21.15.1:0/2332918349 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fe1780c7dd0 msgr2=0x7fe1780c44b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:02:21.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:21.629+0000 7fe175e00640 1 --2- 172.21.15.1:0/2332918349 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fe1780c7dd0 0x7fe1780c44b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:21.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:21.629+0000 7fe175e00640 1 -- 172.21.15.1:0/2332918349 >> v1:172.21.15.7:6789/0 conn(0x7fe17806b250 legacy=0x7fe17806b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:21.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:21.629+0000 7fe175e00640 1 -- 172.21.15.1:0/2332918349 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1780c6d30 con 0x7fe178154790 2024-05-09T08:02:21.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:21.629+0000 7fe175400640 1 --2- 172.21.15.1:0/2332918349 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fe1780c7dd0 0x7fe1780c44b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:02:21.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:21.630+0000 7fe175e00640 1 --2- 172.21.15.1:0/2332918349 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe178154790 0x7fe17815a440 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe1680097d0 tx=0x7fe16802a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a624aec3cc0b469f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:21.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:21.631+0000 7fe174a00640 1 -- 172.21.15.1:0/2332918349 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe168037070 con 0x7fe178154790 2024-05-09T08:02:21.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:21.631+0000 7fe174a00640 1 -- 172.21.15.1:0/2332918349 <== mon.1 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe168032440 con 0x7fe178154790 2024-05-09T08:02:21.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:21.631+0000 7fe174a00640 1 -- 172.21.15.1:0/2332918349 <== mon.1 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe168032810 con 0x7fe178154790 2024-05-09T08:02:21.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:21.631+0000 7fe177200640 1 -- 172.21.15.1:0/2332918349 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe178154790 msgr2=0x7fe17815a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:21.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:21.631+0000 7fe177200640 1 --2- 172.21.15.1:0/2332918349 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe178154790 0x7fe17815a440 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe1680097d0 tx=0x7fe16802a1c0 comp rx=0 tx=0).stop 2024-05-09T08:02:21.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:21.632+0000 7fe177200640 1 -- 172.21.15.1:0/2332918349 shutdown_connections 2024-05-09T08:02:21.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:21.632+0000 7fe177200640 1 --2- 172.21.15.1:0/2332918349 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fe1780c7dd0 0x7fe1780c44b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:21.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:21.632+0000 7fe177200640 1 --2- 172.21.15.1:0/2332918349 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe178154790 0x7fe17815a440 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:21.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:21.632+0000 7fe177200640 1 -- 172.21.15.1:0/2332918349 >> 172.21.15.1:0/2332918349 conn(0x7fe1780cbc10 msgr2=0x7fe1780cc020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:02:21.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:21.632+0000 7fe177200640 1 -- 172.21.15.1:0/2332918349 shutdown_connections 2024-05-09T08:02:21.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:21.632+0000 7fe177200640 1 -- 172.21.15.1:0/2332918349 wait complete. 2024-05-09T08:02:21.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:21.633+0000 7fe177200640 1 Processor -- start 2024-05-09T08:02:21.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:21.634+0000 7fe177200640 1 -- start start 2024-05-09T08:02:21.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:21.634+0000 7fe177200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe17806b250 0x7fe178152600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:21.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:21.634+0000 7fe177200640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fe1780c7dd0 0x7fe178152b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:21.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:21.634+0000 7fe177200640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe178154790 0x7fe1781573a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:21.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:21.634+0000 7fe177200640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fe1780c75d0 con 0x7fe178154790 2024-05-09T08:02:21.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:21.634+0000 7fe177200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe1780c72f0 con 0x7fe17806b250 2024-05-09T08:02:21.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:21.634+0000 7fe177200640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7fe1780c78b0 con 0x7fe1780c7dd0 2024-05-09T08:02:21.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:21.635+0000 7fe176800640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe178154790 0x7fe1781573a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:21.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:21.635+0000 7fe176800640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe178154790 0x7fe1781573a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43416/0 (socket says 172.21.15.1:43416) 2024-05-09T08:02:21.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:21.635+0000 7fe176800640 1 -- 172.21.15.1:0/3714745089 learned_addr learned my addr 172.21.15.1:0/3714745089 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:02:21.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:21.635+0000 7fe175400640 1 --2- 172.21.15.1:0/3714745089 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fe1780c7dd0 0x7fe178152b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:21.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:21.635+0000 7fe175e00640 1 --2- 172.21.15.1:0/3714745089 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe17806b250 0x7fe178152600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:21.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:21.635+0000 7fe176800640 1 -- 172.21.15.1:0/3714745089 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fe1780c7dd0 msgr2=0x7fe178152b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:21.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:21.635+0000 7fe176800640 1 --2- 172.21.15.1:0/3714745089 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fe1780c7dd0 0x7fe178152b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:21.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:21.635+0000 7fe176800640 1 -- 172.21.15.1:0/3714745089 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe17806b250 msgr2=0x7fe178152600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:21.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:21.635+0000 7fe176800640 1 --2- 172.21.15.1:0/3714745089 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe17806b250 0x7fe178152600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:21.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:21.635+0000 7fe176800640 1 -- 172.21.15.1:0/3714745089 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1781f0600 con 0x7fe178154790 2024-05-09T08:02:21.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:21.636+0000 7fe176800640 1 --2- 172.21.15.1:0/3714745089 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe178154790 0x7fe1781573a0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fe17000cbc0 tx=0x7fe17000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:21.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:21.636+0000 7fe16e000640 1 -- 172.21.15.1:0/3714745089 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe170017070 con 0x7fe178154790 2024-05-09T08:02:21.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:21.636+0000 7fe16e000640 1 -- 172.21.15.1:0/3714745089 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe17000ad40 con 0x7fe178154790 2024-05-09T08:02:21.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:21.636+0000 7fe16e000640 1 -- 172.21.15.1:0/3714745089 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe170012600 con 0x7fe178154790 2024-05-09T08:02:21.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:21.636+0000 7fe177200640 1 -- 172.21.15.1:0/3714745089 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1781f08e0 con 0x7fe178154790 2024-05-09T08:02:21.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:21.636+0000 7fe177200640 1 -- 172.21.15.1:0/3714745089 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1781f0ea0 con 0x7fe178154790 2024-05-09T08:02:21.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:21.637+0000 7fe177200640 1 -- 172.21.15.1:0/3714745089 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1781f10e0 con 0x7fe178154790 2024-05-09T08:02:21.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:21.638+0000 7fe16e000640 1 -- 172.21.15.1:0/3714745089 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe17001a070 con 0x7fe178154790 2024-05-09T08:02:21.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:21.639+0000 7fe16e000640 1 --2- 172.21.15.1:0/3714745089 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fe15806be60 0x7fe15806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:21.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:21.639+0000 7fe16e000640 1 -- 172.21.15.1:0/3714745089 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fe17008af40 con 0x7fe178154790 2024-05-09T08:02:21.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:21.639+0000 7fe175e00640 1 --2- 172.21.15.1:0/3714745089 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fe15806be60 0x7fe15806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:21.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:21.640+0000 7fe175e00640 1 --2- 172.21.15.1:0/3714745089 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fe15806be60 0x7fe15806e320 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fe168034000 tx=0x7fe16802e000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:21.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:21.644+0000 7fe16e000640 1 -- 172.21.15.1:0/3714745089 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fe1781f10e0 con 0x7fe178154790 2024-05-09T08:02:21.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:21.851+0000 7fe177200640 1 -- 172.21.15.1:0/3714745089 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe17804f660 con 0x7fe178154790 2024-05-09T08:02:21.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:21.854+0000 7fe16e000640 1 -- 172.21.15.1:0/3714745089 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18382 (secure 0 0 0) 0x7fe170058e30 con 0x7fe178154790 2024-05-09T08:02:21.871 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T08:02:21.871 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":25,"fsid":"54329ec8-3270-4043-9a82-c6cd7420aa7f","created":"2024-05-09T08:01:56.666970+0000","modified":"2024-05-09T08:02:20.403456+0000","last_up_change":"2024-05-09T08:02:07.146082+0000","last_in_change":"2024-05-09T08:01:59.008225+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T08:02:09.165511+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T08:02:12.925485+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T08:02:14.367455+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-09T08:02:17.967064+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c064454e-a9b5-4a24-9a77-17f0a4085952","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6800","nonce":1513349547},{"type":"v1","addr":"172.21.15.1:6801","nonce":1513349547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":1513349547},{"type":"v1","addr":"172.21.15.1:6803","nonce":1513349547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":1513349547},{"type":"v1","addr":"172.21.15.1:6807","nonce":1513349547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":1513349547},{"type":"v1","addr":"172.21.15.1:6805","nonce":1513349547}]},"public_addr":"172.21.15.1:6801/1513349547","cluster_addr":"172.21.15.1:6803/1513349547","heartbeat_back_addr":"172.21.15.1:6807/1513349547","heartbeat_front_addr":"172.21.15.1:6805/1513349547","state":["exists","up"]},{"osd":1,"uuid":"1a9ea687-2ff6-407a-949b-0e7e1083af21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":2942255259},{"type":"v1","addr":"172.21.15.1:6817","nonce":2942255259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":2942255259},{"type":"v1","addr":"172.21.15.1:6819","nonce":2942255259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":2942255259},{"type":"v1","addr":"172.21.15.1:6823","nonce":2942255259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6820","nonce":2942255259},{"type":"v1","addr":"172.21.15.1:6821","nonce":2942255259}]},"public_addr":"172.21.15.1:6817/2942255259","cluster_addr":"172.21.15.1:6819/2942255259","heartbeat_back_addr":"172.21.15.1:6823/2942255259","heartbeat_front_addr":"172.21.15.1:6821/2942255259","state":["exists","up"]},{"osd":2,"uuid":"c88af08b-7c9a-4dd8-9640-44138239addb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":2925031811},{"type":"v1","addr":"172.21.15.1:6809","nonce":2925031811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":2925031811},{"type":"v1","addr":"172.21.15.1:6811","nonce":2925031811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":2925031811},{"type":"v1","addr":"172.21.15.1:6815","nonce":2925031811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":2925031811},{"type":"v1","addr":"172.21.15.1:6813","nonce":2925031811}]},"public_addr":"172.21.15.1:6809/2925031811","cluster_addr":"172.21.15.1:6811/2925031811","heartbeat_back_addr":"172.21.15.1:6815/2925031811","heartbeat_front_addr":"172.21.15.1:6813/2925031811","state":["exists","up"]},{"osd":3,"uuid":"c087b7e3-a41d-4333-9365-32e3bd8c8e6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6824","nonce":2497776654},{"type":"v1","addr":"172.21.15.1:6825","nonce":2497776654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":2497776654},{"type":"v1","addr":"172.21.15.1:6827","nonce":2497776654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":2497776654},{"type":"v1","addr":"172.21.15.1:6831","nonce":2497776654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6828","nonce":2497776654},{"type":"v1","addr":"172.21.15.1:6829","nonce":2497776654}]},"public_addr":"172.21.15.1:6825/2497776654","cluster_addr":"172.21.15.1:6827/2497776654","heartbeat_back_addr":"172.21.15.1:6831/2497776654","heartbeat_front_addr":"172.21.15.1:6829/2497776654","state":["exists","up"]},{"osd":4,"uuid":"b4576b9c-871d-47a8-8d36-65af525d0200","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":2957026670},{"type":"v1","addr":"172.21.15.7:6801","nonce":2957026670}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2957026670},{"type":"v1","addr":"172.21.15.7:6803","nonce":2957026670}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":2957026670},{"type":"v1","addr":"172.21.15.7:6807","nonce":2957026670}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2957026670},{"type":"v1","addr":"172.21.15.7:6805","nonce":2957026670}]},"public_addr":"172.21.15.7:6801/2957026670","cluster_addr":"172.21.15.7:6803/2957026670","heartbeat_back_addr":"172.21.15.7:6807/2957026670","heartbeat_front_addr":"172.21.15.7:6805/2957026670","state":["exists","up"]},{"osd":5,"uuid":"902b3d94-72f3-4499-b21a-e32453ef5144","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1670295242},{"type":"v1","addr":"172.21.15.7:6809","nonce":1670295242}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1670295242},{"type":"v1","addr":"172.21.15.7:6811","nonce":1670295242}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1670295242},{"type":"v1","addr":"172.21.15.7:6815","nonce":1670295242}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1670295242},{"type":"v1","addr":"172.21.15.7:6813","nonce":1670295242}]},"public_addr":"172.21.15.7:6809/1670295242","cluster_addr":"172.21.15.7:6811/1670295242","heartbeat_back_addr":"172.21.15.7:6815/1670295242","heartbeat_front_addr":"172.21.15.7:6813/1670295242","state":["exists","up"]},{"osd":6,"uuid":"ead1d771-1728-4f85-adb4-26be21cedff3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":2852086880},{"type":"v1","addr":"172.21.15.7:6817","nonce":2852086880}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":2852086880},{"type":"v1","addr":"172.21.15.7:6819","nonce":2852086880}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":2852086880},{"type":"v1","addr":"172.21.15.7:6823","nonce":2852086880}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":2852086880},{"type":"v1","addr":"172.21.15.7:6821","nonce":2852086880}]},"public_addr":"172.21.15.7:6817/2852086880","cluster_addr":"172.21.15.7:6819/2852086880","heartbeat_back_addr":"172.21.15.7:6823/2852086880","heartbeat_front_addr":"172.21.15.7:6821/2852086880","state":["exists","up"]},{"osd":7,"uuid":"3ab5a201-9394-48a8-8918-e95197893669","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":1442884138},{"type":"v1","addr":"172.21.15.7:6825","nonce":1442884138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1442884138},{"type":"v1","addr":"172.21.15.7:6827","nonce":1442884138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1442884138},{"type":"v1","addr":"172.21.15.7:6831","nonce":1442884138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1442884138},{"type":"v1","addr":"172.21.15.7:6829","nonce":1442884138}]},"public_addr":"172.21.15.7:6825/1442884138","cluster_addr":"172.21.15.7:6827/1442884138","heartbeat_back_addr":"172.21.15.7:6831/1442884138","heartbeat_front_addr":"172.21.15.7:6829/1442884138","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:02:04.615675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:02:04.385974+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:02:04.619217+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:02:04.401681+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:02:04.097081+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:02:04.129981+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:02:04.225327+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:02:04.193351+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T08:02:21.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:21.857+0000 7fe177200640 1 -- 172.21.15.1:0/3714745089 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fe15806be60 msgr2=0x7fe15806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:21.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:21.857+0000 7fe177200640 1 --2- 172.21.15.1:0/3714745089 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fe15806be60 0x7fe15806e320 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fe168034000 tx=0x7fe16802e000 comp rx=0 tx=0).stop 2024-05-09T08:02:21.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:21.857+0000 7fe177200640 1 -- 172.21.15.1:0/3714745089 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe178154790 msgr2=0x7fe1781573a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:21.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:21.857+0000 7fe177200640 1 --2- 172.21.15.1:0/3714745089 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe178154790 0x7fe1781573a0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fe17000cbc0 tx=0x7fe17000a430 comp rx=0 tx=0).stop 2024-05-09T08:02:21.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:21.857+0000 7fe177200640 1 -- 172.21.15.1:0/3714745089 shutdown_connections 2024-05-09T08:02:21.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:21.857+0000 7fe177200640 1 --2- 172.21.15.1:0/3714745089 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fe15806be60 0x7fe15806e320 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:21.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:21.857+0000 7fe177200640 1 --2- 172.21.15.1:0/3714745089 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe178154790 0x7fe1781573a0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:21.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:21.857+0000 7fe177200640 1 --2- 172.21.15.1:0/3714745089 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fe1780c7dd0 0x7fe178152b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:21.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:21.857+0000 7fe177200640 1 --2- 172.21.15.1:0/3714745089 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe17806b250 0x7fe178152600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:21.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:21.857+0000 7fe177200640 1 -- 172.21.15.1:0/3714745089 >> 172.21.15.1:0/3714745089 conn(0x7fe1780cbc10 msgr2=0x7fe1781434a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:02:21.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:21.857+0000 7fe177200640 1 -- 172.21.15.1:0/3714745089 shutdown_connections 2024-05-09T08:02:21.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:21.857+0000 7fe177200640 1 -- 172.21.15.1:0/3714745089 wait complete. 2024-05-09T08:02:21.895 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T08:02:22.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:22.094+0000 7f2ba2000640 1 Processor -- start 2024-05-09T08:02:22.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:22.094+0000 7f2ba2000640 1 -- start start 2024-05-09T08:02:22.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:22.096+0000 7f2ba2000640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f2b9c0c7720 0x7f2b9c140200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:22.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:22.096+0000 7f2ba2000640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2b9c157430 con 0x7f2b9c0c5910 2024-05-09T08:02:22.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:22.096+0000 7f2ba2000640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2b9c0c7d60 con 0x7f2b9c0c5cf0 2024-05-09T08:02:22.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:22.096+0000 7f2ba2000640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f2b9c0c7b30 con 0x7f2b9c0c7720 2024-05-09T08:02:22.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:22.096+0000 7f2ba0c00640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f2b9c0c5910 0x7f2b9c15b380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:40274/0 (socket says 172.21.15.1:40274) 2024-05-09T08:02:22.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:22.096+0000 7f2ba0c00640 1 -- 172.21.15.1:0/3110307441 learned_addr learned my addr 172.21.15.1:0/3110307441 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T08:02:22.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:22.096+0000 7f2ba1600640 1 --2- 172.21.15.1:0/3110307441 >> v2:172.21.15.1:3300/0 conn(0x7f2b9c0c7720 0x7f2b9c140200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:22.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:22.097+0000 7f2b9be00640 1 -- 172.21.15.1:0/3110307441 <== mon.1 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 361428300 0 0) 0x7f2b9c0c7d60 con 0x7f2b9c0c5cf0 2024-05-09T08:02:22.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:22.097+0000 7f2b9be00640 1 -- 172.21.15.1:0/3110307441 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2b80003520 con 0x7f2b9c0c5cf0 2024-05-09T08:02:22.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:22.097+0000 7f2b9be00640 1 -- 172.21.15.1:0/3110307441 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2799585113 0 0) 0x7f2b9c157430 con 0x7f2b9c0c5910 2024-05-09T08:02:22.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:22.097+0000 7f2b9be00640 1 -- 172.21.15.1:0/3110307441 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2b9c0c7d60 con 0x7f2b9c0c5910 2024-05-09T08:02:22.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:22.097+0000 7f2ba1600640 1 -- 172.21.15.1:0/3110307441 >> v1:172.21.15.7:6789/0 conn(0x7f2b9c0c5cf0 legacy=0x7f2b9c0c3110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:22.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:22.097+0000 7f2ba1600640 1 -- 172.21.15.1:0/3110307441 >> v1:172.21.15.1:6789/0 conn(0x7f2b9c0c5910 legacy=0x7f2b9c15b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:22.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:22.097+0000 7f2ba1600640 1 -- 172.21.15.1:0/3110307441 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b9c0c8040 con 0x7f2b9c0c7720 2024-05-09T08:02:22.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:22.098+0000 7f2ba1600640 1 --2- 172.21.15.1:0/3110307441 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2b9c0c7720 0x7f2b9c140200 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f2b94009d90 tx=0x7f2b9402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8507016709eb441 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:22.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:22.099+0000 7f2b9be00640 1 -- 172.21.15.1:0/3110307441 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b94007740 con 0x7f2b9c0c7720 2024-05-09T08:02:22.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:22.099+0000 7f2b9be00640 1 -- 172.21.15.1:0/3110307441 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b940078e0 con 0x7f2b9c0c7720 2024-05-09T08:02:22.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:22.099+0000 7f2b9be00640 1 -- 172.21.15.1:0/3110307441 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b94007cb0 con 0x7f2b9c0c7720 2024-05-09T08:02:22.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:22.099+0000 7f2ba2000640 1 -- 172.21.15.1:0/3110307441 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2b9c0c7720 msgr2=0x7f2b9c140200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:22.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:22.099+0000 7f2ba2000640 1 --2- 172.21.15.1:0/3110307441 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2b9c0c7720 0x7f2b9c140200 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f2b94009d90 tx=0x7f2b9402a660 comp rx=0 tx=0).stop 2024-05-09T08:02:22.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:22.099+0000 7f2ba2000640 1 -- 172.21.15.1:0/3110307441 shutdown_connections 2024-05-09T08:02:22.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:22.099+0000 7f2ba2000640 1 --2- 172.21.15.1:0/3110307441 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2b9c0c7720 0x7f2b9c140200 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:22.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:22.099+0000 7f2ba2000640 1 -- 172.21.15.1:0/3110307441 >> 172.21.15.1:0/3110307441 conn(0x7f2b9c0cbc10 msgr2=0x7f2b9c0cc020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:02:22.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:22.100+0000 7f2ba2000640 1 -- 172.21.15.1:0/3110307441 shutdown_connections 2024-05-09T08:02:22.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:22.100+0000 7f2ba2000640 1 -- 172.21.15.1:0/3110307441 wait complete. 2024-05-09T08:02:22.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:22.101+0000 7f2ba2000640 1 Processor -- start 2024-05-09T08:02:22.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:22.102+0000 7f2ba2000640 1 -- start start 2024-05-09T08:02:22.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:22.102+0000 7f2ba2000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2b9c0c5910 0x7f2b9c071b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:22.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:22.102+0000 7f2ba2000640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f2b9c0c5cf0 0x7f2b9c1526d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:22.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:22.103+0000 7f2ba2000640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2b9c0c7720 0x7f2b9c1f34b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:22.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:22.103+0000 7f2ba2000640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f2b9c157cb0 con 0x7f2b9c0c7720 2024-05-09T08:02:22.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:22.103+0000 7f2ba2000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2b9c1579d0 con 0x7f2b9c0c5910 2024-05-09T08:02:22.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:22.103+0000 7f2ba2000640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f2b9c1581d0 con 0x7f2b9c0c5cf0 2024-05-09T08:02:22.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:22.103+0000 7f2ba0c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2b9c0c5910 0x7f2b9c071b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:22.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:22.103+0000 7f2ba1600640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2b9c0c7720 0x7f2b9c1f34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:22.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:22.103+0000 7f2b93e00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f2b9c0c5cf0 0x7f2b9c1526d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:22.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:22.103+0000 7f2ba1600640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2b9c0c7720 0x7f2b9c1f34b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43438/0 (socket says 172.21.15.1:43438) 2024-05-09T08:02:22.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:22.103+0000 7f2ba1600640 1 -- 172.21.15.1:0/3325984811 learned_addr learned my addr 172.21.15.1:0/3325984811 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:02:22.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:22.103+0000 7f2ba1600640 1 -- 172.21.15.1:0/3325984811 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f2b9c0c5cf0 msgr2=0x7f2b9c1526d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:22.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:22.103+0000 7f2ba1600640 1 --2- 172.21.15.1:0/3325984811 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f2b9c0c5cf0 0x7f2b9c1526d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:22.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:22.103+0000 7f2ba1600640 1 -- 172.21.15.1:0/3325984811 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2b9c0c5910 msgr2=0x7f2b9c071b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:22.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:22.103+0000 7f2ba1600640 1 --2- 172.21.15.1:0/3325984811 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2b9c0c5910 0x7f2b9c071b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:22.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:22.103+0000 7f2ba1600640 1 -- 172.21.15.1:0/3325984811 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b9c1e00c0 con 0x7f2b9c0c7720 2024-05-09T08:02:22.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:22.104+0000 7f2ba1600640 1 --2- 172.21.15.1:0/3325984811 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2b9c0c7720 0x7f2b9c1f34b0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f2b94009eb0 tx=0x7f2b94004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:22.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:22.104+0000 7f2b99600640 1 -- 172.21.15.1:0/3325984811 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b94040070 con 0x7f2b9c0c7720 2024-05-09T08:02:22.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:22.104+0000 7f2b99600640 1 -- 172.21.15.1:0/3325984811 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b94031bc0 con 0x7f2b9c0c7720 2024-05-09T08:02:22.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:22.104+0000 7f2ba2000640 1 -- 172.21.15.1:0/3325984811 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b9c1e0360 con 0x7f2b9c0c7720 2024-05-09T08:02:22.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:22.104+0000 7f2b99600640 1 -- 172.21.15.1:0/3325984811 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b94009450 con 0x7f2b9c0c7720 2024-05-09T08:02:22.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:22.104+0000 7f2ba2000640 1 -- 172.21.15.1:0/3325984811 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b9c1e0920 con 0x7f2b9c0c7720 2024-05-09T08:02:22.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:22.105+0000 7f2ba2000640 1 -- 172.21.15.1:0/3325984811 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b9c04f660 con 0x7f2b9c0c7720 2024-05-09T08:02:22.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:22.106+0000 7f2b99600640 1 -- 172.21.15.1:0/3325984811 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2b94039040 con 0x7f2b9c0c7720 2024-05-09T08:02:22.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:22.107+0000 7f2b99600640 1 --2- 172.21.15.1:0/3325984811 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f2b7806be30 0x7f2b7806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:22.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:22.107+0000 7f2b99600640 1 -- 172.21.15.1:0/3325984811 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f2b940ac9a0 con 0x7f2b9c0c7720 2024-05-09T08:02:22.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:22.107+0000 7f2ba0c00640 1 --2- 172.21.15.1:0/3325984811 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f2b7806be30 0x7f2b7806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:22.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:22.108+0000 7f2ba0c00640 1 --2- 172.21.15.1:0/3325984811 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f2b7806be30 0x7f2b7806e2f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f2b8c0011e0 tx=0x7f2b8c007d30 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:22.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:22.112+0000 7f2b99600640 1 -- 172.21.15.1:0/3325984811 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f2b9407a9b0 con 0x7f2b9c0c7720 2024-05-09T08:02:22.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:22.312+0000 7f2ba2000640 1 -- 172.21.15.1:0/3325984811 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2b9c0c43a0 con 0x7f2b9c0c7720 2024-05-09T08:02:22.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:22.314+0000 7f2b99600640 1 -- 172.21.15.1:0/3325984811 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18382 (secure 0 0 0) 0x7f2b9407ecb0 con 0x7f2b9c0c7720 2024-05-09T08:02:22.331 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T08:02:22.332 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":25,"fsid":"54329ec8-3270-4043-9a82-c6cd7420aa7f","created":"2024-05-09T08:01:56.666970+0000","modified":"2024-05-09T08:02:20.403456+0000","last_up_change":"2024-05-09T08:02:07.146082+0000","last_in_change":"2024-05-09T08:01:59.008225+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T08:02:09.165511+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T08:02:12.925485+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T08:02:14.367455+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-09T08:02:17.967064+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c064454e-a9b5-4a24-9a77-17f0a4085952","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6800","nonce":1513349547},{"type":"v1","addr":"172.21.15.1:6801","nonce":1513349547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":1513349547},{"type":"v1","addr":"172.21.15.1:6803","nonce":1513349547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":1513349547},{"type":"v1","addr":"172.21.15.1:6807","nonce":1513349547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":1513349547},{"type":"v1","addr":"172.21.15.1:6805","nonce":1513349547}]},"public_addr":"172.21.15.1:6801/1513349547","cluster_addr":"172.21.15.1:6803/1513349547","heartbeat_back_addr":"172.21.15.1:6807/1513349547","heartbeat_front_addr":"172.21.15.1:6805/1513349547","state":["exists","up"]},{"osd":1,"uuid":"1a9ea687-2ff6-407a-949b-0e7e1083af21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":2942255259},{"type":"v1","addr":"172.21.15.1:6817","nonce":2942255259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":2942255259},{"type":"v1","addr":"172.21.15.1:6819","nonce":2942255259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":2942255259},{"type":"v1","addr":"172.21.15.1:6823","nonce":2942255259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6820","nonce":2942255259},{"type":"v1","addr":"172.21.15.1:6821","nonce":2942255259}]},"public_addr":"172.21.15.1:6817/2942255259","cluster_addr":"172.21.15.1:6819/2942255259","heartbeat_back_addr":"172.21.15.1:6823/2942255259","heartbeat_front_addr":"172.21.15.1:6821/2942255259","state":["exists","up"]},{"osd":2,"uuid":"c88af08b-7c9a-4dd8-9640-44138239addb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":2925031811},{"type":"v1","addr":"172.21.15.1:6809","nonce":2925031811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":2925031811},{"type":"v1","addr":"172.21.15.1:6811","nonce":2925031811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":2925031811},{"type":"v1","addr":"172.21.15.1:6815","nonce":2925031811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":2925031811},{"type":"v1","addr":"172.21.15.1:6813","nonce":2925031811}]},"public_addr":"172.21.15.1:6809/2925031811","cluster_addr":"172.21.15.1:6811/2925031811","heartbeat_back_addr":"172.21.15.1:6815/2925031811","heartbeat_front_addr":"172.21.15.1:6813/2925031811","state":["exists","up"]},{"osd":3,"uuid":"c087b7e3-a41d-4333-9365-32e3bd8c8e6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6824","nonce":2497776654},{"type":"v1","addr":"172.21.15.1:6825","nonce":2497776654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":2497776654},{"type":"v1","addr":"172.21.15.1:6827","nonce":2497776654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":2497776654},{"type":"v1","addr":"172.21.15.1:6831","nonce":2497776654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6828","nonce":2497776654},{"type":"v1","addr":"172.21.15.1:6829","nonce":2497776654}]},"public_addr":"172.21.15.1:6825/2497776654","cluster_addr":"172.21.15.1:6827/2497776654","heartbeat_back_addr":"172.21.15.1:6831/2497776654","heartbeat_front_addr":"172.21.15.1:6829/2497776654","state":["exists","up"]},{"osd":4,"uuid":"b4576b9c-871d-47a8-8d36-65af525d0200","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":2957026670},{"type":"v1","addr":"172.21.15.7:6801","nonce":2957026670}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2957026670},{"type":"v1","addr":"172.21.15.7:6803","nonce":2957026670}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":2957026670},{"type":"v1","addr":"172.21.15.7:6807","nonce":2957026670}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2957026670},{"type":"v1","addr":"172.21.15.7:6805","nonce":2957026670}]},"public_addr":"172.21.15.7:6801/2957026670","cluster_addr":"172.21.15.7:6803/2957026670","heartbeat_back_addr":"172.21.15.7:6807/2957026670","heartbeat_front_addr":"172.21.15.7:6805/2957026670","state":["exists","up"]},{"osd":5,"uuid":"902b3d94-72f3-4499-b21a-e32453ef5144","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1670295242},{"type":"v1","addr":"172.21.15.7:6809","nonce":1670295242}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1670295242},{"type":"v1","addr":"172.21.15.7:6811","nonce":1670295242}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1670295242},{"type":"v1","addr":"172.21.15.7:6815","nonce":1670295242}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1670295242},{"type":"v1","addr":"172.21.15.7:6813","nonce":1670295242}]},"public_addr":"172.21.15.7:6809/1670295242","cluster_addr":"172.21.15.7:6811/1670295242","heartbeat_back_addr":"172.21.15.7:6815/1670295242","heartbeat_front_addr":"172.21.15.7:6813/1670295242","state":["exists","up"]},{"osd":6,"uuid":"ead1d771-1728-4f85-adb4-26be21cedff3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":2852086880},{"type":"v1","addr":"172.21.15.7:6817","nonce":2852086880}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":2852086880},{"type":"v1","addr":"172.21.15.7:6819","nonce":2852086880}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":2852086880},{"type":"v1","addr":"172.21.15.7:6823","nonce":2852086880}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":2852086880},{"type":"v1","addr":"172.21.15.7:6821","nonce":2852086880}]},"public_addr":"172.21.15.7:6817/2852086880","cluster_addr":"172.21.15.7:6819/2852086880","heartbeat_back_addr":"172.21.15.7:6823/2852086880","heartbeat_front_addr":"172.21.15.7:6821/2852086880","state":["exists","up"]},{"osd":7,"uuid":"3ab5a201-9394-48a8-8918-e95197893669","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":1442884138},{"type":"v1","addr":"172.21.15.7:6825","nonce":1442884138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1442884138},{"type":"v1","addr":"172.21.15.7:6827","nonce":1442884138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1442884138},{"type":"v1","addr":"172.21.15.7:6831","nonce":1442884138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1442884138},{"type":"v1","addr":"172.21.15.7:6829","nonce":1442884138}]},"public_addr":"172.21.15.7:6825/1442884138","cluster_addr":"172.21.15.7:6827/1442884138","heartbeat_back_addr":"172.21.15.7:6831/1442884138","heartbeat_front_addr":"172.21.15.7:6829/1442884138","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:02:04.615675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:02:04.385974+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:02:04.619217+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:02:04.401681+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:02:04.097081+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:02:04.129981+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:02:04.225327+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:02:04.193351+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T08:02:22.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:22.317+0000 7f2ba2000640 1 -- 172.21.15.1:0/3325984811 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f2b7806be30 msgr2=0x7f2b7806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:22.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:22.317+0000 7f2ba2000640 1 --2- 172.21.15.1:0/3325984811 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f2b7806be30 0x7f2b7806e2f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f2b8c0011e0 tx=0x7f2b8c007d30 comp rx=0 tx=0).stop 2024-05-09T08:02:22.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:22.317+0000 7f2ba2000640 1 -- 172.21.15.1:0/3325984811 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2b9c0c7720 msgr2=0x7f2b9c1f34b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:22.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:22.317+0000 7f2ba2000640 1 --2- 172.21.15.1:0/3325984811 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2b9c0c7720 0x7f2b9c1f34b0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f2b94009eb0 tx=0x7f2b94004a40 comp rx=0 tx=0).stop 2024-05-09T08:02:22.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:22.317+0000 7f2ba2000640 1 -- 172.21.15.1:0/3325984811 shutdown_connections 2024-05-09T08:02:22.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:22.317+0000 7f2ba2000640 1 --2- 172.21.15.1:0/3325984811 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f2b7806be30 0x7f2b7806e2f0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:22.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:22.317+0000 7f2ba2000640 1 --2- 172.21.15.1:0/3325984811 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2b9c0c7720 0x7f2b9c1f34b0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:22.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:22.317+0000 7f2ba2000640 1 --2- 172.21.15.1:0/3325984811 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f2b9c0c5cf0 0x7f2b9c1526d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:22.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:22.317+0000 7f2ba2000640 1 --2- 172.21.15.1:0/3325984811 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2b9c0c5910 0x7f2b9c071b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:22.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:22.317+0000 7f2ba2000640 1 -- 172.21.15.1:0/3325984811 >> 172.21.15.1:0/3325984811 conn(0x7f2b9c0cbc10 msgr2=0x7f2b9c156900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:02:22.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:22.317+0000 7f2ba2000640 1 -- 172.21.15.1:0/3325984811 shutdown_connections 2024-05-09T08:02:22.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:22.317+0000 7f2ba2000640 1 -- 172.21.15.1:0/3325984811 wait complete. 2024-05-09T08:02:22.366 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T08:02:22.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:22.565+0000 7f3b3f200640 1 Processor -- start 2024-05-09T08:02:22.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:22.566+0000 7f3b3f200640 1 -- start start 2024-05-09T08:02:22.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:22.567+0000 7f3b3f200640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f3b40152500 0x7f3b400c8180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:22.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:22.567+0000 7f3b3f200640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f3b400cc730 0x7f3b4015c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:22.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:22.567+0000 7f3b3f200640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f3b400c7660 0x7f3b400c7ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:22.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:22.567+0000 7f3b3f200640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f3b400c5e50 con 0x7f3b400cc730 2024-05-09T08:02:22.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:22.567+0000 7f3b3f200640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f3b400c22d0 con 0x7f3b40152500 2024-05-09T08:02:22.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:22.567+0000 7f3b3f200640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f3b400ccb50 con 0x7f3b400c7660 2024-05-09T08:02:22.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:22.568+0000 7f3b3d400640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f3b400cc730 0x7f3b4015c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:22.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:22.568+0000 7f3b3de00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f3b40152500 0x7f3b400c8180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:22.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:22.568+0000 7f3b3de00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f3b40152500 0x7f3b400c8180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.1:41530/0 (socket says 172.21.15.1:41530) 2024-05-09T08:02:22.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:22.568+0000 7f3b3d400640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f3b400cc730 0x7f3b4015c080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43446/0 (socket says 172.21.15.1:43446) 2024-05-09T08:02:22.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:22.568+0000 7f3b3de00640 1 -- 172.21.15.1:0/450763023 learned_addr learned my addr 172.21.15.1:0/450763023 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:02:22.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:22.568+0000 7f3b3e800640 1 --2- 172.21.15.1:0/450763023 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f3b400c7660 0x7f3b400c7ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:22.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:22.568+0000 7f3b3de00640 1 -- 172.21.15.1:0/450763023 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f3b400c7660 msgr2=0x7f3b400c7ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:02:22.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:22.568+0000 7f3b3de00640 1 --2- 172.21.15.1:0/450763023 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f3b400c7660 0x7f3b400c7ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:22.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:22.568+0000 7f3b3de00640 1 -- 172.21.15.1:0/450763023 >> v2:172.21.15.1:3300/0 conn(0x7f3b400cc730 msgr2=0x7f3b4015c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:02:22.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:22.568+0000 7f3b3de00640 1 --2- 172.21.15.1:0/450763023 >> v2:172.21.15.1:3300/0 conn(0x7f3b400cc730 0x7f3b4015c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:22.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:22.568+0000 7f3b3de00640 1 -- 172.21.15.1:0/450763023 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b400c6a30 con 0x7f3b40152500 2024-05-09T08:02:22.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:22.569+0000 7f3b3de00640 1 --2- 172.21.15.1:0/450763023 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b40152500 0x7f3b400c8180 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f3b4006eb20 tx=0x7f3b3002a730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c6ad6b622c178c9c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:22.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:22.570+0000 7f3b3ca00640 1 -- 172.21.15.1:0/450763023 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b30036070 con 0x7f3b40152500 2024-05-09T08:02:22.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:22.570+0000 7f3b3ca00640 1 -- 172.21.15.1:0/450763023 <== mon.1 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b30031600 con 0x7f3b40152500 2024-05-09T08:02:22.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:22.570+0000 7f3b3ca00640 1 -- 172.21.15.1:0/450763023 <== mon.1 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b300319d0 con 0x7f3b40152500 2024-05-09T08:02:22.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:22.570+0000 7f3b3f200640 1 -- 172.21.15.1:0/450763023 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b40152500 msgr2=0x7f3b400c8180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:22.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:22.570+0000 7f3b3f200640 1 --2- 172.21.15.1:0/450763023 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b40152500 0x7f3b400c8180 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f3b4006eb20 tx=0x7f3b3002a730 comp rx=0 tx=0).stop 2024-05-09T08:02:22.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:22.570+0000 7f3b3f200640 1 -- 172.21.15.1:0/450763023 shutdown_connections 2024-05-09T08:02:22.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:22.570+0000 7f3b3f200640 1 --2- 172.21.15.1:0/450763023 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f3b400c7660 0x7f3b400c7ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:22.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:22.570+0000 7f3b3f200640 1 --2- 172.21.15.1:0/450763023 >> v2:172.21.15.1:3300/0 conn(0x7f3b400cc730 0x7f3b4015c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:22.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:22.570+0000 7f3b3f200640 1 --2- 172.21.15.1:0/450763023 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b40152500 0x7f3b400c8180 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:22.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:22.570+0000 7f3b3f200640 1 -- 172.21.15.1:0/450763023 >> 172.21.15.1:0/450763023 conn(0x7f3b400cbc10 msgr2=0x7f3b400cc020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:02:22.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:22.571+0000 7f3b3f200640 1 -- 172.21.15.1:0/450763023 shutdown_connections 2024-05-09T08:02:22.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:22.571+0000 7f3b3f200640 1 -- 172.21.15.1:0/450763023 wait complete. 2024-05-09T08:02:22.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:22.573+0000 7f3b3f200640 1 Processor -- start 2024-05-09T08:02:22.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:22.573+0000 7f3b3f200640 1 -- start start 2024-05-09T08:02:22.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:22.574+0000 7f3b3f200640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3b400c7660 0x7f3b401e48e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:22.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:22.574+0000 7f3b3f200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b400cc730 0x7f3b401f3c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:22.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:22.574+0000 7f3b3f200640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f3b40152500 0x7f3b40159a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:22.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:22.574+0000 7f3b3f200640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f3b400c72b0 con 0x7f3b400c7660 2024-05-09T08:02:22.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:22.574+0000 7f3b3f200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f3b400c6fd0 con 0x7f3b400cc730 2024-05-09T08:02:22.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:22.574+0000 7f3b3f200640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f3b4015b880 con 0x7f3b40152500 2024-05-09T08:02:22.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:22.574+0000 7f3b3de00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3b400c7660 0x7f3b401e48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:22.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:22.575+0000 7f3b3d400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b400cc730 0x7f3b401f3c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:22.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:22.575+0000 7f3b3e800640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f3b40152500 0x7f3b40159a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:22.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:22.575+0000 7f3b3d400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b400cc730 0x7f3b401f3c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.1:41546/0 (socket says 172.21.15.1:41546) 2024-05-09T08:02:22.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:22.575+0000 7f3b3de00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3b400c7660 0x7f3b401e48e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43452/0 (socket says 172.21.15.1:43452) 2024-05-09T08:02:22.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:22.575+0000 7f3b3d400640 1 -- 172.21.15.1:0/2014951519 learned_addr learned my addr 172.21.15.1:0/2014951519 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:02:22.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:22.575+0000 7f3b3d400640 1 -- 172.21.15.1:0/2014951519 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f3b40152500 msgr2=0x7f3b40159a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:22.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:22.575+0000 7f3b3d400640 1 --2- 172.21.15.1:0/2014951519 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f3b40152500 0x7f3b40159a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:22.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:22.575+0000 7f3b3d400640 1 -- 172.21.15.1:0/2014951519 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3b400c7660 msgr2=0x7f3b401e48e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:22.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:22.575+0000 7f3b3d400640 1 --2- 172.21.15.1:0/2014951519 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3b400c7660 0x7f3b401e48e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:22.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:22.575+0000 7f3b3d400640 1 -- 172.21.15.1:0/2014951519 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b401e3210 con 0x7f3b400cc730 2024-05-09T08:02:22.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:22.575+0000 7f3b3de00640 1 --2- 172.21.15.1:0/2014951519 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3b400c7660 0x7f3b401e48e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:02:22.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:22.575+0000 7f3b3e800640 1 --2- 172.21.15.1:0/2014951519 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f3b40152500 0x7f3b40159a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:02:22.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:22.575+0000 7f3b3d400640 1 --2- 172.21.15.1:0/2014951519 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b400cc730 0x7f3b401f3c20 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f3b2c00e9e0 tx=0x7f3b2c00eea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:22.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:22.576+0000 7f3b36000640 1 -- 172.21.15.1:0/2014951519 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b2c017070 con 0x7f3b400cc730 2024-05-09T08:02:22.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:22.576+0000 7f3b36000640 1 -- 172.21.15.1:0/2014951519 <== mon.1 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b2c00c9d0 con 0x7f3b400cc730 2024-05-09T08:02:22.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:22.576+0000 7f3b36000640 1 -- 172.21.15.1:0/2014951519 <== mon.1 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b2c00cdc0 con 0x7f3b400cc730 2024-05-09T08:02:22.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:22.576+0000 7f3b3f200640 1 -- 172.21.15.1:0/2014951519 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b401eff40 con 0x7f3b400cc730 2024-05-09T08:02:22.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:22.576+0000 7f3b3f200640 1 -- 172.21.15.1:0/2014951519 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b401f0480 con 0x7f3b400cc730 2024-05-09T08:02:22.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:22.577+0000 7f3b3f200640 1 -- 172.21.15.1:0/2014951519 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b4004f660 con 0x7f3b400cc730 2024-05-09T08:02:22.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:22.578+0000 7f3b36000640 1 -- 172.21.15.1:0/2014951519 <== mon.1 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3b2c01a070 con 0x7f3b400cc730 2024-05-09T08:02:22.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:22.579+0000 7f3b36000640 1 --2- 172.21.15.1:0/2014951519 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f3b2006be30 0x7f3b2006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:22.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:22.579+0000 7f3b36000640 1 -- 172.21.15.1:0/2014951519 <== mon.1 v2:172.21.15.7:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f3b2c08b250 con 0x7f3b400cc730 2024-05-09T08:02:22.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:22.579+0000 7f3b3de00640 1 --2- 172.21.15.1:0/2014951519 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f3b2006be30 0x7f3b2006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:22.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:22.580+0000 7f3b3de00640 1 --2- 172.21.15.1:0/2014951519 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f3b2006be30 0x7f3b2006e2f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f3b30033000 tx=0x7f3b30009000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:22.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:22.584+0000 7f3b36000640 1 -- 172.21.15.1:0/2014951519 <== mon.1 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f3b401e3210 con 0x7f3b400cc730 2024-05-09T08:02:22.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:22.827+0000 7f3b3f200640 1 -- 172.21.15.1:0/2014951519 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3b40158e40 con 0x7f3b400cc730 2024-05-09T08:02:22.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:22.828+0000 7f3b36000640 1 -- 172.21.15.1:0/2014951519 <== mon.1 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+6799 (secure 0 0 0) 0x7f3b2c059140 con 0x7f3b400cc730 2024-05-09T08:02:22.845 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T08:02:22.845 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":5,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4220,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/1157980939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":1157980939},{"type":"v1","addr":"172.21.15.7:6837","nonce":1157980939}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4235,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6833/3269798795","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6832","nonce":3269798795},{"type":"v1","addr":"172.21.15.1:6833","nonce":3269798795}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4241,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/725115499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6835","nonce":725115499},{"type":"v1","addr":"172.21.15.1:6837","nonce":725115499}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4276,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6838/1827115274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1827115274},{"type":"v1","addr":"172.21.15.7:6838","nonce":1827115274}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/1032767352","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":1032767352},{"type":"v1","addr":"172.21.15.7:6839","nonce":1032767352}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":5,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-09T08:02:15.332696+0000","modified":"2024-05-09T08:02:20.403902+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4294},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4294":{"gid":4294,"name":"c","rank":0,"incarnation":4,"state":"up:creating","state_seq":1,"addr":"172.21.15.1:6836/1509134635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":1509134635},{"type":"v1","addr":"172.21.15.1:6836","nonce":1509134635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-05-09T08:02:22.845 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 5 2024-05-09T08:02:22.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:22.830+0000 7f3b3f200640 1 -- 172.21.15.1:0/2014951519 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f3b2006be30 msgr2=0x7f3b2006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:22.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:22.830+0000 7f3b3f200640 1 --2- 172.21.15.1:0/2014951519 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f3b2006be30 0x7f3b2006e2f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f3b30033000 tx=0x7f3b30009000 comp rx=0 tx=0).stop 2024-05-09T08:02:22.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:22.830+0000 7f3b3f200640 1 -- 172.21.15.1:0/2014951519 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b400cc730 msgr2=0x7f3b401f3c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:22.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:22.830+0000 7f3b3f200640 1 --2- 172.21.15.1:0/2014951519 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b400cc730 0x7f3b401f3c20 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f3b2c00e9e0 tx=0x7f3b2c00eea0 comp rx=0 tx=0).stop 2024-05-09T08:02:22.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:22.831+0000 7f3b3f200640 1 -- 172.21.15.1:0/2014951519 shutdown_connections 2024-05-09T08:02:22.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:22.831+0000 7f3b3f200640 1 --2- 172.21.15.1:0/2014951519 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f3b2006be30 0x7f3b2006e2f0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:22.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:22.831+0000 7f3b3f200640 1 --2- 172.21.15.1:0/2014951519 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b400cc730 0x7f3b401f3c20 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:22.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:22.831+0000 7f3b3f200640 1 --2- 172.21.15.1:0/2014951519 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f3b40152500 0x7f3b40159a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:22.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:22.831+0000 7f3b3f200640 1 --2- 172.21.15.1:0/2014951519 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3b400c7660 0x7f3b401e48e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:22.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:22.831+0000 7f3b3f200640 1 -- 172.21.15.1:0/2014951519 >> 172.21.15.1:0/2014951519 conn(0x7f3b400cbc10 msgr2=0x7f3b4015aea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:02:22.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:22.831+0000 7f3b3f200640 1 -- 172.21.15.1:0/2014951519 shutdown_connections 2024-05-09T08:02:22.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:22.831+0000 7f3b3f200640 1 -- 172.21.15.1:0/2014951519 wait complete. 2024-05-09T08:02:22.864 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T08:02:23.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:23.055+0000 7fecd8200640 1 Processor -- start 2024-05-09T08:02:23.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:23.056+0000 7fecd8200640 1 -- start start 2024-05-09T08:02:23.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:23.057+0000 7fecd8200640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fecd015b290 0x7fecd00cc400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:23.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:23.057+0000 7fecd8200640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fecd015e6f0 con 0x7fecd00cbf20 2024-05-09T08:02:23.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:23.057+0000 7fecd8200640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fecd015e980 con 0x7fecd01548e0 2024-05-09T08:02:23.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:23.057+0000 7fecd8200640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fecd015ec60 con 0x7fecd015b290 2024-05-09T08:02:23.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:23.058+0000 7fecd4e00640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7fecd00cbf20 0x7fecd015ab80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:40280/0 (socket says 172.21.15.1:40280) 2024-05-09T08:02:23.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:23.058+0000 7fecd4e00640 1 -- 172.21.15.1:0/361260963 learned_addr learned my addr 172.21.15.1:0/361260963 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T08:02:23.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:23.058+0000 7fecd6200640 1 --2- 172.21.15.1:0/361260963 >> v2:172.21.15.7:3300/0 conn(0x7fecd015b290 0x7fecd00cc400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:23.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:23.058+0000 7fecd6200640 1 -- 172.21.15.1:0/361260963 >> v1:172.21.15.7:6789/0 conn(0x7fecd01548e0 legacy=0x7fecd008bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:23.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:23.058+0000 7fecd6200640 1 -- 172.21.15.1:0/361260963 >> v1:172.21.15.1:6789/0 conn(0x7fecd00cbf20 legacy=0x7fecd015ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:23.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:23.058+0000 7fecd6200640 1 -- 172.21.15.1:0/361260963 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecd00caa10 con 0x7fecd015b290 2024-05-09T08:02:23.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:23.059+0000 7fecd6200640 1 --2- 172.21.15.1:0/361260963 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fecd015b290 0x7fecd00cc400 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fecc8009d90 tx=0x7fecc802a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=38127397a67ceec1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:23.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:23.060+0000 7feccfe00640 1 -- 172.21.15.1:0/361260963 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecc8007740 con 0x7fecd015b290 2024-05-09T08:02:23.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:23.060+0000 7feccfe00640 1 -- 172.21.15.1:0/361260963 <== mon.1 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fecc80078e0 con 0x7fecd015b290 2024-05-09T08:02:23.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:23.060+0000 7feccfe00640 1 -- 172.21.15.1:0/361260963 <== mon.1 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecc8007cb0 con 0x7fecd015b290 2024-05-09T08:02:23.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:23.060+0000 7fecd8200640 1 -- 172.21.15.1:0/361260963 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fecd015b290 msgr2=0x7fecd00cc400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:23.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:23.060+0000 7fecd8200640 1 --2- 172.21.15.1:0/361260963 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fecd015b290 0x7fecd00cc400 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fecc8009d90 tx=0x7fecc802a660 comp rx=0 tx=0).stop 2024-05-09T08:02:23.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:23.060+0000 7fecd8200640 1 -- 172.21.15.1:0/361260963 shutdown_connections 2024-05-09T08:02:23.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:23.060+0000 7fecd8200640 1 --2- 172.21.15.1:0/361260963 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fecd015b290 0x7fecd00cc400 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:23.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:23.060+0000 7fecd8200640 1 -- 172.21.15.1:0/361260963 >> 172.21.15.1:0/361260963 conn(0x7fecd0073010 msgr2=0x7fecd006c880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:02:23.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:23.060+0000 7fecd8200640 1 -- 172.21.15.1:0/361260963 shutdown_connections 2024-05-09T08:02:23.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:23.060+0000 7fecd8200640 1 -- 172.21.15.1:0/361260963 wait complete. 2024-05-09T08:02:23.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:23.062+0000 7fecd8200640 1 Processor -- start 2024-05-09T08:02:23.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:23.062+0000 7fecd8200640 1 -- start start 2024-05-09T08:02:23.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:23.063+0000 7fecd8200640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fecd00cbf20 0x7fecd014a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:23.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:23.063+0000 7fecd8200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fecd01548e0 0x7fecd01589f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:23.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:23.063+0000 7fecd8200640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fecd015b290 0x7fecd00ced90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:23.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:23.063+0000 7fecd8200640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fecd0154e40 con 0x7fecd00cbf20 2024-05-09T08:02:23.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:23.063+0000 7fecd8200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fecd0154cc0 con 0x7fecd01548e0 2024-05-09T08:02:23.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:23.063+0000 7fecd8200640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7fecd008c700 con 0x7fecd015b290 2024-05-09T08:02:23.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:23.064+0000 7fecd5800640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fecd00cbf20 0x7fecd014a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:23.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:23.064+0000 7fecd4e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fecd01548e0 0x7fecd01589f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:23.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:23.064+0000 7fecd6200640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fecd015b290 0x7fecd00ced90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:23.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:23.064+0000 7fecd5800640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fecd00cbf20 0x7fecd014a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43456/0 (socket says 172.21.15.1:43456) 2024-05-09T08:02:23.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:23.064+0000 7fecd4e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fecd01548e0 0x7fecd01589f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.1:41570/0 (socket says 172.21.15.1:41570) 2024-05-09T08:02:23.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:23.064+0000 7fecd5800640 1 -- 172.21.15.1:0/1167490390 learned_addr learned my addr 172.21.15.1:0/1167490390 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:02:23.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:23.064+0000 7fecd6200640 1 -- 172.21.15.1:0/1167490390 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fecd01548e0 msgr2=0x7fecd01589f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:23.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:23.064+0000 7fecd6200640 1 --2- 172.21.15.1:0/1167490390 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fecd01548e0 0x7fecd01589f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:23.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:23.064+0000 7fecd6200640 1 -- 172.21.15.1:0/1167490390 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fecd00cbf20 msgr2=0x7fecd014a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:23.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:23.064+0000 7fecd6200640 1 --2- 172.21.15.1:0/1167490390 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fecd00cbf20 0x7fecd014a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:23.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:23.064+0000 7fecd6200640 1 -- 172.21.15.1:0/1167490390 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecd014ab00 con 0x7fecd015b290 2024-05-09T08:02:23.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:23.064+0000 7fecd5800640 1 --2- 172.21.15.1:0/1167490390 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fecd00cbf20 0x7fecd014a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:02:23.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:23.065+0000 7fecd6200640 1 --2- 172.21.15.1:0/1167490390 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fecd015b290 0x7fecd00ced90 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fecd008cdd0 tx=0x7fecc8004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:23.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:23.065+0000 7feccd600640 1 -- 172.21.15.1:0/1167490390 <== mon.2 v2:172.21.15.7:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecc803f070 con 0x7fecd015b290 2024-05-09T08:02:23.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:23.065+0000 7feccd600640 1 -- 172.21.15.1:0/1167490390 <== mon.2 v2:172.21.15.7:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fecc8031c60 con 0x7fecd015b290 2024-05-09T08:02:23.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:23.065+0000 7feccd600640 1 -- 172.21.15.1:0/1167490390 <== mon.2 v2:172.21.15.7:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecc80446a0 con 0x7fecd015b290 2024-05-09T08:02:23.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:23.065+0000 7fecd8200640 1 -- 172.21.15.1:0/1167490390 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fecd0157f40 con 0x7fecd015b290 2024-05-09T08:02:23.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:23.065+0000 7fecd8200640 1 -- 172.21.15.1:0/1167490390 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fecd015a3c0 con 0x7fecd015b290 2024-05-09T08:02:23.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:23.066+0000 7fecd8200640 1 -- 172.21.15.1:0/1167490390 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fecd0158350 con 0x7fecd015b290 2024-05-09T08:02:23.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:23.067+0000 7feccd600640 1 -- 172.21.15.1:0/1167490390 <== mon.2 v2:172.21.15.7:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fecc803b410 con 0x7fecd015b290 2024-05-09T08:02:23.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:23.068+0000 7feccd600640 1 --2- 172.21.15.1:0/1167490390 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fecb006be30 0x7fecb006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:23.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:23.068+0000 7feccd600640 1 -- 172.21.15.1:0/1167490390 <== mon.2 v2:172.21.15.7:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fecc80ac8f0 con 0x7fecd015b290 2024-05-09T08:02:23.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:23.068+0000 7fecd5800640 1 --2- 172.21.15.1:0/1167490390 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fecb006be30 0x7fecb006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:23.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:23.069+0000 7fecd5800640 1 --2- 172.21.15.1:0/1167490390 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fecb006be30 0x7fecb006e2f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fecc4007fd0 tx=0x7fecc4009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:23.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:23.073+0000 7feccd600640 1 -- 172.21.15.1:0/1167490390 <== mon.2 v2:172.21.15.7:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fecc807aa20 con 0x7fecd015b290 2024-05-09T08:02:23.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:23.276+0000 7fecd8200640 1 -- 172.21.15.1:0/1167490390 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fecd0074d70 con 0x7fecd015b290 2024-05-09T08:02:23.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:23.278+0000 7feccd600640 1 -- 172.21.15.1:0/1167490390 <== mon.2 v2:172.21.15.7:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18382 (secure 0 0 0) 0x7fecc807ed20 con 0x7fecd015b290 2024-05-09T08:02:23.295 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T08:02:23.295 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":25,"fsid":"54329ec8-3270-4043-9a82-c6cd7420aa7f","created":"2024-05-09T08:01:56.666970+0000","modified":"2024-05-09T08:02:20.403456+0000","last_up_change":"2024-05-09T08:02:07.146082+0000","last_in_change":"2024-05-09T08:01:59.008225+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T08:02:09.165511+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T08:02:12.925485+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T08:02:14.367455+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-09T08:02:17.967064+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c064454e-a9b5-4a24-9a77-17f0a4085952","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6800","nonce":1513349547},{"type":"v1","addr":"172.21.15.1:6801","nonce":1513349547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":1513349547},{"type":"v1","addr":"172.21.15.1:6803","nonce":1513349547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":1513349547},{"type":"v1","addr":"172.21.15.1:6807","nonce":1513349547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":1513349547},{"type":"v1","addr":"172.21.15.1:6805","nonce":1513349547}]},"public_addr":"172.21.15.1:6801/1513349547","cluster_addr":"172.21.15.1:6803/1513349547","heartbeat_back_addr":"172.21.15.1:6807/1513349547","heartbeat_front_addr":"172.21.15.1:6805/1513349547","state":["exists","up"]},{"osd":1,"uuid":"1a9ea687-2ff6-407a-949b-0e7e1083af21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":2942255259},{"type":"v1","addr":"172.21.15.1:6817","nonce":2942255259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":2942255259},{"type":"v1","addr":"172.21.15.1:6819","nonce":2942255259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":2942255259},{"type":"v1","addr":"172.21.15.1:6823","nonce":2942255259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6820","nonce":2942255259},{"type":"v1","addr":"172.21.15.1:6821","nonce":2942255259}]},"public_addr":"172.21.15.1:6817/2942255259","cluster_addr":"172.21.15.1:6819/2942255259","heartbeat_back_addr":"172.21.15.1:6823/2942255259","heartbeat_front_addr":"172.21.15.1:6821/2942255259","state":["exists","up"]},{"osd":2,"uuid":"c88af08b-7c9a-4dd8-9640-44138239addb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":2925031811},{"type":"v1","addr":"172.21.15.1:6809","nonce":2925031811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":2925031811},{"type":"v1","addr":"172.21.15.1:6811","nonce":2925031811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":2925031811},{"type":"v1","addr":"172.21.15.1:6815","nonce":2925031811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":2925031811},{"type":"v1","addr":"172.21.15.1:6813","nonce":2925031811}]},"public_addr":"172.21.15.1:6809/2925031811","cluster_addr":"172.21.15.1:6811/2925031811","heartbeat_back_addr":"172.21.15.1:6815/2925031811","heartbeat_front_addr":"172.21.15.1:6813/2925031811","state":["exists","up"]},{"osd":3,"uuid":"c087b7e3-a41d-4333-9365-32e3bd8c8e6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6824","nonce":2497776654},{"type":"v1","addr":"172.21.15.1:6825","nonce":2497776654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":2497776654},{"type":"v1","addr":"172.21.15.1:6827","nonce":2497776654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":2497776654},{"type":"v1","addr":"172.21.15.1:6831","nonce":2497776654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6828","nonce":2497776654},{"type":"v1","addr":"172.21.15.1:6829","nonce":2497776654}]},"public_addr":"172.21.15.1:6825/2497776654","cluster_addr":"172.21.15.1:6827/2497776654","heartbeat_back_addr":"172.21.15.1:6831/2497776654","heartbeat_front_addr":"172.21.15.1:6829/2497776654","state":["exists","up"]},{"osd":4,"uuid":"b4576b9c-871d-47a8-8d36-65af525d0200","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":2957026670},{"type":"v1","addr":"172.21.15.7:6801","nonce":2957026670}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2957026670},{"type":"v1","addr":"172.21.15.7:6803","nonce":2957026670}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":2957026670},{"type":"v1","addr":"172.21.15.7:6807","nonce":2957026670}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2957026670},{"type":"v1","addr":"172.21.15.7:6805","nonce":2957026670}]},"public_addr":"172.21.15.7:6801/2957026670","cluster_addr":"172.21.15.7:6803/2957026670","heartbeat_back_addr":"172.21.15.7:6807/2957026670","heartbeat_front_addr":"172.21.15.7:6805/2957026670","state":["exists","up"]},{"osd":5,"uuid":"902b3d94-72f3-4499-b21a-e32453ef5144","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1670295242},{"type":"v1","addr":"172.21.15.7:6809","nonce":1670295242}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1670295242},{"type":"v1","addr":"172.21.15.7:6811","nonce":1670295242}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1670295242},{"type":"v1","addr":"172.21.15.7:6815","nonce":1670295242}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1670295242},{"type":"v1","addr":"172.21.15.7:6813","nonce":1670295242}]},"public_addr":"172.21.15.7:6809/1670295242","cluster_addr":"172.21.15.7:6811/1670295242","heartbeat_back_addr":"172.21.15.7:6815/1670295242","heartbeat_front_addr":"172.21.15.7:6813/1670295242","state":["exists","up"]},{"osd":6,"uuid":"ead1d771-1728-4f85-adb4-26be21cedff3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":2852086880},{"type":"v1","addr":"172.21.15.7:6817","nonce":2852086880}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":2852086880},{"type":"v1","addr":"172.21.15.7:6819","nonce":2852086880}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":2852086880},{"type":"v1","addr":"172.21.15.7:6823","nonce":2852086880}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":2852086880},{"type":"v1","addr":"172.21.15.7:6821","nonce":2852086880}]},"public_addr":"172.21.15.7:6817/2852086880","cluster_addr":"172.21.15.7:6819/2852086880","heartbeat_back_addr":"172.21.15.7:6823/2852086880","heartbeat_front_addr":"172.21.15.7:6821/2852086880","state":["exists","up"]},{"osd":7,"uuid":"3ab5a201-9394-48a8-8918-e95197893669","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":1442884138},{"type":"v1","addr":"172.21.15.7:6825","nonce":1442884138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1442884138},{"type":"v1","addr":"172.21.15.7:6827","nonce":1442884138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1442884138},{"type":"v1","addr":"172.21.15.7:6831","nonce":1442884138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1442884138},{"type":"v1","addr":"172.21.15.7:6829","nonce":1442884138}]},"public_addr":"172.21.15.7:6825/1442884138","cluster_addr":"172.21.15.7:6827/1442884138","heartbeat_back_addr":"172.21.15.7:6831/1442884138","heartbeat_front_addr":"172.21.15.7:6829/1442884138","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:02:04.615675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:02:04.385974+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:02:04.619217+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:02:04.401681+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:02:04.097081+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:02:04.129981+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:02:04.225327+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:02:04.193351+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T08:02:23.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:23.280+0000 7fecd8200640 1 -- 172.21.15.1:0/1167490390 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fecb006be30 msgr2=0x7fecb006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:23.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:23.280+0000 7fecd8200640 1 --2- 172.21.15.1:0/1167490390 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fecb006be30 0x7fecb006e2f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fecc4007fd0 tx=0x7fecc4009040 comp rx=0 tx=0).stop 2024-05-09T08:02:23.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:23.280+0000 7fecd8200640 1 -- 172.21.15.1:0/1167490390 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fecd015b290 msgr2=0x7fecd00ced90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:23.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:23.280+0000 7fecd8200640 1 --2- 172.21.15.1:0/1167490390 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fecd015b290 0x7fecd00ced90 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fecd008cdd0 tx=0x7fecc8004a40 comp rx=0 tx=0).stop 2024-05-09T08:02:23.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:23.281+0000 7fecd8200640 1 -- 172.21.15.1:0/1167490390 shutdown_connections 2024-05-09T08:02:23.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:23.281+0000 7fecd8200640 1 --2- 172.21.15.1:0/1167490390 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fecb006be30 0x7fecb006e2f0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:23.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:23.281+0000 7fecd8200640 1 --2- 172.21.15.1:0/1167490390 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fecd01548e0 0x7fecd01589f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:23.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:23.281+0000 7fecd8200640 1 --2- 172.21.15.1:0/1167490390 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fecd015b290 0x7fecd00ced90 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:23.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:23.281+0000 7fecd8200640 1 --2- 172.21.15.1:0/1167490390 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fecd00cbf20 0x7fecd014a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:23.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:23.281+0000 7fecd8200640 1 -- 172.21.15.1:0/1167490390 >> 172.21.15.1:0/1167490390 conn(0x7fecd0073010 msgr2=0x7fecd00744e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:02:23.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:23.281+0000 7fecd8200640 1 -- 172.21.15.1:0/1167490390 shutdown_connections 2024-05-09T08:02:23.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:23.281+0000 7fecd8200640 1 -- 172.21.15.1:0/1167490390 wait complete. 2024-05-09T08:02:23.921 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 5, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-09T08:02:15.332696+0000', 'modified': '2024-05-09T08:02:20.403902+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4294}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4294': {'gid': 4294, 'name': 'c', 'rank': 0, 'incarnation': 4, 'state': 'up:creating', 'state_seq': 1, 'addr': '172.21.15.1:6836/1509134635', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6834', 'nonce': 1509134635}, {'type': 'v1', 'addr': '172.21.15.1:6836', 'nonce': 1509134635}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-05-09T08:02:23.922 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_4294:up:creating 2024-05-09T08:02:24.923 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T08:02:25.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:25.128+0000 7f6269000640 1 Processor -- start 2024-05-09T08:02:25.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:25.128+0000 7f6269000640 1 -- start start 2024-05-09T08:02:25.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:25.129+0000 7f6269000640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f62640ca5c0 0x7f62640ca9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:25.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:25.129+0000 7f6269000640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f62640ccf70 0x7f626415a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:25.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:25.129+0000 7f6269000640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f626415aaf0 con 0x7f62640cf370 2024-05-09T08:02:25.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:25.129+0000 7f6269000640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f626406c7d0 con 0x7f62640ca5c0 2024-05-09T08:02:25.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:25.129+0000 7f6269000640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f626415ad80 con 0x7f62640ccf70 2024-05-09T08:02:25.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:25.130+0000 7f6263400640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f62640ca5c0 0x7f62640ca9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:25.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:25.130+0000 7f6263400640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f62640ca5c0 0x7f62640ca9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:36712/0 (socket says 172.21.15.1:36712) 2024-05-09T08:02:25.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:25.130+0000 7f6263e00640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f62640cf370 0x7f626406d220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:52560/0 (socket says 172.21.15.1:52560) 2024-05-09T08:02:25.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:25.130+0000 7f6263400640 1 -- 172.21.15.1:0/58252690 learned_addr learned my addr 172.21.15.1:0/58252690 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:02:25.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:25.130+0000 7f6262a00640 1 --2- 172.21.15.1:0/58252690 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f62640ccf70 0x7f626415a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:25.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:25.130+0000 7f6263400640 1 -- 172.21.15.1:0/58252690 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f62640ccf70 msgr2=0x7f626415a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:02:25.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:25.130+0000 7f6263400640 1 --2- 172.21.15.1:0/58252690 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f62640ccf70 0x7f626415a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:25.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:25.130+0000 7f6263400640 1 -- 172.21.15.1:0/58252690 >> v1:172.21.15.1:6789/0 conn(0x7f62640cf370 legacy=0x7f626406d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:25.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:25.130+0000 7f6263400640 1 -- 172.21.15.1:0/58252690 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f626415b060 con 0x7f62640ca5c0 2024-05-09T08:02:25.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:25.131+0000 7f6263400640 1 --2- 172.21.15.1:0/58252690 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f62640ca5c0 0x7f62640ca9a0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f6254009870 tx=0x7f625402a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b600907b661bbe14 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:25.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:25.132+0000 7f6262000640 1 -- 172.21.15.1:0/58252690 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6254037070 con 0x7f62640ca5c0 2024-05-09T08:02:25.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:25.132+0000 7f6262000640 1 -- 172.21.15.1:0/58252690 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f625402ae50 con 0x7f62640ca5c0 2024-05-09T08:02:25.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:25.132+0000 7f6262000640 1 -- 172.21.15.1:0/58252690 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6254032860 con 0x7f62640ca5c0 2024-05-09T08:02:25.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:25.132+0000 7f6269000640 1 -- 172.21.15.1:0/58252690 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f62640ca5c0 msgr2=0x7f62640ca9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:25.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:25.132+0000 7f6269000640 1 --2- 172.21.15.1:0/58252690 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f62640ca5c0 0x7f62640ca9a0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f6254009870 tx=0x7f625402a300 comp rx=0 tx=0).stop 2024-05-09T08:02:25.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:25.133+0000 7f6269000640 1 -- 172.21.15.1:0/58252690 shutdown_connections 2024-05-09T08:02:25.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:25.133+0000 7f6269000640 1 --2- 172.21.15.1:0/58252690 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f62640ccf70 0x7f626415a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:25.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:25.133+0000 7f6269000640 1 --2- 172.21.15.1:0/58252690 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f62640ca5c0 0x7f62640ca9a0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:25.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:25.133+0000 7f6269000640 1 -- 172.21.15.1:0/58252690 >> 172.21.15.1:0/58252690 conn(0x7f62640cfc40 msgr2=0x7f62640d0050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:02:25.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:25.133+0000 7f6269000640 1 -- 172.21.15.1:0/58252690 shutdown_connections 2024-05-09T08:02:25.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:25.133+0000 7f6269000640 1 -- 172.21.15.1:0/58252690 wait complete. 2024-05-09T08:02:25.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:25.135+0000 7f6269000640 1 Processor -- start 2024-05-09T08:02:25.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:25.135+0000 7f6269000640 1 -- start start 2024-05-09T08:02:25.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:25.136+0000 7f6269000640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f62640ca5c0 0x7f62641f4270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:25.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:25.136+0000 7f6269000640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f62640ccf70 0x7f62641572b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:25.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:25.136+0000 7f6269000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62640cf370 0x7f6264159420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:25.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:25.136+0000 7f6269000640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f62640cf800 con 0x7f62640ca5c0 2024-05-09T08:02:25.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:25.136+0000 7f6269000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f626406ca20 con 0x7f62640cf370 2024-05-09T08:02:25.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:25.136+0000 7f6269000640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f62640cf980 con 0x7f62640ccf70 2024-05-09T08:02:25.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:25.136+0000 7f6263400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f62640ca5c0 0x7f62641f4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:25.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:25.136+0000 7f6262a00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f62640ccf70 0x7f62641572b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:25.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:25.136+0000 7f6263e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62640cf370 0x7f6264159420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:25.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:25.136+0000 7f6262a00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f62640ccf70 0x7f62641572b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3301/0 says I am v2:172.21.15.1:43568/0 (socket says 172.21.15.1:43568) 2024-05-09T08:02:25.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:25.136+0000 7f6263400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f62640ca5c0 0x7f62641f4270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:36722/0 (socket says 172.21.15.1:36722) 2024-05-09T08:02:25.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:25.136+0000 7f6262a00640 1 -- 172.21.15.1:0/4281560204 learned_addr learned my addr 172.21.15.1:0/4281560204 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:02:25.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:25.136+0000 7f6262a00640 1 -- 172.21.15.1:0/4281560204 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62640cf370 msgr2=0x7f6264159420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:25.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:25.136+0000 7f6262a00640 1 --2- 172.21.15.1:0/4281560204 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62640cf370 0x7f6264159420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:25.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:25.136+0000 7f6262a00640 1 -- 172.21.15.1:0/4281560204 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f62640ca5c0 msgr2=0x7f62641f4270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:25.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:25.136+0000 7f6262a00640 1 --2- 172.21.15.1:0/4281560204 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f62640ca5c0 0x7f62641f4270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:25.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:25.136+0000 7f6262a00640 1 -- 172.21.15.1:0/4281560204 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62641f47b0 con 0x7f62640ccf70 2024-05-09T08:02:25.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:25.137+0000 7f6262a00640 1 --2- 172.21.15.1:0/4281560204 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f62640ccf70 0x7f62641572b0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f624c00e9e0 tx=0x7f624c00eea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:25.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:25.137+0000 7f625b400640 1 -- 172.21.15.1:0/4281560204 <== mon.2 v2:172.21.15.7:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f624c017070 con 0x7f62640ccf70 2024-05-09T08:02:25.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:25.137+0000 7f625b400640 1 -- 172.21.15.1:0/4281560204 <== mon.2 v2:172.21.15.7:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f624c00c9d0 con 0x7f62640ccf70 2024-05-09T08:02:25.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:25.137+0000 7f625b400640 1 -- 172.21.15.1:0/4281560204 <== mon.2 v2:172.21.15.7:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f624c00cdc0 con 0x7f62640ccf70 2024-05-09T08:02:25.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:25.137+0000 7f6269000640 1 -- 172.21.15.1:0/4281560204 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62640b3650 con 0x7f62640ccf70 2024-05-09T08:02:25.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:25.137+0000 7f6269000640 1 -- 172.21.15.1:0/4281560204 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f626414e3a0 con 0x7f62640ccf70 2024-05-09T08:02:25.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:25.138+0000 7f6269000640 1 -- 172.21.15.1:0/4281560204 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f626404f660 con 0x7f62640ccf70 2024-05-09T08:02:25.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:25.139+0000 7f625b400640 1 -- 172.21.15.1:0/4281560204 <== mon.2 v2:172.21.15.7:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f624c01a070 con 0x7f62640ccf70 2024-05-09T08:02:25.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:25.140+0000 7f625b400640 1 --2- 172.21.15.1:0/4281560204 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f624006be30 0x7f624006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:25.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:25.140+0000 7f625b400640 1 -- 172.21.15.1:0/4281560204 <== mon.2 v2:172.21.15.7:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f624c08b1c0 con 0x7f62640ccf70 2024-05-09T08:02:25.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:25.140+0000 7f6263400640 1 --2- 172.21.15.1:0/4281560204 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f624006be30 0x7f624006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:25.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:25.141+0000 7f6263400640 1 --2- 172.21.15.1:0/4281560204 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f624006be30 0x7f624006e2f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6254034000 tx=0x7f625402e000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:25.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:25.145+0000 7f625b400640 1 -- 172.21.15.1:0/4281560204 <== mon.2 v2:172.21.15.7:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f624c059140 con 0x7f62640ccf70 2024-05-09T08:02:25.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:25.395+0000 7f6269000640 1 -- 172.21.15.1:0/4281560204 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f626414c2d0 con 0x7f62640ccf70 2024-05-09T08:02:25.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:25.396+0000 7f625b400640 1 -- 172.21.15.1:0/4281560204 <== mon.2 v2:172.21.15.7:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+6804 (secure 0 0 0) 0x7f624c05d440 con 0x7f62640ccf70 2024-05-09T08:02:25.413 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T08:02:25.413 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4220,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/1157980939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":1157980939},{"type":"v1","addr":"172.21.15.7:6837","nonce":1157980939}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4235,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6833/3269798795","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6832","nonce":3269798795},{"type":"v1","addr":"172.21.15.1:6833","nonce":3269798795}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4241,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/725115499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6835","nonce":725115499},{"type":"v1","addr":"172.21.15.1:6837","nonce":725115499}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4276,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6838/1827115274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1827115274},{"type":"v1","addr":"172.21.15.7:6838","nonce":1827115274}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/1032767352","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":1032767352},{"type":"v1","addr":"172.21.15.7:6839","nonce":1032767352}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-09T08:02:15.332696+0000","modified":"2024-05-09T08:02:24.941332+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4294},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4294":{"gid":4294,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":6,"addr":"172.21.15.1:6836/1509134635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":1509134635},{"type":"v1","addr":"172.21.15.1:6836","nonce":1509134635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":4294,"qdb_cluster":[4294]},"id":1}]} 2024-05-09T08:02:25.414 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 7 2024-05-09T08:02:25.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:25.399+0000 7f6269000640 1 -- 172.21.15.1:0/4281560204 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f624006be30 msgr2=0x7f624006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:25.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:25.399+0000 7f6269000640 1 --2- 172.21.15.1:0/4281560204 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f624006be30 0x7f624006e2f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6254034000 tx=0x7f625402e000 comp rx=0 tx=0).stop 2024-05-09T08:02:25.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:25.399+0000 7f6269000640 1 -- 172.21.15.1:0/4281560204 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f62640ccf70 msgr2=0x7f62641572b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:25.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:25.399+0000 7f6269000640 1 --2- 172.21.15.1:0/4281560204 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f62640ccf70 0x7f62641572b0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f624c00e9e0 tx=0x7f624c00eea0 comp rx=0 tx=0).stop 2024-05-09T08:02:25.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:25.399+0000 7f6269000640 1 -- 172.21.15.1:0/4281560204 shutdown_connections 2024-05-09T08:02:25.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:25.399+0000 7f6269000640 1 --2- 172.21.15.1:0/4281560204 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f624006be30 0x7f624006e2f0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:25.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:25.399+0000 7f6269000640 1 --2- 172.21.15.1:0/4281560204 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62640cf370 0x7f6264159420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:25.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:25.399+0000 7f6269000640 1 --2- 172.21.15.1:0/4281560204 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f62640ccf70 0x7f62641572b0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:25.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:25.399+0000 7f6269000640 1 --2- 172.21.15.1:0/4281560204 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f62640ca5c0 0x7f62641f4270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:25.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:25.399+0000 7f6269000640 1 -- 172.21.15.1:0/4281560204 >> 172.21.15.1:0/4281560204 conn(0x7f62640cfc40 msgr2=0x7f62640d0020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:02:25.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:25.399+0000 7f6269000640 1 -- 172.21.15.1:0/4281560204 shutdown_connections 2024-05-09T08:02:25.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:25.399+0000 7f6269000640 1 -- 172.21.15.1:0/4281560204 wait complete. 2024-05-09T08:02:26.041 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 6, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-09T08:02:15.332696+0000', 'modified': '2024-05-09T08:02:24.941332+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4294}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4294': {'gid': 4294, 'name': 'c', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.1:6836/1509134635', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6834', 'nonce': 1509134635}, {'type': 'v1', 'addr': '172.21.15.1:6836', 'nonce': 1509134635}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 4294, 'qdb_cluster': [4294]} 2024-05-09T08:02:26.042 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-09T08:02:26.042 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-05-09T08:02:26.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:26.247+0000 7f2573200640 1 Processor -- start 2024-05-09T08:02:26.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:26.247+0000 7f2573200640 1 -- start start 2024-05-09T08:02:26.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:26.249+0000 7f2573200640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f256c0cc5f0 0x7f256c15a570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:26.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:26.249+0000 7f2573200640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f256c0ce4e0 0x7f256c15aab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:26.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:26.249+0000 7f2573200640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f256c0ccb20 con 0x7f256c0cac60 2024-05-09T08:02:26.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:26.249+0000 7f2573200640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f256c0cce00 con 0x7f256c0cc5f0 2024-05-09T08:02:26.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:26.249+0000 7f2573200640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f256c0cf120 con 0x7f256c0ce4e0 2024-05-09T08:02:26.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:26.249+0000 7f2571400640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f256c0cc5f0 0x7f256c15a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:26.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:26.249+0000 7f2571e00640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f256c0cac60 0x7f256c0cda00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:52562/0 (socket says 172.21.15.1:52562) 2024-05-09T08:02:26.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:26.249+0000 7f2571400640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f256c0cc5f0 0x7f256c15a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:36728/0 (socket says 172.21.15.1:36728) 2024-05-09T08:02:26.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:26.249+0000 7f2571e00640 1 -- 172.21.15.1:0/1413603061 learned_addr learned my addr 172.21.15.1:0/1413603061 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T08:02:26.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:26.249+0000 7f2572800640 1 --2- 172.21.15.1:0/1413603061 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f256c0ce4e0 0x7f256c15aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:26.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:26.250+0000 7f2571400640 1 -- 172.21.15.1:0/1413603061 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f256c0ce4e0 msgr2=0x7f256c15aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:02:26.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:26.250+0000 7f2571400640 1 --2- 172.21.15.1:0/1413603061 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f256c0ce4e0 0x7f256c15aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:26.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:26.250+0000 7f2571400640 1 -- 172.21.15.1:0/1413603061 >> v1:172.21.15.1:6789/0 conn(0x7f256c0cac60 legacy=0x7f256c0cda00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:26.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:26.250+0000 7f2571400640 1 -- 172.21.15.1:0/1413603061 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f256c0cf400 con 0x7f256c0cc5f0 2024-05-09T08:02:26.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:26.251+0000 7f2571400640 1 --2- 172.21.15.1:0/1413603061 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f256c0cc5f0 0x7f256c15a570 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f255c0097a0 tx=0x7f255c02a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7197ff099da69bc server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:26.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:26.251+0000 7f2570a00640 1 -- 172.21.15.1:0/1413603061 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f255c037070 con 0x7f256c0cc5f0 2024-05-09T08:02:26.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:26.251+0000 7f2570a00640 1 -- 172.21.15.1:0/1413603061 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f255c02ada0 con 0x7f256c0cc5f0 2024-05-09T08:02:26.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:26.251+0000 7f2570a00640 1 -- 172.21.15.1:0/1413603061 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f255c032660 con 0x7f256c0cc5f0 2024-05-09T08:02:26.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:26.252+0000 7f2573200640 1 -- 172.21.15.1:0/1413603061 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f256c0cc5f0 msgr2=0x7f256c15a570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:26.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:26.252+0000 7f2573200640 1 --2- 172.21.15.1:0/1413603061 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f256c0cc5f0 0x7f256c15a570 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f255c0097a0 tx=0x7f255c02a110 comp rx=0 tx=0).stop 2024-05-09T08:02:26.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:26.252+0000 7f2573200640 1 -- 172.21.15.1:0/1413603061 shutdown_connections 2024-05-09T08:02:26.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:26.252+0000 7f2573200640 1 --2- 172.21.15.1:0/1413603061 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f256c0ce4e0 0x7f256c15aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:26.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:26.252+0000 7f2573200640 1 --2- 172.21.15.1:0/1413603061 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f256c0cc5f0 0x7f256c15a570 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:26.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:26.252+0000 7f2573200640 1 -- 172.21.15.1:0/1413603061 >> 172.21.15.1:0/1413603061 conn(0x7f256c0cfc20 msgr2=0x7f256c0d0030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:02:26.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:26.252+0000 7f2573200640 1 -- 172.21.15.1:0/1413603061 shutdown_connections 2024-05-09T08:02:26.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:26.252+0000 7f2573200640 1 -- 172.21.15.1:0/1413603061 wait complete. 2024-05-09T08:02:26.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:26.254+0000 7f2573200640 1 Processor -- start 2024-05-09T08:02:26.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:26.254+0000 7f2573200640 1 -- start start 2024-05-09T08:02:26.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:26.255+0000 7f2573200640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f256c0cac60 0x7f256c1e6030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:26.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:26.255+0000 7f2573200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f256c0cc5f0 0x7f256c1f0260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:26.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:26.255+0000 7f2573200640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f256c0ce4e0 0x7f256c1f07a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:26.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:26.255+0000 7f2573200640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f256c0ce110 con 0x7f256c0cac60 2024-05-09T08:02:26.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:26.255+0000 7f2573200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f256c0cd550 con 0x7f256c0cc5f0 2024-05-09T08:02:26.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:26.255+0000 7f2573200640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f256c0ce290 con 0x7f256c0ce4e0 2024-05-09T08:02:26.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:26.255+0000 7f2571e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f256c0cac60 0x7f256c1e6030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:26.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:26.255+0000 7f2571400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f256c0cc5f0 0x7f256c1f0260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:26.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:26.255+0000 7f2571e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f256c0cac60 0x7f256c1e6030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:36736/0 (socket says 172.21.15.1:36736) 2024-05-09T08:02:26.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:26.255+0000 7f2571e00640 1 -- 172.21.15.1:0/2305437419 learned_addr learned my addr 172.21.15.1:0/2305437419 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:02:26.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:26.255+0000 7f2572800640 1 --2- 172.21.15.1:0/2305437419 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f256c0ce4e0 0x7f256c1f07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:26.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:26.256+0000 7f2571e00640 1 -- 172.21.15.1:0/2305437419 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f256c0ce4e0 msgr2=0x7f256c1f07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:26.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:26.256+0000 7f2571e00640 1 --2- 172.21.15.1:0/2305437419 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f256c0ce4e0 0x7f256c1f07a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:26.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:26.256+0000 7f2571e00640 1 -- 172.21.15.1:0/2305437419 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f256c0cc5f0 msgr2=0x7f256c1f0260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:26.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:26.256+0000 7f2571e00640 1 --2- 172.21.15.1:0/2305437419 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f256c0cc5f0 0x7f256c1f0260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:26.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:26.256+0000 7f2571e00640 1 -- 172.21.15.1:0/2305437419 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f256c1e5680 con 0x7f256c0cac60 2024-05-09T08:02:26.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:26.256+0000 7f2571e00640 1 --2- 172.21.15.1:0/2305437419 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f256c0cac60 0x7f256c1e6030 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f256000c7b0 tx=0x7f256000cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:26.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:26.256+0000 7f2566000640 1 -- 172.21.15.1:0/2305437419 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2560018070 con 0x7f256c0cac60 2024-05-09T08:02:26.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:26.256+0000 7f2566000640 1 -- 172.21.15.1:0/2305437419 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25600072f0 con 0x7f256c0cac60 2024-05-09T08:02:26.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:26.256+0000 7f2566000640 1 -- 172.21.15.1:0/2305437419 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25600047e0 con 0x7f256c0cac60 2024-05-09T08:02:26.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:26.256+0000 7f2573200640 1 -- 172.21.15.1:0/2305437419 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f256c1e3aa0 con 0x7f256c0cac60 2024-05-09T08:02:26.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:26.257+0000 7f2573200640 1 -- 172.21.15.1:0/2305437419 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f256c15d8f0 con 0x7f256c0cac60 2024-05-09T08:02:26.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:26.259+0000 7f2566000640 1 -- 172.21.15.1:0/2305437419 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2560004980 con 0x7f256c0cac60 2024-05-09T08:02:26.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:26.260+0000 7f2566000640 1 --2- 172.21.15.1:0/2305437419 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f255006be30 0x7f255006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:26.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:26.260+0000 7f2566000640 1 -- 172.21.15.1:0/2305437419 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f256008a590 con 0x7f256c0cac60 2024-05-09T08:02:26.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:26.261+0000 7f2571400640 1 --2- 172.21.15.1:0/2305437419 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f255006be30 0x7f255006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:26.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:26.261+0000 7f257759e740 1 Processor -- start 2024-05-09T08:02:26.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:26.261+0000 7f257759e740 1 -- start start 2024-05-09T08:02:26.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:26.261+0000 7f2571400640 1 --2- 172.21.15.1:0/2305437419 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f255006be30 0x7f255006e2f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f255c034000 tx=0x7f255c02e000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:26.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:26.262+0000 7f257759e740 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x563da0d1edd0 0x563da0d5dd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:26.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:26.262+0000 7f257759e740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563da0d5e2d0 0x563da0d1f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:26.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:26.262+0000 7f257759e740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563da0d20330 0x563da0d1f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:26.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:26.262+0000 7f257759e740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x563da0c81480 con 0x563da0d5e2d0 2024-05-09T08:02:26.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:26.262+0000 7f257759e740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x563da0c295c0 con 0x563da0d20330 2024-05-09T08:02:26.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:26.262+0000 7f257759e740 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x563da0c7d440 con 0x563da0d1edd0 2024-05-09T08:02:26.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:26.262+0000 7f2571400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563da0d5e2d0 0x563da0d1f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:26.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:26.262+0000 7f2571400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563da0d5e2d0 0x563da0d1f1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:36750/0 (socket says 172.21.15.1:36750) 2024-05-09T08:02:26.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:26.262+0000 7f2571400640 1 -- 172.21.15.1:0/3610674803 learned_addr learned my addr 172.21.15.1:0/3610674803 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:02:26.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:26.262+0000 7f2571e00640 1 --2- 172.21.15.1:0/3610674803 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x563da0d1edd0 0x563da0d5dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:26.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:26.262+0000 7f2572800640 1 --2- 172.21.15.1:0/3610674803 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563da0d20330 0x563da0d1f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:26.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:26.263+0000 7f2571400640 1 -- 172.21.15.1:0/3610674803 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x563da0d1edd0 msgr2=0x563da0d5dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:02:26.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:26.263+0000 7f2571400640 1 --2- 172.21.15.1:0/3610674803 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x563da0d1edd0 0x563da0d5dd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:26.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:26.263+0000 7f2571400640 1 -- 172.21.15.1:0/3610674803 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563da0d20330 msgr2=0x563da0d1f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:02:26.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:26.263+0000 7f2571400640 1 --2- 172.21.15.1:0/3610674803 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563da0d20330 0x563da0d1f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:26.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:26.263+0000 7f2571400640 1 -- 172.21.15.1:0/3610674803 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563da09952d0 con 0x563da0d5e2d0 2024-05-09T08:02:26.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:26.263+0000 7f2571400640 1 --2- 172.21.15.1:0/3610674803 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563da0d5e2d0 0x563da0d1f1b0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f255c0098c0 tx=0x7f255c02c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a411a677f54c3c2c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:26.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:26.263+0000 7f255a600640 1 -- 172.21.15.1:0/3610674803 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f255c04d070 con 0x563da0d5e2d0 2024-05-09T08:02:26.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:26.263+0000 7f255a600640 1 -- 172.21.15.1:0/3610674803 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f255c003810 con 0x563da0d5e2d0 2024-05-09T08:02:26.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:26.263+0000 7f255a600640 1 -- 172.21.15.1:0/3610674803 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f255c037040 con 0x563da0d5e2d0 2024-05-09T08:02:26.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:26.264+0000 7f257759e740 1 -- 172.21.15.1:0/3610674803 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563da0d5e2d0 msgr2=0x563da0d1f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:26.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:26.264+0000 7f257759e740 1 --2- 172.21.15.1:0/3610674803 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563da0d5e2d0 0x563da0d1f1b0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f255c0098c0 tx=0x7f255c02c000 comp rx=0 tx=0).stop 2024-05-09T08:02:26.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:26.264+0000 7f257759e740 1 -- 172.21.15.1:0/3610674803 shutdown_connections 2024-05-09T08:02:26.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:26.264+0000 7f257759e740 1 --2- 172.21.15.1:0/3610674803 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563da0d20330 0x563da0d1f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:26.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:26.264+0000 7f257759e740 1 --2- 172.21.15.1:0/3610674803 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563da0d5e2d0 0x563da0d1f1b0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:26.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:26.264+0000 7f257759e740 1 --2- 172.21.15.1:0/3610674803 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x563da0d1edd0 0x563da0d5dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:26.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:26.264+0000 7f257759e740 1 -- 172.21.15.1:0/3610674803 >> 172.21.15.1:0/3610674803 conn(0x563da0ce58a0 msgr2=0x563da0d62d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:02:26.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:26.264+0000 7f257759e740 1 -- 172.21.15.1:0/3610674803 shutdown_connections 2024-05-09T08:02:26.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:26.264+0000 7f257759e740 1 -- 172.21.15.1:0/3610674803 wait complete. 2024-05-09T08:02:26.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:26.265+0000 7f257759e740 1 -- start start 2024-05-09T08:02:26.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:26.265+0000 7f257759e740 1 Processor -- start 2024-05-09T08:02:26.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:26.266+0000 7f257759e740 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x563da0d1edd0 0x563da0d56810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:26.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:26.266+0000 7f257759e740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563da0d20330 0x563da0d6fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:26.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:26.266+0000 7f257759e740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563da0d5e2d0 0x563da0d18790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:26.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:26.266+0000 7f257759e740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x563da0d1d410 con 0x563da0d5e2d0 2024-05-09T08:02:26.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:26.266+0000 7f257759e740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x563da0c3c980 con 0x563da0d20330 2024-05-09T08:02:26.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:26.266+0000 7f257759e740 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x563da0d20960 con 0x563da0d1edd0 2024-05-09T08:02:26.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:26.266+0000 7f257759e740 10 client.0 _ll_register_callbacks cb 0x563da0c77fc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T08:02:26.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:26.266+0000 7f2572800640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563da0d5e2d0 0x563da0d18790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:26.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:26.266+0000 7f2571400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563da0d20330 0x563da0d6fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:26.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:26.267+0000 7f2571e00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x563da0d1edd0 0x563da0d56810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:26.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:26.267+0000 7f2571e00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x563da0d1edd0 0x563da0d56810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3301/0 says I am v2:172.21.15.1:43610/0 (socket says 172.21.15.1:43610) 2024-05-09T08:02:26.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:26.267+0000 7f2571400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563da0d20330 0x563da0d6fba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.1:36558/0 (socket says 172.21.15.1:36558) 2024-05-09T08:02:26.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:26.267+0000 7f2571e00640 1 -- 172.21.15.1:0/3281834545 learned_addr learned my addr 172.21.15.1:0/3281834545 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:02:26.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:26.267+0000 7f2571e00640 1 -- 172.21.15.1:0/3281834545 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563da0d20330 msgr2=0x563da0d6fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:02:26.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:26.267+0000 7f2571e00640 1 --2- 172.21.15.1:0/3281834545 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563da0d20330 0x563da0d6fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:26.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:26.267+0000 7f2571e00640 1 -- 172.21.15.1:0/3281834545 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563da0d5e2d0 msgr2=0x563da0d18790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:02:26.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:26.267+0000 7f2571e00640 1 --2- 172.21.15.1:0/3281834545 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563da0d5e2d0 0x563da0d18790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:26.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:26.267+0000 7f2571e00640 1 -- 172.21.15.1:0/3281834545 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({osdmap=0}) -- 0x563da0a5f6b0 con 0x563da0d1edd0 2024-05-09T08:02:26.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:26.267+0000 7f2571e00640 1 --2- 172.21.15.1:0/3281834545 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x563da0d1edd0 0x563da0d56810 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f25600584a0 tx=0x7f2560058510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c3ec50c10fa5743f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:26.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:26.267+0000 7f254ea00640 10 client.0 ms_handle_connect on v2:172.21.15.7:3301/0 2024-05-09T08:02:26.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:26.267+0000 7f254ea00640 1 -- 172.21.15.1:0/3281834545 <== mon.2 v2:172.21.15.7:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2560061040 con 0x563da0d1edd0 2024-05-09T08:02:26.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:26.267+0000 7f254ea00640 1 -- 172.21.15.1:0/3281834545 <== mon.2 v2:172.21.15.7:3301/0 2 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f256008a0c0 con 0x563da0d1edd0 2024-05-09T08:02:26.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:26.267+0000 7f257759e740 1 -- 172.21.15.1:0/3281834545 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f255c003810 con 0x563da0d1edd0 2024-05-09T08:02:26.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:26.268+0000 7f254ea00640 1 -- 172.21.15.1:0/3281834545 <== mon.2 v2:172.21.15.7:3301/0 3 ==== mon_get_version_reply(handle=1 version=7) ==== 24+0+0 (secure 0 0 0) 0x7f256008a2c0 con 0x563da0d1edd0 2024-05-09T08:02:26.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:26.268+0000 7f257759e740 10 client.4346 fetch_fsmap learned FSMap version 7 2024-05-09T08:02:26.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:26.268+0000 7f257759e740 1 -- 172.21.15.1:0/3281834545 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) -- 0x563da0d73530 con 0x563da0d1edd0 2024-05-09T08:02:26.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:26.268+0000 7f254ea00640 1 -- 172.21.15.1:0/3281834545 <== mon.2 v2:172.21.15.7:3301/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f255c003810 con 0x563da0d1edd0 2024-05-09T08:02:26.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:26.268+0000 7f254ea00640 1 -- 172.21.15.1:0/3281834545 <== mon.2 v2:172.21.15.7:3301/0 5 ==== fsmap(e 7) ==== 4712+0+0 (secure 0 0 0) 0x7f2560088170 con 0x563da0d1edd0 2024-05-09T08:02:26.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:26.268+0000 7f257759e740 10 client.4346 fetch_fsmap finished waiting for FSMap version 7 2024-05-09T08:02:26.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:26.268+0000 7f257759e740 10 client.4346 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-09T08:02:26.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:26.268+0000 7f257759e740 20 client.4346 populate_metadata read hostname 'smithi001' 2024-05-09T08:02:26.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:26.268+0000 7f257759e740 1 --2- 172.21.15.1:0/3281834545 >> [v2:172.21.15.1:6834/1509134635,v1:172.21.15.1:6836/1509134635] conn(0x563da0d464b0 0x563da0d7dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:26.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:26.268+0000 7f257759e740 4 client.4346 mds_command: new command op to 4294 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T08:02:26.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:26.268+0000 7f257759e740 1 -- 172.21.15.1:0/3281834545 --> [v2:172.21.15.1:6834/1509134635,v1:172.21.15.1:6836/1509134635] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563da0c95060 con 0x563da0d464b0 2024-05-09T08:02:26.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:26.268+0000 7f2571400640 1 --2- 172.21.15.1:0/3281834545 >> [v2:172.21.15.1:6834/1509134635,v1:172.21.15.1:6836/1509134635] conn(0x563da0d464b0 0x563da0d7dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:26.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:26.269+0000 7f2571400640 1 --2- 172.21.15.1:0/3281834545 >> [v2:172.21.15.1:6834/1509134635,v1:172.21.15.1:6836/1509134635] conn(0x563da0d464b0 0x563da0d7dc90 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ecaafb3f6801e3c1 server_cookie=d1c861062d958568 in_seq=0 out_seq=0 2024-05-09T08:02:26.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:26.269+0000 7f254ea00640 10 client.4346 ms_handle_connect on v2:172.21.15.1:6834/1509134635 2024-05-09T08:02:29.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:29.952+0000 7f254ea00640 1 -- 172.21.15.1:0/3281834545 <== mds.0 v2:172.21.15.1:6834/1509134635 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563da0c95060 con 0x563da0d464b0 2024-05-09T08:02:29.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:29.952+0000 7f254ea00640 10 client.4346 handle_command_reply: tid=0 2024-05-09T08:02:29.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:29.952+0000 7f257759e740 1 client.4346 shutdown 2024-05-09T08:02:29.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:29.953+0000 7f257759e740 1 -- 172.21.15.1:0/3281834545 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x563da0d1edd0 msgr2=0x563da0d56810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:29.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:29.953+0000 7f257759e740 1 --2- 172.21.15.1:0/3281834545 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x563da0d1edd0 0x563da0d56810 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f25600584a0 tx=0x7f2560058510 comp rx=0 tx=0).stop 2024-05-09T08:02:29.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:29.953+0000 7f257759e740 1 -- 172.21.15.1:0/3281834545 shutdown_connections 2024-05-09T08:02:29.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:29.953+0000 7f257759e740 1 --2- 172.21.15.1:0/3281834545 >> [v2:172.21.15.1:6834/1509134635,v1:172.21.15.1:6836/1509134635] conn(0x563da0d464b0 0x563da0d7dc90 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:29.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:29.953+0000 7f257759e740 1 --2- 172.21.15.1:0/3281834545 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563da0d20330 0x563da0d6fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:29.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:29.953+0000 7f257759e740 1 --2- 172.21.15.1:0/3281834545 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563da0d5e2d0 0x563da0d18790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:29.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:29.953+0000 7f257759e740 1 --2- 172.21.15.1:0/3281834545 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x563da0d1edd0 0x563da0d56810 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:29.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:29.953+0000 7f254ea00640 0 client.4346 ms_handle_reset on v2:172.21.15.1:6834/1509134635 2024-05-09T08:02:29.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:29.953+0000 7f257759e740 1 -- 172.21.15.1:0/3281834545 >> 172.21.15.1:0/3281834545 conn(0x563da0ce58a0 msgr2=0x563da0d1fa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:02:29.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:29.953+0000 7f257759e740 1 -- 172.21.15.1:0/3281834545 shutdown_connections 2024-05-09T08:02:29.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:29.953+0000 7f257759e740 1 -- 172.21.15.1:0/3281834545 wait complete. 2024-05-09T08:02:29.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:29.954+0000 7f257759e740 20 client.4346 trim_cache size 0 max 16384 2024-05-09T08:02:29.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:29.981+0000 7f257759e740 1 Processor -- start 2024-05-09T08:02:29.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:29.981+0000 7f257759e740 1 -- start start 2024-05-09T08:02:29.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:29.982+0000 7f257759e740 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x563da0d1edd0 0x563da0d72260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:29.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:29.982+0000 7f257759e740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563da0d20330 0x563da0d727a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:29.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:29.982+0000 7f257759e740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563da0d5e2d0 0x563da0d82cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:29.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:29.982+0000 7f257759e740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f256c1fce00 con 0x563da0d5e2d0 2024-05-09T08:02:30.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:29.982+0000 7f257759e740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x563da0d5cc70 con 0x563da0d20330 2024-05-09T08:02:30.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:29.982+0000 7f257759e740 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x563da0d6f3a0 con 0x563da0d1edd0 2024-05-09T08:02:30.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:29.983+0000 7f2572800640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563da0d5e2d0 0x563da0d82cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:30.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:29.983+0000 7f2571e00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x563da0d1edd0 0x563da0d72260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:30.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:29.983+0000 7f2571400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563da0d20330 0x563da0d727a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:30.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:29.983+0000 7f2571e00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x563da0d1edd0 0x563da0d72260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3301/0 says I am v2:172.21.15.1:43612/0 (socket says 172.21.15.1:43612) 2024-05-09T08:02:30.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:29.983+0000 7f2571e00640 1 -- 172.21.15.1:0/1451334127 learned_addr learned my addr 172.21.15.1:0/1451334127 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:02:30.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:29.983+0000 7f2571400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563da0d20330 0x563da0d727a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.1:36570/0 (socket says 172.21.15.1:36570) 2024-05-09T08:02:30.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:29.983+0000 7f2571e00640 1 -- 172.21.15.1:0/1451334127 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563da0d20330 msgr2=0x563da0d727a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:02:30.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:29.983+0000 7f2571e00640 1 --2- 172.21.15.1:0/1451334127 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563da0d20330 0x563da0d727a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:30.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:29.983+0000 7f2571e00640 1 -- 172.21.15.1:0/1451334127 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563da0d5e2d0 msgr2=0x563da0d82cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:02:30.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:29.983+0000 7f2571e00640 1 --2- 172.21.15.1:0/1451334127 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563da0d5e2d0 0x563da0d82cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:30.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:29.983+0000 7f2571e00640 1 -- 172.21.15.1:0/1451334127 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563da0d6f8f0 con 0x563da0d1edd0 2024-05-09T08:02:30.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:29.984+0000 7f2572800640 1 --2- 172.21.15.1:0/1451334127 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563da0d5e2d0 0x563da0d82cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:02:30.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:29.984+0000 7f2571e00640 1 --2- 172.21.15.1:0/1451334127 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x563da0d1edd0 0x563da0d72260 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f2560058210 tx=0x7f256008a050 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bd6e74c847b32af5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:30.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:29.984+0000 7f2559c00640 1 -- 172.21.15.1:0/1451334127 <== mon.2 v2:172.21.15.7:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25600565b0 con 0x563da0d1edd0 2024-05-09T08:02:30.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:29.984+0000 7f2559c00640 1 -- 172.21.15.1:0/1451334127 <== mon.2 v2:172.21.15.7:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2560004c70 con 0x563da0d1edd0 2024-05-09T08:02:30.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:29.984+0000 7f2559c00640 1 -- 172.21.15.1:0/1451334127 <== mon.2 v2:172.21.15.7:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2560057960 con 0x563da0d1edd0 2024-05-09T08:02:30.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:29.985+0000 7f257759e740 1 -- 172.21.15.1:0/1451334127 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x563da0d1edd0 msgr2=0x563da0d72260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:30.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:29.985+0000 7f257759e740 1 --2- 172.21.15.1:0/1451334127 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x563da0d1edd0 0x563da0d72260 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f2560058210 tx=0x7f256008a050 comp rx=0 tx=0).stop 2024-05-09T08:02:30.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:29.985+0000 7f257759e740 1 -- 172.21.15.1:0/1451334127 shutdown_connections 2024-05-09T08:02:30.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:29.985+0000 7f257759e740 1 --2- 172.21.15.1:0/1451334127 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563da0d20330 0x563da0d727a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:30.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:29.985+0000 7f257759e740 1 --2- 172.21.15.1:0/1451334127 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563da0d5e2d0 0x563da0d82cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:30.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:29.985+0000 7f257759e740 1 --2- 172.21.15.1:0/1451334127 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x563da0d1edd0 0x563da0d72260 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:30.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:29.985+0000 7f257759e740 1 -- 172.21.15.1:0/1451334127 >> 172.21.15.1:0/1451334127 conn(0x563da0ce58a0 msgr2=0x563da0d59610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:02:30.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:29.985+0000 7f257759e740 1 -- 172.21.15.1:0/1451334127 shutdown_connections 2024-05-09T08:02:30.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:29.985+0000 7f257759e740 1 -- 172.21.15.1:0/1451334127 wait complete. 2024-05-09T08:02:30.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:29.986+0000 7f257759e740 1 -- start start 2024-05-09T08:02:30.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:29.986+0000 7f257759e740 1 Processor -- start 2024-05-09T08:02:30.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:29.988+0000 7f257759e740 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x563da0d1edd0 0x563da0d3f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:30.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:29.988+0000 7f257759e740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563da0d20330 0x563da0dd3380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:30.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:29.988+0000 7f257759e740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563da0d5e2d0 0x563da0dd38c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:30.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:29.988+0000 7f257759e740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x563da0d46980 con 0x563da0d20330 2024-05-09T08:02:30.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:29.988+0000 7f257759e740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x563da0d46330 con 0x563da0d5e2d0 2024-05-09T08:02:30.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:29.988+0000 7f257759e740 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x563da0d5b1a0 con 0x563da0d1edd0 2024-05-09T08:02:30.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:29.988+0000 7f257759e740 10 client.0 _ll_register_callbacks cb 0x7f256000c700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T08:02:30.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:29.988+0000 7f2571400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563da0d20330 0x563da0dd3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:30.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:29.988+0000 7f2571400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563da0d20330 0x563da0dd3380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:36774/0 (socket says 172.21.15.1:36774) 2024-05-09T08:02:30.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:29.988+0000 7f2571400640 1 -- 172.21.15.1:0/286198652 learned_addr learned my addr 172.21.15.1:0/286198652 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:02:30.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:29.988+0000 7f2571e00640 1 --2- 172.21.15.1:0/286198652 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x563da0d1edd0 0x563da0d3f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:30.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:29.988+0000 7f2572800640 1 --2- 172.21.15.1:0/286198652 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563da0d5e2d0 0x563da0dd38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:30.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:29.988+0000 7f2571400640 1 -- 172.21.15.1:0/286198652 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x563da0d1edd0 msgr2=0x563da0d3f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:02:30.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:29.988+0000 7f2571400640 1 --2- 172.21.15.1:0/286198652 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x563da0d1edd0 0x563da0d3f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:30.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:29.988+0000 7f2571400640 1 -- 172.21.15.1:0/286198652 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563da0d5e2d0 msgr2=0x563da0dd38c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:02:30.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:29.988+0000 7f2571400640 1 --2- 172.21.15.1:0/286198652 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563da0d5e2d0 0x563da0dd38c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:30.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:29.988+0000 7f2571400640 1 -- 172.21.15.1:0/286198652 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563da0dd3e00 con 0x563da0d20330 2024-05-09T08:02:30.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:29.989+0000 7f2571400640 1 --2- 172.21.15.1:0/286198652 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563da0d20330 0x563da0dd3380 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f255c0098c0 tx=0x7f255c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e624f2d6eef5f55b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:30.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:29.989+0000 7f254ea00640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-09T08:02:30.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:29.989+0000 7f254ea00640 1 -- 172.21.15.1:0/286198652 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f255c04d040 con 0x563da0d20330 2024-05-09T08:02:30.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:29.989+0000 7f254ea00640 1 -- 172.21.15.1:0/286198652 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f255c003ac0 con 0x563da0d20330 2024-05-09T08:02:30.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:29.989+0000 7f257759e740 1 -- 172.21.15.1:0/286198652 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2560004c70 con 0x563da0d20330 2024-05-09T08:02:30.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:29.989+0000 7f254ea00640 1 -- 172.21.15.1:0/286198652 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) ==== 24+0+0 (secure 0 0 0) 0x7f255c05e050 con 0x563da0d20330 2024-05-09T08:02:30.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:29.989+0000 7f257759e740 10 client.4359 fetch_fsmap learned FSMap version 7 2024-05-09T08:02:30.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:29.989+0000 7f257759e740 1 -- 172.21.15.1:0/286198652 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) -- 0x563da0d3f980 con 0x563da0d20330 2024-05-09T08:02:30.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:29.990+0000 7f254ea00640 1 -- 172.21.15.1:0/286198652 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2560004c70 con 0x563da0d20330 2024-05-09T08:02:30.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:29.990+0000 7f254ea00640 1 -- 172.21.15.1:0/286198652 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 7) ==== 4712+0+0 (secure 0 0 0) 0x7f255c061020 con 0x563da0d20330 2024-05-09T08:02:30.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:29.990+0000 7f257759e740 10 client.4359 fetch_fsmap finished waiting for FSMap version 7 2024-05-09T08:02:30.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:29.990+0000 7f257759e740 10 client.4359 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-09T08:02:30.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:29.990+0000 7f257759e740 20 client.4359 populate_metadata read hostname 'smithi001' 2024-05-09T08:02:30.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:29.990+0000 7f257759e740 1 --2- 172.21.15.1:0/286198652 >> [v2:172.21.15.1:6834/1509134635,v1:172.21.15.1:6836/1509134635] conn(0x563da0ddd590 0x563da0de5f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:30.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:29.990+0000 7f257759e740 4 client.4359 mds_command: new command op to 4294 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-09T08:02:30.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:29.990+0000 7f257759e740 1 -- 172.21.15.1:0/286198652 --> [v2:172.21.15.1:6834/1509134635,v1:172.21.15.1:6836/1509134635] -- command(tid 0: {"prefix": "status"}) -- 0x563da0c8ebb0 con 0x563da0ddd590 2024-05-09T08:02:30.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:29.990+0000 7f2571e00640 1 --2- 172.21.15.1:0/286198652 >> [v2:172.21.15.1:6834/1509134635,v1:172.21.15.1:6836/1509134635] conn(0x563da0ddd590 0x563da0de5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:30.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:29.991+0000 7f2571e00640 1 --2- 172.21.15.1:0/286198652 >> [v2:172.21.15.1:6834/1509134635,v1:172.21.15.1:6836/1509134635] conn(0x563da0ddd590 0x563da0de5f20 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c79dbff66cd7fe00 server_cookie=516df6dd4f1e0ea7 in_seq=0 out_seq=0 2024-05-09T08:02:30.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:29.991+0000 7f254ea00640 10 client.4359 ms_handle_connect on v2:172.21.15.1:6834/1509134635 2024-05-09T08:02:30.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:29.991+0000 7f254ea00640 1 -- 172.21.15.1:0/286198652 <== mds.0 v2:172.21.15.1:6834/1509134635 1 ==== command_reply(tid 0: 0 ) ==== 8+0+330 (crc 0 0 0) 0x563da0c8ebb0 con 0x563da0ddd590 2024-05-09T08:02:30.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:29.991+0000 7f254ea00640 10 client.4359 handle_command_reply: tid=0 2024-05-09T08:02:30.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:29.991+0000 7f257759e740 1 client.4359 shutdown 2024-05-09T08:02:30.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:29.991+0000 7f257759e740 1 -- 172.21.15.1:0/286198652 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563da0d20330 msgr2=0x563da0dd3380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:30.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:29.991+0000 7f257759e740 1 --2- 172.21.15.1:0/286198652 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563da0d20330 0x563da0dd3380 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f255c0098c0 tx=0x7f255c004730 comp rx=0 tx=0).stop 2024-05-09T08:02:30.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:29.991+0000 7f257759e740 1 -- 172.21.15.1:0/286198652 shutdown_connections 2024-05-09T08:02:30.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:29.991+0000 7f257759e740 1 --2- 172.21.15.1:0/286198652 >> [v2:172.21.15.1:6834/1509134635,v1:172.21.15.1:6836/1509134635] conn(0x563da0ddd590 0x563da0de5f20 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:30.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:29.991+0000 7f257759e740 1 --2- 172.21.15.1:0/286198652 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563da0d5e2d0 0x563da0dd38c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:30.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:29.991+0000 7f257759e740 1 --2- 172.21.15.1:0/286198652 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563da0d20330 0x563da0dd3380 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:30.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:29.991+0000 7f257759e740 1 --2- 172.21.15.1:0/286198652 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x563da0d1edd0 0x563da0d3f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:30.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:29.991+0000 7f257759e740 1 -- 172.21.15.1:0/286198652 >> 172.21.15.1:0/286198652 conn(0x563da0ce58a0 msgr2=0x563da0d59400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:02:30.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:29.991+0000 7f254ea00640 0 client.4359 ms_handle_reset on v2:172.21.15.1:6834/1509134635 2024-05-09T08:02:30.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:29.992+0000 7f257759e740 1 -- 172.21.15.1:0/286198652 shutdown_connections 2024-05-09T08:02:30.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:29.992+0000 7f257759e740 1 -- 172.21.15.1:0/286198652 wait complete. 2024-05-09T08:02:30.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:29.992+0000 7f257759e740 20 client.4359 trim_cache size 0 max 16384 2024-05-09T08:02:30.014 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-05-09T08:02:30.014 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_fsid": "54329ec8-3270-4043-9a82-c6cd7420aa7f", 2024-05-09T08:02:30.014 INFO:teuthology.orchestra.run.smithi001.stdout: "whoami": 0, 2024-05-09T08:02:30.015 INFO:teuthology.orchestra.run.smithi001.stdout: "id": 4294, 2024-05-09T08:02:30.015 INFO:teuthology.orchestra.run.smithi001.stdout: "want_state": "up:active", 2024-05-09T08:02:30.015 INFO:teuthology.orchestra.run.smithi001.stdout: "state": "up:active", 2024-05-09T08:02:30.015 INFO:teuthology.orchestra.run.smithi001.stdout: "fs_name": "cephfs", 2024-05-09T08:02:30.015 INFO:teuthology.orchestra.run.smithi001.stdout: "rank_uptime": 14.096166976999999, 2024-05-09T08:02:30.015 INFO:teuthology.orchestra.run.smithi001.stdout: "mdsmap_epoch": 6, 2024-05-09T08:02:30.015 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_epoch": 25, 2024-05-09T08:02:30.015 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_epoch_barrier": 25, 2024-05-09T08:02:30.015 INFO:teuthology.orchestra.run.smithi001.stdout: "uptime": 22.157179147000001 2024-05-09T08:02:30.015 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-05-09T08:02:30.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:29.993+0000 7f2573200640 1 -- 172.21.15.1:0/2305437419 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f255006be30 msgr2=0x7f255006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:30.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:29.993+0000 7f2573200640 1 --2- 172.21.15.1:0/2305437419 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f255006be30 0x7f255006e2f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f255c034000 tx=0x7f255c02e000 comp rx=0 tx=0).stop 2024-05-09T08:02:30.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:29.993+0000 7f2573200640 1 -- 172.21.15.1:0/2305437419 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f256c0cac60 msgr2=0x7f256c1e6030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:30.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:29.993+0000 7f2573200640 1 --2- 172.21.15.1:0/2305437419 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f256c0cac60 0x7f256c1e6030 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f256000c7b0 tx=0x7f256000cc70 comp rx=0 tx=0).stop 2024-05-09T08:02:30.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:29.993+0000 7f2573200640 1 -- 172.21.15.1:0/2305437419 shutdown_connections 2024-05-09T08:02:30.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:29.993+0000 7f2573200640 1 --2- 172.21.15.1:0/2305437419 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f255006be30 0x7f255006e2f0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:30.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:29.993+0000 7f2573200640 1 --2- 172.21.15.1:0/2305437419 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f256c0cc5f0 0x7f256c1f0260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:30.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:29.993+0000 7f2573200640 1 --2- 172.21.15.1:0/2305437419 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f256c0ce4e0 0x7f256c1f07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:30.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:29.993+0000 7f2573200640 1 --2- 172.21.15.1:0/2305437419 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f256c0cac60 0x7f256c1e6030 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:30.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:29.993+0000 7f2573200640 1 -- 172.21.15.1:0/2305437419 >> 172.21.15.1:0/2305437419 conn(0x7f256c0cfc20 msgr2=0x7f256c0d0000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:02:30.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:29.993+0000 7f2573200640 1 -- 172.21.15.1:0/2305437419 shutdown_connections 2024-05-09T08:02:30.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:29.993+0000 7f2573200640 1 -- 172.21.15.1:0/2305437419 wait complete. 2024-05-09T08:02:30.031 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-09T08:02:30.032 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-05-09T08:02:30.032 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-09T08:02:30.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:30.221+0000 7fdbc9000640 1 Processor -- start 2024-05-09T08:02:30.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:30.221+0000 7fdbc9000640 1 -- start start 2024-05-09T08:02:30.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:30.223+0000 7fdbc9000640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fdbc40c5cf0 0x7fdbc40c7720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:30.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:30.223+0000 7fdbc9000640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdbc4157430 con 0x7fdbc40c3110 2024-05-09T08:02:30.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:30.223+0000 7fdbc9000640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdbc40c7d60 con 0x7fdbc40c5910 2024-05-09T08:02:30.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:30.223+0000 7fdbc9000640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fdbc40c8040 con 0x7fdbc40c5cf0 2024-05-09T08:02:30.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:30.223+0000 7fdbc3400640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fdbc40c5910 0x7fdbc415b380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.1:38194/0 (socket says 172.21.15.1:38194) 2024-05-09T08:02:30.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:30.223+0000 7fdbc3400640 1 -- 172.21.15.1:0/1894690229 learned_addr learned my addr 172.21.15.1:0/1894690229 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T08:02:30.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:30.223+0000 7fdbbbe00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fdbc40c5cf0 0x7fdbc40c7720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:30.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:30.223+0000 7fdbc2a00640 1 -- 172.21.15.1:0/1894690229 <== mon.1 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3857517540 0 0) 0x7fdbc40c7d60 con 0x7fdbc40c5910 2024-05-09T08:02:30.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:30.224+0000 7fdbc2a00640 1 -- 172.21.15.1:0/1894690229 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdba8005730 con 0x7fdbc40c5910 2024-05-09T08:02:30.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:30.224+0000 7fdbc2a00640 1 -- 172.21.15.1:0/1894690229 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1290209454 0 0) 0x7fdbc4157430 con 0x7fdbc40c3110 2024-05-09T08:02:30.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:30.224+0000 7fdbbbe00640 1 -- 172.21.15.1:0/1894690229 >> v1:172.21.15.7:6789/0 conn(0x7fdbc40c5910 legacy=0x7fdbc415b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:30.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:30.224+0000 7fdbbbe00640 1 -- 172.21.15.1:0/1894690229 >> v1:172.21.15.1:6789/0 conn(0x7fdbc40c3110 legacy=0x7fdbc4140200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:30.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:30.224+0000 7fdbbbe00640 1 -- 172.21.15.1:0/1894690229 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbc41576f0 con 0x7fdbc40c5cf0 2024-05-09T08:02:30.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:30.225+0000 7fdbbbe00640 1 --2- 172.21.15.1:0/1894690229 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdbc40c5cf0 0x7fdbc40c7720 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fdbac0097a0 tx=0x7fdbac02a1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1b4e7b4d8dc8aa90 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:30.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:30.225+0000 7fdbc2a00640 1 -- 172.21.15.1:0/1894690229 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbac037070 con 0x7fdbc40c5cf0 2024-05-09T08:02:30.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:30.225+0000 7fdbc2a00640 1 -- 172.21.15.1:0/1894690229 <== mon.1 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdbac032440 con 0x7fdbc40c5cf0 2024-05-09T08:02:30.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:30.225+0000 7fdbc2a00640 1 -- 172.21.15.1:0/1894690229 <== mon.1 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbac032810 con 0x7fdbc40c5cf0 2024-05-09T08:02:30.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:30.226+0000 7fdbc9000640 1 -- 172.21.15.1:0/1894690229 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdbc40c5cf0 msgr2=0x7fdbc40c7720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:30.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:30.226+0000 7fdbc9000640 1 --2- 172.21.15.1:0/1894690229 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdbc40c5cf0 0x7fdbc40c7720 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fdbac0097a0 tx=0x7fdbac02a1a0 comp rx=0 tx=0).stop 2024-05-09T08:02:30.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:30.226+0000 7fdbc9000640 1 -- 172.21.15.1:0/1894690229 shutdown_connections 2024-05-09T08:02:30.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:30.226+0000 7fdbc9000640 1 --2- 172.21.15.1:0/1894690229 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdbc40c5cf0 0x7fdbc40c7720 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:30.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:30.226+0000 7fdbc9000640 1 -- 172.21.15.1:0/1894690229 >> 172.21.15.1:0/1894690229 conn(0x7fdbc40cbc10 msgr2=0x7fdbc40cc020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:02:30.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:30.226+0000 7fdbc9000640 1 -- 172.21.15.1:0/1894690229 shutdown_connections 2024-05-09T08:02:30.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:30.226+0000 7fdbc9000640 1 -- 172.21.15.1:0/1894690229 wait complete. 2024-05-09T08:02:30.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:30.228+0000 7fdbc9000640 1 Processor -- start 2024-05-09T08:02:30.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:30.228+0000 7fdbc9000640 1 -- start start 2024-05-09T08:02:30.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:30.229+0000 7fdbc9000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdbc40c3110 0x7fdbc4154680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:30.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:30.229+0000 7fdbc9000640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fdbc40c5910 0x7fdbc41522a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:30.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:30.229+0000 7fdbc9000640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdbc40c5cf0 0x7fdbc41527e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:30.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:30.229+0000 7fdbc9000640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fdbc41581d0 con 0x7fdbc40c5cf0 2024-05-09T08:02:30.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:30.229+0000 7fdbc9000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fdbc4157cb0 con 0x7fdbc40c3110 2024-05-09T08:02:30.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:30.229+0000 7fdbc9000640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7fdbc4072b30 con 0x7fdbc40c5910 2024-05-09T08:02:30.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:30.229+0000 7fdbc3e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdbc40c5cf0 0x7fdbc41527e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:30.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:30.229+0000 7fdbc3e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdbc40c5cf0 0x7fdbc41527e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:36790/0 (socket says 172.21.15.1:36790) 2024-05-09T08:02:30.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:30.229+0000 7fdbc3e00640 1 -- 172.21.15.1:0/3600278502 learned_addr learned my addr 172.21.15.1:0/3600278502 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:02:30.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:30.229+0000 7fdbbbe00640 1 --2- 172.21.15.1:0/3600278502 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fdbc40c5910 0x7fdbc41522a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:30.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:30.229+0000 7fdbc3400640 1 --2- 172.21.15.1:0/3600278502 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdbc40c3110 0x7fdbc4154680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:30.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:30.230+0000 7fdbc3e00640 1 -- 172.21.15.1:0/3600278502 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fdbc40c5910 msgr2=0x7fdbc41522a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:30.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:30.230+0000 7fdbc3e00640 1 --2- 172.21.15.1:0/3600278502 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fdbc40c5910 0x7fdbc41522a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:30.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:30.230+0000 7fdbc3e00640 1 -- 172.21.15.1:0/3600278502 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdbc40c3110 msgr2=0x7fdbc4154680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:30.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:30.230+0000 7fdbc3e00640 1 --2- 172.21.15.1:0/3600278502 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdbc40c3110 0x7fdbc4154680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:30.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:30.230+0000 7fdbc3e00640 1 -- 172.21.15.1:0/3600278502 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbc413adc0 con 0x7fdbc40c5cf0 2024-05-09T08:02:30.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:30.230+0000 7fdbc3e00640 1 --2- 172.21.15.1:0/3600278502 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdbc40c5cf0 0x7fdbc41527e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fdbbc00c760 tx=0x7fdbbc00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:30.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:30.230+0000 7fdbbb400640 1 -- 172.21.15.1:0/3600278502 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbbc003da0 con 0x7fdbc40c5cf0 2024-05-09T08:02:30.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:30.230+0000 7fdbbb400640 1 -- 172.21.15.1:0/3600278502 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdbbc005070 con 0x7fdbc40c5cf0 2024-05-09T08:02:30.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:30.230+0000 7fdbc9000640 1 -- 172.21.15.1:0/3600278502 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbc413b0a0 con 0x7fdbc40c5cf0 2024-05-09T08:02:30.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:30.230+0000 7fdbbb400640 1 -- 172.21.15.1:0/3600278502 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbbc005460 con 0x7fdbc40c5cf0 2024-05-09T08:02:30.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:30.230+0000 7fdbc9000640 1 -- 172.21.15.1:0/3600278502 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbc413d860 con 0x7fdbc40c5cf0 2024-05-09T08:02:30.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:30.232+0000 7fdbc9000640 1 -- 172.21.15.1:0/3600278502 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdbc404f660 con 0x7fdbc40c5cf0 2024-05-09T08:02:30.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:30.232+0000 7fdbbb400640 1 -- 172.21.15.1:0/3600278502 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fdbbc005600 con 0x7fdbc40c5cf0 2024-05-09T08:02:30.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:30.233+0000 7fdbbb400640 1 --2- 172.21.15.1:0/3600278502 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fdba006be30 0x7fdba006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:30.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:30.233+0000 7fdbbb400640 1 -- 172.21.15.1:0/3600278502 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fdbbc08b090 con 0x7fdbc40c5cf0 2024-05-09T08:02:30.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:30.234+0000 7fdbc3400640 1 --2- 172.21.15.1:0/3600278502 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fdba006be30 0x7fdba006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:30.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:30.234+0000 7fdbc3400640 1 --2- 172.21.15.1:0/3600278502 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fdba006be30 0x7fdba006e2f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fdbb40011e0 tx=0x7fdbb4007d30 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:30.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:30.238+0000 7fdbbb400640 1 -- 172.21.15.1:0/3600278502 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fdbbc058f80 con 0x7fdbc40c5cf0 2024-05-09T08:02:30.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:30.477+0000 7fdbc9000640 1 -- 172.21.15.1:0/3600278502 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fdbc40734d0 con 0x7fdbc40c5cf0 2024-05-09T08:02:30.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:30.478+0000 7fdbbb400640 1 -- 172.21.15.1:0/3600278502 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) ==== 53+0+88 (secure 0 0 0) 0x7fdbbc05d280 con 0x7fdbc40c5cf0 2024-05-09T08:02:30.495 INFO:teuthology.orchestra.run.smithi001.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-05-09T08:02:30.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:30.480+0000 7fdbc9000640 1 -- 172.21.15.1:0/3600278502 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fdba006be30 msgr2=0x7fdba006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:30.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:30.480+0000 7fdbc9000640 1 --2- 172.21.15.1:0/3600278502 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fdba006be30 0x7fdba006e2f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fdbb40011e0 tx=0x7fdbb4007d30 comp rx=0 tx=0).stop 2024-05-09T08:02:30.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:30.480+0000 7fdbc9000640 1 -- 172.21.15.1:0/3600278502 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdbc40c5cf0 msgr2=0x7fdbc41527e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:30.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:30.480+0000 7fdbc9000640 1 --2- 172.21.15.1:0/3600278502 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdbc40c5cf0 0x7fdbc41527e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fdbbc00c760 tx=0x7fdbbc00cc20 comp rx=0 tx=0).stop 2024-05-09T08:02:30.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:30.480+0000 7fdbc9000640 1 -- 172.21.15.1:0/3600278502 shutdown_connections 2024-05-09T08:02:30.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:30.480+0000 7fdbc9000640 1 --2- 172.21.15.1:0/3600278502 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fdba006be30 0x7fdba006e2f0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:30.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:30.480+0000 7fdbc9000640 1 --2- 172.21.15.1:0/3600278502 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdbc40c5cf0 0x7fdbc41527e0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:30.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:30.480+0000 7fdbc9000640 1 --2- 172.21.15.1:0/3600278502 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fdbc40c5910 0x7fdbc41522a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:30.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:30.480+0000 7fdbc9000640 1 --2- 172.21.15.1:0/3600278502 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdbc40c3110 0x7fdbc4154680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:30.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:30.480+0000 7fdbc9000640 1 -- 172.21.15.1:0/3600278502 >> 172.21.15.1:0/3600278502 conn(0x7fdbc40cbc10 msgr2=0x7fdbc40c71b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:02:30.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:30.481+0000 7fdbc9000640 1 -- 172.21.15.1:0/3600278502 shutdown_connections 2024-05-09T08:02:30.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:30.481+0000 7fdbc9000640 1 -- 172.21.15.1:0/3600278502 wait complete. 2024-05-09T08:02:30.517 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-05-09T08:02:30.517 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-09T08:02:30.518 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi001.front.sepia.ceph.com 2024-05-09T08:02:30.518 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-05-09T08:02:30.518 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-05-09T08:02:30.518 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-05-09T08:02:30.518 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-09T08:02:30.518 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-09T08:02:30.518 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-05-09T08:02:30.518 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T08:02:30.518 DEBUG:teuthology.orchestra.run.smithi001:> ip addr 2024-05-09T08:02:30.572 INFO:teuthology.orchestra.run.smithi001.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-09T08:02:30.573 INFO:teuthology.orchestra.run.smithi001.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-09T08:02:30.573 INFO:teuthology.orchestra.run.smithi001.stdout: inet 127.0.0.1/8 scope host lo 2024-05-09T08:02:30.573 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft forever preferred_lft forever 2024-05-09T08:02:30.573 INFO:teuthology.orchestra.run.smithi001.stdout: inet6 ::1/128 scope host 2024-05-09T08:02:30.573 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft forever preferred_lft forever 2024-05-09T08:02:30.573 INFO:teuthology.orchestra.run.smithi001.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-09T08:02:30.573 INFO:teuthology.orchestra.run.smithi001.stdout: link/ether 0c:c4:7a:6b:f6:7a brd ff:ff:ff:ff:ff:ff 2024-05-09T08:02:30.573 INFO:teuthology.orchestra.run.smithi001.stdout: altname enp7s0f0 2024-05-09T08:02:30.574 INFO:teuthology.orchestra.run.smithi001.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-09T08:02:30.574 INFO:teuthology.orchestra.run.smithi001.stdout: link/ether 0c:c4:7a:6b:f6:7b brd ff:ff:ff:ff:ff:ff 2024-05-09T08:02:30.574 INFO:teuthology.orchestra.run.smithi001.stdout: altname enp7s0f1 2024-05-09T08:02:30.574 INFO:teuthology.orchestra.run.smithi001.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-09T08:02:30.574 INFO:teuthology.orchestra.run.smithi001.stdout: link/ether 0c:c4:7a:bd:15:e8 brd ff:ff:ff:ff:ff:ff 2024-05-09T08:02:30.574 INFO:teuthology.orchestra.run.smithi001.stdout: altname enp1s0f0 2024-05-09T08:02:30.574 INFO:teuthology.orchestra.run.smithi001.stdout: inet 172.21.15.1/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-05-09T08:02:30.574 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft 42639sec preferred_lft 42639sec 2024-05-09T08:02:30.574 INFO:teuthology.orchestra.run.smithi001.stdout: inet6 fe80::ec4:7aff:febd:15e8/64 scope link noprefixroute 2024-05-09T08:02:30.574 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft forever preferred_lft forever 2024-05-09T08:02:30.574 INFO:teuthology.orchestra.run.smithi001.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-09T08:02:30.574 INFO:teuthology.orchestra.run.smithi001.stdout: link/ether 0c:c4:7a:bd:15:e9 brd ff:ff:ff:ff:ff:ff 2024-05-09T08:02:30.574 INFO:teuthology.orchestra.run.smithi001.stdout: altname enp1s0f1 2024-05-09T08:02:30.575 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-05-09T08:02:30.575 DEBUG:teuthology.orchestra.run.smithi001:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T08:02:30.575 DEBUG:teuthology.orchestra.run.smithi001:> set -e 2024-05-09T08:02:30.575 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip link add name ceph-brx type bridge 2024-05-09T08:02:30.576 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip addr flush dev ceph-brx 2024-05-09T08:02:30.576 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip link set ceph-brx up 2024-05-09T08:02:30.576 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-05-09T08:02:30.576 DEBUG:teuthology.orchestra.run.smithi001:> ') 2024-05-09T08:02:30.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:30 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T08:02:31.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T08:02:31.061 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T08:02:31.062 DEBUG:teuthology.orchestra.run.smithi001:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-09T08:02:31.133 INFO:teuthology.orchestra.run.smithi001.stdout:1 2024-05-09T08:02:31.137 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T08:02:31.137 DEBUG:teuthology.orchestra.run.smithi001:> route 2024-05-09T08:02:31.187 INFO:teuthology.orchestra.run.smithi001.stdout:Kernel IP routing table 2024-05-09T08:02:31.187 INFO:teuthology.orchestra.run.smithi001.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-09T08:02:31.187 INFO:teuthology.orchestra.run.smithi001.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-05-09T08:02:31.188 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-05-09T08:02:31.188 INFO:teuthology.orchestra.run.smithi001.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-09T08:02:31.188 DEBUG:teuthology.orchestra.run.smithi001:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T08:02:31.188 DEBUG:teuthology.orchestra.run.smithi001:> set -e 2024-05-09T08:02:31.188 DEBUG:teuthology.orchestra.run.smithi001:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-05-09T08:02:31.188 DEBUG:teuthology.orchestra.run.smithi001:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-05-09T08:02:31.188 DEBUG:teuthology.orchestra.run.smithi001:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-05-09T08:02:31.189 DEBUG:teuthology.orchestra.run.smithi001:> ') 2024-05-09T08:02:31.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:31 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T08:02:31.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T08:02:31.930 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T08:02:31.930 DEBUG:teuthology.orchestra.run.smithi001:> ip netns list 2024-05-09T08:02:31.976 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T08:02:31.976 DEBUG:teuthology.orchestra.run.smithi001:> ip netns list-id 2024-05-09T08:02:32.065 DEBUG:teuthology.orchestra.run.smithi001:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T08:02:32.065 DEBUG:teuthology.orchestra.run.smithi001:> set -e 2024-05-09T08:02:32.065 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-09T08:02:32.065 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-05-09T08:02:32.065 DEBUG:teuthology.orchestra.run.smithi001:> ') 2024-05-09T08:02:32.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:32 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T08:02:32.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T08:02:32.292 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-05-09T08:02:32.293 DEBUG:teuthology.orchestra.run.smithi001:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T08:02:32.293 DEBUG:teuthology.orchestra.run.smithi001:> set -e 2024-05-09T08:02:32.293 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-05-09T08:02:32.293 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-05-09T08:02:32.293 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-05-09T08:02:32.293 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-05-09T08:02:32.293 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-05-09T08:02:32.293 DEBUG:teuthology.orchestra.run.smithi001:> ') 2024-05-09T08:02:32.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:32 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T08:02:32.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T08:02:32.569 DEBUG:teuthology.orchestra.run.smithi001:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T08:02:32.569 DEBUG:teuthology.orchestra.run.smithi001:> set -e 2024-05-09T08:02:32.569 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip link set brx.0 up 2024-05-09T08:02:32.570 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip link set dev brx.0 master ceph-brx 2024-05-09T08:02:32.570 DEBUG:teuthology.orchestra.run.smithi001:> ') 2024-05-09T08:02:32.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:32 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T08:02:32.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T08:02:32.764 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-05-09T08:02:32.765 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-09T08:02:32.765 DEBUG:teuthology.orchestra.run.smithi001:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-05-09T08:02:32.806 INFO:teuthology.orchestra.run.smithi001.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-05-09T08:02:32.806 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-09T08:02:32.807 DEBUG:teuthology.orchestra.run.smithi001:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-05-09T08:02:32.895 DEBUG:teuthology.orchestra.run.smithi001:> sudo modprobe fuse 2024-05-09T08:02:33.016 DEBUG:teuthology.orchestra.run.smithi001:> cat /proc/self/mounts | awk '{print $2}' 2024-05-09T08:02:33.059 INFO:teuthology.orchestra.run.smithi001.stdout:/proc 2024-05-09T08:02:33.059 INFO:teuthology.orchestra.run.smithi001.stdout:/sys 2024-05-09T08:02:33.059 INFO:teuthology.orchestra.run.smithi001.stdout:/dev 2024-05-09T08:02:33.059 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/kernel/security 2024-05-09T08:02:33.059 INFO:teuthology.orchestra.run.smithi001.stdout:/dev/shm 2024-05-09T08:02:33.059 INFO:teuthology.orchestra.run.smithi001.stdout:/dev/pts 2024-05-09T08:02:33.059 INFO:teuthology.orchestra.run.smithi001.stdout:/run 2024-05-09T08:02:33.059 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/cgroup 2024-05-09T08:02:33.059 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/pstore 2024-05-09T08:02:33.060 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/bpf 2024-05-09T08:02:33.060 INFO:teuthology.orchestra.run.smithi001.stdout:/ 2024-05-09T08:02:33.060 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/selinux 2024-05-09T08:02:33.060 INFO:teuthology.orchestra.run.smithi001.stdout:/proc/sys/fs/binfmt_misc 2024-05-09T08:02:33.060 INFO:teuthology.orchestra.run.smithi001.stdout:/dev/hugepages 2024-05-09T08:02:33.060 INFO:teuthology.orchestra.run.smithi001.stdout:/dev/mqueue 2024-05-09T08:02:33.060 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/kernel/debug 2024-05-09T08:02:33.060 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/kernel/tracing 2024-05-09T08:02:33.060 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/fuse/connections 2024-05-09T08:02:33.060 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/kernel/config 2024-05-09T08:02:33.060 INFO:teuthology.orchestra.run.smithi001.stdout:/run/credentials/systemd-sysctl.service 2024-05-09T08:02:33.060 INFO:teuthology.orchestra.run.smithi001.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-09T08:02:33.060 INFO:teuthology.orchestra.run.smithi001.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-09T08:02:33.060 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/nfs/rpc_pipefs 2024-05-09T08:02:33.061 INFO:teuthology.orchestra.run.smithi001.stdout:/run/user/1000 2024-05-09T08:02:33.061 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph 2024-05-09T08:02:33.061 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph/osd/ceph-0 2024-05-09T08:02:33.061 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph/osd/ceph-1 2024-05-09T08:02:33.061 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph/osd/ceph-2 2024-05-09T08:02:33.061 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph/osd/ceph-3 2024-05-09T08:02:33.061 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns 2024-05-09T08:02:33.061 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-09T08:02:33.061 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-09T08:02:33.061 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T08:02:33.061 DEBUG:teuthology.orchestra.run.smithi001:> ls /sys/fs/fuse/connections 2024-05-09T08:02:33.149 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-09T08:02:33.149 DEBUG:teuthology.orchestra.run.smithi001:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-05-09T08:02:33.154 DEBUG:teuthology.orchestra.run.smithi001:> sudo modprobe fuse 2024-05-09T08:02:33.217 DEBUG:teuthology.orchestra.run.smithi001:> cat /proc/self/mounts | awk '{print $2}' 2024-05-09T08:02:33.262 INFO:teuthology.orchestra.run.smithi001.stdout:/proc 2024-05-09T08:02:33.262 INFO:teuthology.orchestra.run.smithi001.stdout:/sys 2024-05-09T08:02:33.262 INFO:teuthology.orchestra.run.smithi001.stdout:/dev 2024-05-09T08:02:33.262 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/kernel/security 2024-05-09T08:02:33.263 INFO:teuthology.orchestra.run.smithi001.stdout:/dev/shm 2024-05-09T08:02:33.263 INFO:teuthology.orchestra.run.smithi001.stdout:/dev/pts 2024-05-09T08:02:33.263 INFO:teuthology.orchestra.run.smithi001.stdout:/run 2024-05-09T08:02:33.263 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/cgroup 2024-05-09T08:02:33.263 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/pstore 2024-05-09T08:02:33.263 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/bpf 2024-05-09T08:02:33.263 INFO:teuthology.orchestra.run.smithi001.stdout:/ 2024-05-09T08:02:33.263 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/selinux 2024-05-09T08:02:33.263 INFO:teuthology.orchestra.run.smithi001.stdout:/proc/sys/fs/binfmt_misc 2024-05-09T08:02:33.263 INFO:teuthology.orchestra.run.smithi001.stdout:/dev/hugepages 2024-05-09T08:02:33.263 INFO:teuthology.orchestra.run.smithi001.stdout:/dev/mqueue 2024-05-09T08:02:33.263 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/kernel/debug 2024-05-09T08:02:33.263 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/kernel/tracing 2024-05-09T08:02:33.263 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/fs/fuse/connections 2024-05-09T08:02:33.264 INFO:teuthology.orchestra.run.smithi001.stdout:/sys/kernel/config 2024-05-09T08:02:33.264 INFO:teuthology.orchestra.run.smithi001.stdout:/run/credentials/systemd-sysctl.service 2024-05-09T08:02:33.264 INFO:teuthology.orchestra.run.smithi001.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-09T08:02:33.264 INFO:teuthology.orchestra.run.smithi001.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-09T08:02:33.264 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/nfs/rpc_pipefs 2024-05-09T08:02:33.264 INFO:teuthology.orchestra.run.smithi001.stdout:/run/user/1000 2024-05-09T08:02:33.264 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph 2024-05-09T08:02:33.264 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph/osd/ceph-0 2024-05-09T08:02:33.264 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph/osd/ceph-1 2024-05-09T08:02:33.264 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph/osd/ceph-2 2024-05-09T08:02:33.264 INFO:teuthology.orchestra.run.smithi001.stdout:/var/lib/ceph/osd/ceph-3 2024-05-09T08:02:33.264 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns 2024-05-09T08:02:33.264 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-09T08:02:33.264 INFO:teuthology.orchestra.run.smithi001.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-09T08:02:33.265 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T08:02:33.265 DEBUG:teuthology.orchestra.run.smithi001:> ls /sys/fs/fuse/connections 2024-05-09T08:02:33.331 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi001.stderr:2024-05-09T08:02:33.314+0000 7f42fb015480 -1 init, newargv = 0x557a05824140 newargc=15 2024-05-09T08:02:33.331 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi001.stderr:ceph-fuse[40910]: starting ceph client 2024-05-09T08:02:33.342 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi001.stderr:ceph-fuse[40910]: starting fuse 2024-05-09T08:02:33.350 INFO:teuthology.orchestra.run.smithi001.stdout:40 2024-05-09T08:02:33.351 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [40] 2024-05-09T08:02:33.351 DEBUG:teuthology.orchestra.run.smithi001:> sudo stdin-killer -- python3 -c ' 2024-05-09T08:02:33.351 DEBUG:teuthology.orchestra.run.smithi001:> import glob 2024-05-09T08:02:33.351 DEBUG:teuthology.orchestra.run.smithi001:> import re 2024-05-09T08:02:33.351 DEBUG:teuthology.orchestra.run.smithi001:> import os 2024-05-09T08:02:33.351 DEBUG:teuthology.orchestra.run.smithi001:> import subprocess 2024-05-09T08:02:33.351 DEBUG:teuthology.orchestra.run.smithi001:> 2024-05-09T08:02:33.351 DEBUG:teuthology.orchestra.run.smithi001:> def _find_admin_socket(client_name): 2024-05-09T08:02:33.351 DEBUG:teuthology.orchestra.run.smithi001:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-05-09T08:02:33.351 DEBUG:teuthology.orchestra.run.smithi001:> files = glob.glob(asok_path) 2024-05-09T08:02:33.351 DEBUG:teuthology.orchestra.run.smithi001:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-05-09T08:02:33.351 DEBUG:teuthology.orchestra.run.smithi001:> 2024-05-09T08:02:33.352 DEBUG:teuthology.orchestra.run.smithi001:> # Given a non-glob path, it better be there 2024-05-09T08:02:33.352 DEBUG:teuthology.orchestra.run.smithi001:> if "*" not in asok_path: 2024-05-09T08:02:33.352 DEBUG:teuthology.orchestra.run.smithi001:> assert(len(files) == 1) 2024-05-09T08:02:33.352 DEBUG:teuthology.orchestra.run.smithi001:> return files[0] 2024-05-09T08:02:33.352 DEBUG:teuthology.orchestra.run.smithi001:> 2024-05-09T08:02:33.352 DEBUG:teuthology.orchestra.run.smithi001:> for f in files: 2024-05-09T08:02:33.352 DEBUG:teuthology.orchestra.run.smithi001:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-09T08:02:33.352 DEBUG:teuthology.orchestra.run.smithi001:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-09T08:02:33.352 DEBUG:teuthology.orchestra.run.smithi001:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-09T08:02:33.352 DEBUG:teuthology.orchestra.run.smithi001:> contents = proc_f.read() 2024-05-09T08:02:33.352 DEBUG:teuthology.orchestra.run.smithi001:> if mountpoint in contents: 2024-05-09T08:02:33.352 DEBUG:teuthology.orchestra.run.smithi001:> return f 2024-05-09T08:02:33.352 DEBUG:teuthology.orchestra.run.smithi001:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-09T08:02:33.352 DEBUG:teuthology.orchestra.run.smithi001:> 2024-05-09T08:02:33.353 DEBUG:teuthology.orchestra.run.smithi001:> print(_find_admin_socket("client.admin")) 2024-05-09T08:02:33.353 DEBUG:teuthology.orchestra.run.smithi001:> ' 2024-05-09T08:02:33.537 INFO:teuthology.orchestra.run.smithi001.stdout:/var/run/ceph/ceph-client.admin.40910.asok 2024-05-09T08:02:33.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T08:02:33.554 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.40910.asok 2024-05-09T08:02:33.554 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T08:02:33.554 DEBUG:teuthology.orchestra.run.smithi001:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.40910.asok status 2024-05-09T08:02:33.782 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-05-09T08:02:33.782 INFO:teuthology.orchestra.run.smithi001.stdout: "metadata": { 2024-05-09T08:02:33.782 INFO:teuthology.orchestra.run.smithi001.stdout: "ceph_sha1": "96805dcf909541c4ca9a1e90258de8ae5b6fa1bc", 2024-05-09T08:02:33.782 INFO:teuthology.orchestra.run.smithi001.stdout: "ceph_version": "ceph version 19.0.0-2515-g96805dcf (96805dcf909541c4ca9a1e90258de8ae5b6fa1bc) squid (dev)", 2024-05-09T08:02:33.783 INFO:teuthology.orchestra.run.smithi001.stdout: "entity_id": "admin", 2024-05-09T08:02:33.783 INFO:teuthology.orchestra.run.smithi001.stdout: "hostname": "smithi001", 2024-05-09T08:02:33.783 INFO:teuthology.orchestra.run.smithi001.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-05-09T08:02:33.783 INFO:teuthology.orchestra.run.smithi001.stdout: "pid": "40910", 2024-05-09T08:02:33.783 INFO:teuthology.orchestra.run.smithi001.stdout: "root": "/" 2024-05-09T08:02:33.783 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:02:33.783 INFO:teuthology.orchestra.run.smithi001.stdout: "dentry_count": 0, 2024-05-09T08:02:33.783 INFO:teuthology.orchestra.run.smithi001.stdout: "dentry_pinned_count": 0, 2024-05-09T08:02:33.783 INFO:teuthology.orchestra.run.smithi001.stdout: "id": 4358, 2024-05-09T08:02:33.783 INFO:teuthology.orchestra.run.smithi001.stdout: "inst": { 2024-05-09T08:02:33.783 INFO:teuthology.orchestra.run.smithi001.stdout: "name": { 2024-05-09T08:02:33.783 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "client", 2024-05-09T08:02:33.783 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 4358 2024-05-09T08:02:33.784 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:02:33.784 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": { 2024-05-09T08:02:33.784 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-05-09T08:02:33.784 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:0", 2024-05-09T08:02:33.784 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 4156979971 2024-05-09T08:02:33.784 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:02:33.784 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:02:33.784 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": { 2024-05-09T08:02:33.784 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-05-09T08:02:33.784 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:0", 2024-05-09T08:02:33.784 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 4156979971 2024-05-09T08:02:33.784 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:02:33.784 INFO:teuthology.orchestra.run.smithi001.stdout: "inst_str": "client.4358 172.21.15.1:0/4156979971", 2024-05-09T08:02:33.784 INFO:teuthology.orchestra.run.smithi001.stdout: "addr_str": "172.21.15.1:0/4156979971", 2024-05-09T08:02:33.784 INFO:teuthology.orchestra.run.smithi001.stdout: "inode_count": 1, 2024-05-09T08:02:33.785 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_epoch": 6, 2024-05-09T08:02:33.785 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_epoch": 25, 2024-05-09T08:02:33.785 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_epoch_barrier": 0, 2024-05-09T08:02:33.785 INFO:teuthology.orchestra.run.smithi001.stdout: "blocklisted": false, 2024-05-09T08:02:33.785 INFO:teuthology.orchestra.run.smithi001.stdout: "fs_name": "cephfs" 2024-05-09T08:02:33.785 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-05-09T08:02:33.796 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T08:02:33.797 DEBUG:teuthology.orchestra.run.smithi001:> stat --file-system '--printf=%T 2024-05-09T08:02:33.797 DEBUG:teuthology.orchestra.run.smithi001:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-05-09T08:02:33.823 INFO:tasks.ceph.mds.c.smithi001.stderr:2024-05-09T08:02:33.806+0000 7f5024800640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-09T08:02:33.849 INFO:teuthology.orchestra.run.smithi001.stdout:fuseblk 2024-05-09T08:02:33.849 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-05-09T08:02:33.849 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T08:02:33.850 DEBUG:teuthology.orchestra.run.smithi001:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-05-09T08:02:33.987 DEBUG:teuthology.orchestra.run.smithi001:> (cd /home/ubuntu/cephtest/mnt.admin && exec stdin-killer --timeout=900 -- bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-05-09T08:02:34.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:34 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-09T08:02:34.145 INFO:teuthology.orchestra.run.smithi001.stdout:# file: . 2024-05-09T08:02:34.145 INFO:teuthology.orchestra.run.smithi001.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-05-09T08:02:34.145 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T08:02:34.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T08:02:34.156 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-05-09T08:02:34.156 DEBUG:teuthology.orchestra.run.smithi001:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T08:02:34.211 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T08:02:34.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:34.407+0000 7fdeff600640 1 Processor -- start 2024-05-09T08:02:34.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:34.407+0000 7fdeff600640 1 -- start start 2024-05-09T08:02:34.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:34.409+0000 7fdeff600640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fdef806d210 0x7fdef815de40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:34.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:34.409+0000 7fdeff600640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fdef80c7d10 0x7fdef815e380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:34.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:34.409+0000 7fdeff600640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdef815a6f0 con 0x7fdef8154730 2024-05-09T08:02:34.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:34.409+0000 7fdeff600640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7fdef80c8ca0 con 0x7fdef806d210 2024-05-09T08:02:34.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:34.409+0000 7fdeff600640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fdef815a490 con 0x7fdef80c7d10 2024-05-09T08:02:34.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:34.409+0000 7fdeefe00640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fdef806d210 0x7fdef815de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:34.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:34.410+0000 7fdefd600640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fdef80c7d10 0x7fdef815e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:34.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:34.410+0000 7fdefd600640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fdef80c7d10 0x7fdef815e380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.1:47004/0 (socket says 172.21.15.1:47004) 2024-05-09T08:02:34.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:34.410+0000 7fdefd600640 1 -- 172.21.15.1:0/2574755133 learned_addr learned my addr 172.21.15.1:0/2574755133 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:02:34.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:34.410+0000 7fdeefe00640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fdef806d210 0x7fdef815de40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56338/0 (socket says 172.21.15.1:56338) 2024-05-09T08:02:34.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:34.410+0000 7fdefd600640 1 -- 172.21.15.1:0/2574755133 >> v2:172.21.15.1:3300/0 conn(0x7fdef806d210 msgr2=0x7fdef815de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:02:34.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:34.410+0000 7fdefd600640 1 --2- 172.21.15.1:0/2574755133 >> v2:172.21.15.1:3300/0 conn(0x7fdef806d210 0x7fdef815de40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:34.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:34.410+0000 7fdefd600640 1 -- 172.21.15.1:0/2574755133 >> v1:172.21.15.1:6789/0 conn(0x7fdef8154730 legacy=0x7fdef80ccb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:34.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:34.410+0000 7fdefd600640 1 -- 172.21.15.1:0/2574755133 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdef815a990 con 0x7fdef80c7d10 2024-05-09T08:02:34.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:34.411+0000 7fdefd600640 1 --2- 172.21.15.1:0/2574755133 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdef80c7d10 0x7fdef815e380 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fdef80c8e20 tx=0x7fdef002a7c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e2fef033929f845d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:34.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:34.412+0000 7fdef7e00640 1 -- 172.21.15.1:0/2574755133 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdef0036070 con 0x7fdef80c7d10 2024-05-09T08:02:34.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:34.412+0000 7fdef7e00640 1 -- 172.21.15.1:0/2574755133 <== mon.1 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdef0031600 con 0x7fdef80c7d10 2024-05-09T08:02:34.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:34.412+0000 7fdef7e00640 1 -- 172.21.15.1:0/2574755133 <== mon.1 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdef00319f0 con 0x7fdef80c7d10 2024-05-09T08:02:34.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:34.412+0000 7fdeff600640 1 -- 172.21.15.1:0/2574755133 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdef80c7d10 msgr2=0x7fdef815e380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:34.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:34.412+0000 7fdeff600640 1 --2- 172.21.15.1:0/2574755133 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdef80c7d10 0x7fdef815e380 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fdef80c8e20 tx=0x7fdef002a7c0 comp rx=0 tx=0).stop 2024-05-09T08:02:34.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:34.413+0000 7fdeff600640 1 -- 172.21.15.1:0/2574755133 shutdown_connections 2024-05-09T08:02:34.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:34.413+0000 7fdeff600640 1 --2- 172.21.15.1:0/2574755133 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdef80c7d10 0x7fdef815e380 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:34.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:34.413+0000 7fdeff600640 1 --2- 172.21.15.1:0/2574755133 >> v2:172.21.15.1:3300/0 conn(0x7fdef806d210 0x7fdef815de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:34.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:34.413+0000 7fdeff600640 1 -- 172.21.15.1:0/2574755133 >> 172.21.15.1:0/2574755133 conn(0x7fdef8067500 msgr2=0x7fdef8072ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:02:34.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:34.413+0000 7fdeff600640 1 -- 172.21.15.1:0/2574755133 shutdown_connections 2024-05-09T08:02:34.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:34.413+0000 7fdeff600640 1 -- 172.21.15.1:0/2574755133 wait complete. 2024-05-09T08:02:34.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:34.414+0000 7fdeff600640 1 Processor -- start 2024-05-09T08:02:34.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:34.415+0000 7fdeff600640 1 -- start start 2024-05-09T08:02:34.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:34.415+0000 7fdeff600640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fdef806d210 0x7fdef81f0d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:34.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:34.415+0000 7fdeff600640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdef80c7d10 0x7fdef8152c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:34.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:34.415+0000 7fdeff600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdef8154730 0x7fdef814fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:34.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:34.415+0000 7fdeff600640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fdef815b230 con 0x7fdef80c7d10 2024-05-09T08:02:34.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:34.415+0000 7fdeff600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fdef815af50 con 0x7fdef8154730 2024-05-09T08:02:34.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:34.415+0000 7fdeff600640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7fdef8154b10 con 0x7fdef806d210 2024-05-09T08:02:34.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:34.416+0000 7fdeefe00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdef80c7d10 0x7fdef8152c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:34.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:34.416+0000 7fdefcc00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fdef806d210 0x7fdef81f0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:34.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:34.416+0000 7fdefd600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdef8154730 0x7fdef814fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:34.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:34.416+0000 7fdefd600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdef8154730 0x7fdef814fc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.1:47016/0 (socket says 172.21.15.1:47016) 2024-05-09T08:02:34.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:34.416+0000 7fdefd600640 1 -- 172.21.15.1:0/1190872828 learned_addr learned my addr 172.21.15.1:0/1190872828 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:02:34.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:34.416+0000 7fdefcc00640 1 --2- 172.21.15.1:0/1190872828 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fdef806d210 0x7fdef81f0d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3301/0 says I am v2:172.21.15.1:40016/0 (socket says 172.21.15.1:40016) 2024-05-09T08:02:34.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:34.416+0000 7fdefd600640 1 -- 172.21.15.1:0/1190872828 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fdef806d210 msgr2=0x7fdef81f0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:34.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:34.416+0000 7fdefd600640 1 --2- 172.21.15.1:0/1190872828 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fdef806d210 0x7fdef81f0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:34.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:34.416+0000 7fdefd600640 1 -- 172.21.15.1:0/1190872828 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdef80c7d10 msgr2=0x7fdef8152c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:34.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:34.416+0000 7fdefd600640 1 --2- 172.21.15.1:0/1190872828 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdef80c7d10 0x7fdef8152c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:34.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:34.416+0000 7fdefd600640 1 -- 172.21.15.1:0/1190872828 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdef81f1270 con 0x7fdef8154730 2024-05-09T08:02:34.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:34.416+0000 7fdeefe00640 1 --2- 172.21.15.1:0/1190872828 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdef80c7d10 0x7fdef8152c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:02:34.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:34.417+0000 7fdefd600640 1 --2- 172.21.15.1:0/1190872828 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdef8154730 0x7fdef814fc90 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fdef0009f10 tx=0x7fdef0004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:34.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:34.417+0000 7fdef5600640 1 -- 172.21.15.1:0/1190872828 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdef0036050 con 0x7fdef8154730 2024-05-09T08:02:34.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:34.417+0000 7fdef5600640 1 -- 172.21.15.1:0/1190872828 <== mon.1 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdef0042070 con 0x7fdef8154730 2024-05-09T08:02:34.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:34.417+0000 7fdef5600640 1 -- 172.21.15.1:0/1190872828 <== mon.1 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdef0009510 con 0x7fdef8154730 2024-05-09T08:02:34.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:34.417+0000 7fdeff600640 1 -- 172.21.15.1:0/1190872828 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdef8150600 con 0x7fdef8154730 2024-05-09T08:02:34.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:34.417+0000 7fdeff600640 1 -- 172.21.15.1:0/1190872828 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdef81531d0 con 0x7fdef8154730 2024-05-09T08:02:34.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:34.418+0000 7fdeff600640 1 -- 172.21.15.1:0/1190872828 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdef8072ab0 con 0x7fdef8154730 2024-05-09T08:02:34.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:34.419+0000 7fdef5600640 1 -- 172.21.15.1:0/1190872828 <== mon.1 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fdef0004470 con 0x7fdef8154730 2024-05-09T08:02:34.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:34.420+0000 7fdef5600640 1 --2- 172.21.15.1:0/1190872828 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fded806beb0 0x7fded806e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:34.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:34.420+0000 7fdef5600640 1 -- 172.21.15.1:0/1190872828 <== mon.1 v2:172.21.15.7:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fdef00abea0 con 0x7fdef8154730 2024-05-09T08:02:34.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:34.420+0000 7fdefcc00640 1 --2- 172.21.15.1:0/1190872828 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fded806beb0 0x7fded806e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:34.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:34.421+0000 7fdefcc00640 1 --2- 172.21.15.1:0/1190872828 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fded806beb0 0x7fded806e370 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fdee80016e0 tx=0x7fdee8009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:34.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:34.425+0000 7fdef5600640 1 -- 172.21.15.1:0/1190872828 <== mon.1 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fdef8072ab0 con 0x7fdef8154730 2024-05-09T08:02:34.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:34.628+0000 7fdeff600640 1 -- 172.21.15.1:0/1190872828 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdef80c88c0 con 0x7fdef8154730 2024-05-09T08:02:34.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:34.630+0000 7fdef5600640 1 -- 172.21.15.1:0/1190872828 <== mon.1 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18382 (secure 0 0 0) 0x7fdef0079ea0 con 0x7fdef8154730 2024-05-09T08:02:34.648 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T08:02:34.648 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":25,"fsid":"54329ec8-3270-4043-9a82-c6cd7420aa7f","created":"2024-05-09T08:01:56.666970+0000","modified":"2024-05-09T08:02:20.403456+0000","last_up_change":"2024-05-09T08:02:07.146082+0000","last_in_change":"2024-05-09T08:01:59.008225+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T08:02:09.165511+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T08:02:12.925485+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T08:02:14.367455+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-09T08:02:17.967064+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c064454e-a9b5-4a24-9a77-17f0a4085952","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6800","nonce":1513349547},{"type":"v1","addr":"172.21.15.1:6801","nonce":1513349547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":1513349547},{"type":"v1","addr":"172.21.15.1:6803","nonce":1513349547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":1513349547},{"type":"v1","addr":"172.21.15.1:6807","nonce":1513349547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":1513349547},{"type":"v1","addr":"172.21.15.1:6805","nonce":1513349547}]},"public_addr":"172.21.15.1:6801/1513349547","cluster_addr":"172.21.15.1:6803/1513349547","heartbeat_back_addr":"172.21.15.1:6807/1513349547","heartbeat_front_addr":"172.21.15.1:6805/1513349547","state":["exists","up"]},{"osd":1,"uuid":"1a9ea687-2ff6-407a-949b-0e7e1083af21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":2942255259},{"type":"v1","addr":"172.21.15.1:6817","nonce":2942255259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":2942255259},{"type":"v1","addr":"172.21.15.1:6819","nonce":2942255259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":2942255259},{"type":"v1","addr":"172.21.15.1:6823","nonce":2942255259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6820","nonce":2942255259},{"type":"v1","addr":"172.21.15.1:6821","nonce":2942255259}]},"public_addr":"172.21.15.1:6817/2942255259","cluster_addr":"172.21.15.1:6819/2942255259","heartbeat_back_addr":"172.21.15.1:6823/2942255259","heartbeat_front_addr":"172.21.15.1:6821/2942255259","state":["exists","up"]},{"osd":2,"uuid":"c88af08b-7c9a-4dd8-9640-44138239addb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":2925031811},{"type":"v1","addr":"172.21.15.1:6809","nonce":2925031811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":2925031811},{"type":"v1","addr":"172.21.15.1:6811","nonce":2925031811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":2925031811},{"type":"v1","addr":"172.21.15.1:6815","nonce":2925031811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":2925031811},{"type":"v1","addr":"172.21.15.1:6813","nonce":2925031811}]},"public_addr":"172.21.15.1:6809/2925031811","cluster_addr":"172.21.15.1:6811/2925031811","heartbeat_back_addr":"172.21.15.1:6815/2925031811","heartbeat_front_addr":"172.21.15.1:6813/2925031811","state":["exists","up"]},{"osd":3,"uuid":"c087b7e3-a41d-4333-9365-32e3bd8c8e6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6824","nonce":2497776654},{"type":"v1","addr":"172.21.15.1:6825","nonce":2497776654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":2497776654},{"type":"v1","addr":"172.21.15.1:6827","nonce":2497776654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":2497776654},{"type":"v1","addr":"172.21.15.1:6831","nonce":2497776654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6828","nonce":2497776654},{"type":"v1","addr":"172.21.15.1:6829","nonce":2497776654}]},"public_addr":"172.21.15.1:6825/2497776654","cluster_addr":"172.21.15.1:6827/2497776654","heartbeat_back_addr":"172.21.15.1:6831/2497776654","heartbeat_front_addr":"172.21.15.1:6829/2497776654","state":["exists","up"]},{"osd":4,"uuid":"b4576b9c-871d-47a8-8d36-65af525d0200","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":2957026670},{"type":"v1","addr":"172.21.15.7:6801","nonce":2957026670}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2957026670},{"type":"v1","addr":"172.21.15.7:6803","nonce":2957026670}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":2957026670},{"type":"v1","addr":"172.21.15.7:6807","nonce":2957026670}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2957026670},{"type":"v1","addr":"172.21.15.7:6805","nonce":2957026670}]},"public_addr":"172.21.15.7:6801/2957026670","cluster_addr":"172.21.15.7:6803/2957026670","heartbeat_back_addr":"172.21.15.7:6807/2957026670","heartbeat_front_addr":"172.21.15.7:6805/2957026670","state":["exists","up"]},{"osd":5,"uuid":"902b3d94-72f3-4499-b21a-e32453ef5144","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1670295242},{"type":"v1","addr":"172.21.15.7:6809","nonce":1670295242}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1670295242},{"type":"v1","addr":"172.21.15.7:6811","nonce":1670295242}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1670295242},{"type":"v1","addr":"172.21.15.7:6815","nonce":1670295242}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1670295242},{"type":"v1","addr":"172.21.15.7:6813","nonce":1670295242}]},"public_addr":"172.21.15.7:6809/1670295242","cluster_addr":"172.21.15.7:6811/1670295242","heartbeat_back_addr":"172.21.15.7:6815/1670295242","heartbeat_front_addr":"172.21.15.7:6813/1670295242","state":["exists","up"]},{"osd":6,"uuid":"ead1d771-1728-4f85-adb4-26be21cedff3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":2852086880},{"type":"v1","addr":"172.21.15.7:6817","nonce":2852086880}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":2852086880},{"type":"v1","addr":"172.21.15.7:6819","nonce":2852086880}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":2852086880},{"type":"v1","addr":"172.21.15.7:6823","nonce":2852086880}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":2852086880},{"type":"v1","addr":"172.21.15.7:6821","nonce":2852086880}]},"public_addr":"172.21.15.7:6817/2852086880","cluster_addr":"172.21.15.7:6819/2852086880","heartbeat_back_addr":"172.21.15.7:6823/2852086880","heartbeat_front_addr":"172.21.15.7:6821/2852086880","state":["exists","up"]},{"osd":7,"uuid":"3ab5a201-9394-48a8-8918-e95197893669","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":1442884138},{"type":"v1","addr":"172.21.15.7:6825","nonce":1442884138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1442884138},{"type":"v1","addr":"172.21.15.7:6827","nonce":1442884138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1442884138},{"type":"v1","addr":"172.21.15.7:6831","nonce":1442884138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1442884138},{"type":"v1","addr":"172.21.15.7:6829","nonce":1442884138}]},"public_addr":"172.21.15.7:6825/1442884138","cluster_addr":"172.21.15.7:6827/1442884138","heartbeat_back_addr":"172.21.15.7:6831/1442884138","heartbeat_front_addr":"172.21.15.7:6829/1442884138","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:02:04.615675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:02:04.385974+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:02:04.619217+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:02:04.401681+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:02:04.097081+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:02:04.129981+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:02:04.225327+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:02:04.193351+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T08:02:34.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:34.633+0000 7fdeff600640 1 -- 172.21.15.1:0/1190872828 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fded806beb0 msgr2=0x7fded806e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:34.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:34.633+0000 7fdeff600640 1 --2- 172.21.15.1:0/1190872828 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fded806beb0 0x7fded806e370 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fdee80016e0 tx=0x7fdee8009040 comp rx=0 tx=0).stop 2024-05-09T08:02:34.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:34.633+0000 7fdeff600640 1 -- 172.21.15.1:0/1190872828 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdef8154730 msgr2=0x7fdef814fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:34.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:34.633+0000 7fdeff600640 1 --2- 172.21.15.1:0/1190872828 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdef8154730 0x7fdef814fc90 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fdef0009f10 tx=0x7fdef0004a40 comp rx=0 tx=0).stop 2024-05-09T08:02:34.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:34.633+0000 7fdeff600640 1 -- 172.21.15.1:0/1190872828 shutdown_connections 2024-05-09T08:02:34.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:34.633+0000 7fdeff600640 1 --2- 172.21.15.1:0/1190872828 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fded806beb0 0x7fded806e370 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:34.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:34.633+0000 7fdeff600640 1 --2- 172.21.15.1:0/1190872828 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdef8154730 0x7fdef814fc90 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:34.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:34.633+0000 7fdeff600640 1 --2- 172.21.15.1:0/1190872828 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdef80c7d10 0x7fdef8152c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:34.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:34.633+0000 7fdeff600640 1 --2- 172.21.15.1:0/1190872828 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fdef806d210 0x7fdef81f0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:34.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:34.633+0000 7fdeff600640 1 -- 172.21.15.1:0/1190872828 >> 172.21.15.1:0/1190872828 conn(0x7fdef8067500 msgr2=0x7fdef81590f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:02:34.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:34.633+0000 7fdeff600640 1 -- 172.21.15.1:0/1190872828 shutdown_connections 2024-05-09T08:02:34.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:34.633+0000 7fdeff600640 1 -- 172.21.15.1:0/1190872828 wait complete. 2024-05-09T08:02:34.670 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T08:02:09.165511+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-09T08:02:12.925485+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-09T08:02:14.367455+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-09T08:02:17.967064+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-09T08:02:34.671 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-09T08:02:34.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:34.862+0000 7f25eca00640 1 Processor -- start 2024-05-09T08:02:34.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:34.862+0000 7f25eca00640 1 -- start start 2024-05-09T08:02:34.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:34.864+0000 7f25eca00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f25e80719e0 0x7f25e815b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:34.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:34.864+0000 7f25eca00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f25e8071dc0 0x7f25e80c8740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:34.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:34.864+0000 7f25eca00640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f25e806c7f0 0x7f25e806cc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:34.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:34.864+0000 7f25eca00640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f25e806d1c0 con 0x7f25e806c7f0 2024-05-09T08:02:34.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:34.864+0000 7f25eca00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f25e815dc90 con 0x7f25e80719e0 2024-05-09T08:02:34.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:34.864+0000 7f25eca00640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f25e815df10 con 0x7f25e8071dc0 2024-05-09T08:02:34.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:34.864+0000 7f25e6800640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f25e806c7f0 0x7f25e806cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:34.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:34.864+0000 7f25e5e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f25e80719e0 0x7f25e815b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:34.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:34.864+0000 7f25e5400640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f25e8071dc0 0x7f25e80c8740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:34.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:34.864+0000 7f25e6800640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f25e806c7f0 0x7f25e806cc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56350/0 (socket says 172.21.15.1:56350) 2024-05-09T08:02:34.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:34.864+0000 7f25e6800640 1 -- 172.21.15.1:0/2273583475 learned_addr learned my addr 172.21.15.1:0/2273583475 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:02:34.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:34.865+0000 7f25e5e00640 1 -- 172.21.15.1:0/2273583475 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f25e8071dc0 msgr2=0x7f25e80c8740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:02:34.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:34.865+0000 7f25e5e00640 1 --2- 172.21.15.1:0/2273583475 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f25e8071dc0 0x7f25e80c8740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:34.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:34.865+0000 7f25e5e00640 1 -- 172.21.15.1:0/2273583475 >> v2:172.21.15.1:3300/0 conn(0x7f25e806c7f0 msgr2=0x7f25e806cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:02:34.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:34.865+0000 7f25e5e00640 1 --2- 172.21.15.1:0/2273583475 >> v2:172.21.15.1:3300/0 conn(0x7f25e806c7f0 0x7f25e806cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:34.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:34.865+0000 7f25e5e00640 1 -- 172.21.15.1:0/2273583475 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25e815e1f0 con 0x7f25e80719e0 2024-05-09T08:02:34.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:34.865+0000 7f25e5400640 1 --2- 172.21.15.1:0/2273583475 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f25e8071dc0 0x7f25e80c8740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:02:34.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:34.865+0000 7f25e6800640 1 --2- 172.21.15.1:0/2273583475 >> v2:172.21.15.1:3300/0 conn(0x7f25e806c7f0 0x7f25e806cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:02:34.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:34.866+0000 7f25e5e00640 1 --2- 172.21.15.1:0/2273583475 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f25e80719e0 0x7f25e815b2b0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f25e80c8400 tx=0x7f25d802a1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6f4b638e3fbe2b73 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:34.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:34.866+0000 7f25e4a00640 1 -- 172.21.15.1:0/2273583475 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25d8037070 con 0x7f25e80719e0 2024-05-09T08:02:34.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:34.866+0000 7f25e4a00640 1 -- 172.21.15.1:0/2273583475 <== mon.1 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25d8032440 con 0x7f25e80719e0 2024-05-09T08:02:34.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:34.866+0000 7f25e4a00640 1 -- 172.21.15.1:0/2273583475 <== mon.1 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25d8032810 con 0x7f25e80719e0 2024-05-09T08:02:34.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:34.867+0000 7f25eca00640 1 -- 172.21.15.1:0/2273583475 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f25e80719e0 msgr2=0x7f25e815b2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:34.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:34.867+0000 7f25eca00640 1 --2- 172.21.15.1:0/2273583475 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f25e80719e0 0x7f25e815b2b0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f25e80c8400 tx=0x7f25d802a1a0 comp rx=0 tx=0).stop 2024-05-09T08:02:34.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:34.867+0000 7f25eca00640 1 -- 172.21.15.1:0/2273583475 shutdown_connections 2024-05-09T08:02:34.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:34.867+0000 7f25eca00640 1 --2- 172.21.15.1:0/2273583475 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f25e8071dc0 0x7f25e80c8740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:34.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:34.867+0000 7f25eca00640 1 --2- 172.21.15.1:0/2273583475 >> v2:172.21.15.1:3300/0 conn(0x7f25e806c7f0 0x7f25e806cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:34.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:34.867+0000 7f25eca00640 1 --2- 172.21.15.1:0/2273583475 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f25e80719e0 0x7f25e815b2b0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:34.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:34.867+0000 7f25eca00640 1 -- 172.21.15.1:0/2273583475 >> 172.21.15.1:0/2273583475 conn(0x7f25e806b880 msgr2=0x7f25e806fb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:02:34.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:34.867+0000 7f25eca00640 1 -- 172.21.15.1:0/2273583475 shutdown_connections 2024-05-09T08:02:34.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:34.867+0000 7f25eca00640 1 -- 172.21.15.1:0/2273583475 wait complete. 2024-05-09T08:02:34.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:34.869+0000 7f25eca00640 1 Processor -- start 2024-05-09T08:02:34.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:34.869+0000 7f25eca00640 1 -- start start 2024-05-09T08:02:34.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:34.870+0000 7f25eca00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f25e806c7f0 0x7f25e80c7780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:34.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:34.870+0000 7f25eca00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f25e80719e0 0x7f25e8152170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:34.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:34.870+0000 7f25eca00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f25e8071dc0 0x7f25e80c5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:34.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:34.870+0000 7f25eca00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f25e815ea90 con 0x7f25e806c7f0 2024-05-09T08:02:34.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:34.870+0000 7f25eca00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f25e815e7b0 con 0x7f25e80719e0 2024-05-09T08:02:34.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:34.870+0000 7f25eca00640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f25e80c8c80 con 0x7f25e8071dc0 2024-05-09T08:02:34.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:34.870+0000 7f25e5e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f25e806c7f0 0x7f25e80c7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:34.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:34.870+0000 7f25e5e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f25e806c7f0 0x7f25e80c7780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56356/0 (socket says 172.21.15.1:56356) 2024-05-09T08:02:34.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:34.870+0000 7f25e5e00640 1 -- 172.21.15.1:0/3502645357 learned_addr learned my addr 172.21.15.1:0/3502645357 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:02:34.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:34.870+0000 7f25e6800640 1 --2- 172.21.15.1:0/3502645357 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f25e8071dc0 0x7f25e80c5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:34.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:34.870+0000 7f25e5400640 1 --2- 172.21.15.1:0/3502645357 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f25e80719e0 0x7f25e8152170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:34.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:34.870+0000 7f25e5e00640 1 -- 172.21.15.1:0/3502645357 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f25e8071dc0 msgr2=0x7f25e80c5920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:34.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:34.870+0000 7f25e5e00640 1 --2- 172.21.15.1:0/3502645357 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f25e8071dc0 0x7f25e80c5920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:34.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:34.870+0000 7f25e5e00640 1 -- 172.21.15.1:0/3502645357 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f25e80719e0 msgr2=0x7f25e8152170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:34.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:34.870+0000 7f25e5e00640 1 --2- 172.21.15.1:0/3502645357 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f25e80719e0 0x7f25e8152170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:34.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:34.870+0000 7f25e5e00640 1 -- 172.21.15.1:0/3502645357 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25e81eff60 con 0x7f25e806c7f0 2024-05-09T08:02:34.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:34.871+0000 7f25e5e00640 1 --2- 172.21.15.1:0/3502645357 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f25e806c7f0 0x7f25e80c7780 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f25d8000c00 tx=0x7f25d80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:34.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:34.871+0000 7f25de000640 1 -- 172.21.15.1:0/3502645357 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25d8041020 con 0x7f25e806c7f0 2024-05-09T08:02:34.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:34.871+0000 7f25de000640 1 -- 172.21.15.1:0/3502645357 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25d803d700 con 0x7f25e806c7f0 2024-05-09T08:02:34.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:34.871+0000 7f25de000640 1 -- 172.21.15.1:0/3502645357 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25d8037040 con 0x7f25e806c7f0 2024-05-09T08:02:34.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:34.871+0000 7f25eca00640 1 -- 172.21.15.1:0/3502645357 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25e81f3f50 con 0x7f25e806c7f0 2024-05-09T08:02:34.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:34.871+0000 7f25eca00640 1 -- 172.21.15.1:0/3502645357 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25e81f04e0 con 0x7f25e806c7f0 2024-05-09T08:02:34.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:34.872+0000 7f25eca00640 1 -- 172.21.15.1:0/3502645357 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25e8074d70 con 0x7f25e806c7f0 2024-05-09T08:02:34.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:34.874+0000 7f25de000640 1 -- 172.21.15.1:0/3502645357 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f25d8044020 con 0x7f25e806c7f0 2024-05-09T08:02:34.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:34.874+0000 7f25de000640 1 --2- 172.21.15.1:0/3502645357 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f25c806be30 0x7f25c806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:34.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:34.875+0000 7f25de000640 1 -- 172.21.15.1:0/3502645357 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f25d80ab8a0 con 0x7f25e806c7f0 2024-05-09T08:02:34.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:34.875+0000 7f25e5400640 1 --2- 172.21.15.1:0/3502645357 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f25c806be30 0x7f25c806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:34.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:34.875+0000 7f25e5400640 1 --2- 172.21.15.1:0/3502645357 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f25c806be30 0x7f25c806e2f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f25d4009bd0 tx=0x7f25d400b040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:34.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:34.880+0000 7f25de000640 1 -- 172.21.15.1:0/3502645357 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f25d804b030 con 0x7f25e806c7f0 2024-05-09T08:02:35.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:35.085+0000 7f25eca00640 1 -- 172.21.15.1:0/3502645357 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f25e80c5700 con 0x7f25e806c7f0 2024-05-09T08:02:35.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:35.087+0000 7f25de000640 1 -- 172.21.15.1:0/3502645357 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) ==== 93+0+10 (secure 0 0 0) 0x7f25d8079830 con 0x7f25e806c7f0 2024-05-09T08:02:35.104 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 1 2024-05-09T08:02:35.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:35.089+0000 7f25eca00640 1 -- 172.21.15.1:0/3502645357 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f25c806be30 msgr2=0x7f25c806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:35.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:35.089+0000 7f25eca00640 1 --2- 172.21.15.1:0/3502645357 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f25c806be30 0x7f25c806e2f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f25d4009bd0 tx=0x7f25d400b040 comp rx=0 tx=0).stop 2024-05-09T08:02:35.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:35.090+0000 7f25eca00640 1 -- 172.21.15.1:0/3502645357 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f25e806c7f0 msgr2=0x7f25e80c7780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:35.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:35.090+0000 7f25eca00640 1 --2- 172.21.15.1:0/3502645357 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f25e806c7f0 0x7f25e80c7780 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f25d8000c00 tx=0x7f25d80046c0 comp rx=0 tx=0).stop 2024-05-09T08:02:35.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:35.090+0000 7f25eca00640 1 -- 172.21.15.1:0/3502645357 shutdown_connections 2024-05-09T08:02:35.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:35.090+0000 7f25eca00640 1 --2- 172.21.15.1:0/3502645357 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f25c806be30 0x7f25c806e2f0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:35.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:35.090+0000 7f25eca00640 1 --2- 172.21.15.1:0/3502645357 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f25e80719e0 0x7f25e8152170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:35.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:35.090+0000 7f25eca00640 1 --2- 172.21.15.1:0/3502645357 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f25e8071dc0 0x7f25e80c5920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:35.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:35.090+0000 7f25eca00640 1 --2- 172.21.15.1:0/3502645357 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f25e806c7f0 0x7f25e80c7780 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:35.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:35.090+0000 7f25eca00640 1 -- 172.21.15.1:0/3502645357 >> 172.21.15.1:0/3502645357 conn(0x7f25e806b880 msgr2=0x7f25e815d330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:02:35.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:35.090+0000 7f25eca00640 1 -- 172.21.15.1:0/3502645357 shutdown_connections 2024-05-09T08:02:35.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:35.090+0000 7f25eca00640 1 -- 172.21.15.1:0/3502645357 wait complete. 2024-05-09T08:02:35.126 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-09T08:02:35.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:35.325+0000 7f3273200640 1 Processor -- start 2024-05-09T08:02:35.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:35.326+0000 7f3273200640 1 -- start start 2024-05-09T08:02:35.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:35.327+0000 7f3273200640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f326c0c9620 0x7f326c0c8850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:35.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:35.328+0000 7f3273200640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f326c0c8d90 0x7f326c15a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:35.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:35.328+0000 7f3273200640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f326c15ae00 con 0x7f326c1546f0 2024-05-09T08:02:35.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:35.328+0000 7f3273200640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f326c0ca870 con 0x7f326c0c9620 2024-05-09T08:02:35.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:35.328+0000 7f3273200640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f326c15b0e0 con 0x7f326c0c8d90 2024-05-09T08:02:35.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:35.328+0000 7f3271400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f326c0c9620 0x7f326c0c8850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:35.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:35.328+0000 7f3271e00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f326c1546f0 0x7f326c0ca160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.1:40444/0 (socket says 172.21.15.1:40444) 2024-05-09T08:02:35.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:35.328+0000 7f3271e00640 1 -- 172.21.15.1:0/279542799 learned_addr learned my addr 172.21.15.1:0/279542799 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T08:02:35.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:35.328+0000 7f3272800640 1 --2- 172.21.15.1:0/279542799 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f326c0c8d90 0x7f326c15a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:35.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:35.328+0000 7f3271400640 1 -- 172.21.15.1:0/279542799 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f326c0c8d90 msgr2=0x7f326c15a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:02:35.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:35.328+0000 7f3271400640 1 --2- 172.21.15.1:0/279542799 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f326c0c8d90 0x7f326c15a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:35.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:35.328+0000 7f3271400640 1 -- 172.21.15.1:0/279542799 >> v1:172.21.15.7:6789/0 conn(0x7f326c1546f0 legacy=0x7f326c0ca160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:35.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:35.328+0000 7f3271400640 1 -- 172.21.15.1:0/279542799 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f326c15db20 con 0x7f326c0c9620 2024-05-09T08:02:35.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:35.328+0000 7f3270a00640 1 -- 172.21.15.1:0/279542799 <== mon.1 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 401619511 0 0) 0x7f326c15ae00 con 0x7f326c1546f0 2024-05-09T08:02:35.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:35.329+0000 7f3271400640 1 --2- 172.21.15.1:0/279542799 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f326c0c9620 0x7f326c0c8850 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f325c0097a0 tx=0x7f325c02a210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cdee36f46cfd9f0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:35.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:35.330+0000 7f3270a00640 1 -- 172.21.15.1:0/279542799 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f325c037070 con 0x7f326c0c9620 2024-05-09T08:02:35.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:35.330+0000 7f3270a00640 1 -- 172.21.15.1:0/279542799 <== mon.1 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f325c032440 con 0x7f326c0c9620 2024-05-09T08:02:35.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:35.330+0000 7f3270a00640 1 -- 172.21.15.1:0/279542799 <== mon.1 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f325c032810 con 0x7f326c0c9620 2024-05-09T08:02:35.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:35.330+0000 7f3273200640 1 -- 172.21.15.1:0/279542799 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f326c0c9620 msgr2=0x7f326c0c8850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:35.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:35.330+0000 7f3273200640 1 --2- 172.21.15.1:0/279542799 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f326c0c9620 0x7f326c0c8850 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f325c0097a0 tx=0x7f325c02a210 comp rx=0 tx=0).stop 2024-05-09T08:02:35.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:35.331+0000 7f3273200640 1 -- 172.21.15.1:0/279542799 shutdown_connections 2024-05-09T08:02:35.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:35.331+0000 7f3273200640 1 --2- 172.21.15.1:0/279542799 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f326c0c8d90 0x7f326c15a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:35.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:35.331+0000 7f3273200640 1 --2- 172.21.15.1:0/279542799 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f326c0c9620 0x7f326c0c8850 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:35.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:35.331+0000 7f3273200640 1 -- 172.21.15.1:0/279542799 >> 172.21.15.1:0/279542799 conn(0x7f326c071620 msgr2=0x7f326c071a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:02:35.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:35.331+0000 7f3273200640 1 -- 172.21.15.1:0/279542799 shutdown_connections 2024-05-09T08:02:35.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:35.331+0000 7f3273200640 1 -- 172.21.15.1:0/279542799 wait complete. 2024-05-09T08:02:35.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:35.332+0000 7f3273200640 1 Processor -- start 2024-05-09T08:02:35.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:35.333+0000 7f3273200640 1 -- start start 2024-05-09T08:02:35.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:35.333+0000 7f3273200640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f326c0c8d90 0x7f326c0c6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:35.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:35.333+0000 7f3273200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f326c0c9620 0x7f326c0c6740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:35.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:35.333+0000 7f3273200640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f326c1546f0 0x7f326c153020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:35.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:35.333+0000 7f3273200640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f326c15d7f0 con 0x7f326c1546f0 2024-05-09T08:02:35.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:35.333+0000 7f3273200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f326c15e0e0 con 0x7f326c0c9620 2024-05-09T08:02:35.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:35.333+0000 7f3273200640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f326c15e600 con 0x7f326c0c8d90 2024-05-09T08:02:35.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:35.334+0000 7f3271e00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f326c0c8d90 0x7f326c0c6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:35.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:35.334+0000 7f3271400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f326c0c9620 0x7f326c0c6740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:35.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:35.334+0000 7f3272800640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f326c1546f0 0x7f326c153020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:35.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:35.334+0000 7f3271e00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f326c0c8d90 0x7f326c0c6200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3301/0 says I am v2:172.21.15.1:40064/0 (socket says 172.21.15.1:40064) 2024-05-09T08:02:35.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:35.334+0000 7f3271e00640 1 -- 172.21.15.1:0/4091490032 learned_addr learned my addr 172.21.15.1:0/4091490032 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:02:35.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:35.334+0000 7f3271e00640 1 -- 172.21.15.1:0/4091490032 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f326c0c9620 msgr2=0x7f326c0c6740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:35.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:35.334+0000 7f3271e00640 1 --2- 172.21.15.1:0/4091490032 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f326c0c9620 0x7f326c0c6740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:35.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:35.334+0000 7f3271e00640 1 -- 172.21.15.1:0/4091490032 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f326c1546f0 msgr2=0x7f326c153020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:35.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:35.334+0000 7f3271e00640 1 --2- 172.21.15.1:0/4091490032 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f326c1546f0 0x7f326c153020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:35.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:35.334+0000 7f3271e00640 1 -- 172.21.15.1:0/4091490032 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f326c0cc7d0 con 0x7f326c0c8d90 2024-05-09T08:02:35.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:35.334+0000 7f3271400640 1 --2- 172.21.15.1:0/4091490032 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f326c0c9620 0x7f326c0c6740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:02:35.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:35.335+0000 7f3271e00640 1 --2- 172.21.15.1:0/4091490032 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f326c0c8d90 0x7f326c0c6200 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f326000c420 tx=0x7f326000c8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:35.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:35.335+0000 7f3266000640 1 -- 172.21.15.1:0/4091490032 <== mon.2 v2:172.21.15.7:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32600074e0 con 0x7f326c0c8d90 2024-05-09T08:02:35.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:35.335+0000 7f3266000640 1 -- 172.21.15.1:0/4091490032 <== mon.2 v2:172.21.15.7:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32600047e0 con 0x7f326c0c8d90 2024-05-09T08:02:35.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:35.335+0000 7f3273200640 1 -- 172.21.15.1:0/4091490032 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f326c0ccab0 con 0x7f326c0c8d90 2024-05-09T08:02:35.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:35.335+0000 7f3266000640 1 -- 172.21.15.1:0/4091490032 <== mon.2 v2:172.21.15.7:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3260004c10 con 0x7f326c0c8d90 2024-05-09T08:02:35.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:35.335+0000 7f3273200640 1 -- 172.21.15.1:0/4091490032 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f326c0ce730 con 0x7f326c0c8d90 2024-05-09T08:02:35.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:35.336+0000 7f3273200640 1 -- 172.21.15.1:0/4091490032 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f326c04f660 con 0x7f326c0c8d90 2024-05-09T08:02:35.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:35.337+0000 7f3266000640 1 -- 172.21.15.1:0/4091490032 <== mon.2 v2:172.21.15.7:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3260011070 con 0x7f326c0c8d90 2024-05-09T08:02:35.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:35.338+0000 7f3266000640 1 --2- 172.21.15.1:0/4091490032 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f325006be30 0x7f325006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:35.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:35.338+0000 7f3266000640 1 -- 172.21.15.1:0/4091490032 <== mon.2 v2:172.21.15.7:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f326008ad30 con 0x7f326c0c8d90 2024-05-09T08:02:35.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:35.339+0000 7f3271400640 1 --2- 172.21.15.1:0/4091490032 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f325006be30 0x7f325006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:35.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:35.339+0000 7f3271400640 1 --2- 172.21.15.1:0/4091490032 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f325006be30 0x7f325006e2f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f325c034000 tx=0x7f325c02e000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:35.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:35.343+0000 7f3266000640 1 -- 172.21.15.1:0/4091490032 <== mon.2 v2:172.21.15.7:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f3260058cb0 con 0x7f326c0c8d90 2024-05-09T08:02:35.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:35.544+0000 7f3273200640 1 -- 172.21.15.1:0/4091490032 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f326c072300 con 0x7f326c0c8d90 2024-05-09T08:02:35.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:35.545+0000 7f3266000640 1 -- 172.21.15.1:0/4091490032 <== mon.2 v2:172.21.15.7:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) ==== 104+0+11 (secure 0 0 0) 0x7f326005cfb0 con 0x7f326c0c8d90 2024-05-09T08:02:35.561 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 32 2024-05-09T08:02:35.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:35.547+0000 7f3273200640 1 -- 172.21.15.1:0/4091490032 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f325006be30 msgr2=0x7f325006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:35.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:35.547+0000 7f3273200640 1 --2- 172.21.15.1:0/4091490032 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f325006be30 0x7f325006e2f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f325c034000 tx=0x7f325c02e000 comp rx=0 tx=0).stop 2024-05-09T08:02:35.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:35.547+0000 7f3273200640 1 -- 172.21.15.1:0/4091490032 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f326c0c8d90 msgr2=0x7f326c0c6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:35.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:35.547+0000 7f3273200640 1 --2- 172.21.15.1:0/4091490032 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f326c0c8d90 0x7f326c0c6200 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f326000c420 tx=0x7f326000c8e0 comp rx=0 tx=0).stop 2024-05-09T08:02:35.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:35.547+0000 7f3273200640 1 -- 172.21.15.1:0/4091490032 shutdown_connections 2024-05-09T08:02:35.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:35.547+0000 7f3273200640 1 --2- 172.21.15.1:0/4091490032 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f325006be30 0x7f325006e2f0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:35.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:35.547+0000 7f3273200640 1 --2- 172.21.15.1:0/4091490032 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f326c0c9620 0x7f326c0c6740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:35.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:35.547+0000 7f3273200640 1 --2- 172.21.15.1:0/4091490032 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f326c1546f0 0x7f326c153020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:35.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:35.547+0000 7f3273200640 1 --2- 172.21.15.1:0/4091490032 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f326c0c8d90 0x7f326c0c6200 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:35.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:35.547+0000 7f3273200640 1 -- 172.21.15.1:0/4091490032 >> 172.21.15.1:0/4091490032 conn(0x7f326c071620 msgr2=0x7f326c158a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:02:35.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:35.547+0000 7f3273200640 1 -- 172.21.15.1:0/4091490032 shutdown_connections 2024-05-09T08:02:35.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:35.548+0000 7f3273200640 1 -- 172.21.15.1:0/4091490032 wait complete. 2024-05-09T08:02:35.584 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-09T08:02:35.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:35.786+0000 7f2b17600640 1 Processor -- start 2024-05-09T08:02:35.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:35.786+0000 7f2b17600640 1 -- start start 2024-05-09T08:02:35.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:35.787+0000 7f2b17600640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f2b10154910 0x7f2b100c7d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:35.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:35.788+0000 7f2b17600640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f2b100c9750 0x7f2b1006f720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:35.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:35.788+0000 7f2b17600640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f2b10072890 0x7f2b1006fc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:35.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:35.788+0000 7f2b17600640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f2b10073750 con 0x7f2b10072890 2024-05-09T08:02:35.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:35.788+0000 7f2b17600640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f2b10073190 con 0x7f2b100c9750 2024-05-09T08:02:35.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:35.788+0000 7f2b17600640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f2b10073470 con 0x7f2b10154910 2024-05-09T08:02:35.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:35.788+0000 7f2b14c00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f2b10154910 0x7f2b100c7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:35.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:35.788+0000 7f2b14c00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f2b10154910 0x7f2b100c7d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3301/0 says I am v2:172.21.15.1:40078/0 (socket says 172.21.15.1:40078) 2024-05-09T08:02:35.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:35.788+0000 7f2b0fe00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f2b100c9750 0x7f2b1006f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:35.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:35.788+0000 7f2b14c00640 1 -- 172.21.15.1:0/3371519051 learned_addr learned my addr 172.21.15.1:0/3371519051 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:02:35.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:35.788+0000 7f2b15600640 1 --2- 172.21.15.1:0/3371519051 >> v2:172.21.15.1:3300/0 conn(0x7f2b10072890 0x7f2b1006fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:35.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:35.789+0000 7f2b14c00640 1 -- 172.21.15.1:0/3371519051 >> v2:172.21.15.7:3300/0 conn(0x7f2b100c9750 msgr2=0x7f2b1006f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:02:35.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:35.789+0000 7f2b14c00640 1 --2- 172.21.15.1:0/3371519051 >> v2:172.21.15.7:3300/0 conn(0x7f2b100c9750 0x7f2b1006f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:35.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:35.789+0000 7f2b14c00640 1 -- 172.21.15.1:0/3371519051 >> v2:172.21.15.1:3300/0 conn(0x7f2b10072890 msgr2=0x7f2b1006fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:02:35.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:35.789+0000 7f2b14c00640 1 --2- 172.21.15.1:0/3371519051 >> v2:172.21.15.1:3300/0 conn(0x7f2b10072890 0x7f2b1006fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:35.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:35.789+0000 7f2b14c00640 1 -- 172.21.15.1:0/3371519051 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b10071d90 con 0x7f2b10154910 2024-05-09T08:02:35.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:35.789+0000 7f2b0fe00640 1 --2- 172.21.15.1:0/3371519051 >> v2:172.21.15.7:3300/0 conn(0x7f2b100c9750 0x7f2b1006f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:02:35.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:35.790+0000 7f2b14c00640 1 --2- 172.21.15.1:0/3371519051 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f2b10154910 0x7f2b100c7d20 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f2b10070360 tx=0x7f2af802a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b541fc88294f5e6d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:35.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:35.790+0000 7f2b0f400640 1 -- 172.21.15.1:0/3371519051 <== mon.2 v2:172.21.15.7:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2af8037070 con 0x7f2b10154910 2024-05-09T08:02:35.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:35.790+0000 7f2b0f400640 1 -- 172.21.15.1:0/3371519051 <== mon.2 v2:172.21.15.7:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2af8032440 con 0x7f2b10154910 2024-05-09T08:02:35.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:35.790+0000 7f2b0f400640 1 -- 172.21.15.1:0/3371519051 <== mon.2 v2:172.21.15.7:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2af8032810 con 0x7f2b10154910 2024-05-09T08:02:35.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:35.791+0000 7f2b17600640 1 -- 172.21.15.1:0/3371519051 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f2b10154910 msgr2=0x7f2b100c7d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:35.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:35.791+0000 7f2b17600640 1 --2- 172.21.15.1:0/3371519051 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f2b10154910 0x7f2b100c7d20 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f2b10070360 tx=0x7f2af802a270 comp rx=0 tx=0).stop 2024-05-09T08:02:35.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:35.791+0000 7f2b17600640 1 -- 172.21.15.1:0/3371519051 shutdown_connections 2024-05-09T08:02:35.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:35.791+0000 7f2b17600640 1 --2- 172.21.15.1:0/3371519051 >> v2:172.21.15.1:3300/0 conn(0x7f2b10072890 0x7f2b1006fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:35.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:35.791+0000 7f2b17600640 1 --2- 172.21.15.1:0/3371519051 >> v2:172.21.15.7:3300/0 conn(0x7f2b100c9750 0x7f2b1006f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:35.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:35.791+0000 7f2b17600640 1 --2- 172.21.15.1:0/3371519051 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f2b10154910 0x7f2b100c7d20 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:35.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:35.791+0000 7f2b17600640 1 -- 172.21.15.1:0/3371519051 >> 172.21.15.1:0/3371519051 conn(0x7f2b1006dad0 msgr2=0x7f2b1006deb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:02:35.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:35.791+0000 7f2b17600640 1 -- 172.21.15.1:0/3371519051 shutdown_connections 2024-05-09T08:02:35.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:35.791+0000 7f2b17600640 1 -- 172.21.15.1:0/3371519051 wait complete. 2024-05-09T08:02:35.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:35.793+0000 7f2b17600640 1 Processor -- start 2024-05-09T08:02:35.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:35.793+0000 7f2b17600640 1 -- start start 2024-05-09T08:02:35.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:35.794+0000 7f2b17600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2b10072890 0x7f2b1014ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:35.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:35.794+0000 7f2b17600640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f2b100c9750 0x7f2b100c9ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:35.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:35.794+0000 7f2b17600640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2b10154910 0x7f2b100cbff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:35.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:35.794+0000 7f2b17600640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f2b1015aa00 con 0x7f2b10154910 2024-05-09T08:02:35.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:35.794+0000 7f2b17600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2b1015a880 con 0x7f2b10072890 2024-05-09T08:02:35.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:35.794+0000 7f2b17600640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f2b1015ab80 con 0x7f2b100c9750 2024-05-09T08:02:35.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:35.795+0000 7f2b14c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2b10072890 0x7f2b1014ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:35.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:35.795+0000 7f2b14c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2b10072890 0x7f2b1014ac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.1:47074/0 (socket says 172.21.15.1:47074) 2024-05-09T08:02:35.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:35.795+0000 7f2b14c00640 1 -- 172.21.15.1:0/183538463 learned_addr learned my addr 172.21.15.1:0/183538463 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:02:35.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:35.795+0000 7f2b0fe00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f2b100c9750 0x7f2b100c9ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:35.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:35.795+0000 7f2b15600640 1 --2- 172.21.15.1:0/183538463 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2b10154910 0x7f2b100cbff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:35.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:35.795+0000 7f2b14c00640 1 -- 172.21.15.1:0/183538463 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f2b100c9750 msgr2=0x7f2b100c9ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:35.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:35.795+0000 7f2b14c00640 1 --2- 172.21.15.1:0/183538463 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f2b100c9750 0x7f2b100c9ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:35.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:35.795+0000 7f2b14c00640 1 -- 172.21.15.1:0/183538463 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2b10154910 msgr2=0x7f2b100cbff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:35.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:35.795+0000 7f2b14c00640 1 --2- 172.21.15.1:0/183538463 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2b10154910 0x7f2b100cbff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:35.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:35.795+0000 7f2b14c00640 1 -- 172.21.15.1:0/183538463 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b10147760 con 0x7f2b10072890 2024-05-09T08:02:35.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:35.795+0000 7f2b14c00640 1 --2- 172.21.15.1:0/183538463 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2b10072890 0x7f2b1014ac10 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f2af802a2b0 tx=0x7f2af80046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:35.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:35.796+0000 7f2b0cc00640 1 -- 172.21.15.1:0/183538463 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2af8037040 con 0x7f2b10072890 2024-05-09T08:02:35.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:35.796+0000 7f2b0cc00640 1 -- 172.21.15.1:0/183538463 <== mon.1 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2af8042070 con 0x7f2b10072890 2024-05-09T08:02:35.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:35.796+0000 7f2b0cc00640 1 -- 172.21.15.1:0/183538463 <== mon.1 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2af8030880 con 0x7f2b10072890 2024-05-09T08:02:35.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:35.796+0000 7f2b17600640 1 -- 172.21.15.1:0/183538463 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b101579b0 con 0x7f2b10072890 2024-05-09T08:02:35.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:35.796+0000 7f2b17600640 1 -- 172.21.15.1:0/183538463 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b10147cb0 con 0x7f2b10072890 2024-05-09T08:02:35.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:35.797+0000 7f2b17600640 1 -- 172.21.15.1:0/183538463 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b10074d70 con 0x7f2b10072890 2024-05-09T08:02:35.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:35.798+0000 7f2b0cc00640 1 -- 172.21.15.1:0/183538463 <== mon.1 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2af8030a20 con 0x7f2b10072890 2024-05-09T08:02:35.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:35.799+0000 7f2b0cc00640 1 --2- 172.21.15.1:0/183538463 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f2aec06be30 0x7f2aec06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:35.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:35.799+0000 7f2b0cc00640 1 -- 172.21.15.1:0/183538463 <== mon.1 v2:172.21.15.7:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f2af80aba90 con 0x7f2b10072890 2024-05-09T08:02:35.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:35.799+0000 7f2b0fe00640 1 --2- 172.21.15.1:0/183538463 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f2aec06be30 0x7f2aec06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:35.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:35.800+0000 7f2b0fe00640 1 --2- 172.21.15.1:0/183538463 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f2aec06be30 0x7f2aec06e2f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f2b00009560 tx=0x7f2b00009190 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:35.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:35.804+0000 7f2b0cc00640 1 -- 172.21.15.1:0/183538463 <== mon.1 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f2af8079980 con 0x7f2b10072890 2024-05-09T08:02:36.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:36.009+0000 7f2b17600640 1 -- 172.21.15.1:0/183538463 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f2b100c8fe0 con 0x7f2b10072890 2024-05-09T08:02:36.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:36.010+0000 7f2b0cc00640 1 -- 172.21.15.1:0/183538463 <== mon.1 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) ==== 100+0+11 (secure 0 0 0) 0x7f2af807dc80 con 0x7f2b10072890 2024-05-09T08:02:36.027 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 64 2024-05-09T08:02:36.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:36.012+0000 7f2b17600640 1 -- 172.21.15.1:0/183538463 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f2aec06be30 msgr2=0x7f2aec06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:36.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:36.012+0000 7f2b17600640 1 --2- 172.21.15.1:0/183538463 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f2aec06be30 0x7f2aec06e2f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f2b00009560 tx=0x7f2b00009190 comp rx=0 tx=0).stop 2024-05-09T08:02:36.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:36.012+0000 7f2b17600640 1 -- 172.21.15.1:0/183538463 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2b10072890 msgr2=0x7f2b1014ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:36.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:36.012+0000 7f2b17600640 1 --2- 172.21.15.1:0/183538463 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2b10072890 0x7f2b1014ac10 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f2af802a2b0 tx=0x7f2af80046c0 comp rx=0 tx=0).stop 2024-05-09T08:02:36.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:36.013+0000 7f2b17600640 1 -- 172.21.15.1:0/183538463 shutdown_connections 2024-05-09T08:02:36.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:36.013+0000 7f2b17600640 1 --2- 172.21.15.1:0/183538463 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f2aec06be30 0x7f2aec06e2f0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:36.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:36.013+0000 7f2b17600640 1 --2- 172.21.15.1:0/183538463 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2b10154910 0x7f2b100cbff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:36.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:36.013+0000 7f2b17600640 1 --2- 172.21.15.1:0/183538463 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f2b100c9750 0x7f2b100c9ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:36.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:36.013+0000 7f2b17600640 1 --2- 172.21.15.1:0/183538463 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2b10072890 0x7f2b1014ac10 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:36.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:36.013+0000 7f2b17600640 1 -- 172.21.15.1:0/183538463 >> 172.21.15.1:0/183538463 conn(0x7f2b1006dad0 msgr2=0x7f2b100c1ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:02:36.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:36.013+0000 7f2b17600640 1 -- 172.21.15.1:0/183538463 shutdown_connections 2024-05-09T08:02:36.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:36.013+0000 7f2b17600640 1 -- 172.21.15.1:0/183538463 wait complete. 2024-05-09T08:02:36.047 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-09T08:02:36.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:36.237+0000 7f1775e00640 1 Processor -- start 2024-05-09T08:02:36.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:36.238+0000 7f1775e00640 1 -- start start 2024-05-09T08:02:36.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:36.239+0000 7f1775e00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f177015ac80 0x7f1770165d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:36.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:36.239+0000 7f1775e00640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f17700ccb20 con 0x7f17700cac60 2024-05-09T08:02:36.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:36.239+0000 7f1775e00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f17700cce00 con 0x7f17700cc5f0 2024-05-09T08:02:36.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:36.239+0000 7f1775e00640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f17700cf120 con 0x7f177015ac80 2024-05-09T08:02:36.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:36.239+0000 7f1774a00640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f17700cac60 0x7f17700cda00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:58808/0 (socket says 172.21.15.1:58808) 2024-05-09T08:02:36.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:36.239+0000 7f1774a00640 1 -- 172.21.15.1:0/2366229905 learned_addr learned my addr 172.21.15.1:0/2366229905 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T08:02:36.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:36.239+0000 7f1775400640 1 --2- 172.21.15.1:0/2366229905 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f177015ac80 0x7f1770165d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:36.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:36.240+0000 7f176f400640 1 -- 172.21.15.1:0/2366229905 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2407888558 0 0) 0x7f17700ccb20 con 0x7f17700cac60 2024-05-09T08:02:36.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:36.240+0000 7f176f400640 1 -- 172.21.15.1:0/2366229905 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1754003520 con 0x7f17700cac60 2024-05-09T08:02:36.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:36.240+0000 7f176f400640 1 -- 172.21.15.1:0/2366229905 <== mon.1 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 767009028 0 0) 0x7f17700cce00 con 0x7f17700cc5f0 2024-05-09T08:02:36.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:36.240+0000 7f176f400640 1 -- 172.21.15.1:0/2366229905 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f17700ccb20 con 0x7f17700cc5f0 2024-05-09T08:02:36.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:36.240+0000 7f1775400640 1 -- 172.21.15.1:0/2366229905 >> v1:172.21.15.7:6789/0 conn(0x7f17700cc5f0 legacy=0x7f177015a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:36.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:36.240+0000 7f1775400640 1 -- 172.21.15.1:0/2366229905 >> v1:172.21.15.1:6789/0 conn(0x7f17700cac60 legacy=0x7f17700cda00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:36.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:36.240+0000 7f1775400640 1 -- 172.21.15.1:0/2366229905 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17700cf400 con 0x7f177015ac80 2024-05-09T08:02:36.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:36.241+0000 7f1775400640 1 --2- 172.21.15.1:0/2366229905 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f177015ac80 0x7f1770165d40 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f17680097d0 tx=0x7f176802a290 comp rx=0 tx=0).ready entity=mon.2 client_cookie=50db4b779706a61 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:36.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:36.241+0000 7f176f400640 1 -- 172.21.15.1:0/2366229905 <== mon.2 v2:172.21.15.7:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1768037070 con 0x7f177015ac80 2024-05-09T08:02:36.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:36.241+0000 7f176f400640 1 -- 172.21.15.1:0/2366229905 <== mon.2 v2:172.21.15.7:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1768032440 con 0x7f177015ac80 2024-05-09T08:02:36.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:36.241+0000 7f176f400640 1 -- 172.21.15.1:0/2366229905 <== mon.2 v2:172.21.15.7:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17680327f0 con 0x7f177015ac80 2024-05-09T08:02:36.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:36.242+0000 7f1775e00640 1 -- 172.21.15.1:0/2366229905 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f177015ac80 msgr2=0x7f1770165d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:36.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:36.242+0000 7f1775e00640 1 --2- 172.21.15.1:0/2366229905 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f177015ac80 0x7f1770165d40 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f17680097d0 tx=0x7f176802a290 comp rx=0 tx=0).stop 2024-05-09T08:02:36.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:36.242+0000 7f1775e00640 1 -- 172.21.15.1:0/2366229905 shutdown_connections 2024-05-09T08:02:36.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:36.242+0000 7f1775e00640 1 --2- 172.21.15.1:0/2366229905 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f177015ac80 0x7f1770165d40 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:36.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:36.242+0000 7f1775e00640 1 -- 172.21.15.1:0/2366229905 >> 172.21.15.1:0/2366229905 conn(0x7f17700cfc20 msgr2=0x7f17700d0030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:02:36.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:36.242+0000 7f1775e00640 1 -- 172.21.15.1:0/2366229905 shutdown_connections 2024-05-09T08:02:36.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:36.242+0000 7f1775e00640 1 -- 172.21.15.1:0/2366229905 wait complete. 2024-05-09T08:02:36.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:36.243+0000 7f1775e00640 1 Processor -- start 2024-05-09T08:02:36.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:36.243+0000 7f1775e00640 1 -- start start 2024-05-09T08:02:36.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:36.244+0000 7f1775e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f17700cac60 0x7f17700b0720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:36.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:36.244+0000 7f1775e00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f17700cc5f0 0x7f1770155c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:36.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:36.244+0000 7f1775e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f177015ac80 0x7f1770156160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:36.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:36.244+0000 7f1775e00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f17700ce110 con 0x7f17700cac60 2024-05-09T08:02:36.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:36.244+0000 7f1775e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f17700cd550 con 0x7f177015ac80 2024-05-09T08:02:36.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:36.244+0000 7f1775e00640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f17700ce290 con 0x7f17700cc5f0 2024-05-09T08:02:36.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:36.245+0000 7f1774a00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f17700cac60 0x7f17700b0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:36.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:36.245+0000 7f1775400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f177015ac80 0x7f1770156160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:36.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:36.245+0000 7f176fe00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f17700cc5f0 0x7f1770155c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:36.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:36.245+0000 7f176fe00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f17700cc5f0 0x7f1770155c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3301/0 says I am v2:172.21.15.1:40092/0 (socket says 172.21.15.1:40092) 2024-05-09T08:02:36.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:36.245+0000 7f1775400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f177015ac80 0x7f1770156160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.1:47082/0 (socket says 172.21.15.1:47082) 2024-05-09T08:02:36.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:36.245+0000 7f176fe00640 1 -- 172.21.15.1:0/3950379839 learned_addr learned my addr 172.21.15.1:0/3950379839 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:02:36.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:36.245+0000 7f1774a00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f17700cac60 0x7f17700b0720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56382/0 (socket says 172.21.15.1:56382) 2024-05-09T08:02:36.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:36.245+0000 7f1774a00640 1 -- 172.21.15.1:0/3950379839 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f17700cc5f0 msgr2=0x7f1770155c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:36.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:36.245+0000 7f1774a00640 1 --2- 172.21.15.1:0/3950379839 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f17700cc5f0 0x7f1770155c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:36.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:36.245+0000 7f1774a00640 1 -- 172.21.15.1:0/3950379839 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f177015ac80 msgr2=0x7f1770156160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:36.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:36.245+0000 7f1774a00640 1 --2- 172.21.15.1:0/3950379839 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f177015ac80 0x7f1770156160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:36.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:36.245+0000 7f1774a00640 1 -- 172.21.15.1:0/3950379839 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1770154ea0 con 0x7f17700cac60 2024-05-09T08:02:36.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:36.245+0000 7f176fe00640 1 --2- 172.21.15.1:0/3950379839 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f17700cc5f0 0x7f1770155c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:02:36.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:36.245+0000 7f1775400640 1 --2- 172.21.15.1:0/3950379839 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f177015ac80 0x7f1770156160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:02:36.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:36.245+0000 7f1774a00640 1 --2- 172.21.15.1:0/3950379839 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f17700cac60 0x7f17700b0720 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f176000cbc0 tx=0x7f176000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:36.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:36.246+0000 7f176cc00640 1 -- 172.21.15.1:0/3950379839 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1760017070 con 0x7f17700cac60 2024-05-09T08:02:36.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:36.246+0000 7f176cc00640 1 -- 172.21.15.1:0/3950379839 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f176000ad40 con 0x7f17700cac60 2024-05-09T08:02:36.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:36.246+0000 7f176cc00640 1 -- 172.21.15.1:0/3950379839 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1760012640 con 0x7f17700cac60 2024-05-09T08:02:36.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:36.246+0000 7f1775e00640 1 -- 172.21.15.1:0/3950379839 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17701f0db0 con 0x7f17700cac60 2024-05-09T08:02:36.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:36.246+0000 7f1775e00640 1 -- 172.21.15.1:0/3950379839 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1770155420 con 0x7f17700cac60 2024-05-09T08:02:36.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:36.247+0000 7f1775e00640 1 -- 172.21.15.1:0/3950379839 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1770154ba0 con 0x7f17700cac60 2024-05-09T08:02:36.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:36.248+0000 7f176cc00640 1 -- 172.21.15.1:0/3950379839 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f176001a070 con 0x7f17700cac60 2024-05-09T08:02:36.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:36.249+0000 7f176cc00640 1 --2- 172.21.15.1:0/3950379839 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f174c06be30 0x7f174c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:36.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:36.249+0000 7f176cc00640 1 -- 172.21.15.1:0/3950379839 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f176008afe0 con 0x7f17700cac60 2024-05-09T08:02:36.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:36.249+0000 7f176fe00640 1 --2- 172.21.15.1:0/3950379839 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f174c06be30 0x7f174c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:36.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:36.250+0000 7f176fe00640 1 --2- 172.21.15.1:0/3950379839 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f174c06be30 0x7f174c06e2f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f17701510e0 tx=0x7f1758009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:36.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:36.253+0000 7f176cc00640 1 -- 172.21.15.1:0/3950379839 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f1760058ef0 con 0x7f17700cac60 2024-05-09T08:02:36.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:36.448+0000 7f1775e00640 1 -- 172.21.15.1:0/3950379839 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f177004f660 con 0x7f17700cac60 2024-05-09T08:02:36.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:36.449+0000 7f176cc00640 1 -- 172.21.15.1:0/3950379839 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) ==== 103+0+11 (secure 0 0 0) 0x7f176005d1f0 con 0x7f17700cac60 2024-05-09T08:02:36.466 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 32 2024-05-09T08:02:36.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:36.451+0000 7f1775e00640 1 -- 172.21.15.1:0/3950379839 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f174c06be30 msgr2=0x7f174c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:36.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:36.451+0000 7f1775e00640 1 --2- 172.21.15.1:0/3950379839 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f174c06be30 0x7f174c06e2f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f17701510e0 tx=0x7f1758009040 comp rx=0 tx=0).stop 2024-05-09T08:02:36.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:36.451+0000 7f1775e00640 1 -- 172.21.15.1:0/3950379839 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f17700cac60 msgr2=0x7f17700b0720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:36.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:36.451+0000 7f1775e00640 1 --2- 172.21.15.1:0/3950379839 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f17700cac60 0x7f17700b0720 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f176000cbc0 tx=0x7f176000a430 comp rx=0 tx=0).stop 2024-05-09T08:02:36.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:36.451+0000 7f1775e00640 1 -- 172.21.15.1:0/3950379839 shutdown_connections 2024-05-09T08:02:36.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:36.451+0000 7f1775e00640 1 --2- 172.21.15.1:0/3950379839 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f174c06be30 0x7f174c06e2f0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:36.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:36.451+0000 7f1775e00640 1 --2- 172.21.15.1:0/3950379839 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f177015ac80 0x7f1770156160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:36.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:36.451+0000 7f1775e00640 1 --2- 172.21.15.1:0/3950379839 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f17700cc5f0 0x7f1770155c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:36.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:36.451+0000 7f1775e00640 1 --2- 172.21.15.1:0/3950379839 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f17700cac60 0x7f17700b0720 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:36.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:36.451+0000 7f1775e00640 1 -- 172.21.15.1:0/3950379839 >> 172.21.15.1:0/3950379839 conn(0x7f17700cfc20 msgr2=0x7f17701590b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:02:36.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:36.452+0000 7f1775e00640 1 -- 172.21.15.1:0/3950379839 shutdown_connections 2024-05-09T08:02:36.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:36.452+0000 7f1775e00640 1 -- 172.21.15.1:0/3950379839 wait complete. 2024-05-09T08:02:36.487 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-09T08:02:36.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:36.672+0000 7f4f9fe00640 1 Processor -- start 2024-05-09T08:02:36.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:36.673+0000 7f4f9fe00640 1 -- start start 2024-05-09T08:02:36.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:36.674+0000 7f4f9fe00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f4f98072d90 0x7f4f980c5fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:36.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:36.674+0000 7f4f9fe00640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f4f98157d80 0x7f4f98073170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:36.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:36.674+0000 7f4f9fe00640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4f980c1230 con 0x7f4f980c2e50 2024-05-09T08:02:36.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:36.674+0000 7f4f9fe00640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f4f980c1510 con 0x7f4f98157d80 2024-05-09T08:02:36.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:36.674+0000 7f4f9fe00640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f4f9806f830 con 0x7f4f98072d90 2024-05-09T08:02:36.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:36.675+0000 7f4f9de00640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f4f98157d80 0x7f4f98073170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:36.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:36.675+0000 7f4f9ca00640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f4f980c2e50 0x7f4f98070030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:58810/0 (socket says 172.21.15.1:58810) 2024-05-09T08:02:36.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:36.675+0000 7f4f9ca00640 1 -- 172.21.15.1:0/274788861 learned_addr learned my addr 172.21.15.1:0/274788861 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T08:02:36.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:36.675+0000 7f4f9d400640 1 --2- 172.21.15.1:0/274788861 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f4f98072d90 0x7f4f980c5fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:36.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:36.675+0000 7f4f9de00640 1 -- 172.21.15.1:0/274788861 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f4f98072d90 msgr2=0x7f4f980c5fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:02:36.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:36.675+0000 7f4f9de00640 1 --2- 172.21.15.1:0/274788861 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f4f98072d90 0x7f4f980c5fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:36.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:36.675+0000 7f4f9de00640 1 -- 172.21.15.1:0/274788861 >> v1:172.21.15.1:6789/0 conn(0x7f4f980c2e50 legacy=0x7f4f98070030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:36.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:36.675+0000 7f4f9de00640 1 -- 172.21.15.1:0/274788861 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f9806fb10 con 0x7f4f98157d80 2024-05-09T08:02:36.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:36.675+0000 7f4f97e00640 1 -- 172.21.15.1:0/274788861 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 671990661 0 0) 0x7f4f980c1230 con 0x7f4f980c2e50 2024-05-09T08:02:36.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:36.676+0000 7f4f9de00640 1 --2- 172.21.15.1:0/274788861 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4f98157d80 0x7f4f98073170 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f4f90009d90 tx=0x7f4f9002a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=733e8389cd636259 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:36.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:36.676+0000 7f4f97e00640 1 -- 172.21.15.1:0/274788861 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f90007740 con 0x7f4f98157d80 2024-05-09T08:02:36.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:36.677+0000 7f4f97e00640 1 -- 172.21.15.1:0/274788861 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f900078e0 con 0x7f4f98157d80 2024-05-09T08:02:36.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:36.677+0000 7f4f97e00640 1 -- 172.21.15.1:0/274788861 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f90007cb0 con 0x7f4f98157d80 2024-05-09T08:02:36.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:36.677+0000 7f4f9fe00640 1 -- 172.21.15.1:0/274788861 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4f98157d80 msgr2=0x7f4f98073170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:36.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:36.677+0000 7f4f9fe00640 1 --2- 172.21.15.1:0/274788861 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4f98157d80 0x7f4f98073170 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f4f90009d90 tx=0x7f4f9002a6f0 comp rx=0 tx=0).stop 2024-05-09T08:02:36.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:36.677+0000 7f4f9fe00640 1 -- 172.21.15.1:0/274788861 shutdown_connections 2024-05-09T08:02:36.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:36.677+0000 7f4f9fe00640 1 --2- 172.21.15.1:0/274788861 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4f98157d80 0x7f4f98073170 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:36.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:36.677+0000 7f4f9fe00640 1 --2- 172.21.15.1:0/274788861 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f4f98072d90 0x7f4f980c5fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:36.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:36.677+0000 7f4f9fe00640 1 -- 172.21.15.1:0/274788861 >> 172.21.15.1:0/274788861 conn(0x7f4f9806dad0 msgr2=0x7f4f9806deb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:02:36.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:36.677+0000 7f4f9fe00640 1 -- 172.21.15.1:0/274788861 shutdown_connections 2024-05-09T08:02:36.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:36.677+0000 7f4f9fe00640 1 -- 172.21.15.1:0/274788861 wait complete. 2024-05-09T08:02:36.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:36.679+0000 7f4f9fe00640 1 Processor -- start 2024-05-09T08:02:36.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:36.679+0000 7f4f9fe00640 1 -- start start 2024-05-09T08:02:36.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:36.680+0000 7f4f9fe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4f98072d90 0x7f4f981e1790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:36.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:36.680+0000 7f4f9fe00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4f980c2e50 0x7f4f980c44a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:36.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:36.680+0000 7f4f9fe00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f4f98157d80 0x7f4f9814bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:36.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:36.680+0000 7f4f9fe00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f4f98151a60 con 0x7f4f980c2e50 2024-05-09T08:02:36.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:36.680+0000 7f4f9fe00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f4f981518e0 con 0x7f4f98072d90 2024-05-09T08:02:36.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:36.680+0000 7f4f9fe00640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f4f980c6a00 con 0x7f4f98157d80 2024-05-09T08:02:36.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:36.680+0000 7f4f9ca00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4f980c2e50 0x7f4f980c44a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:36.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:36.680+0000 7f4f9ca00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4f980c2e50 0x7f4f980c44a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56402/0 (socket says 172.21.15.1:56402) 2024-05-09T08:02:36.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:36.680+0000 7f4f9ca00640 1 -- 172.21.15.1:0/2733825488 learned_addr learned my addr 172.21.15.1:0/2733825488 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:02:36.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:36.680+0000 7f4f9de00640 1 --2- 172.21.15.1:0/2733825488 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f4f98157d80 0x7f4f9814bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:36.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:36.680+0000 7f4f9d400640 1 --2- 172.21.15.1:0/2733825488 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4f98072d90 0x7f4f981e1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:36.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:36.681+0000 7f4f9ca00640 1 -- 172.21.15.1:0/2733825488 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f4f98157d80 msgr2=0x7f4f9814bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:36.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:36.681+0000 7f4f9ca00640 1 --2- 172.21.15.1:0/2733825488 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f4f98157d80 0x7f4f9814bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:36.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:36.681+0000 7f4f9ca00640 1 -- 172.21.15.1:0/2733825488 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4f98072d90 msgr2=0x7f4f981e1790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:36.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:36.681+0000 7f4f9ca00640 1 --2- 172.21.15.1:0/2733825488 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4f98072d90 0x7f4f981e1790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:36.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:36.681+0000 7f4f9ca00640 1 -- 172.21.15.1:0/2733825488 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f980c3400 con 0x7f4f980c2e50 2024-05-09T08:02:36.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:36.681+0000 7f4f9ca00640 1 --2- 172.21.15.1:0/2733825488 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4f980c2e50 0x7f4f980c44a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f4f8400c430 tx=0x7f4f8400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:36.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:36.681+0000 7f4f95600640 1 -- 172.21.15.1:0/2733825488 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f840074e0 con 0x7f4f980c2e50 2024-05-09T08:02:36.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:36.681+0000 7f4f95600640 1 -- 172.21.15.1:0/2733825488 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f840047e0 con 0x7f4f980c2e50 2024-05-09T08:02:36.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:36.681+0000 7f4f95600640 1 -- 172.21.15.1:0/2733825488 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f84004bd0 con 0x7f4f980c2e50 2024-05-09T08:02:36.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:36.681+0000 7f4f9fe00640 1 -- 172.21.15.1:0/2733825488 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f980c36e0 con 0x7f4f980c2e50 2024-05-09T08:02:36.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:36.682+0000 7f4f9fe00640 1 -- 172.21.15.1:0/2733825488 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f980c3ca0 con 0x7f4f980c2e50 2024-05-09T08:02:36.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:36.683+0000 7f4f9fe00640 1 -- 172.21.15.1:0/2733825488 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f98074d70 con 0x7f4f980c2e50 2024-05-09T08:02:36.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:36.683+0000 7f4f95600640 1 -- 172.21.15.1:0/2733825488 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4f84011070 con 0x7f4f980c2e50 2024-05-09T08:02:36.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:36.684+0000 7f4f95600640 1 --2- 172.21.15.1:0/2733825488 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f4f7806be30 0x7f4f7806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:36.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:36.684+0000 7f4f95600640 1 -- 172.21.15.1:0/2733825488 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f4f8408abb0 con 0x7f4f980c2e50 2024-05-09T08:02:36.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:36.685+0000 7f4f9d400640 1 --2- 172.21.15.1:0/2733825488 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f4f7806be30 0x7f4f7806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:36.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:36.685+0000 7f4f9d400640 1 --2- 172.21.15.1:0/2733825488 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f4f7806be30 0x7f4f7806e2f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4f8c009aa0 tx=0x7f4f8c009190 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:36.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:36.690+0000 7f4f95600640 1 -- 172.21.15.1:0/2733825488 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f4f84058c40 con 0x7f4f980c2e50 2024-05-09T08:02:36.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:36.895+0000 7f4f9fe00640 1 -- 172.21.15.1:0/2733825488 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f4f980c5b80 con 0x7f4f980c2e50 2024-05-09T08:02:36.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:36.896+0000 7f4f95600640 1 -- 172.21.15.1:0/2733825488 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v25) ==== 80+0+0 (secure 0 0 0) 0x7f4f8405cf40 con 0x7f4f980c2e50 2024-05-09T08:02:36.913 INFO:teuthology.orchestra.run.smithi001.stderr:listed 0 entries 2024-05-09T08:02:36.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:36.898+0000 7f4f9fe00640 1 -- 172.21.15.1:0/2733825488 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f4f7806be30 msgr2=0x7f4f7806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:36.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:36.898+0000 7f4f9fe00640 1 --2- 172.21.15.1:0/2733825488 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f4f7806be30 0x7f4f7806e2f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4f8c009aa0 tx=0x7f4f8c009190 comp rx=0 tx=0).stop 2024-05-09T08:02:36.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:36.898+0000 7f4f9fe00640 1 -- 172.21.15.1:0/2733825488 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4f980c2e50 msgr2=0x7f4f980c44a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:36.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:36.898+0000 7f4f9fe00640 1 --2- 172.21.15.1:0/2733825488 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4f980c2e50 0x7f4f980c44a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f4f8400c430 tx=0x7f4f8400c8f0 comp rx=0 tx=0).stop 2024-05-09T08:02:36.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:36.898+0000 7f4f9fe00640 1 -- 172.21.15.1:0/2733825488 shutdown_connections 2024-05-09T08:02:36.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:36.898+0000 7f4f9fe00640 1 --2- 172.21.15.1:0/2733825488 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f4f7806be30 0x7f4f7806e2f0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:36.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:36.898+0000 7f4f9fe00640 1 --2- 172.21.15.1:0/2733825488 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f4f98157d80 0x7f4f9814bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:36.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:36.898+0000 7f4f9fe00640 1 --2- 172.21.15.1:0/2733825488 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4f980c2e50 0x7f4f980c44a0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:36.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:36.898+0000 7f4f9fe00640 1 --2- 172.21.15.1:0/2733825488 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4f98072d90 0x7f4f981e1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:36.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:36.898+0000 7f4f9fe00640 1 -- 172.21.15.1:0/2733825488 >> 172.21.15.1:0/2733825488 conn(0x7f4f9806dad0 msgr2=0x7f4f980cb910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:02:36.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:36.899+0000 7f4f9fe00640 1 -- 172.21.15.1:0/2733825488 shutdown_connections 2024-05-09T08:02:36.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:36.899+0000 7f4f9fe00640 1 -- 172.21.15.1:0/2733825488 wait complete. 2024-05-09T08:02:36.934 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-05-09T08:02:36.934 DEBUG:teuthology.orchestra.run.smithi001:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T08:02:36.990 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T08:02:37.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:37.181+0000 7faa65600640 1 Processor -- start 2024-05-09T08:02:37.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:37.182+0000 7faa65600640 1 -- start start 2024-05-09T08:02:37.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:37.183+0000 7faa65600640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7faa600c8180 0x7faa600cc730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:37.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:37.183+0000 7faa65600640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faa6015c790 con 0x7faa600c7660 2024-05-09T08:02:37.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:37.184+0000 7faa65600640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faa600c6a30 con 0x7faa60152500 2024-05-09T08:02:37.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:37.184+0000 7faa65600640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7faa600c5e50 con 0x7faa600c8180 2024-05-09T08:02:37.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:37.184+0000 7faa57e00640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7faa600c8180 0x7faa600cc730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:37.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:37.184+0000 7faa57e00640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7faa600c8180 0x7faa600cc730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56404/0 (socket says 172.21.15.1:56404) 2024-05-09T08:02:37.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:37.184+0000 7faa57e00640 1 -- 172.21.15.1:0/635299867 learned_addr learned my addr 172.21.15.1:0/635299867 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:02:37.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:37.184+0000 7faa5fe00640 1 --1- 172.21.15.1:0/635299867 >> v1:172.21.15.7:6789/0 conn(0x7faa60152500 0x7faa6015c080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.1:40450/0 (socket says 172.21.15.1:40450) 2024-05-09T08:02:37.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:37.184+0000 7faa57e00640 1 -- 172.21.15.1:0/635299867 >> v1:172.21.15.7:6789/0 conn(0x7faa60152500 legacy=0x7faa6015c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:37.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:37.184+0000 7faa57e00640 1 -- 172.21.15.1:0/635299867 >> v1:172.21.15.1:6789/0 conn(0x7faa600c7660 legacy=0x7faa60162bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:37.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:37.184+0000 7faa57e00640 1 -- 172.21.15.1:0/635299867 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa600c6cf0 con 0x7faa600c8180 2024-05-09T08:02:37.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:37.185+0000 7faa57e00640 1 --2- 172.21.15.1:0/635299867 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faa600c8180 0x7faa600cc730 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7faa48009870 tx=0x7faa4802a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccf79865f291a238 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:37.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:37.188+0000 7faa5f400640 1 -- 172.21.15.1:0/635299867 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa48037070 con 0x7faa600c8180 2024-05-09T08:02:37.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:37.189+0000 7faa5f400640 1 -- 172.21.15.1:0/635299867 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa4802ae50 con 0x7faa600c8180 2024-05-09T08:02:37.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:37.189+0000 7faa5f400640 1 -- 172.21.15.1:0/635299867 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa48032860 con 0x7faa600c8180 2024-05-09T08:02:37.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:37.189+0000 7faa65600640 1 -- 172.21.15.1:0/635299867 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faa600c8180 msgr2=0x7faa600cc730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:37.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:37.189+0000 7faa65600640 1 --2- 172.21.15.1:0/635299867 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faa600c8180 0x7faa600cc730 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7faa48009870 tx=0x7faa4802a300 comp rx=0 tx=0).stop 2024-05-09T08:02:37.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:37.189+0000 7faa65600640 1 -- 172.21.15.1:0/635299867 shutdown_connections 2024-05-09T08:02:37.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:37.189+0000 7faa65600640 1 --2- 172.21.15.1:0/635299867 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faa600c8180 0x7faa600cc730 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:37.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:37.189+0000 7faa65600640 1 -- 172.21.15.1:0/635299867 >> 172.21.15.1:0/635299867 conn(0x7faa600cbc10 msgr2=0x7faa600cc020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:02:37.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:37.189+0000 7faa65600640 1 -- 172.21.15.1:0/635299867 shutdown_connections 2024-05-09T08:02:37.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:37.190+0000 7faa65600640 1 -- 172.21.15.1:0/635299867 wait complete. 2024-05-09T08:02:37.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:37.191+0000 7faa65600640 1 Processor -- start 2024-05-09T08:02:37.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:37.191+0000 7faa65600640 1 -- start start 2024-05-09T08:02:37.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:37.192+0000 7faa65600640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faa600c7660 0x7faa601f3c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:37.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:37.192+0000 7faa65600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faa600c8180 0x7faa60072110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:37.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:37.192+0000 7faa65600640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7faa60152500 0x7faa601f0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:37.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:37.192+0000 7faa65600640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7faa600c72b0 con 0x7faa600c7660 2024-05-09T08:02:37.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:37.192+0000 7faa65600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7faa600c22d0 con 0x7faa600c8180 2024-05-09T08:02:37.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:37.192+0000 7faa65600640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7faa6015b880 con 0x7faa60152500 2024-05-09T08:02:37.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:37.193+0000 7faa5fe00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faa600c7660 0x7faa601f3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:37.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:37.193+0000 7faa64c00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7faa60152500 0x7faa601f0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:37.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:37.193+0000 7faa5fe00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faa600c7660 0x7faa601f3c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56414/0 (socket says 172.21.15.1:56414) 2024-05-09T08:02:37.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:37.193+0000 7faa57e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faa600c8180 0x7faa60072110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:37.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:37.193+0000 7faa64c00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7faa60152500 0x7faa601f0650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3301/0 says I am v2:172.21.15.1:40116/0 (socket says 172.21.15.1:40116) 2024-05-09T08:02:37.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:37.193+0000 7faa5fe00640 1 -- 172.21.15.1:0/872033726 learned_addr learned my addr 172.21.15.1:0/872033726 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:02:37.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:37.193+0000 7faa5fe00640 1 -- 172.21.15.1:0/872033726 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7faa60152500 msgr2=0x7faa601f0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:37.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:37.193+0000 7faa5fe00640 1 --2- 172.21.15.1:0/872033726 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7faa60152500 0x7faa601f0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:37.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:37.193+0000 7faa5fe00640 1 -- 172.21.15.1:0/872033726 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faa600c8180 msgr2=0x7faa60072110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:37.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:37.193+0000 7faa5fe00640 1 --2- 172.21.15.1:0/872033726 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faa600c8180 0x7faa60072110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:37.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:37.193+0000 7faa5fe00640 1 -- 172.21.15.1:0/872033726 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa601f41b0 con 0x7faa600c7660 2024-05-09T08:02:37.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:37.193+0000 7faa57e00640 1 --2- 172.21.15.1:0/872033726 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faa600c8180 0x7faa60072110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:02:37.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:37.194+0000 7faa5fe00640 1 --2- 172.21.15.1:0/872033726 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faa600c7660 0x7faa601f3c70 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7faa5000c3c0 tx=0x7faa5000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:37.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:37.194+0000 7faa5cc00640 1 -- 172.21.15.1:0/872033726 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa500074e0 con 0x7faa600c7660 2024-05-09T08:02:37.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:37.194+0000 7faa5cc00640 1 -- 172.21.15.1:0/872033726 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa50007680 con 0x7faa600c7660 2024-05-09T08:02:37.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:37.194+0000 7faa5cc00640 1 -- 172.21.15.1:0/872033726 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa50004a00 con 0x7faa600c7660 2024-05-09T08:02:37.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:37.194+0000 7faa65600640 1 -- 172.21.15.1:0/872033726 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa60155ce0 con 0x7faa600c7660 2024-05-09T08:02:37.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:37.194+0000 7faa65600640 1 -- 172.21.15.1:0/872033726 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa60157c50 con 0x7faa600c7660 2024-05-09T08:02:37.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:37.195+0000 7faa65600640 1 -- 172.21.15.1:0/872033726 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa6004f660 con 0x7faa600c7660 2024-05-09T08:02:37.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:37.196+0000 7faa5cc00640 1 -- 172.21.15.1:0/872033726 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7faa5001a020 con 0x7faa600c7660 2024-05-09T08:02:37.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:37.197+0000 7faa5cc00640 1 --2- 172.21.15.1:0/872033726 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7faa3c06be60 0x7faa3c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:37.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:37.197+0000 7faa5cc00640 1 -- 172.21.15.1:0/872033726 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7faa5008a7c0 con 0x7faa600c7660 2024-05-09T08:02:37.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:37.197+0000 7faa57e00640 1 --2- 172.21.15.1:0/872033726 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7faa3c06be60 0x7faa3c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:37.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:37.198+0000 7faa57e00640 1 --2- 172.21.15.1:0/872033726 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7faa3c06be60 0x7faa3c06e320 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7faa48034000 tx=0x7faa4802e000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:37.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:37.202+0000 7faa5cc00640 1 -- 172.21.15.1:0/872033726 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7faa500587d0 con 0x7faa600c7660 2024-05-09T08:02:37.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:37.397+0000 7faa65600640 1 -- 172.21.15.1:0/872033726 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7faa600c2a40 con 0x7faa600c7660 2024-05-09T08:02:37.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:37.401+0000 7faa5cc00640 1 -- 172.21.15.1:0/872033726 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18382 (secure 0 0 0) 0x7faa5005cad0 con 0x7faa600c7660 2024-05-09T08:02:37.418 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T08:02:37.418 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":25,"fsid":"54329ec8-3270-4043-9a82-c6cd7420aa7f","created":"2024-05-09T08:01:56.666970+0000","modified":"2024-05-09T08:02:20.403456+0000","last_up_change":"2024-05-09T08:02:07.146082+0000","last_in_change":"2024-05-09T08:01:59.008225+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T08:02:09.165511+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T08:02:12.925485+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T08:02:14.367455+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-09T08:02:17.967064+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c064454e-a9b5-4a24-9a77-17f0a4085952","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6800","nonce":1513349547},{"type":"v1","addr":"172.21.15.1:6801","nonce":1513349547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":1513349547},{"type":"v1","addr":"172.21.15.1:6803","nonce":1513349547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":1513349547},{"type":"v1","addr":"172.21.15.1:6807","nonce":1513349547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":1513349547},{"type":"v1","addr":"172.21.15.1:6805","nonce":1513349547}]},"public_addr":"172.21.15.1:6801/1513349547","cluster_addr":"172.21.15.1:6803/1513349547","heartbeat_back_addr":"172.21.15.1:6807/1513349547","heartbeat_front_addr":"172.21.15.1:6805/1513349547","state":["exists","up"]},{"osd":1,"uuid":"1a9ea687-2ff6-407a-949b-0e7e1083af21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":2942255259},{"type":"v1","addr":"172.21.15.1:6817","nonce":2942255259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":2942255259},{"type":"v1","addr":"172.21.15.1:6819","nonce":2942255259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":2942255259},{"type":"v1","addr":"172.21.15.1:6823","nonce":2942255259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6820","nonce":2942255259},{"type":"v1","addr":"172.21.15.1:6821","nonce":2942255259}]},"public_addr":"172.21.15.1:6817/2942255259","cluster_addr":"172.21.15.1:6819/2942255259","heartbeat_back_addr":"172.21.15.1:6823/2942255259","heartbeat_front_addr":"172.21.15.1:6821/2942255259","state":["exists","up"]},{"osd":2,"uuid":"c88af08b-7c9a-4dd8-9640-44138239addb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":2925031811},{"type":"v1","addr":"172.21.15.1:6809","nonce":2925031811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":2925031811},{"type":"v1","addr":"172.21.15.1:6811","nonce":2925031811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":2925031811},{"type":"v1","addr":"172.21.15.1:6815","nonce":2925031811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":2925031811},{"type":"v1","addr":"172.21.15.1:6813","nonce":2925031811}]},"public_addr":"172.21.15.1:6809/2925031811","cluster_addr":"172.21.15.1:6811/2925031811","heartbeat_back_addr":"172.21.15.1:6815/2925031811","heartbeat_front_addr":"172.21.15.1:6813/2925031811","state":["exists","up"]},{"osd":3,"uuid":"c087b7e3-a41d-4333-9365-32e3bd8c8e6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6824","nonce":2497776654},{"type":"v1","addr":"172.21.15.1:6825","nonce":2497776654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":2497776654},{"type":"v1","addr":"172.21.15.1:6827","nonce":2497776654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":2497776654},{"type":"v1","addr":"172.21.15.1:6831","nonce":2497776654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6828","nonce":2497776654},{"type":"v1","addr":"172.21.15.1:6829","nonce":2497776654}]},"public_addr":"172.21.15.1:6825/2497776654","cluster_addr":"172.21.15.1:6827/2497776654","heartbeat_back_addr":"172.21.15.1:6831/2497776654","heartbeat_front_addr":"172.21.15.1:6829/2497776654","state":["exists","up"]},{"osd":4,"uuid":"b4576b9c-871d-47a8-8d36-65af525d0200","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":2957026670},{"type":"v1","addr":"172.21.15.7:6801","nonce":2957026670}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2957026670},{"type":"v1","addr":"172.21.15.7:6803","nonce":2957026670}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":2957026670},{"type":"v1","addr":"172.21.15.7:6807","nonce":2957026670}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2957026670},{"type":"v1","addr":"172.21.15.7:6805","nonce":2957026670}]},"public_addr":"172.21.15.7:6801/2957026670","cluster_addr":"172.21.15.7:6803/2957026670","heartbeat_back_addr":"172.21.15.7:6807/2957026670","heartbeat_front_addr":"172.21.15.7:6805/2957026670","state":["exists","up"]},{"osd":5,"uuid":"902b3d94-72f3-4499-b21a-e32453ef5144","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1670295242},{"type":"v1","addr":"172.21.15.7:6809","nonce":1670295242}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1670295242},{"type":"v1","addr":"172.21.15.7:6811","nonce":1670295242}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1670295242},{"type":"v1","addr":"172.21.15.7:6815","nonce":1670295242}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1670295242},{"type":"v1","addr":"172.21.15.7:6813","nonce":1670295242}]},"public_addr":"172.21.15.7:6809/1670295242","cluster_addr":"172.21.15.7:6811/1670295242","heartbeat_back_addr":"172.21.15.7:6815/1670295242","heartbeat_front_addr":"172.21.15.7:6813/1670295242","state":["exists","up"]},{"osd":6,"uuid":"ead1d771-1728-4f85-adb4-26be21cedff3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":2852086880},{"type":"v1","addr":"172.21.15.7:6817","nonce":2852086880}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":2852086880},{"type":"v1","addr":"172.21.15.7:6819","nonce":2852086880}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":2852086880},{"type":"v1","addr":"172.21.15.7:6823","nonce":2852086880}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":2852086880},{"type":"v1","addr":"172.21.15.7:6821","nonce":2852086880}]},"public_addr":"172.21.15.7:6817/2852086880","cluster_addr":"172.21.15.7:6819/2852086880","heartbeat_back_addr":"172.21.15.7:6823/2852086880","heartbeat_front_addr":"172.21.15.7:6821/2852086880","state":["exists","up"]},{"osd":7,"uuid":"3ab5a201-9394-48a8-8918-e95197893669","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":1442884138},{"type":"v1","addr":"172.21.15.7:6825","nonce":1442884138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1442884138},{"type":"v1","addr":"172.21.15.7:6827","nonce":1442884138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1442884138},{"type":"v1","addr":"172.21.15.7:6831","nonce":1442884138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1442884138},{"type":"v1","addr":"172.21.15.7:6829","nonce":1442884138}]},"public_addr":"172.21.15.7:6825/1442884138","cluster_addr":"172.21.15.7:6827/1442884138","heartbeat_back_addr":"172.21.15.7:6831/1442884138","heartbeat_front_addr":"172.21.15.7:6829/1442884138","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:02:04.615675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:02:04.385974+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:02:04.619217+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:02:04.401681+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:02:04.097081+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:02:04.129981+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:02:04.225327+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:02:04.193351+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T08:02:37.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:37.404+0000 7faa65600640 1 -- 172.21.15.1:0/872033726 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7faa3c06be60 msgr2=0x7faa3c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:37.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:37.404+0000 7faa65600640 1 --2- 172.21.15.1:0/872033726 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7faa3c06be60 0x7faa3c06e320 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7faa48034000 tx=0x7faa4802e000 comp rx=0 tx=0).stop 2024-05-09T08:02:37.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:37.404+0000 7faa65600640 1 -- 172.21.15.1:0/872033726 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faa600c7660 msgr2=0x7faa601f3c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:37.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:37.404+0000 7faa65600640 1 --2- 172.21.15.1:0/872033726 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faa600c7660 0x7faa601f3c70 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7faa5000c3c0 tx=0x7faa5000c880 comp rx=0 tx=0).stop 2024-05-09T08:02:37.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:37.404+0000 7faa65600640 1 -- 172.21.15.1:0/872033726 shutdown_connections 2024-05-09T08:02:37.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:37.404+0000 7faa65600640 1 --2- 172.21.15.1:0/872033726 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7faa3c06be60 0x7faa3c06e320 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:37.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:37.404+0000 7faa65600640 1 --2- 172.21.15.1:0/872033726 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faa600c8180 0x7faa60072110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:37.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:37.404+0000 7faa65600640 1 --2- 172.21.15.1:0/872033726 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7faa60152500 0x7faa601f0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:37.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:37.404+0000 7faa65600640 1 --2- 172.21.15.1:0/872033726 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faa600c7660 0x7faa601f3c70 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:37.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:37.404+0000 7faa65600640 1 -- 172.21.15.1:0/872033726 >> 172.21.15.1:0/872033726 conn(0x7faa600cbc10 msgr2=0x7faa600cb280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:02:37.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:37.404+0000 7faa65600640 1 -- 172.21.15.1:0/872033726 shutdown_connections 2024-05-09T08:02:37.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:37.404+0000 7faa65600640 1 -- 172.21.15.1:0/872033726 wait complete. 2024-05-09T08:02:37.439 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T08:02:09.165511+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-09T08:02:12.925485+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-09T08:02:14.367455+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-09T08:02:17.967064+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-09T08:02:37.439 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-09T08:02:37.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:37.622+0000 7fe1e7200640 1 Processor -- start 2024-05-09T08:02:37.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:37.622+0000 7fe1e7200640 1 -- start start 2024-05-09T08:02:37.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:37.623+0000 7fe1e7200640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fe1e80cac60 0x7fe1e80cda00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:37.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:37.624+0000 7fe1e7200640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fe1e815ac80 0x7fe1e80cdf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:37.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:37.624+0000 7fe1e7200640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe1e80ccb20 con 0x7fe1e80cc5f0 2024-05-09T08:02:37.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:37.624+0000 7fe1e7200640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7fe1e80cce00 con 0x7fe1e815ac80 2024-05-09T08:02:37.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:37.624+0000 7fe1e7200640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7fe1e80cf120 con 0x7fe1e80cac60 2024-05-09T08:02:37.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:37.624+0000 7fe1e5e00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fe1e80cac60 0x7fe1e80cda00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:37.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:37.624+0000 7fe1e5e00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fe1e80cac60 0x7fe1e80cda00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3301/0 says I am v2:172.21.15.1:40120/0 (socket says 172.21.15.1:40120) 2024-05-09T08:02:37.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:37.624+0000 7fe1e5e00640 1 -- 172.21.15.1:0/3171412079 learned_addr learned my addr 172.21.15.1:0/3171412079 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:02:37.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:37.624+0000 7fe1e6800640 1 --2- 172.21.15.1:0/3171412079 >> v2:172.21.15.1:3300/0 conn(0x7fe1e815ac80 0x7fe1e80cdf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:37.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:37.624+0000 7fe1e5e00640 1 -- 172.21.15.1:0/3171412079 >> v2:172.21.15.1:3300/0 conn(0x7fe1e815ac80 msgr2=0x7fe1e80cdf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:02:37.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:37.624+0000 7fe1e5e00640 1 --2- 172.21.15.1:0/3171412079 >> v2:172.21.15.1:3300/0 conn(0x7fe1e815ac80 0x7fe1e80cdf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:37.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:37.624+0000 7fe1e5e00640 1 -- 172.21.15.1:0/3171412079 >> v1:172.21.15.1:6789/0 conn(0x7fe1e80cc5f0 legacy=0x7fe1e815a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:37.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:37.624+0000 7fe1e5e00640 1 -- 172.21.15.1:0/3171412079 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1e80cf400 con 0x7fe1e80cac60 2024-05-09T08:02:37.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:37.624+0000 7fe1e6800640 1 --2- 172.21.15.1:0/3171412079 >> v2:172.21.15.1:3300/0 conn(0x7fe1e815ac80 0x7fe1e80cdf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T08:02:37.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:37.626+0000 7fe1e5e00640 1 --2- 172.21.15.1:0/3171412079 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fe1e80cac60 0x7fe1e80cda00 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fe1d8009870 tx=0x7fe1d802a300 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1684ab4c6602025d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:37.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:37.626+0000 7fe1e5400640 1 -- 172.21.15.1:0/3171412079 <== mon.2 v2:172.21.15.7:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1d8037070 con 0x7fe1e80cac60 2024-05-09T08:02:37.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:37.626+0000 7fe1e5400640 1 -- 172.21.15.1:0/3171412079 <== mon.2 v2:172.21.15.7:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1d802ae50 con 0x7fe1e80cac60 2024-05-09T08:02:37.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:37.626+0000 7fe1e5400640 1 -- 172.21.15.1:0/3171412079 <== mon.2 v2:172.21.15.7:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1d8032860 con 0x7fe1e80cac60 2024-05-09T08:02:37.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:37.626+0000 7fe1e7200640 1 -- 172.21.15.1:0/3171412079 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fe1e80cac60 msgr2=0x7fe1e80cda00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:37.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:37.626+0000 7fe1e7200640 1 --2- 172.21.15.1:0/3171412079 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fe1e80cac60 0x7fe1e80cda00 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fe1d8009870 tx=0x7fe1d802a300 comp rx=0 tx=0).stop 2024-05-09T08:02:37.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:37.627+0000 7fe1e7200640 1 -- 172.21.15.1:0/3171412079 shutdown_connections 2024-05-09T08:02:37.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:37.627+0000 7fe1e7200640 1 --2- 172.21.15.1:0/3171412079 >> v2:172.21.15.1:3300/0 conn(0x7fe1e815ac80 0x7fe1e80cdf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:37.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:37.627+0000 7fe1e7200640 1 --2- 172.21.15.1:0/3171412079 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fe1e80cac60 0x7fe1e80cda00 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:37.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:37.627+0000 7fe1e7200640 1 -- 172.21.15.1:0/3171412079 >> 172.21.15.1:0/3171412079 conn(0x7fe1e80cfc20 msgr2=0x7fe1e80d0030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:02:37.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:37.627+0000 7fe1e7200640 1 -- 172.21.15.1:0/3171412079 shutdown_connections 2024-05-09T08:02:37.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:37.627+0000 7fe1e7200640 1 -- 172.21.15.1:0/3171412079 wait complete. 2024-05-09T08:02:37.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:37.628+0000 7fe1e7200640 1 Processor -- start 2024-05-09T08:02:37.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:37.628+0000 7fe1e7200640 1 -- start start 2024-05-09T08:02:37.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:37.629+0000 7fe1e7200640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fe1e80cac60 0x7fe1e8151b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:37.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:37.629+0000 7fe1e7200640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe1e80cc5f0 0x7fe1e8157280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:37.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:37.629+0000 7fe1e7200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe1e815ac80 0x7fe1e81577c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:37.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:37.629+0000 7fe1e7200640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fe1e80ce4e0 con 0x7fe1e80cc5f0 2024-05-09T08:02:37.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:37.629+0000 7fe1e7200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe1e80cd550 con 0x7fe1e815ac80 2024-05-09T08:02:37.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:37.629+0000 7fe1e7200640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7fe1e80ce660 con 0x7fe1e80cac60 2024-05-09T08:02:37.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:37.630+0000 7fe1dd400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe1e80cc5f0 0x7fe1e8157280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:37.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:37.630+0000 7fe1dd400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe1e80cc5f0 0x7fe1e8157280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56428/0 (socket says 172.21.15.1:56428) 2024-05-09T08:02:37.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:37.630+0000 7fe1dd400640 1 -- 172.21.15.1:0/3123207086 learned_addr learned my addr 172.21.15.1:0/3123207086 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:02:37.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:37.630+0000 7fe1e5e00640 1 --2- 172.21.15.1:0/3123207086 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fe1e80cac60 0x7fe1e8151b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:37.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:37.630+0000 7fe1e6800640 1 --2- 172.21.15.1:0/3123207086 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe1e815ac80 0x7fe1e81577c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:37.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:37.630+0000 7fe1dd400640 1 -- 172.21.15.1:0/3123207086 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fe1e80cac60 msgr2=0x7fe1e8151b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:37.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:37.630+0000 7fe1dd400640 1 --2- 172.21.15.1:0/3123207086 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fe1e80cac60 0x7fe1e8151b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:37.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:37.630+0000 7fe1dd400640 1 -- 172.21.15.1:0/3123207086 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe1e815ac80 msgr2=0x7fe1e81577c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:37.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:37.630+0000 7fe1dd400640 1 --2- 172.21.15.1:0/3123207086 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe1e815ac80 0x7fe1e81577c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:37.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:37.630+0000 7fe1dd400640 1 -- 172.21.15.1:0/3123207086 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1e8152080 con 0x7fe1e80cc5f0 2024-05-09T08:02:37.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:37.630+0000 7fe1e5e00640 1 --2- 172.21.15.1:0/3123207086 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fe1e80cac60 0x7fe1e8151b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:02:37.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:37.630+0000 7fe1dd400640 1 --2- 172.21.15.1:0/3123207086 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe1e80cc5f0 0x7fe1e8157280 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe1d000c7c0 tx=0x7fe1d000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:37.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:37.631+0000 7fe1dea00640 1 -- 172.21.15.1:0/3123207086 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1d0016020 con 0x7fe1e80cc5f0 2024-05-09T08:02:37.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:37.631+0000 7fe1dea00640 1 -- 172.21.15.1:0/3123207086 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1d0006d50 con 0x7fe1e80cc5f0 2024-05-09T08:02:37.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:37.631+0000 7fe1dea00640 1 -- 172.21.15.1:0/3123207086 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1d0003bb0 con 0x7fe1e80cc5f0 2024-05-09T08:02:37.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:37.631+0000 7fe1e7200640 1 -- 172.21.15.1:0/3123207086 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1e81507c0 con 0x7fe1e80cc5f0 2024-05-09T08:02:37.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:37.631+0000 7fe1e7200640 1 -- 172.21.15.1:0/3123207086 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1e8150d80 con 0x7fe1e80cc5f0 2024-05-09T08:02:37.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:37.632+0000 7fe1e7200640 1 -- 172.21.15.1:0/3123207086 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1e80ca740 con 0x7fe1e80cc5f0 2024-05-09T08:02:37.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:37.633+0000 7fe1dea00640 1 -- 172.21.15.1:0/3123207086 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe1d000a850 con 0x7fe1e80cc5f0 2024-05-09T08:02:37.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:37.634+0000 7fe1dea00640 1 --2- 172.21.15.1:0/3123207086 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fe1c406be30 0x7fe1c406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:37.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:37.634+0000 7fe1dea00640 1 -- 172.21.15.1:0/3123207086 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fe1d008ad80 con 0x7fe1e80cc5f0 2024-05-09T08:02:37.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:37.634+0000 7fe1e5e00640 1 --2- 172.21.15.1:0/3123207086 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fe1c406be30 0x7fe1c406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:37.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:37.635+0000 7fe1e5e00640 1 --2- 172.21.15.1:0/3123207086 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fe1c406be30 0x7fe1c406e2f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe1d8034000 tx=0x7fe1d802e000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:37.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:37.638+0000 7fe1dea00640 1 -- 172.21.15.1:0/3123207086 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fe1e80ca740 con 0x7fe1e80cc5f0 2024-05-09T08:02:37.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:37.843+0000 7fe1e7200640 1 -- 172.21.15.1:0/3123207086 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fe1e804f660 con 0x7fe1e80cc5f0 2024-05-09T08:02:37.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:37.844+0000 7fe1dea00640 1 -- 172.21.15.1:0/3123207086 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) ==== 93+0+10 (secure 0 0 0) 0x7fe1d0020020 con 0x7fe1e80cc5f0 2024-05-09T08:02:37.861 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 1 2024-05-09T08:02:37.863 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:37.847+0000 7fe1e7200640 1 -- 172.21.15.1:0/3123207086 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fe1c406be30 msgr2=0x7fe1c406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:37.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:37.847+0000 7fe1e7200640 1 --2- 172.21.15.1:0/3123207086 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fe1c406be30 0x7fe1c406e2f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe1d8034000 tx=0x7fe1d802e000 comp rx=0 tx=0).stop 2024-05-09T08:02:37.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:37.847+0000 7fe1e7200640 1 -- 172.21.15.1:0/3123207086 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe1e80cc5f0 msgr2=0x7fe1e8157280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:37.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:37.847+0000 7fe1e7200640 1 --2- 172.21.15.1:0/3123207086 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe1e80cc5f0 0x7fe1e8157280 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe1d000c7c0 tx=0x7fe1d000cc80 comp rx=0 tx=0).stop 2024-05-09T08:02:37.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:37.847+0000 7fe1e7200640 1 -- 172.21.15.1:0/3123207086 shutdown_connections 2024-05-09T08:02:37.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:37.847+0000 7fe1e7200640 1 --2- 172.21.15.1:0/3123207086 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fe1c406be30 0x7fe1c406e2f0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:37.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:37.847+0000 7fe1e7200640 1 --2- 172.21.15.1:0/3123207086 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe1e815ac80 0x7fe1e81577c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:37.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:37.847+0000 7fe1e7200640 1 --2- 172.21.15.1:0/3123207086 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe1e80cc5f0 0x7fe1e8157280 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:37.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:37.847+0000 7fe1e7200640 1 --2- 172.21.15.1:0/3123207086 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fe1e80cac60 0x7fe1e8151b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:37.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:37.847+0000 7fe1e7200640 1 -- 172.21.15.1:0/3123207086 >> 172.21.15.1:0/3123207086 conn(0x7fe1e80cfc20 msgr2=0x7fe1e80c6f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:02:37.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:37.847+0000 7fe1e7200640 1 -- 172.21.15.1:0/3123207086 shutdown_connections 2024-05-09T08:02:37.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:37.847+0000 7fe1e7200640 1 -- 172.21.15.1:0/3123207086 wait complete. 2024-05-09T08:02:37.883 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-09T08:02:38.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:38.080+0000 7f5fd5600640 1 Processor -- start 2024-05-09T08:02:38.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:38.080+0000 7f5fd5600640 1 -- start start 2024-05-09T08:02:38.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:38.081+0000 7f5fd5600640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f5fd015c360 0x7f5fd00c9570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:38.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:38.081+0000 7f5fd5600640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f5fd0072c20 0x7f5fd0073050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:38.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:38.081+0000 7f5fd5600640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5fd0071c80 con 0x7f5fd00c5e40 2024-05-09T08:02:38.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:38.081+0000 7f5fd5600640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f5fd015b980 con 0x7f5fd0072c20 2024-05-09T08:02:38.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:38.081+0000 7f5fd5600640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f5fd0071f20 con 0x7f5fd015c360 2024-05-09T08:02:38.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:38.082+0000 7f5fcf400640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f5fd0072c20 0x7f5fd0073050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:38.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:38.082+0000 7f5fcea00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f5fd015c360 0x7f5fd00c9570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:38.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:38.082+0000 7f5fcf400640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f5fd0072c20 0x7f5fd0073050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56434/0 (socket says 172.21.15.1:56434) 2024-05-09T08:02:38.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:38.082+0000 7f5fc7e00640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f5fd00c5e40 0x7f5fd00cc5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:58844/0 (socket says 172.21.15.1:58844) 2024-05-09T08:02:38.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:38.082+0000 7f5fcf400640 1 -- 172.21.15.1:0/1725118681 learned_addr learned my addr 172.21.15.1:0/1725118681 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:02:38.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:38.082+0000 7f5fcf400640 1 -- 172.21.15.1:0/1725118681 >> v2:172.21.15.7:3300/0 conn(0x7f5fd015c360 msgr2=0x7f5fd00c9570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:02:38.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:38.082+0000 7f5fcf400640 1 --2- 172.21.15.1:0/1725118681 >> v2:172.21.15.7:3300/0 conn(0x7f5fd015c360 0x7f5fd00c9570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:38.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:38.082+0000 7f5fcf400640 1 -- 172.21.15.1:0/1725118681 >> v1:172.21.15.1:6789/0 conn(0x7f5fd00c5e40 legacy=0x7f5fd00cc5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:38.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:38.083+0000 7f5fcf400640 1 -- 172.21.15.1:0/1725118681 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fd00721e0 con 0x7f5fd0072c20 2024-05-09T08:02:38.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:38.083+0000 7f5fce000640 1 -- 172.21.15.1:0/1725118681 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 585298061 0 0) 0x7f5fd0071c80 con 0x7f5fd00c5e40 2024-05-09T08:02:38.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:38.084+0000 7f5fcf400640 1 --2- 172.21.15.1:0/1725118681 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5fd0072c20 0x7f5fd0073050 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f5fd00c8c60 tx=0x7f5fc802a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a443e7643b170d2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:38.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:38.084+0000 7f5fce000640 1 -- 172.21.15.1:0/1725118681 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fc8036070 con 0x7f5fd0072c20 2024-05-09T08:02:38.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:38.084+0000 7f5fce000640 1 -- 172.21.15.1:0/1725118681 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5fc8031600 con 0x7f5fd0072c20 2024-05-09T08:02:38.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:38.084+0000 7f5fce000640 1 -- 172.21.15.1:0/1725118681 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fc80319d0 con 0x7f5fd0072c20 2024-05-09T08:02:38.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:38.084+0000 7f5fd5600640 1 -- 172.21.15.1:0/1725118681 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5fd0072c20 msgr2=0x7f5fd0073050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:38.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:38.084+0000 7f5fd5600640 1 --2- 172.21.15.1:0/1725118681 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5fd0072c20 0x7f5fd0073050 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f5fd00c8c60 tx=0x7f5fc802a610 comp rx=0 tx=0).stop 2024-05-09T08:02:38.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:38.085+0000 7f5fd5600640 1 -- 172.21.15.1:0/1725118681 shutdown_connections 2024-05-09T08:02:38.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:38.085+0000 7f5fd5600640 1 --2- 172.21.15.1:0/1725118681 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5fd0072c20 0x7f5fd0073050 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:38.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:38.085+0000 7f5fd5600640 1 --2- 172.21.15.1:0/1725118681 >> v2:172.21.15.7:3300/0 conn(0x7f5fd015c360 0x7f5fd00c9570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:38.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:38.085+0000 7f5fd5600640 1 -- 172.21.15.1:0/1725118681 >> 172.21.15.1:0/1725118681 conn(0x7f5fd006dad0 msgr2=0x7f5fd006deb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:02:38.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:38.085+0000 7f5fd5600640 1 -- 172.21.15.1:0/1725118681 shutdown_connections 2024-05-09T08:02:38.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:38.085+0000 7f5fd5600640 1 -- 172.21.15.1:0/1725118681 wait complete. 2024-05-09T08:02:38.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:38.087+0000 7f5fd5600640 1 Processor -- start 2024-05-09T08:02:38.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:38.087+0000 7f5fd5600640 1 -- start start 2024-05-09T08:02:38.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:38.088+0000 7f5fd5600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5fd0072c20 0x7f5fd01f0d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:38.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:38.088+0000 7f5fd5600640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f5fd00c5e40 0x7f5fd0156070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:38.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:38.088+0000 7f5fd5600640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5fd015c360 0x7f5fd0142350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:38.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:38.088+0000 7f5fd5600640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f5fd00cc0d0 con 0x7f5fd015c360 2024-05-09T08:02:38.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:38.088+0000 7f5fd5600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5fd00727a0 con 0x7f5fd0072c20 2024-05-09T08:02:38.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:38.088+0000 7f5fd5600640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f5fd00ccd00 con 0x7f5fd00c5e40 2024-05-09T08:02:38.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:38.088+0000 7f5fc7e00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f5fd00c5e40 0x7f5fd0156070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:38.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:38.088+0000 7f5fcea00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5fd0072c20 0x7f5fd01f0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:38.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:38.088+0000 7f5fcf400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5fd015c360 0x7f5fd0142350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:38.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:38.089+0000 7f5fc7e00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f5fd00c5e40 0x7f5fd0156070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3301/0 says I am v2:172.21.15.1:40136/0 (socket says 172.21.15.1:40136) 2024-05-09T08:02:38.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:38.089+0000 7f5fc7e00640 1 -- 172.21.15.1:0/2632004337 learned_addr learned my addr 172.21.15.1:0/2632004337 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:02:38.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:38.089+0000 7f5fc7e00640 1 -- 172.21.15.1:0/2632004337 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5fd0072c20 msgr2=0x7f5fd01f0d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:38.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:38.089+0000 7f5fc7e00640 1 --2- 172.21.15.1:0/2632004337 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5fd0072c20 0x7f5fd01f0d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:38.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:38.089+0000 7f5fc7e00640 1 -- 172.21.15.1:0/2632004337 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5fd015c360 msgr2=0x7f5fd0142350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:38.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:38.089+0000 7f5fc7e00640 1 --2- 172.21.15.1:0/2632004337 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5fd015c360 0x7f5fd0142350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:38.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:38.089+0000 7f5fc7e00640 1 -- 172.21.15.1:0/2632004337 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fd0157b10 con 0x7f5fd00c5e40 2024-05-09T08:02:38.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:38.089+0000 7f5fcf400640 1 --2- 172.21.15.1:0/2632004337 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5fd015c360 0x7f5fd0142350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:02:38.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:38.089+0000 7f5fc7e00640 1 --2- 172.21.15.1:0/2632004337 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f5fd00c5e40 0x7f5fd0156070 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f5fb800c440 tx=0x7f5fb800c900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:38.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:38.090+0000 7f5fc6a00640 1 -- 172.21.15.1:0/2632004337 <== mon.2 v2:172.21.15.7:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fb8018070 con 0x7f5fd00c5e40 2024-05-09T08:02:38.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:38.090+0000 7f5fc6a00640 1 -- 172.21.15.1:0/2632004337 <== mon.2 v2:172.21.15.7:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5fb80074a0 con 0x7f5fd00c5e40 2024-05-09T08:02:38.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:38.090+0000 7f5fc6a00640 1 -- 172.21.15.1:0/2632004337 <== mon.2 v2:172.21.15.7:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fb8004a20 con 0x7f5fd00c5e40 2024-05-09T08:02:38.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:38.090+0000 7f5fd5600640 1 -- 172.21.15.1:0/2632004337 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fd0157db0 con 0x7f5fd00c5e40 2024-05-09T08:02:38.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:38.090+0000 7f5fd5600640 1 -- 172.21.15.1:0/2632004337 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fd0159990 con 0x7f5fd00c5e40 2024-05-09T08:02:38.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:38.091+0000 7f5fd5600640 1 -- 172.21.15.1:0/2632004337 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5fd0074d70 con 0x7f5fd00c5e40 2024-05-09T08:02:38.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:38.092+0000 7f5fc6a00640 1 -- 172.21.15.1:0/2632004337 <== mon.2 v2:172.21.15.7:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5fb8004bc0 con 0x7f5fd00c5e40 2024-05-09T08:02:38.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:38.093+0000 7f5fc6a00640 1 --2- 172.21.15.1:0/2632004337 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f5fac06be60 0x7f5fac06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:38.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:38.093+0000 7f5fc6a00640 1 -- 172.21.15.1:0/2632004337 <== mon.2 v2:172.21.15.7:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f5fb808a390 con 0x7f5fd00c5e40 2024-05-09T08:02:38.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:38.093+0000 7f5fcea00640 1 --2- 172.21.15.1:0/2632004337 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f5fac06be60 0x7f5fac06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:38.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:38.094+0000 7f5fcea00640 1 --2- 172.21.15.1:0/2632004337 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f5fac06be60 0x7f5fac06e320 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f5fc0009aa0 tx=0x7f5fc0009190 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:38.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:38.098+0000 7f5fc6a00640 1 -- 172.21.15.1:0/2632004337 <== mon.2 v2:172.21.15.7:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f5fb8007650 con 0x7f5fd00c5e40 2024-05-09T08:02:38.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:38.299+0000 7f5fd5600640 1 -- 172.21.15.1:0/2632004337 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f5fd01565b0 con 0x7f5fd00c5e40 2024-05-09T08:02:38.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:38.300+0000 7f5fc6a00640 1 -- 172.21.15.1:0/2632004337 <== mon.2 v2:172.21.15.7:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) ==== 104+0+11 (secure 0 0 0) 0x7f5fb8058310 con 0x7f5fd00c5e40 2024-05-09T08:02:38.317 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 32 2024-05-09T08:02:38.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:38.303+0000 7f5fd5600640 1 -- 172.21.15.1:0/2632004337 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f5fac06be60 msgr2=0x7f5fac06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:38.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:38.303+0000 7f5fd5600640 1 --2- 172.21.15.1:0/2632004337 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f5fac06be60 0x7f5fac06e320 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f5fc0009aa0 tx=0x7f5fc0009190 comp rx=0 tx=0).stop 2024-05-09T08:02:38.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:38.303+0000 7f5fd5600640 1 -- 172.21.15.1:0/2632004337 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f5fd00c5e40 msgr2=0x7f5fd0156070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:38.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:38.303+0000 7f5fd5600640 1 --2- 172.21.15.1:0/2632004337 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f5fd00c5e40 0x7f5fd0156070 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f5fb800c440 tx=0x7f5fb800c900 comp rx=0 tx=0).stop 2024-05-09T08:02:38.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:38.303+0000 7f5fd5600640 1 -- 172.21.15.1:0/2632004337 shutdown_connections 2024-05-09T08:02:38.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:38.303+0000 7f5fd5600640 1 --2- 172.21.15.1:0/2632004337 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f5fac06be60 0x7f5fac06e320 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:38.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:38.303+0000 7f5fd5600640 1 --2- 172.21.15.1:0/2632004337 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5fd015c360 0x7f5fd0142350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:38.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:38.303+0000 7f5fd5600640 1 --2- 172.21.15.1:0/2632004337 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f5fd00c5e40 0x7f5fd0156070 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:38.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:38.303+0000 7f5fd5600640 1 --2- 172.21.15.1:0/2632004337 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5fd0072c20 0x7f5fd01f0d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:38.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:38.303+0000 7f5fd5600640 1 -- 172.21.15.1:0/2632004337 >> 172.21.15.1:0/2632004337 conn(0x7f5fd006dad0 msgr2=0x7f5fd0158460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:02:38.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:38.303+0000 7f5fd5600640 1 -- 172.21.15.1:0/2632004337 shutdown_connections 2024-05-09T08:02:38.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:38.303+0000 7f5fd5600640 1 -- 172.21.15.1:0/2632004337 wait complete. 2024-05-09T08:02:38.339 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-09T08:02:38.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:38.537+0000 7ff4cfe00640 1 Processor -- start 2024-05-09T08:02:38.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:38.537+0000 7ff4cfe00640 1 -- start start 2024-05-09T08:02:38.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:38.538+0000 7ff4cfe00640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7ff4c815c360 0x7ff4c80c9570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:38.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:38.539+0000 7ff4cfe00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff4c8072c20 0x7ff4c8073050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:38.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:38.539+0000 7ff4cfe00640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4c8071c80 con 0x7ff4c80c5e40 2024-05-09T08:02:38.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:38.539+0000 7ff4cfe00640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7ff4c815b980 con 0x7ff4c815c360 2024-05-09T08:02:38.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:38.539+0000 7ff4cfe00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7ff4c8071f20 con 0x7ff4c8072c20 2024-05-09T08:02:38.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:38.539+0000 7ff4cd400640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7ff4c815c360 0x7ff4c80c9570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:38.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:38.539+0000 7ff4cca00640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7ff4c80c5e40 0x7ff4c80cc5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:58860/0 (socket says 172.21.15.1:58860) 2024-05-09T08:02:38.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:38.539+0000 7ff4cca00640 1 -- 172.21.15.1:0/1082876001 learned_addr learned my addr 172.21.15.1:0/1082876001 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T08:02:38.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:38.539+0000 7ff4cde00640 1 --2- 172.21.15.1:0/1082876001 >> v2:172.21.15.7:3300/0 conn(0x7ff4c8072c20 0x7ff4c8073050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:38.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:38.540+0000 7ff4cd400640 1 -- 172.21.15.1:0/1082876001 >> v2:172.21.15.7:3300/0 conn(0x7ff4c8072c20 msgr2=0x7ff4c8073050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:02:38.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:38.540+0000 7ff4cd400640 1 --2- 172.21.15.1:0/1082876001 >> v2:172.21.15.7:3300/0 conn(0x7ff4c8072c20 0x7ff4c8073050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:38.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:38.540+0000 7ff4cd400640 1 -- 172.21.15.1:0/1082876001 >> v1:172.21.15.1:6789/0 conn(0x7ff4c80c5e40 legacy=0x7ff4c80cc5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:38.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:38.540+0000 7ff4cd400640 1 -- 172.21.15.1:0/1082876001 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4c80721e0 con 0x7ff4c815c360 2024-05-09T08:02:38.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:38.541+0000 7ff4cd400640 1 --2- 172.21.15.1:0/1082876001 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff4c815c360 0x7ff4c80c9570 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff4c80c6d30 tx=0x7ff4b402a080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0862c294f6514d2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:38.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:38.541+0000 7ff4c7e00640 1 -- 172.21.15.1:0/1082876001 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4b4037070 con 0x7ff4c815c360 2024-05-09T08:02:38.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:38.541+0000 7ff4c7e00640 1 -- 172.21.15.1:0/1082876001 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4b402add0 con 0x7ff4c815c360 2024-05-09T08:02:38.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:38.541+0000 7ff4c7e00640 1 -- 172.21.15.1:0/1082876001 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4b4032660 con 0x7ff4c815c360 2024-05-09T08:02:38.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:38.542+0000 7ff4cfe00640 1 -- 172.21.15.1:0/1082876001 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff4c815c360 msgr2=0x7ff4c80c9570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:38.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:38.542+0000 7ff4cfe00640 1 --2- 172.21.15.1:0/1082876001 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff4c815c360 0x7ff4c80c9570 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff4c80c6d30 tx=0x7ff4b402a080 comp rx=0 tx=0).stop 2024-05-09T08:02:38.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:38.542+0000 7ff4cfe00640 1 -- 172.21.15.1:0/1082876001 shutdown_connections 2024-05-09T08:02:38.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:38.542+0000 7ff4cfe00640 1 --2- 172.21.15.1:0/1082876001 >> v2:172.21.15.7:3300/0 conn(0x7ff4c8072c20 0x7ff4c8073050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:38.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:38.542+0000 7ff4cfe00640 1 --2- 172.21.15.1:0/1082876001 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff4c815c360 0x7ff4c80c9570 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:38.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:38.542+0000 7ff4cfe00640 1 -- 172.21.15.1:0/1082876001 >> 172.21.15.1:0/1082876001 conn(0x7ff4c806dad0 msgr2=0x7ff4c806deb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:02:38.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:38.542+0000 7ff4cfe00640 1 -- 172.21.15.1:0/1082876001 shutdown_connections 2024-05-09T08:02:38.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:38.542+0000 7ff4cfe00640 1 -- 172.21.15.1:0/1082876001 wait complete. 2024-05-09T08:02:38.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:38.544+0000 7ff4cfe00640 1 Processor -- start 2024-05-09T08:02:38.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:38.545+0000 7ff4cfe00640 1 -- start start 2024-05-09T08:02:38.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:38.546+0000 7ff4cfe00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff4c8072c20 0x7ff4c80ca5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:38.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:38.546+0000 7ff4cfe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4c80c5e40 0x7ff4c80cb6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:38.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:38.546+0000 7ff4cfe00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ff4c815c360 0x7ff4c81f17d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:38.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:38.546+0000 7ff4cfe00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ff4c80cc0d0 con 0x7ff4c8072c20 2024-05-09T08:02:38.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:38.546+0000 7ff4cfe00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff4c80727a0 con 0x7ff4c80c5e40 2024-05-09T08:02:38.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:38.546+0000 7ff4cfe00640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7ff4c80ccd00 con 0x7ff4c815c360 2024-05-09T08:02:38.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:38.546+0000 7ff4cd400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff4c8072c20 0x7ff4c80ca5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:38.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:38.546+0000 7ff4cca00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4c80c5e40 0x7ff4c80cb6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:38.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:38.546+0000 7ff4cd400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff4c8072c20 0x7ff4c80ca5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56456/0 (socket says 172.21.15.1:56456) 2024-05-09T08:02:38.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:38.546+0000 7ff4cd400640 1 -- 172.21.15.1:0/2408753989 learned_addr learned my addr 172.21.15.1:0/2408753989 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:02:38.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:38.546+0000 7ff4cde00640 1 --2- 172.21.15.1:0/2408753989 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ff4c815c360 0x7ff4c81f17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:38.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:38.547+0000 7ff4cd400640 1 -- 172.21.15.1:0/2408753989 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ff4c815c360 msgr2=0x7ff4c81f17d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:38.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:38.547+0000 7ff4cd400640 1 --2- 172.21.15.1:0/2408753989 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ff4c815c360 0x7ff4c81f17d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:38.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:38.547+0000 7ff4cd400640 1 -- 172.21.15.1:0/2408753989 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4c80c5e40 msgr2=0x7ff4c80cb6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:38.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:38.547+0000 7ff4cd400640 1 --2- 172.21.15.1:0/2408753989 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4c80c5e40 0x7ff4c80cb6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:38.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:38.547+0000 7ff4cd400640 1 -- 172.21.15.1:0/2408753989 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4c8156880 con 0x7ff4c8072c20 2024-05-09T08:02:38.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:38.547+0000 7ff4cca00640 1 --2- 172.21.15.1:0/2408753989 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4c80c5e40 0x7ff4c80cb6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:02:38.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:38.547+0000 7ff4cd400640 1 --2- 172.21.15.1:0/2408753989 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff4c8072c20 0x7ff4c80ca5a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff4b4000c00 tx=0x7ff4b40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:38.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:38.547+0000 7ff4c5600640 1 -- 172.21.15.1:0/2408753989 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4b4041030 con 0x7ff4c8072c20 2024-05-09T08:02:38.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:38.547+0000 7ff4c5600640 1 -- 172.21.15.1:0/2408753989 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4b402abd0 con 0x7ff4c8072c20 2024-05-09T08:02:38.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:38.547+0000 7ff4c5600640 1 -- 172.21.15.1:0/2408753989 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4b4037040 con 0x7ff4c8072c20 2024-05-09T08:02:38.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:38.547+0000 7ff4cfe00640 1 -- 172.21.15.1:0/2408753989 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4c80c67b0 con 0x7ff4c8072c20 2024-05-09T08:02:38.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:38.548+0000 7ff4cfe00640 1 -- 172.21.15.1:0/2408753989 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4c81562c0 con 0x7ff4c8072c20 2024-05-09T08:02:38.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:38.549+0000 7ff4cfe00640 1 -- 172.21.15.1:0/2408753989 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4c8074d70 con 0x7ff4c8072c20 2024-05-09T08:02:38.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:38.550+0000 7ff4c5600640 1 -- 172.21.15.1:0/2408753989 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff4b4044070 con 0x7ff4c8072c20 2024-05-09T08:02:38.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:38.551+0000 7ff4c5600640 1 --2- 172.21.15.1:0/2408753989 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7ff4a806be30 0x7ff4a806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:38.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:38.551+0000 7ff4c5600640 1 -- 172.21.15.1:0/2408753989 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7ff4b40ab400 con 0x7ff4c8072c20 2024-05-09T08:02:38.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:38.551+0000 7ff4cca00640 1 --2- 172.21.15.1:0/2408753989 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7ff4a806be30 0x7ff4a806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:38.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:38.552+0000 7ff4cca00640 1 --2- 172.21.15.1:0/2408753989 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7ff4a806be30 0x7ff4a806e2f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff4c80cb5a0 tx=0x7ff4bc007e80 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:38.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:38.555+0000 7ff4c5600640 1 -- 172.21.15.1:0/2408753989 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ff4c8156880 con 0x7ff4c8072c20 2024-05-09T08:02:38.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:38.762+0000 7ff4cfe00640 1 -- 172.21.15.1:0/2408753989 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7ff4c80683a0 con 0x7ff4c8072c20 2024-05-09T08:02:38.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:38.763+0000 7ff4c5600640 1 -- 172.21.15.1:0/2408753989 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) ==== 100+0+11 (secure 0 0 0) 0x7ff4b4079530 con 0x7ff4c8072c20 2024-05-09T08:02:38.780 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 64 2024-05-09T08:02:38.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:38.766+0000 7ff4cfe00640 1 -- 172.21.15.1:0/2408753989 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7ff4a806be30 msgr2=0x7ff4a806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:38.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:38.766+0000 7ff4cfe00640 1 --2- 172.21.15.1:0/2408753989 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7ff4a806be30 0x7ff4a806e2f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff4c80cb5a0 tx=0x7ff4bc007e80 comp rx=0 tx=0).stop 2024-05-09T08:02:38.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:38.766+0000 7ff4cfe00640 1 -- 172.21.15.1:0/2408753989 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff4c8072c20 msgr2=0x7ff4c80ca5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:38.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:38.766+0000 7ff4cfe00640 1 --2- 172.21.15.1:0/2408753989 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff4c8072c20 0x7ff4c80ca5a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff4b4000c00 tx=0x7ff4b40046c0 comp rx=0 tx=0).stop 2024-05-09T08:02:38.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:38.767+0000 7ff4cfe00640 1 -- 172.21.15.1:0/2408753989 shutdown_connections 2024-05-09T08:02:38.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:38.767+0000 7ff4cfe00640 1 --2- 172.21.15.1:0/2408753989 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7ff4a806be30 0x7ff4a806e2f0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:38.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:38.767+0000 7ff4cfe00640 1 --2- 172.21.15.1:0/2408753989 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4c80c5e40 0x7ff4c80cb6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:38.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:38.767+0000 7ff4cfe00640 1 --2- 172.21.15.1:0/2408753989 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ff4c815c360 0x7ff4c81f17d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:38.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:38.767+0000 7ff4cfe00640 1 --2- 172.21.15.1:0/2408753989 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff4c8072c20 0x7ff4c80ca5a0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:38.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:38.767+0000 7ff4cfe00640 1 -- 172.21.15.1:0/2408753989 >> 172.21.15.1:0/2408753989 conn(0x7ff4c806dad0 msgr2=0x7ff4c80c6dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:02:38.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:38.767+0000 7ff4cfe00640 1 -- 172.21.15.1:0/2408753989 shutdown_connections 2024-05-09T08:02:38.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:38.767+0000 7ff4cfe00640 1 -- 172.21.15.1:0/2408753989 wait complete. 2024-05-09T08:02:38.804 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-09T08:02:39.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:39.003+0000 7f17d9600640 1 Processor -- start 2024-05-09T08:02:39.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:39.003+0000 7f17d9600640 1 -- start start 2024-05-09T08:02:39.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:39.005+0000 7f17d9600640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f17d4152680 0x7f17d40c87d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:39.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:39.005+0000 7f17d9600640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f17d415ba10 con 0x7f17d40c3090 2024-05-09T08:02:39.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:39.005+0000 7f17d9600640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f17d415bcd0 con 0x7f17d41589b0 2024-05-09T08:02:39.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:39.005+0000 7f17d9600640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f17d4158520 con 0x7f17d4152680 2024-05-09T08:02:39.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:39.006+0000 7f17d2a00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f17d4152680 0x7f17d40c87d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:39.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:39.006+0000 7f17ca000640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f17d40c3090 0x7f17d40c3770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:58866/0 (socket says 172.21.15.1:58866) 2024-05-09T08:02:39.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:39.006+0000 7f17ca000640 1 -- 172.21.15.1:0/3765105964 learned_addr learned my addr 172.21.15.1:0/3765105964 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T08:02:39.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:39.006+0000 7f17d2a00640 1 -- 172.21.15.1:0/3765105964 >> v1:172.21.15.7:6789/0 conn(0x7f17d41589b0 legacy=0x7f17d4158e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:39.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:39.006+0000 7f17d2a00640 1 -- 172.21.15.1:0/3765105964 >> v1:172.21.15.1:6789/0 conn(0x7f17d40c3090 legacy=0x7f17d40c3770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:39.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:39.006+0000 7f17d2000640 1 -- 172.21.15.1:0/3765105964 <== mon.1 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 897766179 0 0) 0x7f17d415bcd0 con 0x7f17d41589b0 2024-05-09T08:02:39.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:39.006+0000 7f17d2a00640 1 -- 172.21.15.1:0/3765105964 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17d415bfb0 con 0x7f17d4152680 2024-05-09T08:02:39.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:39.007+0000 7f17d2a00640 1 --2- 172.21.15.1:0/3765105964 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f17d4152680 0x7f17d40c87d0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f17c40097d0 tx=0x7f17c402a1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=faf36a1d99adda39 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:39.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:39.008+0000 7f17d2000640 1 -- 172.21.15.1:0/3765105964 <== mon.2 v2:172.21.15.7:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17c4037070 con 0x7f17d4152680 2024-05-09T08:02:39.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:39.008+0000 7f17d2000640 1 -- 172.21.15.1:0/3765105964 <== mon.2 v2:172.21.15.7:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17c402ae30 con 0x7f17d4152680 2024-05-09T08:02:39.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:39.008+0000 7f17d2000640 1 -- 172.21.15.1:0/3765105964 <== mon.2 v2:172.21.15.7:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17c4032640 con 0x7f17d4152680 2024-05-09T08:02:39.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:39.008+0000 7f17d9600640 1 -- 172.21.15.1:0/3765105964 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f17d4152680 msgr2=0x7f17d40c87d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:39.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:39.008+0000 7f17d9600640 1 --2- 172.21.15.1:0/3765105964 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f17d4152680 0x7f17d40c87d0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f17c40097d0 tx=0x7f17c402a1a0 comp rx=0 tx=0).stop 2024-05-09T08:02:39.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:39.009+0000 7f17d9600640 1 -- 172.21.15.1:0/3765105964 shutdown_connections 2024-05-09T08:02:39.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:39.009+0000 7f17d9600640 1 --2- 172.21.15.1:0/3765105964 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f17d4152680 0x7f17d40c87d0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:39.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:39.009+0000 7f17d9600640 1 -- 172.21.15.1:0/3765105964 >> 172.21.15.1:0/3765105964 conn(0x7f17d4071870 msgr2=0x7f17d4071c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:02:39.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:39.009+0000 7f17d9600640 1 -- 172.21.15.1:0/3765105964 shutdown_connections 2024-05-09T08:02:39.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:39.009+0000 7f17d9600640 1 -- 172.21.15.1:0/3765105964 wait complete. 2024-05-09T08:02:39.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:39.011+0000 7f17d9600640 1 Processor -- start 2024-05-09T08:02:39.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:39.011+0000 7f17d9600640 1 -- start start 2024-05-09T08:02:39.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:39.012+0000 7f17d9600640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f17d40c3090 0x7f17d41f4560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:39.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:39.012+0000 7f17d9600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17d4152680 0x7f17d4146900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:39.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:39.012+0000 7f17d9600640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f17d41589b0 0x7f17d4146e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:39.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:39.013+0000 7f17d9600640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f17d415c850 con 0x7f17d40c3090 2024-05-09T08:02:39.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:39.013+0000 7f17d9600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f17d415c570 con 0x7f17d4152680 2024-05-09T08:02:39.033 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:39.013+0000 7f17d9600640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f17d40c8d10 con 0x7f17d41589b0 2024-05-09T08:02:39.033 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:39.013+0000 7f17d2a00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f17d40c3090 0x7f17d41f4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:39.033 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:39.013+0000 7f17d2a00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f17d40c3090 0x7f17d41f4560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56472/0 (socket says 172.21.15.1:56472) 2024-05-09T08:02:39.033 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:39.013+0000 7f17d2a00640 1 -- 172.21.15.1:0/1619074914 learned_addr learned my addr 172.21.15.1:0/1619074914 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:02:39.033 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:39.013+0000 7f17ca000640 1 --2- 172.21.15.1:0/1619074914 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17d4152680 0x7f17d4146900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:39.033 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:39.013+0000 7f17d3400640 1 --2- 172.21.15.1:0/1619074914 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f17d41589b0 0x7f17d4146e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:39.033 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:39.013+0000 7f17d2a00640 1 -- 172.21.15.1:0/1619074914 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f17d41589b0 msgr2=0x7f17d4146e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:39.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:39.013+0000 7f17d2a00640 1 --2- 172.21.15.1:0/1619074914 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f17d41589b0 0x7f17d4146e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:39.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:39.013+0000 7f17d2a00640 1 -- 172.21.15.1:0/1619074914 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17d4152680 msgr2=0x7f17d4146900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:39.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:39.013+0000 7f17d2a00640 1 --2- 172.21.15.1:0/1619074914 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17d4152680 0x7f17d4146900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:39.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:39.013+0000 7f17d2a00640 1 -- 172.21.15.1:0/1619074914 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17d41f4aa0 con 0x7f17d40c3090 2024-05-09T08:02:39.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:39.013+0000 7f17ca000640 1 --2- 172.21.15.1:0/1619074914 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17d4152680 0x7f17d4146900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:02:39.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:39.014+0000 7f17d2a00640 1 --2- 172.21.15.1:0/1619074914 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f17d40c3090 0x7f17d41f4560 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f17c40098f0 tx=0x7f17c40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:39.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:39.014+0000 7f17cb400640 1 -- 172.21.15.1:0/1619074914 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17c4037040 con 0x7f17d40c3090 2024-05-09T08:02:39.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:39.014+0000 7f17cb400640 1 -- 172.21.15.1:0/1619074914 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17c4004030 con 0x7f17d40c3090 2024-05-09T08:02:39.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:39.014+0000 7f17cb400640 1 -- 172.21.15.1:0/1619074914 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17c4004420 con 0x7f17d40c3090 2024-05-09T08:02:39.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:39.014+0000 7f17d9600640 1 -- 172.21.15.1:0/1619074914 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17d41f0850 con 0x7f17d40c3090 2024-05-09T08:02:39.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:39.014+0000 7f17d9600640 1 -- 172.21.15.1:0/1619074914 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17d41f0e10 con 0x7f17d40c3090 2024-05-09T08:02:39.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:39.016+0000 7f17d9600640 1 -- 172.21.15.1:0/1619074914 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17d4074d70 con 0x7f17d40c3090 2024-05-09T08:02:39.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:39.017+0000 7f17cb400640 1 -- 172.21.15.1:0/1619074914 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f17c403a040 con 0x7f17d40c3090 2024-05-09T08:02:39.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:39.018+0000 7f17cb400640 1 --2- 172.21.15.1:0/1619074914 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f17b006be30 0x7f17b006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:39.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:39.018+0000 7f17cb400640 1 -- 172.21.15.1:0/1619074914 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f17c40ab690 con 0x7f17d40c3090 2024-05-09T08:02:39.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:39.018+0000 7f17ca000640 1 --2- 172.21.15.1:0/1619074914 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f17b006be30 0x7f17b006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:39.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:39.019+0000 7f17ca000640 1 --2- 172.21.15.1:0/1619074914 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f17b006be30 0x7f17b006e2f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f17d41eaf80 tx=0x7f17bc007e60 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:39.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:39.023+0000 7f17cb400640 1 -- 172.21.15.1:0/1619074914 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f17c4042020 con 0x7f17d40c3090 2024-05-09T08:02:39.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:39.230+0000 7f17d9600640 1 -- 172.21.15.1:0/1619074914 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f17d4150970 con 0x7f17d40c3090 2024-05-09T08:02:39.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:39.231+0000 7f17cb400640 1 -- 172.21.15.1:0/1619074914 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) ==== 103+0+11 (secure 0 0 0) 0x7f17c40796a0 con 0x7f17d40c3090 2024-05-09T08:02:39.248 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 32 2024-05-09T08:02:39.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:39.234+0000 7f17d9600640 1 -- 172.21.15.1:0/1619074914 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f17b006be30 msgr2=0x7f17b006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:39.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:39.234+0000 7f17d9600640 1 --2- 172.21.15.1:0/1619074914 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f17b006be30 0x7f17b006e2f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f17d41eaf80 tx=0x7f17bc007e60 comp rx=0 tx=0).stop 2024-05-09T08:02:39.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:39.234+0000 7f17d9600640 1 -- 172.21.15.1:0/1619074914 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f17d40c3090 msgr2=0x7f17d41f4560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:39.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:39.234+0000 7f17d9600640 1 --2- 172.21.15.1:0/1619074914 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f17d40c3090 0x7f17d41f4560 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f17c40098f0 tx=0x7f17c40046c0 comp rx=0 tx=0).stop 2024-05-09T08:02:39.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:39.234+0000 7f17d9600640 1 -- 172.21.15.1:0/1619074914 shutdown_connections 2024-05-09T08:02:39.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:39.234+0000 7f17d9600640 1 --2- 172.21.15.1:0/1619074914 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f17b006be30 0x7f17b006e2f0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:39.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:39.234+0000 7f17d9600640 1 --2- 172.21.15.1:0/1619074914 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17d4152680 0x7f17d4146900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:39.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:39.234+0000 7f17d9600640 1 --2- 172.21.15.1:0/1619074914 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f17d41589b0 0x7f17d4146e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:39.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:39.234+0000 7f17d9600640 1 --2- 172.21.15.1:0/1619074914 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f17d40c3090 0x7f17d41f4560 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:39.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:39.234+0000 7f17d9600640 1 -- 172.21.15.1:0/1619074914 >> 172.21.15.1:0/1619074914 conn(0x7f17d4071870 msgr2=0x7f17d4151450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:02:39.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:39.234+0000 7f17d9600640 1 -- 172.21.15.1:0/1619074914 shutdown_connections 2024-05-09T08:02:39.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:39.235+0000 7f17d9600640 1 -- 172.21.15.1:0/1619074914 wait complete. 2024-05-09T08:02:39.270 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-09T08:02:39.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:39.461+0000 7fbb28400640 1 Processor -- start 2024-05-09T08:02:39.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:39.461+0000 7fbb28400640 1 -- start start 2024-05-09T08:02:39.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:39.463+0000 7fbb28400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fbb200c5e40 0x7fbb200c9570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:39.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:39.463+0000 7fbb28400640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fbb20072c20 0x7fbb20073050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:39.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:39.463+0000 7fbb28400640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbb20071c80 con 0x7fbb2015c360 2024-05-09T08:02:39.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:39.463+0000 7fbb28400640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7fbb2015b980 con 0x7fbb20072c20 2024-05-09T08:02:39.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:39.463+0000 7fbb28400640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fbb20071f20 con 0x7fbb200c5e40 2024-05-09T08:02:39.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:39.463+0000 7fbb26400640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fbb20072c20 0x7fbb20073050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:39.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:39.463+0000 7fbb26400640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fbb20072c20 0x7fbb20073050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56476/0 (socket says 172.21.15.1:56476) 2024-05-09T08:02:39.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:39.463+0000 7fbb26400640 1 -- 172.21.15.1:0/1517425214 learned_addr learned my addr 172.21.15.1:0/1517425214 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:02:39.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:39.463+0000 7fbb25000640 1 --2- 172.21.15.1:0/1517425214 >> v2:172.21.15.7:3300/0 conn(0x7fbb200c5e40 0x7fbb200c9570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:39.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:39.464+0000 7fbb1fe00640 1 -- 172.21.15.1:0/1517425214 <== mon.1 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1164458129 0 0) 0x7fbb20071c80 con 0x7fbb2015c360 2024-05-09T08:02:39.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:39.464+0000 7fbb1fe00640 1 -- 172.21.15.1:0/1517425214 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbb08003520 con 0x7fbb2015c360 2024-05-09T08:02:39.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:39.464+0000 7fbb25000640 1 -- 172.21.15.1:0/1517425214 >> v2:172.21.15.1:3300/0 conn(0x7fbb20072c20 msgr2=0x7fbb20073050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:02:39.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:39.464+0000 7fbb25000640 1 --2- 172.21.15.1:0/1517425214 >> v2:172.21.15.1:3300/0 conn(0x7fbb20072c20 0x7fbb20073050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:39.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:39.464+0000 7fbb25000640 1 -- 172.21.15.1:0/1517425214 >> v1:172.21.15.7:6789/0 conn(0x7fbb2015c360 legacy=0x7fbb200cc5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:39.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:39.464+0000 7fbb25000640 1 -- 172.21.15.1:0/1517425214 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb200721e0 con 0x7fbb200c5e40 2024-05-09T08:02:39.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:39.464+0000 7fbb26400640 1 --2- 172.21.15.1:0/1517425214 >> v2:172.21.15.1:3300/0 conn(0x7fbb20072c20 0x7fbb20073050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:02:39.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:39.465+0000 7fbb25000640 1 --2- 172.21.15.1:0/1517425214 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb200c5e40 0x7fbb200c9570 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fbb20073990 tx=0x7fbb0c02a080 comp rx=0 tx=0).ready entity=mon.1 client_cookie=29ce9f88515d637e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:39.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:39.465+0000 7fbb1fe00640 1 -- 172.21.15.1:0/1517425214 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb0c037070 con 0x7fbb200c5e40 2024-05-09T08:02:39.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:39.466+0000 7fbb1fe00640 1 -- 172.21.15.1:0/1517425214 <== mon.1 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb0c02add0 con 0x7fbb200c5e40 2024-05-09T08:02:39.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:39.466+0000 7fbb1fe00640 1 -- 172.21.15.1:0/1517425214 <== mon.1 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb0c032640 con 0x7fbb200c5e40 2024-05-09T08:02:39.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:39.466+0000 7fbb28400640 1 -- 172.21.15.1:0/1517425214 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb200c5e40 msgr2=0x7fbb200c9570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:39.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:39.466+0000 7fbb28400640 1 --2- 172.21.15.1:0/1517425214 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb200c5e40 0x7fbb200c9570 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fbb20073990 tx=0x7fbb0c02a080 comp rx=0 tx=0).stop 2024-05-09T08:02:39.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:39.466+0000 7fbb28400640 1 -- 172.21.15.1:0/1517425214 shutdown_connections 2024-05-09T08:02:39.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:39.466+0000 7fbb28400640 1 --2- 172.21.15.1:0/1517425214 >> v2:172.21.15.1:3300/0 conn(0x7fbb20072c20 0x7fbb20073050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:39.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:39.466+0000 7fbb28400640 1 --2- 172.21.15.1:0/1517425214 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb200c5e40 0x7fbb200c9570 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:39.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:39.466+0000 7fbb28400640 1 -- 172.21.15.1:0/1517425214 >> 172.21.15.1:0/1517425214 conn(0x7fbb2006dad0 msgr2=0x7fbb2006deb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:02:39.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:39.467+0000 7fbb28400640 1 -- 172.21.15.1:0/1517425214 shutdown_connections 2024-05-09T08:02:39.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:39.467+0000 7fbb28400640 1 -- 172.21.15.1:0/1517425214 wait complete. 2024-05-09T08:02:39.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:39.469+0000 7fbb28400640 1 Processor -- start 2024-05-09T08:02:39.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:39.469+0000 7fbb28400640 1 -- start start 2024-05-09T08:02:39.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:39.470+0000 7fbb28400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbb20072c20 0x7fbb20155360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:39.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:39.470+0000 7fbb28400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb200c5e40 0x7fbb2014a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:39.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:39.470+0000 7fbb28400640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fbb2015c360 0x7fbb20156650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:39.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:39.470+0000 7fbb28400640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fbb200cc0d0 con 0x7fbb20072c20 2024-05-09T08:02:39.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:39.470+0000 7fbb28400640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fbb200727a0 con 0x7fbb200c5e40 2024-05-09T08:02:39.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:39.470+0000 7fbb28400640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7fbb200ccd00 con 0x7fbb2015c360 2024-05-09T08:02:39.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:39.471+0000 7fbb25a00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbb20072c20 0x7fbb20155360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:39.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:39.471+0000 7fbb25a00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbb20072c20 0x7fbb20155360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56484/0 (socket says 172.21.15.1:56484) 2024-05-09T08:02:39.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:39.471+0000 7fbb25a00640 1 -- 172.21.15.1:0/901253156 learned_addr learned my addr 172.21.15.1:0/901253156 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:02:39.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:39.471+0000 7fbb25000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb200c5e40 0x7fbb2014a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:39.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:39.471+0000 7fbb26400640 1 --2- 172.21.15.1:0/901253156 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fbb2015c360 0x7fbb20156650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:39.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:39.471+0000 7fbb25a00640 1 -- 172.21.15.1:0/901253156 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fbb2015c360 msgr2=0x7fbb20156650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:39.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:39.471+0000 7fbb25a00640 1 --2- 172.21.15.1:0/901253156 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fbb2015c360 0x7fbb20156650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:39.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:39.471+0000 7fbb25a00640 1 -- 172.21.15.1:0/901253156 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb200c5e40 msgr2=0x7fbb2014a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:39.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:39.471+0000 7fbb25a00640 1 --2- 172.21.15.1:0/901253156 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb200c5e40 0x7fbb2014a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:39.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:39.471+0000 7fbb25a00640 1 -- 172.21.15.1:0/901253156 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb2014cdf0 con 0x7fbb20072c20 2024-05-09T08:02:39.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:39.471+0000 7fbb25000640 1 --2- 172.21.15.1:0/901253156 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb200c5e40 0x7fbb2014a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:02:39.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:39.471+0000 7fbb25a00640 1 --2- 172.21.15.1:0/901253156 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbb20072c20 0x7fbb20155360 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fbb1400c460 tx=0x7fbb1400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:39.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:39.472+0000 7fbb1d600640 1 -- 172.21.15.1:0/901253156 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb14017070 con 0x7fbb20072c20 2024-05-09T08:02:39.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:39.472+0000 7fbb1d600640 1 -- 172.21.15.1:0/901253156 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb140052f0 con 0x7fbb20072c20 2024-05-09T08:02:39.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:39.472+0000 7fbb1d600640 1 -- 172.21.15.1:0/901253156 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb14003830 con 0x7fbb20072c20 2024-05-09T08:02:39.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:39.472+0000 7fbb28400640 1 -- 172.21.15.1:0/901253156 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb201f0ab0 con 0x7fbb20072c20 2024-05-09T08:02:39.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:39.472+0000 7fbb28400640 1 -- 172.21.15.1:0/901253156 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb2014d370 con 0x7fbb20072c20 2024-05-09T08:02:39.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:39.474+0000 7fbb28400640 1 -- 172.21.15.1:0/901253156 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb20074d70 con 0x7fbb20072c20 2024-05-09T08:02:39.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:39.474+0000 7fbb1d600640 1 -- 172.21.15.1:0/901253156 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fbb1401a070 con 0x7fbb20072c20 2024-05-09T08:02:39.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:39.475+0000 7fbb1d600640 1 --2- 172.21.15.1:0/901253156 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fbb0006be80 0x7fbb0006e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:39.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:39.475+0000 7fbb1d600640 1 -- 172.21.15.1:0/901253156 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fbb1408a530 con 0x7fbb20072c20 2024-05-09T08:02:39.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:39.475+0000 7fbb25000640 1 --2- 172.21.15.1:0/901253156 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fbb0006be80 0x7fbb0006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:39.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:39.476+0000 7fbb25000640 1 --2- 172.21.15.1:0/901253156 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fbb0006be80 0x7fbb0006e340 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c034000 tx=0x7fbb0c02e000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:39.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:39.480+0000 7fbb1d600640 1 -- 172.21.15.1:0/901253156 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fbb14058660 con 0x7fbb20072c20 2024-05-09T08:02:39.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:39.698+0000 7fbb28400640 1 -- 172.21.15.1:0/901253156 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fbb200c5160 con 0x7fbb20072c20 2024-05-09T08:02:39.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:39.699+0000 7fbb1d600640 1 -- 172.21.15.1:0/901253156 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v25) ==== 80+0+0 (secure 0 0 0) 0x7fbb1405c960 con 0x7fbb20072c20 2024-05-09T08:02:39.716 INFO:teuthology.orchestra.run.smithi001.stderr:listed 0 entries 2024-05-09T08:02:39.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:39.701+0000 7fbb28400640 1 -- 172.21.15.1:0/901253156 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fbb0006be80 msgr2=0x7fbb0006e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:39.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:39.701+0000 7fbb28400640 1 --2- 172.21.15.1:0/901253156 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fbb0006be80 0x7fbb0006e340 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c034000 tx=0x7fbb0c02e000 comp rx=0 tx=0).stop 2024-05-09T08:02:39.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:39.701+0000 7fbb28400640 1 -- 172.21.15.1:0/901253156 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbb20072c20 msgr2=0x7fbb20155360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:39.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:39.701+0000 7fbb28400640 1 --2- 172.21.15.1:0/901253156 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbb20072c20 0x7fbb20155360 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fbb1400c460 tx=0x7fbb1400c920 comp rx=0 tx=0).stop 2024-05-09T08:02:39.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:39.701+0000 7fbb28400640 1 -- 172.21.15.1:0/901253156 shutdown_connections 2024-05-09T08:02:39.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:39.702+0000 7fbb28400640 1 --2- 172.21.15.1:0/901253156 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fbb0006be80 0x7fbb0006e340 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:39.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:39.702+0000 7fbb28400640 1 --2- 172.21.15.1:0/901253156 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb200c5e40 0x7fbb2014a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:39.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:39.702+0000 7fbb28400640 1 --2- 172.21.15.1:0/901253156 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fbb2015c360 0x7fbb20156650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:39.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:39.702+0000 7fbb28400640 1 --2- 172.21.15.1:0/901253156 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbb20072c20 0x7fbb20155360 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:39.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:39.702+0000 7fbb28400640 1 -- 172.21.15.1:0/901253156 >> 172.21.15.1:0/901253156 conn(0x7fbb2006dad0 msgr2=0x7fbb20073000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:02:39.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:39.702+0000 7fbb28400640 1 -- 172.21.15.1:0/901253156 shutdown_connections 2024-05-09T08:02:39.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:39.702+0000 7fbb28400640 1 -- 172.21.15.1:0/901253156 wait complete. 2024-05-09T08:02:39.735 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi001.front.sepia.ceph.com... 2024-05-09T08:02:39.736 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T08:02:39.736 DEBUG:teuthology.orchestra.run.smithi001:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-05-09T08:02:39.815 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi001.stderr:ceph-fuse[40910]: fuse finished with error 0 and tester_r 0 2024-05-09T08:02:39.821 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T08:02:45.923 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi001.front.sepia.ceph.com 2024-05-09T08:02:45.923 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T08:02:45.923 DEBUG:teuthology.orchestra.run.smithi001:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-05-09T08:02:45.998 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T08:02:46.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:46.199+0000 7faa91600640 1 Processor -- start 2024-05-09T08:02:46.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:46.199+0000 7faa91600640 1 -- start start 2024-05-09T08:02:46.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:46.201+0000 7faa91600640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7faa8c152500 0x7faa8c0c8180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:46.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:46.201+0000 7faa91600640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7faa8c0cc730 0x7faa8c15c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:46.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:46.201+0000 7faa91600640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7faa8c0c7660 0x7faa8c0c7af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:46.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:46.201+0000 7faa91600640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7faa8c0c5e50 con 0x7faa8c152500 2024-05-09T08:02:46.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:46.201+0000 7faa91600640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7faa8c0c22d0 con 0x7faa8c0c7660 2024-05-09T08:02:46.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:46.201+0000 7faa91600640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7faa8c0ccb50 con 0x7faa8c0cc730 2024-05-09T08:02:46.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:46.201+0000 7faa83e00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7faa8c0cc730 0x7faa8c15c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:46.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:46.202+0000 7faa90c00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7faa8c0c7660 0x7faa8c0c7af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:46.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:46.202+0000 7faa83e00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7faa8c0cc730 0x7faa8c15c080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3301/0 says I am v2:172.21.15.1:41440/0 (socket says 172.21.15.1:41440) 2024-05-09T08:02:46.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:46.202+0000 7faa83e00640 1 -- 172.21.15.1:0/3326797259 learned_addr learned my addr 172.21.15.1:0/3326797259 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:02:46.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:46.202+0000 7faa8be00640 1 --2- 172.21.15.1:0/3326797259 >> v2:172.21.15.1:3300/0 conn(0x7faa8c152500 0x7faa8c0c8180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:46.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:46.202+0000 7faa83e00640 1 -- 172.21.15.1:0/3326797259 >> v2:172.21.15.7:3300/0 conn(0x7faa8c0c7660 msgr2=0x7faa8c0c7af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:02:46.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:46.202+0000 7faa83e00640 1 --2- 172.21.15.1:0/3326797259 >> v2:172.21.15.7:3300/0 conn(0x7faa8c0c7660 0x7faa8c0c7af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:46.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:46.202+0000 7faa83e00640 1 -- 172.21.15.1:0/3326797259 >> v2:172.21.15.1:3300/0 conn(0x7faa8c152500 msgr2=0x7faa8c0c8180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:02:46.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:46.202+0000 7faa83e00640 1 --2- 172.21.15.1:0/3326797259 >> v2:172.21.15.1:3300/0 conn(0x7faa8c152500 0x7faa8c0c8180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:46.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:46.202+0000 7faa83e00640 1 -- 172.21.15.1:0/3326797259 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa8c0c6a30 con 0x7faa8c0cc730 2024-05-09T08:02:46.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:46.203+0000 7faa83e00640 1 --2- 172.21.15.1:0/3326797259 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7faa8c0cc730 0x7faa8c15c080 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7faa8c06d680 tx=0x7faa7402a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c69aa62a0bbee259 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:46.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:46.204+0000 7faa8b400640 1 -- 172.21.15.1:0/3326797259 <== mon.2 v2:172.21.15.7:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa74037070 con 0x7faa8c0cc730 2024-05-09T08:02:46.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:46.204+0000 7faa8b400640 1 -- 172.21.15.1:0/3326797259 <== mon.2 v2:172.21.15.7:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa74032440 con 0x7faa8c0cc730 2024-05-09T08:02:46.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:46.204+0000 7faa8b400640 1 -- 172.21.15.1:0/3326797259 <== mon.2 v2:172.21.15.7:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa74032810 con 0x7faa8c0cc730 2024-05-09T08:02:46.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:46.204+0000 7faa91600640 1 -- 172.21.15.1:0/3326797259 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7faa8c0cc730 msgr2=0x7faa8c15c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:46.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:46.204+0000 7faa91600640 1 --2- 172.21.15.1:0/3326797259 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7faa8c0cc730 0x7faa8c15c080 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7faa8c06d680 tx=0x7faa7402a270 comp rx=0 tx=0).stop 2024-05-09T08:02:46.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:46.204+0000 7faa91600640 1 -- 172.21.15.1:0/3326797259 shutdown_connections 2024-05-09T08:02:46.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:46.204+0000 7faa91600640 1 --2- 172.21.15.1:0/3326797259 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7faa8c0cc730 0x7faa8c15c080 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:46.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:46.204+0000 7faa91600640 1 --2- 172.21.15.1:0/3326797259 >> v2:172.21.15.7:3300/0 conn(0x7faa8c0c7660 0x7faa8c0c7af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:46.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:46.204+0000 7faa91600640 1 --2- 172.21.15.1:0/3326797259 >> v2:172.21.15.1:3300/0 conn(0x7faa8c152500 0x7faa8c0c8180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:46.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:46.204+0000 7faa91600640 1 -- 172.21.15.1:0/3326797259 >> 172.21.15.1:0/3326797259 conn(0x7faa8c0cbc10 msgr2=0x7faa8c0cc020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:02:46.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:46.205+0000 7faa91600640 1 -- 172.21.15.1:0/3326797259 shutdown_connections 2024-05-09T08:02:46.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:46.205+0000 7faa91600640 1 -- 172.21.15.1:0/3326797259 wait complete. 2024-05-09T08:02:46.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:46.207+0000 7faa91600640 1 Processor -- start 2024-05-09T08:02:46.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:46.207+0000 7faa91600640 1 -- start start 2024-05-09T08:02:46.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:46.208+0000 7faa91600640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7faa8c0c7660 0x7faa8c153e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:46.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:46.208+0000 7faa91600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faa8c0cc730 0x7faa8c159d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:46.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:46.208+0000 7faa91600640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faa8c152500 0x7faa8c0c3210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:46.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:46.208+0000 7faa91600640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7faa8c0c72b0 con 0x7faa8c152500 2024-05-09T08:02:46.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:46.208+0000 7faa91600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7faa8c0c6fd0 con 0x7faa8c0cc730 2024-05-09T08:02:46.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:46.208+0000 7faa91600640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7faa8c15b880 con 0x7faa8c0c7660 2024-05-09T08:02:46.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:46.208+0000 7faa90c00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faa8c152500 0x7faa8c0c3210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:46.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:46.208+0000 7faa90c00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faa8c152500 0x7faa8c0c3210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:52902/0 (socket says 172.21.15.1:52902) 2024-05-09T08:02:46.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:46.208+0000 7faa90c00640 1 -- 172.21.15.1:0/971929335 learned_addr learned my addr 172.21.15.1:0/971929335 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:02:46.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:46.208+0000 7faa8be00640 1 --2- 172.21.15.1:0/971929335 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7faa8c0c7660 0x7faa8c153e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:46.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:46.208+0000 7faa83e00640 1 --2- 172.21.15.1:0/971929335 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faa8c0cc730 0x7faa8c159d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:46.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:46.209+0000 7faa90c00640 1 -- 172.21.15.1:0/971929335 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7faa8c0c7660 msgr2=0x7faa8c153e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:46.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:46.209+0000 7faa90c00640 1 --2- 172.21.15.1:0/971929335 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7faa8c0c7660 0x7faa8c153e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:46.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:46.209+0000 7faa90c00640 1 -- 172.21.15.1:0/971929335 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faa8c0cc730 msgr2=0x7faa8c159d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:46.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:46.209+0000 7faa90c00640 1 --2- 172.21.15.1:0/971929335 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faa8c0cc730 0x7faa8c159d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:46.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:46.209+0000 7faa90c00640 1 -- 172.21.15.1:0/971929335 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa8c1f3820 con 0x7faa8c152500 2024-05-09T08:02:46.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:46.209+0000 7faa90c00640 1 --2- 172.21.15.1:0/971929335 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faa8c152500 0x7faa8c0c3210 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7faa8400ef00 tx=0x7faa8400b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:46.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:46.209+0000 7faa88c00640 1 -- 172.21.15.1:0/971929335 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa84018020 con 0x7faa8c152500 2024-05-09T08:02:46.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:46.209+0000 7faa88c00640 1 -- 172.21.15.1:0/971929335 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa84014400 con 0x7faa8c152500 2024-05-09T08:02:46.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:46.209+0000 7faa88c00640 1 -- 172.21.15.1:0/971929335 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa840147f0 con 0x7faa8c152500 2024-05-09T08:02:46.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:46.209+0000 7faa91600640 1 -- 172.21.15.1:0/971929335 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa8c155370 con 0x7faa8c152500 2024-05-09T08:02:46.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:46.210+0000 7faa91600640 1 -- 172.21.15.1:0/971929335 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa8c1f3da0 con 0x7faa8c152500 2024-05-09T08:02:46.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:46.211+0000 7faa91600640 1 -- 172.21.15.1:0/971929335 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa8c06e8b0 con 0x7faa8c152500 2024-05-09T08:02:46.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:46.212+0000 7faa88c00640 1 -- 172.21.15.1:0/971929335 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7faa84005d10 con 0x7faa8c152500 2024-05-09T08:02:46.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:46.213+0000 7faa88c00640 1 --2- 172.21.15.1:0/971929335 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7faa6806be60 0x7faa6806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:46.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:46.213+0000 7faa88c00640 1 -- 172.21.15.1:0/971929335 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7faa8408b770 con 0x7faa8c152500 2024-05-09T08:02:46.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:46.213+0000 7faa8be00640 1 --2- 172.21.15.1:0/971929335 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7faa6806be60 0x7faa6806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:46.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:46.214+0000 7faa8be00640 1 --2- 172.21.15.1:0/971929335 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7faa6806be60 0x7faa6806e320 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7faa7c009aa0 tx=0x7faa7c009190 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:46.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:46.219+0000 7faa88c00640 1 -- 172.21.15.1:0/971929335 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7faa8c06e8b0 con 0x7faa8c152500 2024-05-09T08:02:46.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:46.428+0000 7faa91600640 1 -- 172.21.15.1:0/971929335 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7faa8c158bb0 con 0x7faa8c152500 2024-05-09T08:02:46.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:46.431+0000 7faa88c00640 1 -- 172.21.15.1:0/971929335 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18382 (secure 0 0 0) 0x7faa84022070 con 0x7faa8c152500 2024-05-09T08:02:46.449 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T08:02:46.449 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":25,"fsid":"54329ec8-3270-4043-9a82-c6cd7420aa7f","created":"2024-05-09T08:01:56.666970+0000","modified":"2024-05-09T08:02:20.403456+0000","last_up_change":"2024-05-09T08:02:07.146082+0000","last_in_change":"2024-05-09T08:01:59.008225+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T08:02:09.165511+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T08:02:12.925485+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T08:02:14.367455+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-09T08:02:17.967064+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c064454e-a9b5-4a24-9a77-17f0a4085952","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6800","nonce":1513349547},{"type":"v1","addr":"172.21.15.1:6801","nonce":1513349547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":1513349547},{"type":"v1","addr":"172.21.15.1:6803","nonce":1513349547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":1513349547},{"type":"v1","addr":"172.21.15.1:6807","nonce":1513349547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":1513349547},{"type":"v1","addr":"172.21.15.1:6805","nonce":1513349547}]},"public_addr":"172.21.15.1:6801/1513349547","cluster_addr":"172.21.15.1:6803/1513349547","heartbeat_back_addr":"172.21.15.1:6807/1513349547","heartbeat_front_addr":"172.21.15.1:6805/1513349547","state":["exists","up"]},{"osd":1,"uuid":"1a9ea687-2ff6-407a-949b-0e7e1083af21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":2942255259},{"type":"v1","addr":"172.21.15.1:6817","nonce":2942255259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":2942255259},{"type":"v1","addr":"172.21.15.1:6819","nonce":2942255259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":2942255259},{"type":"v1","addr":"172.21.15.1:6823","nonce":2942255259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6820","nonce":2942255259},{"type":"v1","addr":"172.21.15.1:6821","nonce":2942255259}]},"public_addr":"172.21.15.1:6817/2942255259","cluster_addr":"172.21.15.1:6819/2942255259","heartbeat_back_addr":"172.21.15.1:6823/2942255259","heartbeat_front_addr":"172.21.15.1:6821/2942255259","state":["exists","up"]},{"osd":2,"uuid":"c88af08b-7c9a-4dd8-9640-44138239addb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":2925031811},{"type":"v1","addr":"172.21.15.1:6809","nonce":2925031811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":2925031811},{"type":"v1","addr":"172.21.15.1:6811","nonce":2925031811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":2925031811},{"type":"v1","addr":"172.21.15.1:6815","nonce":2925031811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":2925031811},{"type":"v1","addr":"172.21.15.1:6813","nonce":2925031811}]},"public_addr":"172.21.15.1:6809/2925031811","cluster_addr":"172.21.15.1:6811/2925031811","heartbeat_back_addr":"172.21.15.1:6815/2925031811","heartbeat_front_addr":"172.21.15.1:6813/2925031811","state":["exists","up"]},{"osd":3,"uuid":"c087b7e3-a41d-4333-9365-32e3bd8c8e6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6824","nonce":2497776654},{"type":"v1","addr":"172.21.15.1:6825","nonce":2497776654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":2497776654},{"type":"v1","addr":"172.21.15.1:6827","nonce":2497776654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":2497776654},{"type":"v1","addr":"172.21.15.1:6831","nonce":2497776654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6828","nonce":2497776654},{"type":"v1","addr":"172.21.15.1:6829","nonce":2497776654}]},"public_addr":"172.21.15.1:6825/2497776654","cluster_addr":"172.21.15.1:6827/2497776654","heartbeat_back_addr":"172.21.15.1:6831/2497776654","heartbeat_front_addr":"172.21.15.1:6829/2497776654","state":["exists","up"]},{"osd":4,"uuid":"b4576b9c-871d-47a8-8d36-65af525d0200","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":2957026670},{"type":"v1","addr":"172.21.15.7:6801","nonce":2957026670}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2957026670},{"type":"v1","addr":"172.21.15.7:6803","nonce":2957026670}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":2957026670},{"type":"v1","addr":"172.21.15.7:6807","nonce":2957026670}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2957026670},{"type":"v1","addr":"172.21.15.7:6805","nonce":2957026670}]},"public_addr":"172.21.15.7:6801/2957026670","cluster_addr":"172.21.15.7:6803/2957026670","heartbeat_back_addr":"172.21.15.7:6807/2957026670","heartbeat_front_addr":"172.21.15.7:6805/2957026670","state":["exists","up"]},{"osd":5,"uuid":"902b3d94-72f3-4499-b21a-e32453ef5144","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1670295242},{"type":"v1","addr":"172.21.15.7:6809","nonce":1670295242}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1670295242},{"type":"v1","addr":"172.21.15.7:6811","nonce":1670295242}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1670295242},{"type":"v1","addr":"172.21.15.7:6815","nonce":1670295242}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1670295242},{"type":"v1","addr":"172.21.15.7:6813","nonce":1670295242}]},"public_addr":"172.21.15.7:6809/1670295242","cluster_addr":"172.21.15.7:6811/1670295242","heartbeat_back_addr":"172.21.15.7:6815/1670295242","heartbeat_front_addr":"172.21.15.7:6813/1670295242","state":["exists","up"]},{"osd":6,"uuid":"ead1d771-1728-4f85-adb4-26be21cedff3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":2852086880},{"type":"v1","addr":"172.21.15.7:6817","nonce":2852086880}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":2852086880},{"type":"v1","addr":"172.21.15.7:6819","nonce":2852086880}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":2852086880},{"type":"v1","addr":"172.21.15.7:6823","nonce":2852086880}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":2852086880},{"type":"v1","addr":"172.21.15.7:6821","nonce":2852086880}]},"public_addr":"172.21.15.7:6817/2852086880","cluster_addr":"172.21.15.7:6819/2852086880","heartbeat_back_addr":"172.21.15.7:6823/2852086880","heartbeat_front_addr":"172.21.15.7:6821/2852086880","state":["exists","up"]},{"osd":7,"uuid":"3ab5a201-9394-48a8-8918-e95197893669","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":1442884138},{"type":"v1","addr":"172.21.15.7:6825","nonce":1442884138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1442884138},{"type":"v1","addr":"172.21.15.7:6827","nonce":1442884138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1442884138},{"type":"v1","addr":"172.21.15.7:6831","nonce":1442884138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1442884138},{"type":"v1","addr":"172.21.15.7:6829","nonce":1442884138}]},"public_addr":"172.21.15.7:6825/1442884138","cluster_addr":"172.21.15.7:6827/1442884138","heartbeat_back_addr":"172.21.15.7:6831/1442884138","heartbeat_front_addr":"172.21.15.7:6829/1442884138","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:02:04.615675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:02:04.385974+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:02:04.619217+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:02:04.401681+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:02:04.097081+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:02:04.129981+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:02:04.225327+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:02:04.193351+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T08:02:46.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:46.434+0000 7faa91600640 1 -- 172.21.15.1:0/971929335 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7faa6806be60 msgr2=0x7faa6806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:46.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:46.434+0000 7faa91600640 1 --2- 172.21.15.1:0/971929335 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7faa6806be60 0x7faa6806e320 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7faa7c009aa0 tx=0x7faa7c009190 comp rx=0 tx=0).stop 2024-05-09T08:02:46.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:46.434+0000 7faa91600640 1 -- 172.21.15.1:0/971929335 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faa8c152500 msgr2=0x7faa8c0c3210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:46.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:46.434+0000 7faa91600640 1 --2- 172.21.15.1:0/971929335 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faa8c152500 0x7faa8c0c3210 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7faa8400ef00 tx=0x7faa8400b480 comp rx=0 tx=0).stop 2024-05-09T08:02:46.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:46.434+0000 7faa91600640 1 -- 172.21.15.1:0/971929335 shutdown_connections 2024-05-09T08:02:46.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:46.434+0000 7faa91600640 1 --2- 172.21.15.1:0/971929335 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7faa6806be60 0x7faa6806e320 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:46.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:46.434+0000 7faa91600640 1 --2- 172.21.15.1:0/971929335 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faa8c0cc730 0x7faa8c159d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:46.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:46.434+0000 7faa91600640 1 --2- 172.21.15.1:0/971929335 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faa8c152500 0x7faa8c0c3210 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:46.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:46.434+0000 7faa91600640 1 --2- 172.21.15.1:0/971929335 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7faa8c0c7660 0x7faa8c153e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:46.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:46.434+0000 7faa91600640 1 -- 172.21.15.1:0/971929335 >> 172.21.15.1:0/971929335 conn(0x7faa8c0cbc10 msgr2=0x7faa8c1e38a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:02:46.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:46.434+0000 7faa91600640 1 -- 172.21.15.1:0/971929335 shutdown_connections 2024-05-09T08:02:46.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:46.434+0000 7faa91600640 1 -- 172.21.15.1:0/971929335 wait complete. 2024-05-09T08:02:46.472 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T08:02:46.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:46.665+0000 7fa4f6200640 1 Processor -- start 2024-05-09T08:02:46.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:46.666+0000 7fa4f6200640 1 -- start start 2024-05-09T08:02:46.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:46.667+0000 7fa4f6200640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa4f00ca5c0 0x7fa4f00ca9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:46.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:46.667+0000 7fa4f6200640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fa4f00cf370 0x7fa4f006d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:46.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:46.667+0000 7fa4f6200640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa4f015ad80 con 0x7fa4f00ccf70 2024-05-09T08:02:46.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:46.667+0000 7fa4f6200640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fa4f006c7d0 con 0x7fa4f00ca5c0 2024-05-09T08:02:46.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:46.667+0000 7fa4f6200640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7fa4f015b060 con 0x7fa4f00cf370 2024-05-09T08:02:46.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:46.668+0000 7fa4f4e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa4f00ca5c0 0x7fa4f00ca9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:46.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:46.668+0000 7fa4f5800640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fa4f00cf370 0x7fa4f006d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:46.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:46.668+0000 7fa4f4e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa4f00ca5c0 0x7fa4f00ca9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.1:57798/0 (socket says 172.21.15.1:57798) 2024-05-09T08:02:46.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:46.668+0000 7fa4f4e00640 1 -- 172.21.15.1:0/1340300025 learned_addr learned my addr 172.21.15.1:0/1340300025 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:02:46.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:46.668+0000 7fa4f5800640 1 -- 172.21.15.1:0/1340300025 >> v2:172.21.15.7:3300/0 conn(0x7fa4f00ca5c0 msgr2=0x7fa4f00ca9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:02:46.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:46.668+0000 7fa4f5800640 1 --2- 172.21.15.1:0/1340300025 >> v2:172.21.15.7:3300/0 conn(0x7fa4f00ca5c0 0x7fa4f00ca9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:46.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:46.668+0000 7fa4f5800640 1 -- 172.21.15.1:0/1340300025 >> v1:172.21.15.1:6789/0 conn(0x7fa4f00ccf70 legacy=0x7fa4f015a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:46.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:46.668+0000 7fa4f5800640 1 -- 172.21.15.1:0/1340300025 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4f015b340 con 0x7fa4f00cf370 2024-05-09T08:02:46.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:46.668+0000 7fa4f4e00640 1 --2- 172.21.15.1:0/1340300025 >> v2:172.21.15.7:3300/0 conn(0x7fa4f00ca5c0 0x7fa4f00ca9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:02:46.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:46.669+0000 7fa4f5800640 1 --2- 172.21.15.1:0/1340300025 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fa4f00cf370 0x7fa4f006d220 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa4e8009d90 tx=0x7fa4e802a6f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=96629fe5f0575049 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:46.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:46.670+0000 7fa4ef400640 1 -- 172.21.15.1:0/1340300025 <== mon.2 v2:172.21.15.7:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4e8007740 con 0x7fa4f00cf370 2024-05-09T08:02:46.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:46.670+0000 7fa4ef400640 1 -- 172.21.15.1:0/1340300025 <== mon.2 v2:172.21.15.7:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4e80078e0 con 0x7fa4f00cf370 2024-05-09T08:02:46.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:46.670+0000 7fa4ef400640 1 -- 172.21.15.1:0/1340300025 <== mon.2 v2:172.21.15.7:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4e8007cb0 con 0x7fa4f00cf370 2024-05-09T08:02:46.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:46.670+0000 7fa4f6200640 1 -- 172.21.15.1:0/1340300025 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fa4f00cf370 msgr2=0x7fa4f006d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:46.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:46.670+0000 7fa4f6200640 1 --2- 172.21.15.1:0/1340300025 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fa4f00cf370 0x7fa4f006d220 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa4e8009d90 tx=0x7fa4e802a6f0 comp rx=0 tx=0).stop 2024-05-09T08:02:46.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:46.670+0000 7fa4f6200640 1 -- 172.21.15.1:0/1340300025 shutdown_connections 2024-05-09T08:02:46.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:46.670+0000 7fa4f6200640 1 --2- 172.21.15.1:0/1340300025 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fa4f00cf370 0x7fa4f006d220 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:46.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:46.670+0000 7fa4f6200640 1 --2- 172.21.15.1:0/1340300025 >> v2:172.21.15.7:3300/0 conn(0x7fa4f00ca5c0 0x7fa4f00ca9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:46.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:46.670+0000 7fa4f6200640 1 -- 172.21.15.1:0/1340300025 >> 172.21.15.1:0/1340300025 conn(0x7fa4f00cfc40 msgr2=0x7fa4f00d0050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:02:46.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:46.671+0000 7fa4f6200640 1 -- 172.21.15.1:0/1340300025 shutdown_connections 2024-05-09T08:02:46.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:46.671+0000 7fa4f6200640 1 -- 172.21.15.1:0/1340300025 wait complete. 2024-05-09T08:02:46.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:46.672+0000 7fa4f6200640 1 Processor -- start 2024-05-09T08:02:46.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:46.672+0000 7fa4f6200640 1 -- start start 2024-05-09T08:02:46.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:46.673+0000 7fa4f6200640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fa4f00ca5c0 0x7fa4f01f0210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:46.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:46.673+0000 7fa4f6200640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa4f00ccf70 0x7fa4f01e5450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:46.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:46.673+0000 7fa4f6200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa4f00cf370 0x7fa4f0151c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:46.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:46.673+0000 7fa4f6200640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fa4f00cf940 con 0x7fa4f00ccf70 2024-05-09T08:02:46.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:46.673+0000 7fa4f6200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa4f00cf7c0 con 0x7fa4f00cf370 2024-05-09T08:02:46.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:46.673+0000 7fa4f6200640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7fa4f006d760 con 0x7fa4f00ca5c0 2024-05-09T08:02:46.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:46.674+0000 7fa4efe00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa4f00ccf70 0x7fa4f01e5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:46.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:46.674+0000 7fa4f4e00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fa4f00ca5c0 0x7fa4f01f0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:46.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:46.674+0000 7fa4f5800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa4f00cf370 0x7fa4f0151c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:46.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:46.674+0000 7fa4f4e00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fa4f00ca5c0 0x7fa4f01f0210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3301/0 says I am v2:172.21.15.1:41466/0 (socket says 172.21.15.1:41466) 2024-05-09T08:02:46.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:46.674+0000 7fa4f5800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa4f00cf370 0x7fa4f0151c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.1:57814/0 (socket says 172.21.15.1:57814) 2024-05-09T08:02:46.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:46.674+0000 7fa4f4e00640 1 -- 172.21.15.1:0/3062543442 learned_addr learned my addr 172.21.15.1:0/3062543442 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:02:46.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:46.674+0000 7fa4efe00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa4f00ccf70 0x7fa4f01e5450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:52914/0 (socket says 172.21.15.1:52914) 2024-05-09T08:02:46.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:46.674+0000 7fa4f4e00640 1 -- 172.21.15.1:0/3062543442 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa4f00cf370 msgr2=0x7fa4f0151c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:46.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:46.674+0000 7fa4f4e00640 1 --2- 172.21.15.1:0/3062543442 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa4f00cf370 0x7fa4f0151c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:46.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:46.674+0000 7fa4f4e00640 1 -- 172.21.15.1:0/3062543442 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa4f00ccf70 msgr2=0x7fa4f01e5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:46.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:46.674+0000 7fa4f4e00640 1 --2- 172.21.15.1:0/3062543442 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa4f00ccf70 0x7fa4f01e5450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:46.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:46.674+0000 7fa4f4e00640 1 -- 172.21.15.1:0/3062543442 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4f01e3a50 con 0x7fa4f00ca5c0 2024-05-09T08:02:46.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:46.674+0000 7fa4efe00640 1 --2- 172.21.15.1:0/3062543442 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa4f00ccf70 0x7fa4f01e5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:02:46.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:46.675+0000 7fa4f4e00640 1 --2- 172.21.15.1:0/3062543442 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fa4f00ca5c0 0x7fa4f01f0210 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa4e000e9d0 tx=0x7fa4e000ee90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:46.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:46.675+0000 7fa4ecc00640 1 -- 172.21.15.1:0/3062543442 <== mon.2 v2:172.21.15.7:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4e0017070 con 0x7fa4f00ca5c0 2024-05-09T08:02:46.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:46.675+0000 7fa4ecc00640 1 -- 172.21.15.1:0/3062543442 <== mon.2 v2:172.21.15.7:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4e000ca60 con 0x7fa4f00ca5c0 2024-05-09T08:02:46.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:46.675+0000 7fa4ecc00640 1 -- 172.21.15.1:0/3062543442 <== mon.2 v2:172.21.15.7:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4e000ce10 con 0x7fa4f00ca5c0 2024-05-09T08:02:46.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:46.675+0000 7fa4f6200640 1 -- 172.21.15.1:0/3062543442 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4f01e5d90 con 0x7fa4f00ca5c0 2024-05-09T08:02:46.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:46.675+0000 7fa4f6200640 1 -- 172.21.15.1:0/3062543442 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4f01e6330 con 0x7fa4f00ca5c0 2024-05-09T08:02:46.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:46.676+0000 7fa4f6200640 1 -- 172.21.15.1:0/3062543442 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4f004f660 con 0x7fa4f00ca5c0 2024-05-09T08:02:46.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:46.677+0000 7fa4ecc00640 1 -- 172.21.15.1:0/3062543442 <== mon.2 v2:172.21.15.7:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa4e001a070 con 0x7fa4f00ca5c0 2024-05-09T08:02:46.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:46.678+0000 7fa4ecc00640 1 --2- 172.21.15.1:0/3062543442 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fa4d006be30 0x7fa4d006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:46.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:46.678+0000 7fa4ecc00640 1 -- 172.21.15.1:0/3062543442 <== mon.2 v2:172.21.15.7:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fa4e008b170 con 0x7fa4f00ca5c0 2024-05-09T08:02:46.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:46.678+0000 7fa4efe00640 1 --2- 172.21.15.1:0/3062543442 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fa4d006be30 0x7fa4d006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:46.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:46.679+0000 7fa4efe00640 1 --2- 172.21.15.1:0/3062543442 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fa4d006be30 0x7fa4d006e2f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa4f0152ae0 tx=0x7fa4dc009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:46.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:46.683+0000 7fa4ecc00640 1 -- 172.21.15.1:0/3062543442 <== mon.2 v2:172.21.15.7:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fa4e0059180 con 0x7fa4f00ca5c0 2024-05-09T08:02:46.907 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:46.890+0000 7fa4f6200640 1 -- 172.21.15.1:0/3062543442 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa4f006e140 con 0x7fa4f00ca5c0 2024-05-09T08:02:46.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:46.892+0000 7fa4ecc00640 1 -- 172.21.15.1:0/3062543442 <== mon.2 v2:172.21.15.7:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18382 (secure 0 0 0) 0x7fa4e005d480 con 0x7fa4f00ca5c0 2024-05-09T08:02:46.909 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T08:02:46.909 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":25,"fsid":"54329ec8-3270-4043-9a82-c6cd7420aa7f","created":"2024-05-09T08:01:56.666970+0000","modified":"2024-05-09T08:02:20.403456+0000","last_up_change":"2024-05-09T08:02:07.146082+0000","last_in_change":"2024-05-09T08:01:59.008225+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T08:02:09.165511+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T08:02:12.925485+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T08:02:14.367455+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-09T08:02:17.967064+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c064454e-a9b5-4a24-9a77-17f0a4085952","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6800","nonce":1513349547},{"type":"v1","addr":"172.21.15.1:6801","nonce":1513349547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":1513349547},{"type":"v1","addr":"172.21.15.1:6803","nonce":1513349547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":1513349547},{"type":"v1","addr":"172.21.15.1:6807","nonce":1513349547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":1513349547},{"type":"v1","addr":"172.21.15.1:6805","nonce":1513349547}]},"public_addr":"172.21.15.1:6801/1513349547","cluster_addr":"172.21.15.1:6803/1513349547","heartbeat_back_addr":"172.21.15.1:6807/1513349547","heartbeat_front_addr":"172.21.15.1:6805/1513349547","state":["exists","up"]},{"osd":1,"uuid":"1a9ea687-2ff6-407a-949b-0e7e1083af21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":2942255259},{"type":"v1","addr":"172.21.15.1:6817","nonce":2942255259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":2942255259},{"type":"v1","addr":"172.21.15.1:6819","nonce":2942255259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":2942255259},{"type":"v1","addr":"172.21.15.1:6823","nonce":2942255259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6820","nonce":2942255259},{"type":"v1","addr":"172.21.15.1:6821","nonce":2942255259}]},"public_addr":"172.21.15.1:6817/2942255259","cluster_addr":"172.21.15.1:6819/2942255259","heartbeat_back_addr":"172.21.15.1:6823/2942255259","heartbeat_front_addr":"172.21.15.1:6821/2942255259","state":["exists","up"]},{"osd":2,"uuid":"c88af08b-7c9a-4dd8-9640-44138239addb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":2925031811},{"type":"v1","addr":"172.21.15.1:6809","nonce":2925031811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":2925031811},{"type":"v1","addr":"172.21.15.1:6811","nonce":2925031811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":2925031811},{"type":"v1","addr":"172.21.15.1:6815","nonce":2925031811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":2925031811},{"type":"v1","addr":"172.21.15.1:6813","nonce":2925031811}]},"public_addr":"172.21.15.1:6809/2925031811","cluster_addr":"172.21.15.1:6811/2925031811","heartbeat_back_addr":"172.21.15.1:6815/2925031811","heartbeat_front_addr":"172.21.15.1:6813/2925031811","state":["exists","up"]},{"osd":3,"uuid":"c087b7e3-a41d-4333-9365-32e3bd8c8e6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6824","nonce":2497776654},{"type":"v1","addr":"172.21.15.1:6825","nonce":2497776654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":2497776654},{"type":"v1","addr":"172.21.15.1:6827","nonce":2497776654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":2497776654},{"type":"v1","addr":"172.21.15.1:6831","nonce":2497776654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6828","nonce":2497776654},{"type":"v1","addr":"172.21.15.1:6829","nonce":2497776654}]},"public_addr":"172.21.15.1:6825/2497776654","cluster_addr":"172.21.15.1:6827/2497776654","heartbeat_back_addr":"172.21.15.1:6831/2497776654","heartbeat_front_addr":"172.21.15.1:6829/2497776654","state":["exists","up"]},{"osd":4,"uuid":"b4576b9c-871d-47a8-8d36-65af525d0200","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":2957026670},{"type":"v1","addr":"172.21.15.7:6801","nonce":2957026670}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2957026670},{"type":"v1","addr":"172.21.15.7:6803","nonce":2957026670}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":2957026670},{"type":"v1","addr":"172.21.15.7:6807","nonce":2957026670}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2957026670},{"type":"v1","addr":"172.21.15.7:6805","nonce":2957026670}]},"public_addr":"172.21.15.7:6801/2957026670","cluster_addr":"172.21.15.7:6803/2957026670","heartbeat_back_addr":"172.21.15.7:6807/2957026670","heartbeat_front_addr":"172.21.15.7:6805/2957026670","state":["exists","up"]},{"osd":5,"uuid":"902b3d94-72f3-4499-b21a-e32453ef5144","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1670295242},{"type":"v1","addr":"172.21.15.7:6809","nonce":1670295242}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1670295242},{"type":"v1","addr":"172.21.15.7:6811","nonce":1670295242}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1670295242},{"type":"v1","addr":"172.21.15.7:6815","nonce":1670295242}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1670295242},{"type":"v1","addr":"172.21.15.7:6813","nonce":1670295242}]},"public_addr":"172.21.15.7:6809/1670295242","cluster_addr":"172.21.15.7:6811/1670295242","heartbeat_back_addr":"172.21.15.7:6815/1670295242","heartbeat_front_addr":"172.21.15.7:6813/1670295242","state":["exists","up"]},{"osd":6,"uuid":"ead1d771-1728-4f85-adb4-26be21cedff3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":2852086880},{"type":"v1","addr":"172.21.15.7:6817","nonce":2852086880}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":2852086880},{"type":"v1","addr":"172.21.15.7:6819","nonce":2852086880}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":2852086880},{"type":"v1","addr":"172.21.15.7:6823","nonce":2852086880}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":2852086880},{"type":"v1","addr":"172.21.15.7:6821","nonce":2852086880}]},"public_addr":"172.21.15.7:6817/2852086880","cluster_addr":"172.21.15.7:6819/2852086880","heartbeat_back_addr":"172.21.15.7:6823/2852086880","heartbeat_front_addr":"172.21.15.7:6821/2852086880","state":["exists","up"]},{"osd":7,"uuid":"3ab5a201-9394-48a8-8918-e95197893669","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":1442884138},{"type":"v1","addr":"172.21.15.7:6825","nonce":1442884138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1442884138},{"type":"v1","addr":"172.21.15.7:6827","nonce":1442884138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1442884138},{"type":"v1","addr":"172.21.15.7:6831","nonce":1442884138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1442884138},{"type":"v1","addr":"172.21.15.7:6829","nonce":1442884138}]},"public_addr":"172.21.15.7:6825/1442884138","cluster_addr":"172.21.15.7:6827/1442884138","heartbeat_back_addr":"172.21.15.7:6831/1442884138","heartbeat_front_addr":"172.21.15.7:6829/1442884138","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:02:04.615675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:02:04.385974+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:02:04.619217+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:02:04.401681+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:02:04.097081+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:02:04.129981+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:02:04.225327+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:02:04.193351+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T08:02:46.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:46.894+0000 7fa4f6200640 1 -- 172.21.15.1:0/3062543442 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fa4d006be30 msgr2=0x7fa4d006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:46.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:46.894+0000 7fa4f6200640 1 --2- 172.21.15.1:0/3062543442 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fa4d006be30 0x7fa4d006e2f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa4f0152ae0 tx=0x7fa4dc009040 comp rx=0 tx=0).stop 2024-05-09T08:02:46.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:46.894+0000 7fa4f6200640 1 -- 172.21.15.1:0/3062543442 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fa4f00ca5c0 msgr2=0x7fa4f01f0210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:46.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:46.894+0000 7fa4f6200640 1 --2- 172.21.15.1:0/3062543442 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fa4f00ca5c0 0x7fa4f01f0210 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa4e000e9d0 tx=0x7fa4e000ee90 comp rx=0 tx=0).stop 2024-05-09T08:02:46.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:46.894+0000 7fa4f6200640 1 -- 172.21.15.1:0/3062543442 shutdown_connections 2024-05-09T08:02:46.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:46.894+0000 7fa4f6200640 1 --2- 172.21.15.1:0/3062543442 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fa4d006be30 0x7fa4d006e2f0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:46.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:46.894+0000 7fa4f6200640 1 --2- 172.21.15.1:0/3062543442 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa4f00cf370 0x7fa4f0151c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:46.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:46.894+0000 7fa4f6200640 1 --2- 172.21.15.1:0/3062543442 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa4f00ccf70 0x7fa4f01e5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:46.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:46.894+0000 7fa4f6200640 1 --2- 172.21.15.1:0/3062543442 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fa4f00ca5c0 0x7fa4f01f0210 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:46.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:46.894+0000 7fa4f6200640 1 -- 172.21.15.1:0/3062543442 >> 172.21.15.1:0/3062543442 conn(0x7fa4f00cfc40 msgr2=0x7fa4f0158420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:02:46.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:46.895+0000 7fa4f6200640 1 -- 172.21.15.1:0/3062543442 shutdown_connections 2024-05-09T08:02:46.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:46.895+0000 7fa4f6200640 1 -- 172.21.15.1:0/3062543442 wait complete. 2024-05-09T08:02:46.931 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-09T08:02:47.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:47.129+0000 7f4fb2600640 1 Processor -- start 2024-05-09T08:02:47.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:47.129+0000 7f4fb2600640 1 -- start start 2024-05-09T08:02:47.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:47.131+0000 7f4fb2600640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f4fac1523b0 0x7f4fac158910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:47.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:47.131+0000 7f4fb2600640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f4fac0ca570 0x7f4fac070750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:47.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:47.131+0000 7f4fb2600640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4fac0b3060 con 0x7f4fac070c90 2024-05-09T08:02:47.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:47.131+0000 7f4fb2600640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f4fac152790 con 0x7f4fac1523b0 2024-05-09T08:02:47.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:47.131+0000 7f4fb2600640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f4fac0caad0 con 0x7f4fac0ca570 2024-05-09T08:02:47.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:47.131+0000 7f4fb1200640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f4fac1523b0 0x7f4fac158910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:47.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:47.131+0000 7f4fb1200640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f4fac1523b0 0x7f4fac158910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:52930/0 (socket says 172.21.15.1:52930) 2024-05-09T08:02:47.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:47.131+0000 7f4fb1200640 1 -- 172.21.15.1:0/1789313037 learned_addr learned my addr 172.21.15.1:0/1789313037 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:02:47.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:47.131+0000 7f4fabe00640 1 --2- 172.21.15.1:0/1789313037 >> v2:172.21.15.7:3300/0 conn(0x7f4fac0ca570 0x7f4fac070750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:47.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:47.132+0000 7f4fb1200640 1 -- 172.21.15.1:0/1789313037 >> v2:172.21.15.7:3300/0 conn(0x7f4fac0ca570 msgr2=0x7f4fac070750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:02:47.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:47.132+0000 7f4fb1200640 1 --2- 172.21.15.1:0/1789313037 >> v2:172.21.15.7:3300/0 conn(0x7f4fac0ca570 0x7f4fac070750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:47.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:47.132+0000 7f4fb1200640 1 -- 172.21.15.1:0/1789313037 >> v1:172.21.15.1:6789/0 conn(0x7f4fac070c90 legacy=0x7f4fac0cdfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:47.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:47.132+0000 7f4fb1200640 1 -- 172.21.15.1:0/1789313037 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fac0cadb0 con 0x7f4fac1523b0 2024-05-09T08:02:47.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:47.132+0000 7f4fab400640 1 -- 172.21.15.1:0/1789313037 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3315142292 0 0) 0x7f4fa4001490 con 0x7f4fac070c90 2024-05-09T08:02:47.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:47.133+0000 7f4fb1200640 1 --2- 172.21.15.1:0/1789313037 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4fac1523b0 0x7f4fac158910 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f4fac071130 tx=0x7f4fa002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b99370eb9769f21e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:47.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:47.133+0000 7f4fab400640 1 -- 172.21.15.1:0/1789313037 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fa0037070 con 0x7f4fac1523b0 2024-05-09T08:02:47.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:47.133+0000 7f4fab400640 1 -- 172.21.15.1:0/1789313037 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4fa0032440 con 0x7f4fac1523b0 2024-05-09T08:02:47.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:47.133+0000 7f4fab400640 1 -- 172.21.15.1:0/1789313037 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fa0032810 con 0x7f4fac1523b0 2024-05-09T08:02:47.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:47.134+0000 7f4fb2600640 1 -- 172.21.15.1:0/1789313037 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4fac1523b0 msgr2=0x7f4fac158910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:47.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:47.134+0000 7f4fb2600640 1 --2- 172.21.15.1:0/1789313037 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4fac1523b0 0x7f4fac158910 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f4fac071130 tx=0x7f4fa002a270 comp rx=0 tx=0).stop 2024-05-09T08:02:47.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:47.134+0000 7f4fb2600640 1 -- 172.21.15.1:0/1789313037 shutdown_connections 2024-05-09T08:02:47.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:47.134+0000 7f4fb2600640 1 --2- 172.21.15.1:0/1789313037 >> v2:172.21.15.7:3300/0 conn(0x7f4fac0ca570 0x7f4fac070750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:47.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:47.134+0000 7f4fb2600640 1 --2- 172.21.15.1:0/1789313037 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4fac1523b0 0x7f4fac158910 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:47.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:47.134+0000 7f4fb2600640 1 -- 172.21.15.1:0/1789313037 >> 172.21.15.1:0/1789313037 conn(0x7f4fac0672f0 msgr2=0x7f4fac067700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:02:47.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:47.134+0000 7f4fb2600640 1 -- 172.21.15.1:0/1789313037 shutdown_connections 2024-05-09T08:02:47.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:47.134+0000 7f4fb2600640 1 -- 172.21.15.1:0/1789313037 wait complete. 2024-05-09T08:02:47.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:47.136+0000 7f4fb2600640 1 Processor -- start 2024-05-09T08:02:47.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:47.136+0000 7f4fb2600640 1 -- start start 2024-05-09T08:02:47.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:47.137+0000 7f4fb2600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4fac070c90 0x7f4fac06c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:47.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:47.137+0000 7f4fb2600640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4fac0ca570 0x7f4fac06ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:47.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:47.137+0000 7f4fb2600640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f4fac1523b0 0x7f4fac143370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:47.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:47.137+0000 7f4fb2600640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f4fac1583f0 con 0x7f4fac0ca570 2024-05-09T08:02:47.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:47.137+0000 7f4fb2600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f4fac158110 con 0x7f4fac070c90 2024-05-09T08:02:47.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:47.137+0000 7f4fb2600640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f4fac15b5f0 con 0x7f4fac1523b0 2024-05-09T08:02:47.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:47.137+0000 7f4fabe00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4fac0ca570 0x7f4fac06ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:47.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:47.137+0000 7f4fabe00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4fac0ca570 0x7f4fac06ced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:52946/0 (socket says 172.21.15.1:52946) 2024-05-09T08:02:47.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:47.137+0000 7f4fb1200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4fac070c90 0x7f4fac06c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:47.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:47.137+0000 7f4fabe00640 1 -- 172.21.15.1:0/15653498 learned_addr learned my addr 172.21.15.1:0/15653498 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:02:47.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:47.137+0000 7f4fb1c00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f4fac1523b0 0x7f4fac143370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:47.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:47.138+0000 7f4fabe00640 1 -- 172.21.15.1:0/15653498 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f4fac1523b0 msgr2=0x7f4fac143370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:47.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:47.138+0000 7f4fabe00640 1 --2- 172.21.15.1:0/15653498 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f4fac1523b0 0x7f4fac143370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:47.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:47.138+0000 7f4fabe00640 1 -- 172.21.15.1:0/15653498 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4fac070c90 msgr2=0x7f4fac06c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:47.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:47.138+0000 7f4fabe00640 1 --2- 172.21.15.1:0/15653498 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4fac070c90 0x7f4fac06c990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:47.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:47.138+0000 7f4fabe00640 1 -- 172.21.15.1:0/15653498 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fac0c6c20 con 0x7f4fac0ca570 2024-05-09T08:02:47.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:47.138+0000 7f4fabe00640 1 --2- 172.21.15.1:0/15653498 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4fac0ca570 0x7f4fac06ced0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f4f9800e8a0 tx=0x7f4f9800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:47.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:47.139+0000 7f4fa8c00640 1 -- 172.21.15.1:0/15653498 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f98017070 con 0x7f4fac0ca570 2024-05-09T08:02:47.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:47.139+0000 7f4fa8c00640 1 -- 172.21.15.1:0/15653498 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f9800c940 con 0x7f4fac0ca570 2024-05-09T08:02:47.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:47.139+0000 7f4fa8c00640 1 -- 172.21.15.1:0/15653498 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f9800cd30 con 0x7f4fac0ca570 2024-05-09T08:02:47.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:47.139+0000 7f4fb2600640 1 -- 172.21.15.1:0/15653498 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fac14de30 con 0x7f4fac0ca570 2024-05-09T08:02:47.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:47.139+0000 7f4fb2600640 1 -- 172.21.15.1:0/15653498 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fac0c6940 con 0x7f4fac0ca570 2024-05-09T08:02:47.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:47.140+0000 7f4fb2600640 1 -- 172.21.15.1:0/15653498 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4fac04f660 con 0x7f4fac0ca570 2024-05-09T08:02:47.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:47.141+0000 7f4fa8c00640 1 -- 172.21.15.1:0/15653498 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4f9801a070 con 0x7f4fac0ca570 2024-05-09T08:02:47.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:47.142+0000 7f4fa8c00640 1 --2- 172.21.15.1:0/15653498 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f4f8c06be30 0x7f4f8c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:47.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:47.142+0000 7f4fa8c00640 1 -- 172.21.15.1:0/15653498 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f4f9808afe0 con 0x7f4fac0ca570 2024-05-09T08:02:47.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:47.142+0000 7f4fb1200640 1 --2- 172.21.15.1:0/15653498 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f4f8c06be30 0x7f4f8c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:47.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:47.143+0000 7f4fb1200640 1 --2- 172.21.15.1:0/15653498 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f4f8c06be30 0x7f4f8c06e2f0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f4fa0034000 tx=0x7f4fa002e000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:47.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:47.147+0000 7f4fa8c00640 1 -- 172.21.15.1:0/15653498 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f4f98058fa0 con 0x7f4fac0ca570 2024-05-09T08:02:47.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:47.389+0000 7f4fb2600640 1 -- 172.21.15.1:0/15653498 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f4fac0c44f0 con 0x7f4fac0ca570 2024-05-09T08:02:48.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:48.229+0000 7f4fa8c00640 1 -- 172.21.15.1:0/15653498 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v8) ==== 118+0+0 (secure 0 0 0) 0x7f4f9805d2a0 con 0x7f4fac0ca570 2024-05-09T08:02:48.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:48.233+0000 7f4fb2600640 1 -- 172.21.15.1:0/15653498 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f4f8c06be30 msgr2=0x7f4f8c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:48.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:48.233+0000 7f4fb2600640 1 --2- 172.21.15.1:0/15653498 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f4f8c06be30 0x7f4f8c06e2f0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f4fa0034000 tx=0x7f4fa002e000 comp rx=0 tx=0).stop 2024-05-09T08:02:48.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:48.233+0000 7f4fb2600640 1 -- 172.21.15.1:0/15653498 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4fac0ca570 msgr2=0x7f4fac06ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:48.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:48.233+0000 7f4fb2600640 1 --2- 172.21.15.1:0/15653498 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4fac0ca570 0x7f4fac06ced0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f4f9800e8a0 tx=0x7f4f9800ed60 comp rx=0 tx=0).stop 2024-05-09T08:02:48.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:48.233+0000 7f4fb2600640 1 -- 172.21.15.1:0/15653498 shutdown_connections 2024-05-09T08:02:48.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:48.233+0000 7f4fb2600640 1 --2- 172.21.15.1:0/15653498 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f4f8c06be30 0x7f4f8c06e2f0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:48.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:48.233+0000 7f4fb2600640 1 --2- 172.21.15.1:0/15653498 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f4fac1523b0 0x7f4fac143370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:48.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:48.233+0000 7f4fb2600640 1 --2- 172.21.15.1:0/15653498 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4fac0ca570 0x7f4fac06ced0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:48.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:48.233+0000 7f4fb2600640 1 --2- 172.21.15.1:0/15653498 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4fac070c90 0x7f4fac06c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:48.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:48.233+0000 7f4fb2600640 1 -- 172.21.15.1:0/15653498 >> 172.21.15.1:0/15653498 conn(0x7f4fac0672f0 msgr2=0x7f4fac0c9d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:02:48.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:48.234+0000 7f4fb2600640 1 -- 172.21.15.1:0/15653498 shutdown_connections 2024-05-09T08:02:48.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:48.234+0000 7f4fb2600640 1 -- 172.21.15.1:0/15653498 wait complete. 2024-05-09T08:02:48.272 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'session_timeout': 300} 2024-05-09T08:02:48.272 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-05-09T08:02:48.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:48.455+0000 7f8456600640 1 Processor -- start 2024-05-09T08:02:48.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:48.456+0000 7f8456600640 1 -- start start 2024-05-09T08:02:48.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:48.457+0000 7f8456600640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f84500c54f0 0x7f845006f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:48.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:48.457+0000 7f8456600640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f84500c58d0 0x7f84501589e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:48.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:48.457+0000 7f8456600640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f84500718e0 con 0x7f84500c6470 2024-05-09T08:02:48.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:48.457+0000 7f8456600640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f84500726e0 con 0x7f84500c58d0 2024-05-09T08:02:48.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:48.457+0000 7f8456600640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f8450071b60 con 0x7f84500c54f0 2024-05-09T08:02:48.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:48.458+0000 7f844fe00640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f84500c58d0 0x7f84501589e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:48.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:48.458+0000 7f844fe00640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f84500c58d0 0x7f84501589e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:52952/0 (socket says 172.21.15.1:52952) 2024-05-09T08:02:48.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:48.458+0000 7f844fe00640 1 -- 172.21.15.1:0/3497062314 learned_addr learned my addr 172.21.15.1:0/3497062314 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:02:48.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:48.458+0000 7f8455200640 1 --2- 172.21.15.1:0/3497062314 >> v2:172.21.15.7:3300/0 conn(0x7f84500c54f0 0x7f845006f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:48.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:48.458+0000 7f844fe00640 1 -- 172.21.15.1:0/3497062314 >> v2:172.21.15.7:3300/0 conn(0x7f84500c54f0 msgr2=0x7f845006f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:02:48.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:48.458+0000 7f844fe00640 1 --2- 172.21.15.1:0/3497062314 >> v2:172.21.15.7:3300/0 conn(0x7f84500c54f0 0x7f845006f930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:48.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:48.458+0000 7f844fe00640 1 -- 172.21.15.1:0/3497062314 >> v1:172.21.15.1:6789/0 conn(0x7f84500c6470 legacy=0x7f8450151d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:48.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:48.458+0000 7f844fe00640 1 -- 172.21.15.1:0/3497062314 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8450071e40 con 0x7f84500c58d0 2024-05-09T08:02:48.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:48.458+0000 7f8455200640 1 --2- 172.21.15.1:0/3497062314 >> v2:172.21.15.7:3300/0 conn(0x7f84500c54f0 0x7f845006f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:02:48.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:48.458+0000 7f844f400640 1 -- 172.21.15.1:0/3497062314 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3292546945 0 0) 0x7f8448001110 con 0x7f84500c6470 2024-05-09T08:02:48.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:48.459+0000 7f844fe00640 1 --2- 172.21.15.1:0/3497062314 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f84500c58d0 0x7f84501589e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f84500c96b0 tx=0x7f843c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bceb9075270b0d8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:48.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:48.460+0000 7f844f400640 1 -- 172.21.15.1:0/3497062314 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f843c037070 con 0x7f84500c58d0 2024-05-09T08:02:48.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:48.460+0000 7f844f400640 1 -- 172.21.15.1:0/3497062314 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f843c032440 con 0x7f84500c58d0 2024-05-09T08:02:48.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:48.460+0000 7f844f400640 1 -- 172.21.15.1:0/3497062314 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f843c032810 con 0x7f84500c58d0 2024-05-09T08:02:48.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:48.460+0000 7f8456600640 1 -- 172.21.15.1:0/3497062314 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f84500c58d0 msgr2=0x7f84501589e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:48.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:48.460+0000 7f8456600640 1 --2- 172.21.15.1:0/3497062314 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f84500c58d0 0x7f84501589e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f84500c96b0 tx=0x7f843c02a270 comp rx=0 tx=0).stop 2024-05-09T08:02:48.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:48.460+0000 7f8456600640 1 -- 172.21.15.1:0/3497062314 shutdown_connections 2024-05-09T08:02:48.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:48.460+0000 7f8456600640 1 --2- 172.21.15.1:0/3497062314 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f84500c58d0 0x7f84501589e0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:48.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:48.460+0000 7f8456600640 1 --2- 172.21.15.1:0/3497062314 >> v2:172.21.15.7:3300/0 conn(0x7f84500c54f0 0x7f845006f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:48.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:48.460+0000 7f8456600640 1 -- 172.21.15.1:0/3497062314 >> 172.21.15.1:0/3497062314 conn(0x7f845006b250 msgr2=0x7f845006b630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:02:48.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:48.460+0000 7f8456600640 1 -- 172.21.15.1:0/3497062314 shutdown_connections 2024-05-09T08:02:48.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:48.461+0000 7f8456600640 1 -- 172.21.15.1:0/3497062314 wait complete. 2024-05-09T08:02:48.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:48.462+0000 7f8456600640 1 Processor -- start 2024-05-09T08:02:48.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:48.462+0000 7f8456600640 1 -- start start 2024-05-09T08:02:48.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:48.463+0000 7f8456600640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f84500c54f0 0x7f8450156b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:48.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:48.463+0000 7f8456600640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f84500c58d0 0x7f845014f950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:48.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:48.463+0000 7f8456600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84500c6470 0x7f84501f3d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:48.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:48.463+0000 7f8456600640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f84501584c0 con 0x7f84500c54f0 2024-05-09T08:02:48.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:48.463+0000 7f8456600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f8450072400 con 0x7f84500c6470 2024-05-09T08:02:48.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:48.463+0000 7f8456600640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f845006fe70 con 0x7f84500c58d0 2024-05-09T08:02:48.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:48.464+0000 7f8455200640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f84500c54f0 0x7f8450156b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:48.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:48.464+0000 7f8455200640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f84500c54f0 0x7f8450156b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:52960/0 (socket says 172.21.15.1:52960) 2024-05-09T08:02:48.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:48.464+0000 7f8455200640 1 -- 172.21.15.1:0/4187647307 learned_addr learned my addr 172.21.15.1:0/4187647307 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:02:48.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:48.464+0000 7f8455c00640 1 --2- 172.21.15.1:0/4187647307 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84500c6470 0x7f84501f3d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:48.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:48.464+0000 7f844fe00640 1 --2- 172.21.15.1:0/4187647307 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f84500c58d0 0x7f845014f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:48.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:48.464+0000 7f8455200640 1 -- 172.21.15.1:0/4187647307 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f84500c58d0 msgr2=0x7f845014f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:48.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:48.464+0000 7f8455200640 1 --2- 172.21.15.1:0/4187647307 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f84500c58d0 0x7f845014f950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:48.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:48.464+0000 7f8455200640 1 -- 172.21.15.1:0/4187647307 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84500c6470 msgr2=0x7f84501f3d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:48.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:48.464+0000 7f8455200640 1 --2- 172.21.15.1:0/4187647307 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84500c6470 0x7f84501f3d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:48.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:48.464+0000 7f8455200640 1 -- 172.21.15.1:0/4187647307 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84501e36b0 con 0x7f84500c54f0 2024-05-09T08:02:48.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:48.464+0000 7f8455200640 1 --2- 172.21.15.1:0/4187647307 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f84500c54f0 0x7f8450156b70 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f844400efc0 tx=0x7f844400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:48.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:48.465+0000 7f844cc00640 1 -- 172.21.15.1:0/4187647307 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8444018020 con 0x7f84500c54f0 2024-05-09T08:02:48.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:48.465+0000 7f844cc00640 1 -- 172.21.15.1:0/4187647307 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8444014680 con 0x7f84500c54f0 2024-05-09T08:02:48.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:48.465+0000 7f8456600640 1 -- 172.21.15.1:0/4187647307 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84501e3990 con 0x7f84500c54f0 2024-05-09T08:02:48.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:48.465+0000 7f844cc00640 1 -- 172.21.15.1:0/4187647307 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8444014a50 con 0x7f84500c54f0 2024-05-09T08:02:48.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:48.465+0000 7f8456600640 1 -- 172.21.15.1:0/4187647307 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84501e3f50 con 0x7f84500c54f0 2024-05-09T08:02:48.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:48.466+0000 7f8456600640 1 -- 172.21.15.1:0/4187647307 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f845004f660 con 0x7f84500c54f0 2024-05-09T08:02:48.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:48.467+0000 7f844cc00640 1 -- 172.21.15.1:0/4187647307 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8444005d10 con 0x7f84500c54f0 2024-05-09T08:02:48.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:48.468+0000 7f844cc00640 1 --2- 172.21.15.1:0/4187647307 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f843006be30 0x7f843006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:48.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:48.468+0000 7f844cc00640 1 -- 172.21.15.1:0/4187647307 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f844408bf50 con 0x7f84500c54f0 2024-05-09T08:02:48.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:48.468+0000 7f844fe00640 1 --2- 172.21.15.1:0/4187647307 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f843006be30 0x7f843006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:48.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:48.469+0000 7f844fe00640 1 --2- 172.21.15.1:0/4187647307 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f843006be30 0x7f843006e2f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f843c034000 tx=0x7f843c02e000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:48.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:48.473+0000 7f844cc00640 1 -- 172.21.15.1:0/4187647307 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f8444022030 con 0x7f84500c54f0 2024-05-09T08:02:48.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:48.717+0000 7f8456600640 1 -- 172.21.15.1:0/4187647307 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f8450155c80 con 0x7f84500c54f0 2024-05-09T08:02:49.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:49.240+0000 7f844cc00640 1 -- 172.21.15.1:0/4187647307 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v9) ==== 122+0+0 (secure 0 0 0) 0x7f8444010040 con 0x7f84500c54f0 2024-05-09T08:02:49.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:49.243+0000 7f8456600640 1 -- 172.21.15.1:0/4187647307 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f843006be30 msgr2=0x7f843006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:49.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:49.243+0000 7f8456600640 1 --2- 172.21.15.1:0/4187647307 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f843006be30 0x7f843006e2f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f843c034000 tx=0x7f843c02e000 comp rx=0 tx=0).stop 2024-05-09T08:02:49.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:49.243+0000 7f8456600640 1 -- 172.21.15.1:0/4187647307 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f84500c54f0 msgr2=0x7f8450156b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:49.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:49.243+0000 7f8456600640 1 --2- 172.21.15.1:0/4187647307 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f84500c54f0 0x7f8450156b70 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f844400efc0 tx=0x7f844400c490 comp rx=0 tx=0).stop 2024-05-09T08:02:49.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:49.243+0000 7f8456600640 1 -- 172.21.15.1:0/4187647307 shutdown_connections 2024-05-09T08:02:49.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:49.243+0000 7f8456600640 1 --2- 172.21.15.1:0/4187647307 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f843006be30 0x7f843006e2f0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:49.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:49.243+0000 7f8456600640 1 --2- 172.21.15.1:0/4187647307 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84500c6470 0x7f84501f3d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:49.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:49.243+0000 7f8456600640 1 --2- 172.21.15.1:0/4187647307 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f84500c58d0 0x7f845014f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:49.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:49.243+0000 7f8456600640 1 --2- 172.21.15.1:0/4187647307 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f84500c54f0 0x7f8450156b70 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:49.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:49.244+0000 7f8456600640 1 -- 172.21.15.1:0/4187647307 >> 172.21.15.1:0/4187647307 conn(0x7f845006b250 msgr2=0x7f8450150fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:02:49.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:49.244+0000 7f8456600640 1 -- 172.21.15.1:0/4187647307 shutdown_connections 2024-05-09T08:02:49.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:49.244+0000 7f8456600640 1 -- 172.21.15.1:0/4187647307 wait complete. 2024-05-09T08:02:49.283 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-05-09T08:02:49.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:49.477+0000 7faa6a200640 1 Processor -- start 2024-05-09T08:02:49.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:49.478+0000 7faa6a200640 1 -- start start 2024-05-09T08:02:49.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:49.479+0000 7faa6a200640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7faa640cac60 0x7faa640cda00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:49.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:49.479+0000 7faa6a200640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7faa640cc5f0 0x7faa640cdf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:49.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:49.479+0000 7faa6a200640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faa640ccb20 con 0x7faa640ce4e0 2024-05-09T08:02:49.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:49.479+0000 7faa6a200640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7faa640cce00 con 0x7faa640cc5f0 2024-05-09T08:02:49.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:49.479+0000 7faa6a200640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7faa640cf120 con 0x7faa640cac60 2024-05-09T08:02:49.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:49.480+0000 7faa63e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7faa640cc5f0 0x7faa640cdf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:49.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:49.480+0000 7faa68e00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7faa640cac60 0x7faa640cda00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:49.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:49.480+0000 7faa69800640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7faa640ce4e0 0x7faa6415a570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.1:56406/0 (socket says 172.21.15.1:56406) 2024-05-09T08:02:49.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:49.480+0000 7faa69800640 1 -- 172.21.15.1:0/1082514787 learned_addr learned my addr 172.21.15.1:0/1082514787 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T08:02:49.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:49.480+0000 7faa63400640 1 -- 172.21.15.1:0/1082514787 <== mon.1 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1702879419 0 0) 0x7faa640ccb20 con 0x7faa640ce4e0 2024-05-09T08:02:49.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:49.480+0000 7faa63400640 1 -- 172.21.15.1:0/1082514787 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faa48003520 con 0x7faa640ce4e0 2024-05-09T08:02:49.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:49.480+0000 7faa63e00640 1 -- 172.21.15.1:0/1082514787 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7faa640cac60 msgr2=0x7faa640cda00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:02:49.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:49.480+0000 7faa63e00640 1 --2- 172.21.15.1:0/1082514787 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7faa640cac60 0x7faa640cda00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:49.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:49.480+0000 7faa63e00640 1 -- 172.21.15.1:0/1082514787 >> v1:172.21.15.7:6789/0 conn(0x7faa640ce4e0 legacy=0x7faa6415a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:49.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:49.480+0000 7faa63e00640 1 -- 172.21.15.1:0/1082514787 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa640cf400 con 0x7faa640cc5f0 2024-05-09T08:02:49.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:49.481+0000 7faa63e00640 1 --2- 172.21.15.1:0/1082514787 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faa640cc5f0 0x7faa640cdf40 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7faa4c0097a0 tx=0x7faa4c02a210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1070f5210e17110e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:49.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:49.481+0000 7faa63400640 1 -- 172.21.15.1:0/1082514787 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa4c037070 con 0x7faa640cc5f0 2024-05-09T08:02:49.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:49.481+0000 7faa63400640 1 -- 172.21.15.1:0/1082514787 <== mon.1 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa4c032440 con 0x7faa640cc5f0 2024-05-09T08:02:49.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:49.481+0000 7faa63400640 1 -- 172.21.15.1:0/1082514787 <== mon.1 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa4c032810 con 0x7faa640cc5f0 2024-05-09T08:02:49.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:49.482+0000 7faa6a200640 1 -- 172.21.15.1:0/1082514787 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faa640cc5f0 msgr2=0x7faa640cdf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:49.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:49.482+0000 7faa6a200640 1 --2- 172.21.15.1:0/1082514787 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faa640cc5f0 0x7faa640cdf40 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7faa4c0097a0 tx=0x7faa4c02a210 comp rx=0 tx=0).stop 2024-05-09T08:02:49.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:49.482+0000 7faa6a200640 1 -- 172.21.15.1:0/1082514787 shutdown_connections 2024-05-09T08:02:49.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:49.482+0000 7faa6a200640 1 --2- 172.21.15.1:0/1082514787 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faa640cc5f0 0x7faa640cdf40 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:49.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:49.482+0000 7faa6a200640 1 --2- 172.21.15.1:0/1082514787 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7faa640cac60 0x7faa640cda00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:49.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:49.482+0000 7faa6a200640 1 -- 172.21.15.1:0/1082514787 >> 172.21.15.1:0/1082514787 conn(0x7faa640cfc20 msgr2=0x7faa640d0030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:02:49.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:49.482+0000 7faa6a200640 1 -- 172.21.15.1:0/1082514787 shutdown_connections 2024-05-09T08:02:49.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:49.482+0000 7faa6a200640 1 -- 172.21.15.1:0/1082514787 wait complete. 2024-05-09T08:02:49.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:49.484+0000 7faa6a200640 1 Processor -- start 2024-05-09T08:02:49.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:49.484+0000 7faa6a200640 1 -- start start 2024-05-09T08:02:49.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:49.485+0000 7faa6a200640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7faa640cac60 0x7faa641532e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:49.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:49.485+0000 7faa6a200640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faa640cc5f0 0x7faa640c81e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:49.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:49.485+0000 7faa6a200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faa640ce4e0 0x7faa6414f530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:49.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:49.485+0000 7faa6a200640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7faa6415ac80 con 0x7faa640cc5f0 2024-05-09T08:02:49.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:49.485+0000 7faa6a200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7faa640cd550 con 0x7faa640ce4e0 2024-05-09T08:02:49.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:49.485+0000 7faa6a200640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7faa6415ae00 con 0x7faa640cac60 2024-05-09T08:02:49.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:49.485+0000 7faa63e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faa640cc5f0 0x7faa640c81e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:49.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:49.485+0000 7faa63e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faa640cc5f0 0x7faa640c81e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:52964/0 (socket says 172.21.15.1:52964) 2024-05-09T08:02:49.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:49.485+0000 7faa63e00640 1 -- 172.21.15.1:0/2411163530 learned_addr learned my addr 172.21.15.1:0/2411163530 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:02:49.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:49.485+0000 7faa68e00640 1 --2- 172.21.15.1:0/2411163530 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7faa640cac60 0x7faa641532e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:49.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:49.485+0000 7faa69800640 1 --2- 172.21.15.1:0/2411163530 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faa640ce4e0 0x7faa6414f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:49.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:49.486+0000 7faa63e00640 1 -- 172.21.15.1:0/2411163530 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7faa640cac60 msgr2=0x7faa641532e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:49.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:49.486+0000 7faa63e00640 1 --2- 172.21.15.1:0/2411163530 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7faa640cac60 0x7faa641532e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:49.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:49.486+0000 7faa63e00640 1 -- 172.21.15.1:0/2411163530 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faa640ce4e0 msgr2=0x7faa6414f530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:49.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:49.486+0000 7faa63e00640 1 --2- 172.21.15.1:0/2411163530 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faa640ce4e0 0x7faa6414f530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:49.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:49.486+0000 7faa63e00640 1 -- 172.21.15.1:0/2411163530 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa640b4ae0 con 0x7faa640cc5f0 2024-05-09T08:02:49.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:49.486+0000 7faa63e00640 1 --2- 172.21.15.1:0/2411163530 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faa640cc5f0 0x7faa640c81e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7faa4c0098c0 tx=0x7faa4c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:49.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:49.486+0000 7faa60c00640 1 -- 172.21.15.1:0/2411163530 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa4c041070 con 0x7faa640cc5f0 2024-05-09T08:02:49.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:49.486+0000 7faa60c00640 1 -- 172.21.15.1:0/2411163530 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa4c03c870 con 0x7faa640cc5f0 2024-05-09T08:02:49.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:49.486+0000 7faa6a200640 1 -- 172.21.15.1:0/2411163530 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa640b4dc0 con 0x7faa640cc5f0 2024-05-09T08:02:49.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:49.486+0000 7faa60c00640 1 -- 172.21.15.1:0/2411163530 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa4c037040 con 0x7faa640cc5f0 2024-05-09T08:02:49.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:49.487+0000 7faa6a200640 1 -- 172.21.15.1:0/2411163530 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa640b5380 con 0x7faa640cc5f0 2024-05-09T08:02:49.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:49.488+0000 7faa6a200640 1 -- 172.21.15.1:0/2411163530 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa6406e370 con 0x7faa640cc5f0 2024-05-09T08:02:49.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:49.489+0000 7faa60c00640 1 -- 172.21.15.1:0/2411163530 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7faa4c044070 con 0x7faa640cc5f0 2024-05-09T08:02:49.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:49.490+0000 7faa60c00640 1 --2- 172.21.15.1:0/2411163530 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7faa4006be60 0x7faa4006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:49.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:49.490+0000 7faa60c00640 1 -- 172.21.15.1:0/2411163530 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7faa4c0abc20 con 0x7faa640cc5f0 2024-05-09T08:02:49.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:49.491+0000 7faa68e00640 1 --2- 172.21.15.1:0/2411163530 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7faa4006be60 0x7faa4006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:49.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:49.491+0000 7faa68e00640 1 --2- 172.21.15.1:0/2411163530 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7faa4006be60 0x7faa4006e320 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7faa540113e0 tx=0x7faa5400b040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:49.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:49.495+0000 7faa60c00640 1 -- 172.21.15.1:0/2411163530 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7faa4c079c30 con 0x7faa640cc5f0 2024-05-09T08:02:49.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:49.732+0000 7faa6a200640 1 -- 172.21.15.1:0/2411163530 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7faa640cba30 con 0x7faa640cc5f0 2024-05-09T08:02:49.812 DEBUG:teuthology.orchestra.run.smithi001:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T08:02:49.820 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T08:02:50.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:50.250+0000 7faa60c00640 1 -- 172.21.15.1:0/2411163530 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v10) ==== 115+0+0 (secure 0 0 0) 0x7faa640cba30 con 0x7faa640cc5f0 2024-05-09T08:02:50.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:50.254+0000 7faa6a200640 1 -- 172.21.15.1:0/2411163530 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7faa4006be60 msgr2=0x7faa4006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:50.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:50.254+0000 7faa6a200640 1 --2- 172.21.15.1:0/2411163530 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7faa4006be60 0x7faa4006e320 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7faa540113e0 tx=0x7faa5400b040 comp rx=0 tx=0).stop 2024-05-09T08:02:50.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:50.254+0000 7faa6a200640 1 -- 172.21.15.1:0/2411163530 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faa640cc5f0 msgr2=0x7faa640c81e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:50.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:50.254+0000 7faa6a200640 1 --2- 172.21.15.1:0/2411163530 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faa640cc5f0 0x7faa640c81e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7faa4c0098c0 tx=0x7faa4c0046c0 comp rx=0 tx=0).stop 2024-05-09T08:02:50.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:50.254+0000 7faa6a200640 1 -- 172.21.15.1:0/2411163530 shutdown_connections 2024-05-09T08:02:50.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:50.254+0000 7faa6a200640 1 --2- 172.21.15.1:0/2411163530 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7faa4006be60 0x7faa4006e320 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:50.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:50.254+0000 7faa6a200640 1 --2- 172.21.15.1:0/2411163530 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faa640ce4e0 0x7faa6414f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:50.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:50.254+0000 7faa6a200640 1 --2- 172.21.15.1:0/2411163530 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faa640cc5f0 0x7faa640c81e0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:50.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:50.254+0000 7faa6a200640 1 --2- 172.21.15.1:0/2411163530 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7faa640cac60 0x7faa641532e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:50.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:50.254+0000 7faa6a200640 1 -- 172.21.15.1:0/2411163530 >> 172.21.15.1:0/2411163530 conn(0x7faa640cfc20 msgr2=0x7faa6415a230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:02:50.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:50.255+0000 7faa6a200640 1 -- 172.21.15.1:0/2411163530 shutdown_connections 2024-05-09T08:02:50.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:50.255+0000 7faa6a200640 1 -- 172.21.15.1:0/2411163530 wait complete. 2024-05-09T08:02:50.292 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-05-09T08:02:50.293 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T08:02:50.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:50.483+0000 7f5ee3e00640 1 Processor -- start 2024-05-09T08:02:50.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:50.483+0000 7f5ee3e00640 1 -- start start 2024-05-09T08:02:50.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:50.484+0000 7f5ee3e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f5ee40cd040 0x7f5ee415ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:50.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:50.484+0000 7f5ee3e00640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ee415b550 con 0x7f5ee415a5e0 2024-05-09T08:02:50.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:50.484+0000 7f5ee3e00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ee406c8b0 con 0x7f5ee406d350 2024-05-09T08:02:50.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:50.484+0000 7f5ee3e00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f5ee406cec0 con 0x7f5ee40cd040 2024-05-09T08:02:50.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:50.485+0000 7f5eda000640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f5ee406d350 0x7f5ee406da60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.1:56410/0 (socket says 172.21.15.1:56410) 2024-05-09T08:02:50.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:50.485+0000 7f5eda000640 1 -- 172.21.15.1:0/1092670063 learned_addr learned my addr 172.21.15.1:0/1092670063 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T08:02:50.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:50.485+0000 7f5ee2a00640 1 --2- 172.21.15.1:0/1092670063 >> v2:172.21.15.7:3300/0 conn(0x7f5ee40cd040 0x7f5ee415ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:50.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:50.485+0000 7f5ee2000640 1 -- 172.21.15.1:0/1092670063 <== mon.1 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1800478856 0 0) 0x7f5ee406c8b0 con 0x7f5ee406d350 2024-05-09T08:02:50.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:50.485+0000 7f5ee2000640 1 -- 172.21.15.1:0/1092670063 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5ec8003520 con 0x7f5ee406d350 2024-05-09T08:02:50.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:50.485+0000 7f5ee2000640 1 -- 172.21.15.1:0/1092670063 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1781873273 0 0) 0x7f5ee415b550 con 0x7f5ee415a5e0 2024-05-09T08:02:50.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:50.485+0000 7f5ee2000640 1 -- 172.21.15.1:0/1092670063 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5ee406c8b0 con 0x7f5ee415a5e0 2024-05-09T08:02:50.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:50.485+0000 7f5ee2a00640 1 -- 172.21.15.1:0/1092670063 >> v1:172.21.15.7:6789/0 conn(0x7f5ee406d350 legacy=0x7f5ee406da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:50.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:50.486+0000 7f5ee2a00640 1 -- 172.21.15.1:0/1092670063 >> v1:172.21.15.1:6789/0 conn(0x7f5ee415a5e0 legacy=0x7f5ee40ca5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:50.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:50.486+0000 7f5ee2a00640 1 -- 172.21.15.1:0/1092670063 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ee415af90 con 0x7f5ee40cd040 2024-05-09T08:02:50.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:50.487+0000 7f5ee2a00640 1 --2- 172.21.15.1:0/1092670063 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5ee40cd040 0x7f5ee415ba50 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f5ed40097a0 tx=0x7f5ed402a110 comp rx=0 tx=0).ready entity=mon.1 client_cookie=69e5f391481d4fc8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:50.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:50.487+0000 7f5ee2000640 1 -- 172.21.15.1:0/1092670063 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ed4037070 con 0x7f5ee40cd040 2024-05-09T08:02:50.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:50.487+0000 7f5ee2000640 1 -- 172.21.15.1:0/1092670063 <== mon.1 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ed4032440 con 0x7f5ee40cd040 2024-05-09T08:02:50.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:50.487+0000 7f5ee2000640 1 -- 172.21.15.1:0/1092670063 <== mon.1 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ed402ae60 con 0x7f5ee40cd040 2024-05-09T08:02:50.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:50.487+0000 7f5ee3e00640 1 -- 172.21.15.1:0/1092670063 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5ee40cd040 msgr2=0x7f5ee415ba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:50.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:50.487+0000 7f5ee3e00640 1 --2- 172.21.15.1:0/1092670063 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5ee40cd040 0x7f5ee415ba50 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f5ed40097a0 tx=0x7f5ed402a110 comp rx=0 tx=0).stop 2024-05-09T08:02:50.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:50.488+0000 7f5ee3e00640 1 -- 172.21.15.1:0/1092670063 shutdown_connections 2024-05-09T08:02:50.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:50.488+0000 7f5ee3e00640 1 --2- 172.21.15.1:0/1092670063 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5ee40cd040 0x7f5ee415ba50 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:50.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:50.488+0000 7f5ee3e00640 1 -- 172.21.15.1:0/1092670063 >> 172.21.15.1:0/1092670063 conn(0x7f5ee40b1780 msgr2=0x7f5ee40b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:02:50.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:50.488+0000 7f5ee3e00640 1 -- 172.21.15.1:0/1092670063 shutdown_connections 2024-05-09T08:02:50.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:50.488+0000 7f5ee3e00640 1 -- 172.21.15.1:0/1092670063 wait complete. 2024-05-09T08:02:50.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:50.489+0000 7f5ee3e00640 1 Processor -- start 2024-05-09T08:02:50.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:50.490+0000 7f5ee3e00640 1 -- start start 2024-05-09T08:02:50.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:50.490+0000 7f5ee3e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5ee406d350 0x7f5ee40c6f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:50.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:50.490+0000 7f5ee3e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5ee40cd040 0x7f5ee4157550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:50.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:50.490+0000 7f5ee3e00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f5ee415a5e0 0x7f5ee4157a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:50.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:50.490+0000 7f5ee3e00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f5ee406cb50 con 0x7f5ee40cd040 2024-05-09T08:02:50.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:50.490+0000 7f5ee3e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5ee406d7b0 con 0x7f5ee406d350 2024-05-09T08:02:50.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:50.490+0000 7f5ee3e00640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f5ee40cad00 con 0x7f5ee415a5e0 2024-05-09T08:02:50.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:50.491+0000 7f5eda000640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5ee40cd040 0x7f5ee4157550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:50.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:50.491+0000 7f5eda000640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5ee40cd040 0x7f5ee4157550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:52978/0 (socket says 172.21.15.1:52978) 2024-05-09T08:02:50.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:50.491+0000 7f5eda000640 1 -- 172.21.15.1:0/3686681827 learned_addr learned my addr 172.21.15.1:0/3686681827 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:02:50.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:50.491+0000 7f5ee2a00640 1 --2- 172.21.15.1:0/3686681827 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5ee406d350 0x7f5ee40c6f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:50.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:50.491+0000 7f5ee3400640 1 --2- 172.21.15.1:0/3686681827 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f5ee415a5e0 0x7f5ee4157a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:50.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:50.491+0000 7f5ee2a00640 1 -- 172.21.15.1:0/3686681827 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f5ee415a5e0 msgr2=0x7f5ee4157a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:50.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:50.491+0000 7f5ee2a00640 1 --2- 172.21.15.1:0/3686681827 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f5ee415a5e0 0x7f5ee4157a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:50.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:50.491+0000 7f5ee2a00640 1 -- 172.21.15.1:0/3686681827 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5ee40cd040 msgr2=0x7f5ee4157550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:50.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:50.491+0000 7f5ee2a00640 1 --2- 172.21.15.1:0/3686681827 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5ee40cd040 0x7f5ee4157550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:50.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:50.491+0000 7f5ee2a00640 1 -- 172.21.15.1:0/3686681827 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ee4155ae0 con 0x7f5ee406d350 2024-05-09T08:02:50.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:50.492+0000 7f5ee2a00640 1 --2- 172.21.15.1:0/3686681827 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5ee406d350 0x7f5ee40c6f70 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f5ed4031000 tx=0x7f5ed40046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:50.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:50.492+0000 7f5edb400640 1 -- 172.21.15.1:0/3686681827 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ed4041030 con 0x7f5ee406d350 2024-05-09T08:02:50.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:50.492+0000 7f5edb400640 1 -- 172.21.15.1:0/3686681827 <== mon.1 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ed403d4e0 con 0x7f5ee406d350 2024-05-09T08:02:50.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:50.492+0000 7f5edb400640 1 -- 172.21.15.1:0/3686681827 <== mon.1 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ed4037040 con 0x7f5ee406d350 2024-05-09T08:02:50.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:50.492+0000 7f5ee3e00640 1 -- 172.21.15.1:0/3686681827 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ee41f0870 con 0x7f5ee406d350 2024-05-09T08:02:50.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:50.493+0000 7f5ee3e00640 1 -- 172.21.15.1:0/3686681827 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ee41f0e10 con 0x7f5ee406d350 2024-05-09T08:02:50.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:50.494+0000 7f5ee3e00640 1 -- 172.21.15.1:0/3686681827 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ee404f660 con 0x7f5ee406d350 2024-05-09T08:02:50.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:50.495+0000 7f5edb400640 1 -- 172.21.15.1:0/3686681827 <== mon.1 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5ed403d8b0 con 0x7f5ee406d350 2024-05-09T08:02:50.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:50.496+0000 7f5edb400640 1 --2- 172.21.15.1:0/3686681827 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f5ec006be30 0x7f5ec006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:50.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:50.496+0000 7f5edb400640 1 -- 172.21.15.1:0/3686681827 <== mon.1 v2:172.21.15.7:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f5ed40ab390 con 0x7f5ee406d350 2024-05-09T08:02:50.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:50.496+0000 7f5eda000640 1 --2- 172.21.15.1:0/3686681827 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f5ec006be30 0x7f5ec006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:50.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:50.496+0000 7f5eda000640 1 --2- 172.21.15.1:0/3686681827 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f5ec006be30 0x7f5ec006e2f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5ee4159b20 tx=0x7f5ecc007e10 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:50.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:50.501+0000 7f5edb400640 1 -- 172.21.15.1:0/3686681827 <== mon.1 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f5ed404b020 con 0x7f5ee406d350 2024-05-09T08:02:50.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:50.746+0000 7f5ee3e00640 1 -- 172.21.15.1:0/3686681827 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5ee40c9090 con 0x7f5ee406d350 2024-05-09T08:02:50.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:50.747+0000 7f5edb400640 1 -- 172.21.15.1:0/3686681827 <== mon.1 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+6807 (secure 0 0 0) 0x7f5ed4079280 con 0x7f5ee406d350 2024-05-09T08:02:50.764 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T08:02:50.764 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":10,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4220,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/1157980939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":1157980939},{"type":"v1","addr":"172.21.15.7:6837","nonce":1157980939}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4235,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6833/3269798795","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6832","nonce":3269798795},{"type":"v1","addr":"172.21.15.1:6833","nonce":3269798795}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4241,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/725115499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6835","nonce":725115499},{"type":"v1","addr":"172.21.15.1:6837","nonce":725115499}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4276,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6838/1827115274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1827115274},{"type":"v1","addr":"172.21.15.7:6838","nonce":1827115274}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/1032767352","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":1032767352},{"type":"v1","addr":"172.21.15.7:6839","nonce":1032767352}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":10,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-09T08:02:15.332696+0000","modified":"2024-05-09T08:02:49.734561+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4294},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4294":{"gid":4294,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":6,"addr":"172.21.15.1:6836/1509134635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":1509134635},{"type":"v1","addr":"172.21.15.1:6836","nonce":1509134635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4294,"qdb_cluster":[4294]},"id":1}]} 2024-05-09T08:02:50.764 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 10 2024-05-09T08:02:50.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:50.750+0000 7f5ee3e00640 1 -- 172.21.15.1:0/3686681827 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f5ec006be30 msgr2=0x7f5ec006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:50.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:50.750+0000 7f5ee3e00640 1 --2- 172.21.15.1:0/3686681827 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f5ec006be30 0x7f5ec006e2f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5ee4159b20 tx=0x7f5ecc007e10 comp rx=0 tx=0).stop 2024-05-09T08:02:50.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:50.750+0000 7f5ee3e00640 1 -- 172.21.15.1:0/3686681827 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5ee406d350 msgr2=0x7f5ee40c6f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:50.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:50.750+0000 7f5ee3e00640 1 --2- 172.21.15.1:0/3686681827 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5ee406d350 0x7f5ee40c6f70 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f5ed4031000 tx=0x7f5ed40046c0 comp rx=0 tx=0).stop 2024-05-09T08:02:50.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:50.750+0000 7f5ee3e00640 1 -- 172.21.15.1:0/3686681827 shutdown_connections 2024-05-09T08:02:50.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:50.750+0000 7f5ee3e00640 1 --2- 172.21.15.1:0/3686681827 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f5ec006be30 0x7f5ec006e2f0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:50.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:50.750+0000 7f5ee3e00640 1 --2- 172.21.15.1:0/3686681827 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f5ee415a5e0 0x7f5ee4157a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:50.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:50.750+0000 7f5ee3e00640 1 --2- 172.21.15.1:0/3686681827 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5ee40cd040 0x7f5ee4157550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:50.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:50.750+0000 7f5ee3e00640 1 --2- 172.21.15.1:0/3686681827 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5ee406d350 0x7f5ee40c6f70 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:50.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:50.750+0000 7f5ee3e00640 1 -- 172.21.15.1:0/3686681827 >> 172.21.15.1:0/3686681827 conn(0x7f5ee40b1780 msgr2=0x7f5ee40ce0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:02:50.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:50.750+0000 7f5ee3e00640 1 -- 172.21.15.1:0/3686681827 shutdown_connections 2024-05-09T08:02:50.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:50.750+0000 7f5ee3e00640 1 -- 172.21.15.1:0/3686681827 wait complete. 2024-05-09T08:02:50.787 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T08:02:51.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:50.984+0000 7f1c77e00640 1 Processor -- start 2024-05-09T08:02:51.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:50.984+0000 7f1c77e00640 1 -- start start 2024-05-09T08:02:51.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:50.986+0000 7f1c77e00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f1c780cc730 0x7f1c78163bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:51.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:50.986+0000 7f1c77e00640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1c7815c790 con 0x7f1c780c8180 2024-05-09T08:02:51.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:50.986+0000 7f1c77e00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1c780c6a30 con 0x7f1c78152500 2024-05-09T08:02:51.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:50.986+0000 7f1c77e00640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f1c780ccb80 con 0x7f1c780cc730 2024-05-09T08:02:51.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:50.986+0000 7f1c6e000640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f1c780c8180 0x7f1c780c7660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:38390/0 (socket says 172.21.15.1:38390) 2024-05-09T08:02:51.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:50.986+0000 7f1c6e000640 1 -- 172.21.15.1:0/1042766432 learned_addr learned my addr 172.21.15.1:0/1042766432 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T08:02:51.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:50.987+0000 7f1c77400640 1 --2- 172.21.15.1:0/1042766432 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f1c780cc730 0x7f1c78163bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:51.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:50.987+0000 7f1c76000640 1 -- 172.21.15.1:0/1042766432 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2809450555 0 0) 0x7f1c60001110 con 0x7f1c780c8180 2024-05-09T08:02:51.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:50.987+0000 7f1c76000640 1 -- 172.21.15.1:0/1042766432 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1c5c003520 con 0x7f1c780c8180 2024-05-09T08:02:51.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:50.987+0000 7f1c76000640 1 -- 172.21.15.1:0/1042766432 <== mon.1 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 837780924 0 0) 0x7f1c780c6a30 con 0x7f1c78152500 2024-05-09T08:02:51.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:50.987+0000 7f1c77400640 1 -- 172.21.15.1:0/1042766432 >> v1:172.21.15.7:6789/0 conn(0x7f1c78152500 legacy=0x7f1c7815c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:51.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:50.987+0000 7f1c77400640 1 -- 172.21.15.1:0/1042766432 >> v1:172.21.15.1:6789/0 conn(0x7f1c780c8180 legacy=0x7f1c780c7660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:51.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:50.987+0000 7f1c77400640 1 -- 172.21.15.1:0/1042766432 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c780c6cf0 con 0x7f1c780cc730 2024-05-09T08:02:51.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:50.988+0000 7f1c77400640 1 --2- 172.21.15.1:0/1042766432 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f1c780cc730 0x7f1c78163bc0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f1c70009dc0 tx=0x7f1c7002a760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f50528ee778d52c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:51.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:50.988+0000 7f1c76000640 1 -- 172.21.15.1:0/1042766432 <== mon.2 v2:172.21.15.7:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c70007740 con 0x7f1c780cc730 2024-05-09T08:02:51.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:50.988+0000 7f1c76000640 1 -- 172.21.15.1:0/1042766432 <== mon.2 v2:172.21.15.7:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c700078e0 con 0x7f1c780cc730 2024-05-09T08:02:51.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:50.988+0000 7f1c76000640 1 -- 172.21.15.1:0/1042766432 <== mon.2 v2:172.21.15.7:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c70007c90 con 0x7f1c780cc730 2024-05-09T08:02:51.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:50.989+0000 7f1c77e00640 1 -- 172.21.15.1:0/1042766432 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f1c780cc730 msgr2=0x7f1c78163bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:51.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:50.989+0000 7f1c77e00640 1 --2- 172.21.15.1:0/1042766432 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f1c780cc730 0x7f1c78163bc0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f1c70009dc0 tx=0x7f1c7002a760 comp rx=0 tx=0).stop 2024-05-09T08:02:51.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:50.989+0000 7f1c77e00640 1 -- 172.21.15.1:0/1042766432 shutdown_connections 2024-05-09T08:02:51.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:50.989+0000 7f1c77e00640 1 --2- 172.21.15.1:0/1042766432 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f1c780cc730 0x7f1c78163bc0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:51.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:50.989+0000 7f1c77e00640 1 -- 172.21.15.1:0/1042766432 >> 172.21.15.1:0/1042766432 conn(0x7f1c780cbc10 msgr2=0x7f1c780cc020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:02:51.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:50.989+0000 7f1c77e00640 1 -- 172.21.15.1:0/1042766432 shutdown_connections 2024-05-09T08:02:51.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:50.989+0000 7f1c77e00640 1 -- 172.21.15.1:0/1042766432 wait complete. 2024-05-09T08:02:51.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:50.991+0000 7f1c77e00640 1 Processor -- start 2024-05-09T08:02:51.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:50.992+0000 7f1c77e00640 1 -- start start 2024-05-09T08:02:51.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:50.993+0000 7f1c77e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1c780c8180 0x7f1c781f2160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:51.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:50.993+0000 7f1c77e00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f1c780cc730 0x7f1c78159470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:51.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:50.993+0000 7f1c77e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c78152500 0x7f1c7814f920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:51.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:50.993+0000 7f1c77e00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f1c780c22d0 con 0x7f1c780c8180 2024-05-09T08:02:51.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:50.993+0000 7f1c77e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1c780c72b0 con 0x7f1c78152500 2024-05-09T08:02:51.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:50.993+0000 7f1c77e00640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f1c7815b880 con 0x7f1c780cc730 2024-05-09T08:02:51.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:50.993+0000 7f1c76a00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1c780c8180 0x7f1c781f2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:51.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:50.993+0000 7f1c76a00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1c780c8180 0x7f1c781f2160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:52988/0 (socket says 172.21.15.1:52988) 2024-05-09T08:02:51.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:50.993+0000 7f1c76a00640 1 -- 172.21.15.1:0/3347302610 learned_addr learned my addr 172.21.15.1:0/3347302610 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:02:51.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:50.993+0000 7f1c6e000640 1 --2- 172.21.15.1:0/3347302610 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f1c780cc730 0x7f1c78159470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:51.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:50.993+0000 7f1c77400640 1 --2- 172.21.15.1:0/3347302610 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c78152500 0x7f1c7814f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:51.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:50.994+0000 7f1c76a00640 1 -- 172.21.15.1:0/3347302610 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f1c780cc730 msgr2=0x7f1c78159470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:51.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:50.994+0000 7f1c76a00640 1 --2- 172.21.15.1:0/3347302610 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f1c780cc730 0x7f1c78159470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:51.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:50.994+0000 7f1c76a00640 1 -- 172.21.15.1:0/3347302610 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c78152500 msgr2=0x7f1c7814f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:51.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:50.994+0000 7f1c76a00640 1 --2- 172.21.15.1:0/3347302610 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c78152500 0x7f1c7814f920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:51.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:50.994+0000 7f1c76a00640 1 -- 172.21.15.1:0/3347302610 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c781f26a0 con 0x7f1c780c8180 2024-05-09T08:02:51.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:50.994+0000 7f1c6e000640 1 --2- 172.21.15.1:0/3347302610 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f1c780cc730 0x7f1c78159470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:02:51.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:50.994+0000 7f1c76a00640 1 --2- 172.21.15.1:0/3347302610 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1c780c8180 0x7f1c781f2160 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1c6800c3c0 tx=0x7f1c6800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:51.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:50.995+0000 7f1c6f400640 1 -- 172.21.15.1:0/3347302610 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c68017070 con 0x7f1c780c8180 2024-05-09T08:02:51.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:50.995+0000 7f1c6f400640 1 -- 172.21.15.1:0/3347302610 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c680035f0 con 0x7f1c780c8180 2024-05-09T08:02:51.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:50.995+0000 7f1c6f400640 1 -- 172.21.15.1:0/3347302610 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c680039e0 con 0x7f1c780c8180 2024-05-09T08:02:51.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:50.995+0000 7f1c77e00640 1 -- 172.21.15.1:0/3347302610 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c780722c0 con 0x7f1c780c8180 2024-05-09T08:02:51.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:50.995+0000 7f1c77e00640 1 -- 172.21.15.1:0/3347302610 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c781ee420 con 0x7f1c780c8180 2024-05-09T08:02:51.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:50.996+0000 7f1c77e00640 1 -- 172.21.15.1:0/3347302610 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c781ee210 con 0x7f1c780c8180 2024-05-09T08:02:51.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:50.998+0000 7f1c6f400640 1 -- 172.21.15.1:0/3347302610 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1c6801a070 con 0x7f1c780c8180 2024-05-09T08:02:51.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:50.998+0000 7f1c6f400640 1 --2- 172.21.15.1:0/3347302610 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f1c5406be60 0x7f1c5406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:51.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:50.999+0000 7f1c6f400640 1 -- 172.21.15.1:0/3347302610 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f1c6808a4b0 con 0x7f1c780c8180 2024-05-09T08:02:51.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:50.999+0000 7f1c6e000640 1 --2- 172.21.15.1:0/3347302610 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f1c5406be60 0x7f1c5406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:51.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:51.000+0000 7f1c6e000640 1 --2- 172.21.15.1:0/3347302610 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f1c5406be60 0x7f1c5406e320 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f1c781504e0 tx=0x7f1c60009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:51.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:51.003+0000 7f1c6f400640 1 -- 172.21.15.1:0/3347302610 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f1c68058430 con 0x7f1c780c8180 2024-05-09T08:02:51.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:51.208+0000 7f1c77e00640 1 -- 172.21.15.1:0/3347302610 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1c7804f660 con 0x7f1c780c8180 2024-05-09T08:02:51.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:51.211+0000 7f1c6f400640 1 -- 172.21.15.1:0/3347302610 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18382 (secure 0 0 0) 0x7f1c6805c730 con 0x7f1c780c8180 2024-05-09T08:02:51.228 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T08:02:51.228 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":25,"fsid":"54329ec8-3270-4043-9a82-c6cd7420aa7f","created":"2024-05-09T08:01:56.666970+0000","modified":"2024-05-09T08:02:20.403456+0000","last_up_change":"2024-05-09T08:02:07.146082+0000","last_in_change":"2024-05-09T08:01:59.008225+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T08:02:09.165511+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T08:02:12.925485+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T08:02:14.367455+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-09T08:02:17.967064+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c064454e-a9b5-4a24-9a77-17f0a4085952","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6800","nonce":1513349547},{"type":"v1","addr":"172.21.15.1:6801","nonce":1513349547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":1513349547},{"type":"v1","addr":"172.21.15.1:6803","nonce":1513349547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":1513349547},{"type":"v1","addr":"172.21.15.1:6807","nonce":1513349547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":1513349547},{"type":"v1","addr":"172.21.15.1:6805","nonce":1513349547}]},"public_addr":"172.21.15.1:6801/1513349547","cluster_addr":"172.21.15.1:6803/1513349547","heartbeat_back_addr":"172.21.15.1:6807/1513349547","heartbeat_front_addr":"172.21.15.1:6805/1513349547","state":["exists","up"]},{"osd":1,"uuid":"1a9ea687-2ff6-407a-949b-0e7e1083af21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":2942255259},{"type":"v1","addr":"172.21.15.1:6817","nonce":2942255259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":2942255259},{"type":"v1","addr":"172.21.15.1:6819","nonce":2942255259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":2942255259},{"type":"v1","addr":"172.21.15.1:6823","nonce":2942255259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6820","nonce":2942255259},{"type":"v1","addr":"172.21.15.1:6821","nonce":2942255259}]},"public_addr":"172.21.15.1:6817/2942255259","cluster_addr":"172.21.15.1:6819/2942255259","heartbeat_back_addr":"172.21.15.1:6823/2942255259","heartbeat_front_addr":"172.21.15.1:6821/2942255259","state":["exists","up"]},{"osd":2,"uuid":"c88af08b-7c9a-4dd8-9640-44138239addb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":2925031811},{"type":"v1","addr":"172.21.15.1:6809","nonce":2925031811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":2925031811},{"type":"v1","addr":"172.21.15.1:6811","nonce":2925031811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":2925031811},{"type":"v1","addr":"172.21.15.1:6815","nonce":2925031811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":2925031811},{"type":"v1","addr":"172.21.15.1:6813","nonce":2925031811}]},"public_addr":"172.21.15.1:6809/2925031811","cluster_addr":"172.21.15.1:6811/2925031811","heartbeat_back_addr":"172.21.15.1:6815/2925031811","heartbeat_front_addr":"172.21.15.1:6813/2925031811","state":["exists","up"]},{"osd":3,"uuid":"c087b7e3-a41d-4333-9365-32e3bd8c8e6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6824","nonce":2497776654},{"type":"v1","addr":"172.21.15.1:6825","nonce":2497776654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":2497776654},{"type":"v1","addr":"172.21.15.1:6827","nonce":2497776654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":2497776654},{"type":"v1","addr":"172.21.15.1:6831","nonce":2497776654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6828","nonce":2497776654},{"type":"v1","addr":"172.21.15.1:6829","nonce":2497776654}]},"public_addr":"172.21.15.1:6825/2497776654","cluster_addr":"172.21.15.1:6827/2497776654","heartbeat_back_addr":"172.21.15.1:6831/2497776654","heartbeat_front_addr":"172.21.15.1:6829/2497776654","state":["exists","up"]},{"osd":4,"uuid":"b4576b9c-871d-47a8-8d36-65af525d0200","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":2957026670},{"type":"v1","addr":"172.21.15.7:6801","nonce":2957026670}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2957026670},{"type":"v1","addr":"172.21.15.7:6803","nonce":2957026670}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":2957026670},{"type":"v1","addr":"172.21.15.7:6807","nonce":2957026670}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2957026670},{"type":"v1","addr":"172.21.15.7:6805","nonce":2957026670}]},"public_addr":"172.21.15.7:6801/2957026670","cluster_addr":"172.21.15.7:6803/2957026670","heartbeat_back_addr":"172.21.15.7:6807/2957026670","heartbeat_front_addr":"172.21.15.7:6805/2957026670","state":["exists","up"]},{"osd":5,"uuid":"902b3d94-72f3-4499-b21a-e32453ef5144","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1670295242},{"type":"v1","addr":"172.21.15.7:6809","nonce":1670295242}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1670295242},{"type":"v1","addr":"172.21.15.7:6811","nonce":1670295242}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1670295242},{"type":"v1","addr":"172.21.15.7:6815","nonce":1670295242}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1670295242},{"type":"v1","addr":"172.21.15.7:6813","nonce":1670295242}]},"public_addr":"172.21.15.7:6809/1670295242","cluster_addr":"172.21.15.7:6811/1670295242","heartbeat_back_addr":"172.21.15.7:6815/1670295242","heartbeat_front_addr":"172.21.15.7:6813/1670295242","state":["exists","up"]},{"osd":6,"uuid":"ead1d771-1728-4f85-adb4-26be21cedff3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":2852086880},{"type":"v1","addr":"172.21.15.7:6817","nonce":2852086880}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":2852086880},{"type":"v1","addr":"172.21.15.7:6819","nonce":2852086880}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":2852086880},{"type":"v1","addr":"172.21.15.7:6823","nonce":2852086880}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":2852086880},{"type":"v1","addr":"172.21.15.7:6821","nonce":2852086880}]},"public_addr":"172.21.15.7:6817/2852086880","cluster_addr":"172.21.15.7:6819/2852086880","heartbeat_back_addr":"172.21.15.7:6823/2852086880","heartbeat_front_addr":"172.21.15.7:6821/2852086880","state":["exists","up"]},{"osd":7,"uuid":"3ab5a201-9394-48a8-8918-e95197893669","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":1442884138},{"type":"v1","addr":"172.21.15.7:6825","nonce":1442884138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1442884138},{"type":"v1","addr":"172.21.15.7:6827","nonce":1442884138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1442884138},{"type":"v1","addr":"172.21.15.7:6831","nonce":1442884138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1442884138},{"type":"v1","addr":"172.21.15.7:6829","nonce":1442884138}]},"public_addr":"172.21.15.7:6825/1442884138","cluster_addr":"172.21.15.7:6827/1442884138","heartbeat_back_addr":"172.21.15.7:6831/1442884138","heartbeat_front_addr":"172.21.15.7:6829/1442884138","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:02:04.615675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:02:04.385974+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:02:04.619217+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:02:04.401681+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:02:04.097081+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:02:04.129981+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:02:04.225327+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:02:04.193351+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T08:02:51.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:51.213+0000 7f1c77e00640 1 -- 172.21.15.1:0/3347302610 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f1c5406be60 msgr2=0x7f1c5406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:51.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:51.213+0000 7f1c77e00640 1 --2- 172.21.15.1:0/3347302610 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f1c5406be60 0x7f1c5406e320 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f1c781504e0 tx=0x7f1c60009040 comp rx=0 tx=0).stop 2024-05-09T08:02:51.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:51.213+0000 7f1c77e00640 1 -- 172.21.15.1:0/3347302610 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1c780c8180 msgr2=0x7f1c781f2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:51.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:51.213+0000 7f1c77e00640 1 --2- 172.21.15.1:0/3347302610 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1c780c8180 0x7f1c781f2160 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1c6800c3c0 tx=0x7f1c6800c880 comp rx=0 tx=0).stop 2024-05-09T08:02:51.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:51.214+0000 7f1c77e00640 1 -- 172.21.15.1:0/3347302610 shutdown_connections 2024-05-09T08:02:51.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:51.214+0000 7f1c77e00640 1 --2- 172.21.15.1:0/3347302610 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f1c5406be60 0x7f1c5406e320 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:51.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:51.214+0000 7f1c77e00640 1 --2- 172.21.15.1:0/3347302610 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c78152500 0x7f1c7814f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:51.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:51.214+0000 7f1c77e00640 1 --2- 172.21.15.1:0/3347302610 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f1c780cc730 0x7f1c78159470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:51.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:51.214+0000 7f1c77e00640 1 --2- 172.21.15.1:0/3347302610 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1c780c8180 0x7f1c781f2160 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:51.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:51.214+0000 7f1c77e00640 1 -- 172.21.15.1:0/3347302610 >> 172.21.15.1:0/3347302610 conn(0x7f1c780cbc10 msgr2=0x7f1c780ca2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:02:51.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:51.214+0000 7f1c77e00640 1 -- 172.21.15.1:0/3347302610 shutdown_connections 2024-05-09T08:02:51.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:51.214+0000 7f1c77e00640 1 -- 172.21.15.1:0/3347302610 wait complete. 2024-05-09T08:02:51.250 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-05-09T08:02:51.250 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-09T08:02:51.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:51.433+0000 7fb46d000640 1 Processor -- start 2024-05-09T08:02:51.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:51.433+0000 7fb46d000640 1 -- start start 2024-05-09T08:02:51.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:51.435+0000 7fb46d000640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fb4680c6ea0 0x7fb468072900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:51.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:51.435+0000 7fb46d000640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb468073200 con 0x7fb46806dca0 2024-05-09T08:02:51.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:51.435+0000 7fb46d000640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb4680734e0 con 0x7fb46806ece0 2024-05-09T08:02:51.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:51.435+0000 7fb46d000640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7fb468072e40 con 0x7fb4680c6ea0 2024-05-09T08:02:51.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:51.435+0000 7fb465e00640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7fb46806dca0 0x7fb46806e080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:38392/0 (socket says 172.21.15.1:38392) 2024-05-09T08:02:51.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:51.435+0000 7fb465e00640 1 -- 172.21.15.1:0/3278195092 learned_addr learned my addr 172.21.15.1:0/3278195092 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T08:02:51.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:51.435+0000 7fb465400640 1 --2- 172.21.15.1:0/3278195092 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fb4680c6ea0 0x7fb468072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:51.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:51.436+0000 7fb465400640 1 -- 172.21.15.1:0/3278195092 >> v1:172.21.15.7:6789/0 conn(0x7fb46806ece0 legacy=0x7fb468152410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:51.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:51.436+0000 7fb465400640 1 -- 172.21.15.1:0/3278195092 >> v1:172.21.15.1:6789/0 conn(0x7fb46806dca0 legacy=0x7fb46806e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:51.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:51.436+0000 7fb465400640 1 -- 172.21.15.1:0/3278195092 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb468071e20 con 0x7fb4680c6ea0 2024-05-09T08:02:51.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:51.436+0000 7fb464a00640 1 -- 172.21.15.1:0/3278195092 <== mon.1 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1496741445 0 0) 0x7fb4680734e0 con 0x7fb46806ece0 2024-05-09T08:02:51.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:51.437+0000 7fb465400640 1 --2- 172.21.15.1:0/3278195092 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fb4680c6ea0 0x7fb468072900 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fb4500097d0 tx=0x7fb45002a1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bb0253a63f7d2ea5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:51.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:51.437+0000 7fb464a00640 1 -- 172.21.15.1:0/3278195092 <== mon.2 v2:172.21.15.7:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb450037070 con 0x7fb4680c6ea0 2024-05-09T08:02:51.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:51.437+0000 7fb464a00640 1 -- 172.21.15.1:0/3278195092 <== mon.2 v2:172.21.15.7:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb45002ae30 con 0x7fb4680c6ea0 2024-05-09T08:02:51.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:51.437+0000 7fb464a00640 1 -- 172.21.15.1:0/3278195092 <== mon.2 v2:172.21.15.7:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb450032640 con 0x7fb4680c6ea0 2024-05-09T08:02:51.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:51.438+0000 7fb46d000640 1 -- 172.21.15.1:0/3278195092 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fb4680c6ea0 msgr2=0x7fb468072900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:51.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:51.438+0000 7fb46d000640 1 --2- 172.21.15.1:0/3278195092 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fb4680c6ea0 0x7fb468072900 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fb4500097d0 tx=0x7fb45002a1a0 comp rx=0 tx=0).stop 2024-05-09T08:02:51.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:51.438+0000 7fb46d000640 1 -- 172.21.15.1:0/3278195092 shutdown_connections 2024-05-09T08:02:51.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:51.438+0000 7fb46d000640 1 --2- 172.21.15.1:0/3278195092 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fb4680c6ea0 0x7fb468072900 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:51.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:51.438+0000 7fb46d000640 1 -- 172.21.15.1:0/3278195092 >> 172.21.15.1:0/3278195092 conn(0x7fb4680cc070 msgr2=0x7fb4680cc480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:02:51.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:51.438+0000 7fb46d000640 1 -- 172.21.15.1:0/3278195092 shutdown_connections 2024-05-09T08:02:51.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:51.438+0000 7fb46d000640 1 -- 172.21.15.1:0/3278195092 wait complete. 2024-05-09T08:02:51.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:51.440+0000 7fb46d000640 1 Processor -- start 2024-05-09T08:02:51.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:51.440+0000 7fb46d000640 1 -- start start 2024-05-09T08:02:51.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:51.441+0000 7fb46d000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb46806dca0 0x7fb468155950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:51.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:51.441+0000 7fb46d000640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fb46806ece0 0x7fb468152f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:51.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:51.441+0000 7fb46d000640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb4680c6ea0 0x7fb4681561f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:51.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:51.441+0000 7fb46d000640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fb468072470 con 0x7fb4680c6ea0 2024-05-09T08:02:51.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:51.441+0000 7fb46d000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb468152b20 con 0x7fb46806dca0 2024-05-09T08:02:51.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:51.441+0000 7fb46d000640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7fb468152ca0 con 0x7fb46806ece0 2024-05-09T08:02:51.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:51.442+0000 7fb465e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb46806dca0 0x7fb468155950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:51.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:51.441+0000 7fb466800640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb4680c6ea0 0x7fb4681561f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:51.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:51.442+0000 7fb465400640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fb46806ece0 0x7fb468152f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:51.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:51.442+0000 7fb465e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb46806dca0 0x7fb468155950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.1:57906/0 (socket says 172.21.15.1:57906) 2024-05-09T08:02:51.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:51.442+0000 7fb465e00640 1 -- 172.21.15.1:0/1377766943 learned_addr learned my addr 172.21.15.1:0/1377766943 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:02:51.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:51.442+0000 7fb465e00640 1 -- 172.21.15.1:0/1377766943 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fb46806ece0 msgr2=0x7fb468152f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:51.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:51.442+0000 7fb465e00640 1 --2- 172.21.15.1:0/1377766943 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fb46806ece0 0x7fb468152f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:51.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:51.442+0000 7fb465e00640 1 -- 172.21.15.1:0/1377766943 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb4680c6ea0 msgr2=0x7fb4681561f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:51.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:51.442+0000 7fb465e00640 1 --2- 172.21.15.1:0/1377766943 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb4680c6ea0 0x7fb4681561f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:51.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:51.442+0000 7fb465e00640 1 -- 172.21.15.1:0/1377766943 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb468158070 con 0x7fb46806dca0 2024-05-09T08:02:51.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:51.442+0000 7fb465e00640 1 --2- 172.21.15.1:0/1377766943 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb46806dca0 0x7fb468155950 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fb45800c910 tx=0x7fb45800cdd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:51.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:51.443+0000 7fb45e000640 1 -- 172.21.15.1:0/1377766943 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb458018070 con 0x7fb46806dca0 2024-05-09T08:02:51.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:51.443+0000 7fb45e000640 1 -- 172.21.15.1:0/1377766943 <== mon.1 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb45800a700 con 0x7fb46806dca0 2024-05-09T08:02:51.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:51.443+0000 7fb45e000640 1 -- 172.21.15.1:0/1377766943 <== mon.1 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb45800aaf0 con 0x7fb46806dca0 2024-05-09T08:02:51.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:51.443+0000 7fb46d000640 1 -- 172.21.15.1:0/1377766943 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb468158310 con 0x7fb46806dca0 2024-05-09T08:02:51.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:51.443+0000 7fb46d000640 1 -- 172.21.15.1:0/1377766943 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4681588d0 con 0x7fb46806dca0 2024-05-09T08:02:51.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:51.444+0000 7fb46d000640 1 -- 172.21.15.1:0/1377766943 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4680c6760 con 0x7fb46806dca0 2024-05-09T08:02:51.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:51.445+0000 7fb45e000640 1 -- 172.21.15.1:0/1377766943 <== mon.1 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb45800ac90 con 0x7fb46806dca0 2024-05-09T08:02:51.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:51.446+0000 7fb45e000640 1 --2- 172.21.15.1:0/1377766943 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fb44406be30 0x7fb44406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:51.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:51.446+0000 7fb45e000640 1 -- 172.21.15.1:0/1377766943 <== mon.1 v2:172.21.15.7:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fb45808a850 con 0x7fb46806dca0 2024-05-09T08:02:51.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:51.446+0000 7fb465400640 1 --2- 172.21.15.1:0/1377766943 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fb44406be30 0x7fb44406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:51.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:51.447+0000 7fb465400640 1 --2- 172.21.15.1:0/1377766943 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fb44406be30 0x7fb44406e2f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fb450034000 tx=0x7fb45002e000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:51.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:51.451+0000 7fb45e000640 1 -- 172.21.15.1:0/1377766943 <== mon.1 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fb4680c6760 con 0x7fb46806dca0 2024-05-09T08:02:51.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:51.651+0000 7fb46d000640 1 -- 172.21.15.1:0/1377766943 --> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb468074d70 con 0x7fb44406be30 2024-05-09T08:02:51.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:51.662+0000 7fb45e000640 1 -- 172.21.15.1:0/1377766943 <== mgr.4114 v2:172.21.15.7:6832/3562612464 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332537 (secure 0 0 0) 0x7fb468074d70 con 0x7fb44406be30 2024-05-09T08:02:51.684 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T08:02:51.686 INFO:teuthology.orchestra.run.smithi001.stderr:dumped all 2024-05-09T08:02:51.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:51.670+0000 7fb46d000640 1 -- 172.21.15.1:0/1377766943 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fb44406be30 msgr2=0x7fb44406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:51.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:51.670+0000 7fb46d000640 1 --2- 172.21.15.1:0/1377766943 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fb44406be30 0x7fb44406e2f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fb450034000 tx=0x7fb45002e000 comp rx=0 tx=0).stop 2024-05-09T08:02:51.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:51.670+0000 7fb46d000640 1 -- 172.21.15.1:0/1377766943 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb46806dca0 msgr2=0x7fb468155950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:51.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:51.670+0000 7fb46d000640 1 --2- 172.21.15.1:0/1377766943 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb46806dca0 0x7fb468155950 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fb45800c910 tx=0x7fb45800cdd0 comp rx=0 tx=0).stop 2024-05-09T08:02:51.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:51.670+0000 7fb46d000640 1 -- 172.21.15.1:0/1377766943 shutdown_connections 2024-05-09T08:02:51.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:51.670+0000 7fb46d000640 1 --2- 172.21.15.1:0/1377766943 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fb44406be30 0x7fb44406e2f0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:51.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:51.670+0000 7fb46d000640 1 --2- 172.21.15.1:0/1377766943 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb4680c6ea0 0x7fb4681561f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:51.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:51.670+0000 7fb46d000640 1 --2- 172.21.15.1:0/1377766943 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fb46806ece0 0x7fb468152f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:51.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:51.670+0000 7fb46d000640 1 --2- 172.21.15.1:0/1377766943 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb46806dca0 0x7fb468155950 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:51.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:51.670+0000 7fb46d000640 1 -- 172.21.15.1:0/1377766943 >> 172.21.15.1:0/1377766943 conn(0x7fb4680cc070 msgr2=0x7fb4680c7380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:02:51.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:51.670+0000 7fb46d000640 1 -- 172.21.15.1:0/1377766943 shutdown_connections 2024-05-09T08:02:51.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:51.670+0000 7fb46d000640 1 -- 172.21.15.1:0/1377766943 wait complete. 2024-05-09T08:02:51.708 INFO:teuthology.orchestra.run.smithi001.stdout:{"pg_ready":true,"pg_map":{"version":40,"stamp":"2024-05-09T08:02:51.105163+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":208,"num_write_kb":2435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160,"ondisk_log_size":160,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":240256,"kb_used_data":3304,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754734464,"statfs":{"total":773094113280,"available":772848091136,"internally_reserved":0,"allocated":3383296,"data_stored":2029432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":242535986},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":141,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003616"},"pg_stats":[{"pgid":"2.1f","version":"24'4","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:21.677957+0000","last_change":"2024-05-09T08:02:14.875108+0000","last_active":"2024-05-09T08:02:21.677957+0000","last_peered":"2024-05-09T08:02:21.677957+0000","last_clean":"2024-05-09T08:02:21.677957+0000","last_became_active":"2024-05-09T08:02:14.874841+0000","last_became_peered":"2024-05-09T08:02:14.874841+0000","last_unstale":"2024-05-09T08:02:21.677957+0000","last_undegraded":"2024-05-09T08:02:21.677957+0000","last_fullsized":"2024-05-09T08:02:21.677957+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:00:40.599475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:20.934410+0000","last_change":"2024-05-09T08:02:15.899818+0000","last_active":"2024-05-09T08:02:20.934410+0000","last_peered":"2024-05-09T08:02:20.934410+0000","last_clean":"2024-05-09T08:02:20.934410+0000","last_became_active":"2024-05-09T08:02:15.899479+0000","last_became_peered":"2024-05-09T08:02:15.899479+0000","last_unstale":"2024-05-09T08:02:20.934410+0000","last_undegraded":"2024-05-09T08:02:20.934410+0000","last_fullsized":"2024-05-09T08:02:20.934410+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:18:36.309627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:20.932807+0000","last_change":"2024-05-09T08:02:20.057944+0000","last_active":"2024-05-09T08:02:20.932807+0000","last_peered":"2024-05-09T08:02:20.932807+0000","last_clean":"2024-05-09T08:02:20.932807+0000","last_became_active":"2024-05-09T08:02:20.057016+0000","last_became_peered":"2024-05-09T08:02:20.057016+0000","last_unstale":"2024-05-09T08:02:20.932807+0000","last_undegraded":"2024-05-09T08:02:20.932807+0000","last_fullsized":"2024-05-09T08:02:20.932807+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:17:01.514168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"24'1","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:20.934743+0000","last_change":"2024-05-09T08:02:14.886137+0000","last_active":"2024-05-09T08:02:20.934743+0000","last_peered":"2024-05-09T08:02:20.934743+0000","last_clean":"2024-05-09T08:02:20.934743+0000","last_became_active":"2024-05-09T08:02:14.885830+0000","last_became_peered":"2024-05-09T08:02:14.885830+0000","last_unstale":"2024-05-09T08:02:20.934743+0000","last_undegraded":"2024-05-09T08:02:20.934743+0000","last_fullsized":"2024-05-09T08:02:20.934743+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:46:11.160815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:21.677645+0000","last_change":"2024-05-09T08:02:15.906280+0000","last_active":"2024-05-09T08:02:21.677645+0000","last_peered":"2024-05-09T08:02:21.677645+0000","last_clean":"2024-05-09T08:02:21.677645+0000","last_became_active":"2024-05-09T08:02:15.905590+0000","last_became_peered":"2024-05-09T08:02:15.905590+0000","last_unstale":"2024-05-09T08:02:21.677645+0000","last_undegraded":"2024-05-09T08:02:21.677645+0000","last_fullsized":"2024-05-09T08:02:21.677645+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:20:07.368327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.617387+0000","last_change":"2024-05-09T08:02:19.954548+0000","last_active":"2024-05-09T08:02:22.617387+0000","last_peered":"2024-05-09T08:02:22.617387+0000","last_clean":"2024-05-09T08:02:22.617387+0000","last_became_active":"2024-05-09T08:02:19.954161+0000","last_became_peered":"2024-05-09T08:02:19.954161+0000","last_unstale":"2024-05-09T08:02:22.617387+0000","last_undegraded":"2024-05-09T08:02:22.617387+0000","last_fullsized":"2024-05-09T08:02:22.617387+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:53:27.246334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"24'2","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:21.077047+0000","last_change":"2024-05-09T08:02:14.867128+0000","last_active":"2024-05-09T08:02:21.077047+0000","last_peered":"2024-05-09T08:02:21.077047+0000","last_clean":"2024-05-09T08:02:21.077047+0000","last_became_active":"2024-05-09T08:02:14.866809+0000","last_became_peered":"2024-05-09T08:02:14.866809+0000","last_unstale":"2024-05-09T08:02:21.077047+0000","last_undegraded":"2024-05-09T08:02:21.077047+0000","last_fullsized":"2024-05-09T08:02:21.077047+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:57:06.093718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.618173+0000","last_change":"2024-05-09T08:02:15.888431+0000","last_active":"2024-05-09T08:02:22.618173+0000","last_peered":"2024-05-09T08:02:22.618173+0000","last_clean":"2024-05-09T08:02:22.618173+0000","last_became_active":"2024-05-09T08:02:15.887509+0000","last_became_peered":"2024-05-09T08:02:15.887509+0000","last_unstale":"2024-05-09T08:02:22.618173+0000","last_undegraded":"2024-05-09T08:02:22.618173+0000","last_fullsized":"2024-05-09T08:02:22.618173+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:25:01.238816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.617803+0000","last_change":"2024-05-09T08:02:19.953600+0000","last_active":"2024-05-09T08:02:22.617803+0000","last_peered":"2024-05-09T08:02:22.617803+0000","last_clean":"2024-05-09T08:02:22.617803+0000","last_became_active":"2024-05-09T08:02:19.952275+0000","last_became_peered":"2024-05-09T08:02:19.952275+0000","last_unstale":"2024-05-09T08:02:22.617803+0000","last_undegraded":"2024-05-09T08:02:22.617803+0000","last_fullsized":"2024-05-09T08:02:22.617803+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:25:05.306541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"24'2","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:21.076799+0000","last_change":"2024-05-09T08:02:14.870819+0000","last_active":"2024-05-09T08:02:21.076799+0000","last_peered":"2024-05-09T08:02:21.076799+0000","last_clean":"2024-05-09T08:02:21.076799+0000","last_became_active":"2024-05-09T08:02:14.870476+0000","last_became_peered":"2024-05-09T08:02:14.870476+0000","last_unstale":"2024-05-09T08:02:21.076799+0000","last_undegraded":"2024-05-09T08:02:21.076799+0000","last_fullsized":"2024-05-09T08:02:21.076799+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:43:01.061801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.618291+0000","last_change":"2024-05-09T08:02:15.886471+0000","last_active":"2024-05-09T08:02:22.618291+0000","last_peered":"2024-05-09T08:02:22.618291+0000","last_clean":"2024-05-09T08:02:22.618291+0000","last_became_active":"2024-05-09T08:02:15.886168+0000","last_became_peered":"2024-05-09T08:02:15.886168+0000","last_unstale":"2024-05-09T08:02:22.618291+0000","last_undegraded":"2024-05-09T08:02:22.618291+0000","last_fullsized":"2024-05-09T08:02:22.618291+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:30:33.351158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.618044+0000","last_change":"2024-05-09T08:02:19.947347+0000","last_active":"2024-05-09T08:02:22.618044+0000","last_peered":"2024-05-09T08:02:22.618044+0000","last_clean":"2024-05-09T08:02:22.618044+0000","last_became_active":"2024-05-09T08:02:19.946906+0000","last_became_peered":"2024-05-09T08:02:19.946906+0000","last_unstale":"2024-05-09T08:02:22.618044+0000","last_undegraded":"2024-05-09T08:02:22.618044+0000","last_fullsized":"2024-05-09T08:02:22.618044+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:30:37.418883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.618712+0000","last_change":"2024-05-09T08:02:15.880765+0000","last_active":"2024-05-09T08:02:22.618712+0000","last_peered":"2024-05-09T08:02:22.618712+0000","last_clean":"2024-05-09T08:02:22.618712+0000","last_became_active":"2024-05-09T08:02:15.880439+0000","last_became_peered":"2024-05-09T08:02:15.880439+0000","last_unstale":"2024-05-09T08:02:22.618712+0000","last_undegraded":"2024-05-09T08:02:22.618712+0000","last_fullsized":"2024-05-09T08:02:22.618712+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:29:59.441314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"24'2","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:21.076978+0000","last_change":"2024-05-09T08:02:14.867587+0000","last_active":"2024-05-09T08:02:21.076978+0000","last_peered":"2024-05-09T08:02:21.076978+0000","last_clean":"2024-05-09T08:02:21.076978+0000","last_became_active":"2024-05-09T08:02:14.867291+0000","last_became_peered":"2024-05-09T08:02:14.867291+0000","last_unstale":"2024-05-09T08:02:21.076978+0000","last_undegraded":"2024-05-09T08:02:21.076978+0000","last_fullsized":"2024-05-09T08:02:21.076978+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:20:08.921752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.619129+0000","last_change":"2024-05-09T08:02:19.954902+0000","last_active":"2024-05-09T08:02:22.619129+0000","last_peered":"2024-05-09T08:02:22.619129+0000","last_clean":"2024-05-09T08:02:22.619129+0000","last_became_active":"2024-05-09T08:02:19.954007+0000","last_became_peered":"2024-05-09T08:02:19.954007+0000","last_unstale":"2024-05-09T08:02:22.619129+0000","last_undegraded":"2024-05-09T08:02:22.619129+0000","last_fullsized":"2024-05-09T08:02:22.619129+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:41:31.470546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:20.934123+0000","last_change":"2024-05-09T08:02:15.915155+0000","last_active":"2024-05-09T08:02:20.934123+0000","last_peered":"2024-05-09T08:02:20.934123+0000","last_clean":"2024-05-09T08:02:20.934123+0000","last_became_active":"2024-05-09T08:02:15.914852+0000","last_became_peered":"2024-05-09T08:02:15.914852+0000","last_unstale":"2024-05-09T08:02:20.934123+0000","last_undegraded":"2024-05-09T08:02:20.934123+0000","last_fullsized":"2024-05-09T08:02:20.934123+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:53:23.178610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:20.934852+0000","last_change":"2024-05-09T08:02:14.881770+0000","last_active":"2024-05-09T08:02:20.934852+0000","last_peered":"2024-05-09T08:02:20.934852+0000","last_clean":"2024-05-09T08:02:20.934852+0000","last_became_active":"2024-05-09T08:02:14.881437+0000","last_became_peered":"2024-05-09T08:02:14.881437+0000","last_unstale":"2024-05-09T08:02:20.934852+0000","last_undegraded":"2024-05-09T08:02:20.934852+0000","last_fullsized":"2024-05-09T08:02:20.934852+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:25:23.306827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.618007+0000","last_change":"2024-05-09T08:02:19.955928+0000","last_active":"2024-05-09T08:02:22.618007+0000","last_peered":"2024-05-09T08:02:22.618007+0000","last_clean":"2024-05-09T08:02:22.618007+0000","last_became_active":"2024-05-09T08:02:19.955432+0000","last_became_peered":"2024-05-09T08:02:19.955432+0000","last_unstale":"2024-05-09T08:02:22.618007+0000","last_undegraded":"2024-05-09T08:02:22.618007+0000","last_fullsized":"2024-05-09T08:02:22.618007+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:53:15.458120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:20.933351+0000","last_change":"2024-05-09T08:02:15.913311+0000","last_active":"2024-05-09T08:02:20.933351+0000","last_peered":"2024-05-09T08:02:20.933351+0000","last_clean":"2024-05-09T08:02:20.933351+0000","last_became_active":"2024-05-09T08:02:15.912484+0000","last_became_peered":"2024-05-09T08:02:15.912484+0000","last_unstale":"2024-05-09T08:02:20.933351+0000","last_undegraded":"2024-05-09T08:02:20.933351+0000","last_fullsized":"2024-05-09T08:02:20.933351+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:43:06.938904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:20.933542+0000","last_change":"2024-05-09T08:02:14.868884+0000","last_active":"2024-05-09T08:02:20.933542+0000","last_peered":"2024-05-09T08:02:20.933542+0000","last_clean":"2024-05-09T08:02:20.933542+0000","last_became_active":"2024-05-09T08:02:14.868553+0000","last_became_peered":"2024-05-09T08:02:14.868553+0000","last_unstale":"2024-05-09T08:02:20.933542+0000","last_undegraded":"2024-05-09T08:02:20.933542+0000","last_fullsized":"2024-05-09T08:02:20.933542+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:32:48.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:20.935948+0000","last_change":"2024-05-09T08:02:20.053648+0000","last_active":"2024-05-09T08:02:20.935948+0000","last_peered":"2024-05-09T08:02:20.935948+0000","last_clean":"2024-05-09T08:02:20.935948+0000","last_became_active":"2024-05-09T08:02:20.053152+0000","last_became_peered":"2024-05-09T08:02:20.053152+0000","last_unstale":"2024-05-09T08:02:20.935948+0000","last_undegraded":"2024-05-09T08:02:20.935948+0000","last_fullsized":"2024-05-09T08:02:20.935948+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:59:19.025530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.618049+0000","last_change":"2024-05-09T08:02:15.891115+0000","last_active":"2024-05-09T08:02:22.618049+0000","last_peered":"2024-05-09T08:02:22.618049+0000","last_clean":"2024-05-09T08:02:22.618049+0000","last_became_active":"2024-05-09T08:02:15.890806+0000","last_became_peered":"2024-05-09T08:02:15.890806+0000","last_unstale":"2024-05-09T08:02:22.618049+0000","last_undegraded":"2024-05-09T08:02:22.618049+0000","last_fullsized":"2024-05-09T08:02:22.618049+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:57:07.107359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:21.076482+0000","last_change":"2024-05-09T08:02:14.874987+0000","last_active":"2024-05-09T08:02:21.076482+0000","last_peered":"2024-05-09T08:02:21.076482+0000","last_clean":"2024-05-09T08:02:21.076482+0000","last_became_active":"2024-05-09T08:02:14.874549+0000","last_became_peered":"2024-05-09T08:02:14.874549+0000","last_unstale":"2024-05-09T08:02:21.076482+0000","last_undegraded":"2024-05-09T08:02:21.076482+0000","last_fullsized":"2024-05-09T08:02:21.076482+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:57:10.338510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.618392+0000","last_change":"2024-05-09T08:02:19.952790+0000","last_active":"2024-05-09T08:02:22.618392+0000","last_peered":"2024-05-09T08:02:22.618392+0000","last_clean":"2024-05-09T08:02:22.618392+0000","last_became_active":"2024-05-09T08:02:19.952378+0000","last_became_peered":"2024-05-09T08:02:19.952378+0000","last_unstale":"2024-05-09T08:02:22.618392+0000","last_undegraded":"2024-05-09T08:02:22.618392+0000","last_fullsized":"2024-05-09T08:02:22.618392+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:42:00.939587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:21.678285+0000","last_change":"2024-05-09T08:02:15.913011+0000","last_active":"2024-05-09T08:02:21.678285+0000","last_peered":"2024-05-09T08:02:21.678285+0000","last_clean":"2024-05-09T08:02:21.678285+0000","last_became_active":"2024-05-09T08:02:15.912590+0000","last_became_peered":"2024-05-09T08:02:15.912590+0000","last_unstale":"2024-05-09T08:02:21.678285+0000","last_undegraded":"2024-05-09T08:02:21.678285+0000","last_fullsized":"2024-05-09T08:02:21.678285+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:43:06.938904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"24'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.619767+0000","last_change":"2024-05-09T08:02:14.864763+0000","last_active":"2024-05-09T08:02:22.619767+0000","last_peered":"2024-05-09T08:02:22.619767+0000","last_clean":"2024-05-09T08:02:22.619767+0000","last_became_active":"2024-05-09T08:02:14.864470+0000","last_became_peered":"2024-05-09T08:02:14.864470+0000","last_unstale":"2024-05-09T08:02:22.619767+0000","last_undegraded":"2024-05-09T08:02:22.619767+0000","last_fullsized":"2024-05-09T08:02:22.619767+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:00:40.599475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.617800+0000","last_change":"2024-05-09T08:02:20.054896+0000","last_active":"2024-05-09T08:02:22.617800+0000","last_peered":"2024-05-09T08:02:22.617800+0000","last_clean":"2024-05-09T08:02:22.617800+0000","last_became_active":"2024-05-09T08:02:20.054424+0000","last_became_peered":"2024-05-09T08:02:20.054424+0000","last_unstale":"2024-05-09T08:02:22.617800+0000","last_undegraded":"2024-05-09T08:02:22.617800+0000","last_fullsized":"2024-05-09T08:02:22.617800+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:23:44.316545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:20.935333+0000","last_change":"2024-05-09T08:02:15.905448+0000","last_active":"2024-05-09T08:02:20.935333+0000","last_peered":"2024-05-09T08:02:20.935333+0000","last_clean":"2024-05-09T08:02:20.935333+0000","last_became_active":"2024-05-09T08:02:15.905131+0000","last_became_peered":"2024-05-09T08:02:15.905131+0000","last_unstale":"2024-05-09T08:02:20.935333+0000","last_undegraded":"2024-05-09T08:02:20.935333+0000","last_fullsized":"2024-05-09T08:02:20.935333+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:25:01.238816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"25'1","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:20.936562+0000","last_change":"2024-05-09T08:02:14.868410+0000","last_active":"2024-05-09T08:02:20.936562+0000","last_peered":"2024-05-09T08:02:20.936562+0000","last_clean":"2024-05-09T08:02:20.936562+0000","last_became_active":"2024-05-09T08:02:14.865621+0000","last_became_peered":"2024-05-09T08:02:14.865621+0000","last_unstale":"2024-05-09T08:02:20.936562+0000","last_undegraded":"2024-05-09T08:02:20.936562+0000","last_fullsized":"2024-05-09T08:02:20.936562+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:43:01.061801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:20.932915+0000","last_change":"2024-05-09T08:02:20.057603+0000","last_active":"2024-05-09T08:02:20.932915+0000","last_peered":"2024-05-09T08:02:20.932915+0000","last_clean":"2024-05-09T08:02:20.932915+0000","last_became_active":"2024-05-09T08:02:20.055365+0000","last_became_peered":"2024-05-09T08:02:20.055365+0000","last_unstale":"2024-05-09T08:02:20.932915+0000","last_undegraded":"2024-05-09T08:02:20.932915+0000","last_fullsized":"2024-05-09T08:02:20.932915+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:48:38.283562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:20.934935+0000","last_change":"2024-05-09T08:02:15.890980+0000","last_active":"2024-05-09T08:02:20.934935+0000","last_peered":"2024-05-09T08:02:20.934935+0000","last_clean":"2024-05-09T08:02:20.934935+0000","last_became_active":"2024-05-09T08:02:15.890346+0000","last_became_peered":"2024-05-09T08:02:15.890346+0000","last_unstale":"2024-05-09T08:02:20.934935+0000","last_undegraded":"2024-05-09T08:02:20.934935+0000","last_fullsized":"2024-05-09T08:02:20.934935+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:29:59.441314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:21.077241+0000","last_change":"2024-05-09T08:02:14.876206+0000","last_active":"2024-05-09T08:02:21.077241+0000","last_peered":"2024-05-09T08:02:21.077241+0000","last_clean":"2024-05-09T08:02:21.077241+0000","last_became_active":"2024-05-09T08:02:14.875894+0000","last_became_peered":"2024-05-09T08:02:14.875894+0000","last_unstale":"2024-05-09T08:02:21.077241+0000","last_undegraded":"2024-05-09T08:02:21.077241+0000","last_fullsized":"2024-05-09T08:02:21.077241+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:49:53.882513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:21.677241+0000","last_change":"2024-05-09T08:02:20.055222+0000","last_active":"2024-05-09T08:02:21.677241+0000","last_peered":"2024-05-09T08:02:21.677241+0000","last_clean":"2024-05-09T08:02:21.677241+0000","last_became_active":"2024-05-09T08:02:20.053802+0000","last_became_peered":"2024-05-09T08:02:20.053802+0000","last_unstale":"2024-05-09T08:02:21.677241+0000","last_undegraded":"2024-05-09T08:02:21.677241+0000","last_fullsized":"2024-05-09T08:02:21.677241+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:42:00.939587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.618451+0000","last_change":"2024-05-09T08:02:15.904528+0000","last_active":"2024-05-09T08:02:22.618451+0000","last_peered":"2024-05-09T08:02:22.618451+0000","last_clean":"2024-05-09T08:02:22.618451+0000","last_became_active":"2024-05-09T08:02:15.904049+0000","last_became_peered":"2024-05-09T08:02:15.904049+0000","last_unstale":"2024-05-09T08:02:22.618451+0000","last_undegraded":"2024-05-09T08:02:22.618451+0000","last_fullsized":"2024-05-09T08:02:22.618451+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:53:11.390395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:20.935040+0000","last_change":"2024-05-09T08:02:14.872897+0000","last_active":"2024-05-09T08:02:20.935040+0000","last_peered":"2024-05-09T08:02:20.935040+0000","last_clean":"2024-05-09T08:02:20.935040+0000","last_became_active":"2024-05-09T08:02:14.872096+0000","last_became_peered":"2024-05-09T08:02:14.872096+0000","last_unstale":"2024-05-09T08:02:20.935040+0000","last_undegraded":"2024-05-09T08:02:20.935040+0000","last_fullsized":"2024-05-09T08:02:20.935040+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:00:40.599475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:20.933696+0000","last_change":"2024-05-09T08:02:19.963716+0000","last_active":"2024-05-09T08:02:20.933696+0000","last_peered":"2024-05-09T08:02:20.933696+0000","last_clean":"2024-05-09T08:02:20.933696+0000","last_became_active":"2024-05-09T08:02:19.962922+0000","last_became_peered":"2024-05-09T08:02:19.962922+0000","last_unstale":"2024-05-09T08:02:20.933696+0000","last_undegraded":"2024-05-09T08:02:20.933696+0000","last_fullsized":"2024-05-09T08:02:20.933696+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:42:00.939587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.617918+0000","last_change":"2024-05-09T08:02:15.901450+0000","last_active":"2024-05-09T08:02:22.617918+0000","last_peered":"2024-05-09T08:02:22.617918+0000","last_clean":"2024-05-09T08:02:22.617918+0000","last_became_active":"2024-05-09T08:02:15.901180+0000","last_became_peered":"2024-05-09T08:02:15.901180+0000","last_unstale":"2024-05-09T08:02:22.617918+0000","last_undegraded":"2024-05-09T08:02:22.617918+0000","last_fullsized":"2024-05-09T08:02:22.617918+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:43:02.075442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.618076+0000","last_change":"2024-05-09T08:02:14.876518+0000","last_active":"2024-05-09T08:02:22.618076+0000","last_peered":"2024-05-09T08:02:22.618076+0000","last_clean":"2024-05-09T08:02:22.618076+0000","last_became_active":"2024-05-09T08:02:14.876175+0000","last_became_peered":"2024-05-09T08:02:14.876175+0000","last_unstale":"2024-05-09T08:02:22.618076+0000","last_undegraded":"2024-05-09T08:02:22.618076+0000","last_fullsized":"2024-05-09T08:02:22.618076+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:00:40.599475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:20.934897+0000","last_change":"2024-05-09T08:02:19.960940+0000","last_active":"2024-05-09T08:02:20.934897+0000","last_peered":"2024-05-09T08:02:20.934897+0000","last_clean":"2024-05-09T08:02:20.934897+0000","last_became_active":"2024-05-09T08:02:19.960496+0000","last_became_peered":"2024-05-09T08:02:19.960496+0000","last_unstale":"2024-05-09T08:02:20.934897+0000","last_undegraded":"2024-05-09T08:02:20.934897+0000","last_fullsized":"2024-05-09T08:02:20.934897+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:52:40.529638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:20.932615+0000","last_change":"2024-05-09T08:02:15.891774+0000","last_active":"2024-05-09T08:02:20.932615+0000","last_peered":"2024-05-09T08:02:20.932615+0000","last_clean":"2024-05-09T08:02:20.932615+0000","last_became_active":"2024-05-09T08:02:15.891407+0000","last_became_peered":"2024-05-09T08:02:15.891407+0000","last_unstale":"2024-05-09T08:02:20.932615+0000","last_undegraded":"2024-05-09T08:02:20.932615+0000","last_fullsized":"2024-05-09T08:02:20.932615+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:05:02.181367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.618768+0000","last_change":"2024-05-09T08:02:14.883655+0000","last_active":"2024-05-09T08:02:22.618768+0000","last_peered":"2024-05-09T08:02:22.618768+0000","last_clean":"2024-05-09T08:02:22.618768+0000","last_became_active":"2024-05-09T08:02:14.882359+0000","last_became_peered":"2024-05-09T08:02:14.882359+0000","last_unstale":"2024-05-09T08:02:22.618768+0000","last_undegraded":"2024-05-09T08:02:22.618768+0000","last_fullsized":"2024-05-09T08:02:22.618768+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:46:11.160815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:20.936081+0000","last_change":"2024-05-09T08:02:19.953319+0000","last_active":"2024-05-09T08:02:20.936081+0000","last_peered":"2024-05-09T08:02:20.936081+0000","last_clean":"2024-05-09T08:02:20.936081+0000","last_became_active":"2024-05-09T08:02:19.952740+0000","last_became_peered":"2024-05-09T08:02:19.952740+0000","last_unstale":"2024-05-09T08:02:20.936081+0000","last_undegraded":"2024-05-09T08:02:20.936081+0000","last_fullsized":"2024-05-09T08:02:20.936081+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:25:28.804048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:21.676635+0000","last_change":"2024-05-09T08:02:15.913828+0000","last_active":"2024-05-09T08:02:21.676635+0000","last_peered":"2024-05-09T08:02:21.676635+0000","last_clean":"2024-05-09T08:02:21.676635+0000","last_became_active":"2024-05-09T08:02:15.913313+0000","last_became_peered":"2024-05-09T08:02:15.913313+0000","last_unstale":"2024-05-09T08:02:21.676635+0000","last_undegraded":"2024-05-09T08:02:21.676635+0000","last_fullsized":"2024-05-09T08:02:21.676635+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:20:09.935394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:21.076590+0000","last_change":"2024-05-09T08:02:14.876438+0000","last_active":"2024-05-09T08:02:21.076590+0000","last_peered":"2024-05-09T08:02:21.076590+0000","last_clean":"2024-05-09T08:02:21.076590+0000","last_became_active":"2024-05-09T08:02:14.876146+0000","last_became_peered":"2024-05-09T08:02:14.876146+0000","last_unstale":"2024-05-09T08:02:21.076590+0000","last_undegraded":"2024-05-09T08:02:21.076590+0000","last_fullsized":"2024-05-09T08:02:21.076590+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:36:49.629991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:21.678130+0000","last_change":"2024-05-09T08:02:20.051067+0000","last_active":"2024-05-09T08:02:21.678130+0000","last_peered":"2024-05-09T08:02:21.678130+0000","last_clean":"2024-05-09T08:02:21.678130+0000","last_became_active":"2024-05-09T08:02:20.050593+0000","last_became_peered":"2024-05-09T08:02:20.050593+0000","last_unstale":"2024-05-09T08:02:21.678130+0000","last_undegraded":"2024-05-09T08:02:21.678130+0000","last_fullsized":"2024-05-09T08:02:21.678130+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:25:28.804048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:20.933046+0000","last_change":"2024-05-09T08:02:15.899137+0000","last_active":"2024-05-09T08:02:20.933046+0000","last_peered":"2024-05-09T08:02:20.933046+0000","last_clean":"2024-05-09T08:02:20.933046+0000","last_became_active":"2024-05-09T08:02:15.898766+0000","last_became_peered":"2024-05-09T08:02:15.898766+0000","last_unstale":"2024-05-09T08:02:20.933046+0000","last_undegraded":"2024-05-09T08:02:20.933046+0000","last_fullsized":"2024-05-09T08:02:20.933046+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:47:32.605288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.617934+0000","last_change":"2024-05-09T08:02:14.868850+0000","last_active":"2024-05-09T08:02:22.617934+0000","last_peered":"2024-05-09T08:02:22.617934+0000","last_clean":"2024-05-09T08:02:22.617934+0000","last_became_active":"2024-05-09T08:02:14.868384+0000","last_became_peered":"2024-05-09T08:02:14.868384+0000","last_unstale":"2024-05-09T08:02:22.617934+0000","last_undegraded":"2024-05-09T08:02:22.617934+0000","last_fullsized":"2024-05-09T08:02:22.617934+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:14:52.920972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":177,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:21.076748+0000","last_change":"2024-05-09T08:02:11.559804+0000","last_active":"2024-05-09T08:02:21.076748+0000","last_peered":"2024-05-09T08:02:21.076748+0000","last_clean":"2024-05-09T08:02:21.076748+0000","last_became_active":"2024-05-09T08:02:11.558761+0000","last_became_peered":"2024-05-09T08:02:11.558761+0000","last_unstale":"2024-05-09T08:02:21.076748+0000","last_undegraded":"2024-05-09T08:02:21.076748+0000","last_fullsized":"2024-05-09T08:02:21.076748+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:09.800143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:09.800143+0000","last_clean_scrub_stamp":"2024-05-09T08:02:09.800143+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:18:35.964149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.618755+0000","last_change":"2024-05-09T08:02:20.054746+0000","last_active":"2024-05-09T08:02:22.618755+0000","last_peered":"2024-05-09T08:02:22.618755+0000","last_clean":"2024-05-09T08:02:22.618755+0000","last_became_active":"2024-05-09T08:02:20.054186+0000","last_became_peered":"2024-05-09T08:02:20.054186+0000","last_unstale":"2024-05-09T08:02:22.618755+0000","last_undegraded":"2024-05-09T08:02:22.618755+0000","last_fullsized":"2024-05-09T08:02:22.618755+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:53:15.458120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:21.076861+0000","last_change":"2024-05-09T08:02:15.889657+0000","last_active":"2024-05-09T08:02:21.076861+0000","last_peered":"2024-05-09T08:02:21.076861+0000","last_clean":"2024-05-09T08:02:21.076861+0000","last_became_active":"2024-05-09T08:02:15.889381+0000","last_became_peered":"2024-05-09T08:02:15.889381+0000","last_unstale":"2024-05-09T08:02:21.076861+0000","last_undegraded":"2024-05-09T08:02:21.076861+0000","last_fullsized":"2024-05-09T08:02:21.076861+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:59:14.957806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.619305+0000","last_change":"2024-05-09T08:02:14.868892+0000","last_active":"2024-05-09T08:02:22.619305+0000","last_peered":"2024-05-09T08:02:22.619305+0000","last_clean":"2024-05-09T08:02:22.619305+0000","last_became_active":"2024-05-09T08:02:14.868513+0000","last_became_peered":"2024-05-09T08:02:14.868513+0000","last_unstale":"2024-05-09T08:02:22.619305+0000","last_undegraded":"2024-05-09T08:02:22.619305+0000","last_fullsized":"2024-05-09T08:02:22.619305+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:19:31.913236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.618654+0000","last_change":"2024-05-09T08:02:20.053097+0000","last_active":"2024-05-09T08:02:22.618654+0000","last_peered":"2024-05-09T08:02:22.618654+0000","last_clean":"2024-05-09T08:02:22.618654+0000","last_became_active":"2024-05-09T08:02:20.052653+0000","last_became_peered":"2024-05-09T08:02:20.052653+0000","last_unstale":"2024-05-09T08:02:22.618654+0000","last_undegraded":"2024-05-09T08:02:22.618654+0000","last_fullsized":"2024-05-09T08:02:22.618654+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:25:05.306541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.618299+0000","last_change":"2024-05-09T08:02:15.899059+0000","last_active":"2024-05-09T08:02:22.618299+0000","last_peered":"2024-05-09T08:02:22.618299+0000","last_clean":"2024-05-09T08:02:22.618299+0000","last_became_active":"2024-05-09T08:02:15.898692+0000","last_became_peered":"2024-05-09T08:02:15.898692+0000","last_unstale":"2024-05-09T08:02:22.618299+0000","last_undegraded":"2024-05-09T08:02:22.618299+0000","last_fullsized":"2024-05-09T08:02:22.618299+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:52:36.461913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"24'1","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:20.935331+0000","last_change":"2024-05-09T08:02:14.873071+0000","last_active":"2024-05-09T08:02:20.935331+0000","last_peered":"2024-05-09T08:02:20.935331+0000","last_clean":"2024-05-09T08:02:20.935331+0000","last_became_active":"2024-05-09T08:02:14.872556+0000","last_became_peered":"2024-05-09T08:02:14.872556+0000","last_unstale":"2024-05-09T08:02:20.935331+0000","last_undegraded":"2024-05-09T08:02:20.935331+0000","last_fullsized":"2024-05-09T08:02:20.935331+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:57:06.093718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:21.677388+0000","last_change":"2024-05-09T08:02:19.960414+0000","last_active":"2024-05-09T08:02:21.677388+0000","last_peered":"2024-05-09T08:02:21.677388+0000","last_clean":"2024-05-09T08:02:21.677388+0000","last_became_active":"2024-05-09T08:02:19.957941+0000","last_became_peered":"2024-05-09T08:02:19.957941+0000","last_unstale":"2024-05-09T08:02:21.677388+0000","last_undegraded":"2024-05-09T08:02:21.677388+0000","last_fullsized":"2024-05-09T08:02:21.677388+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:41:31.470546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:21.077406+0000","last_change":"2024-05-09T08:02:15.884395+0000","last_active":"2024-05-09T08:02:21.077406+0000","last_peered":"2024-05-09T08:02:21.077406+0000","last_clean":"2024-05-09T08:02:21.077406+0000","last_became_active":"2024-05-09T08:02:15.883788+0000","last_became_peered":"2024-05-09T08:02:15.883788+0000","last_unstale":"2024-05-09T08:02:21.077406+0000","last_undegraded":"2024-05-09T08:02:21.077406+0000","last_fullsized":"2024-05-09T08:02:21.077406+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:48:34.215837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"24'2","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.619345+0000","last_change":"2024-05-09T08:02:14.885044+0000","last_active":"2024-05-09T08:02:22.619345+0000","last_peered":"2024-05-09T08:02:22.619345+0000","last_clean":"2024-05-09T08:02:22.619345+0000","last_became_active":"2024-05-09T08:02:14.884706+0000","last_became_peered":"2024-05-09T08:02:14.884706+0000","last_unstale":"2024-05-09T08:02:22.619345+0000","last_undegraded":"2024-05-09T08:02:22.619345+0000","last_fullsized":"2024-05-09T08:02:22.619345+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:14:52.920972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.617383+0000","last_change":"2024-05-09T08:02:20.050865+0000","last_active":"2024-05-09T08:02:22.617383+0000","last_peered":"2024-05-09T08:02:22.617383+0000","last_clean":"2024-05-09T08:02:22.617383+0000","last_became_active":"2024-05-09T08:02:20.050270+0000","last_became_peered":"2024-05-09T08:02:20.050270+0000","last_unstale":"2024-05-09T08:02:22.617383+0000","last_undegraded":"2024-05-09T08:02:22.617383+0000","last_fullsized":"2024-05-09T08:02:22.617383+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:48:38.283562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:21.077190+0000","last_change":"2024-05-09T08:02:15.884737+0000","last_active":"2024-05-09T08:02:21.077190+0000","last_peered":"2024-05-09T08:02:21.077190+0000","last_clean":"2024-05-09T08:02:21.077190+0000","last_became_active":"2024-05-09T08:02:15.884465+0000","last_became_peered":"2024-05-09T08:02:15.884465+0000","last_unstale":"2024-05-09T08:02:21.077190+0000","last_undegraded":"2024-05-09T08:02:21.077190+0000","last_fullsized":"2024-05-09T08:02:21.077190+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:52:36.461913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"24'1","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.618658+0000","last_change":"2024-05-09T08:02:14.875003+0000","last_active":"2024-05-09T08:02:22.618658+0000","last_peered":"2024-05-09T08:02:22.618658+0000","last_clean":"2024-05-09T08:02:22.618658+0000","last_became_active":"2024-05-09T08:02:14.874665+0000","last_became_peered":"2024-05-09T08:02:14.874665+0000","last_unstale":"2024-05-09T08:02:22.618658+0000","last_undegraded":"2024-05-09T08:02:22.618658+0000","last_fullsized":"2024-05-09T08:02:22.618658+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:18:39.991319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.617254+0000","last_change":"2024-05-09T08:02:19.952692+0000","last_active":"2024-05-09T08:02:22.617254+0000","last_peered":"2024-05-09T08:02:22.617254+0000","last_clean":"2024-05-09T08:02:22.617254+0000","last_became_active":"2024-05-09T08:02:19.952224+0000","last_became_peered":"2024-05-09T08:02:19.952224+0000","last_unstale":"2024-05-09T08:02:22.617254+0000","last_undegraded":"2024-05-09T08:02:22.617254+0000","last_fullsized":"2024-05-09T08:02:22.617254+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:47:36.673013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:21.076666+0000","last_change":"2024-05-09T08:02:15.883408+0000","last_active":"2024-05-09T08:02:21.076666+0000","last_peered":"2024-05-09T08:02:21.076666+0000","last_clean":"2024-05-09T08:02:21.076666+0000","last_became_active":"2024-05-09T08:02:15.883103+0000","last_became_peered":"2024-05-09T08:02:15.883103+0000","last_unstale":"2024-05-09T08:02:21.076666+0000","last_undegraded":"2024-05-09T08:02:21.076666+0000","last_fullsized":"2024-05-09T08:02:21.076666+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:53:11.390395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"24'8","reported_seq":29,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.618529+0000","last_change":"2024-05-09T08:02:14.873184+0000","last_active":"2024-05-09T08:02:22.618529+0000","last_peered":"2024-05-09T08:02:22.618529+0000","last_clean":"2024-05-09T08:02:22.618529+0000","last_became_active":"2024-05-09T08:02:14.871055+0000","last_became_peered":"2024-05-09T08:02:14.871055+0000","last_unstale":"2024-05-09T08:02:22.618529+0000","last_undegraded":"2024-05-09T08:02:22.618529+0000","last_fullsized":"2024-05-09T08:02:22.618529+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:19:56.315972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:21.678092+0000","last_change":"2024-05-09T08:02:19.950823+0000","last_active":"2024-05-09T08:02:21.678092+0000","last_peered":"2024-05-09T08:02:21.678092+0000","last_clean":"2024-05-09T08:02:21.678092+0000","last_became_active":"2024-05-09T08:02:19.950352+0000","last_became_peered":"2024-05-09T08:02:19.950352+0000","last_unstale":"2024-05-09T08:02:21.678092+0000","last_undegraded":"2024-05-09T08:02:21.678092+0000","last_fullsized":"2024-05-09T08:02:21.678092+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:59:19.025530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.619335+0000","last_change":"2024-05-09T08:02:15.899115+0000","last_active":"2024-05-09T08:02:22.619335+0000","last_peered":"2024-05-09T08:02:22.619335+0000","last_clean":"2024-05-09T08:02:22.619335+0000","last_became_active":"2024-05-09T08:02:15.898811+0000","last_became_peered":"2024-05-09T08:02:15.898811+0000","last_unstale":"2024-05-09T08:02:22.619335+0000","last_undegraded":"2024-05-09T08:02:22.619335+0000","last_fullsized":"2024-05-09T08:02:22.619335+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:43:06.938904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.619023+0000","last_change":"2024-05-09T08:02:14.860818+0000","last_active":"2024-05-09T08:02:22.619023+0000","last_peered":"2024-05-09T08:02:22.619023+0000","last_clean":"2024-05-09T08:02:22.619023+0000","last_became_active":"2024-05-09T08:02:14.860518+0000","last_became_peered":"2024-05-09T08:02:14.860518+0000","last_unstale":"2024-05-09T08:02:22.619023+0000","last_undegraded":"2024-05-09T08:02:22.619023+0000","last_fullsized":"2024-05-09T08:02:22.619023+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:25:23.306827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:20.935214+0000","last_change":"2024-05-09T08:02:20.053251+0000","last_active":"2024-05-09T08:02:20.935214+0000","last_peered":"2024-05-09T08:02:20.935214+0000","last_clean":"2024-05-09T08:02:20.935214+0000","last_became_active":"2024-05-09T08:02:20.052783+0000","last_became_peered":"2024-05-09T08:02:20.052783+0000","last_unstale":"2024-05-09T08:02:20.935214+0000","last_undegraded":"2024-05-09T08:02:20.935214+0000","last_fullsized":"2024-05-09T08:02:20.935214+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:16:41.863341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:21.076854+0000","last_change":"2024-05-09T08:02:15.883972+0000","last_active":"2024-05-09T08:02:21.076854+0000","last_peered":"2024-05-09T08:02:21.076854+0000","last_clean":"2024-05-09T08:02:21.076854+0000","last_became_active":"2024-05-09T08:02:15.883613+0000","last_became_peered":"2024-05-09T08:02:15.883613+0000","last_unstale":"2024-05-09T08:02:21.076854+0000","last_undegraded":"2024-05-09T08:02:21.076854+0000","last_fullsized":"2024-05-09T08:02:21.076854+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:41:27.402821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.618038+0000","last_change":"2024-05-09T08:02:14.883366+0000","last_active":"2024-05-09T08:02:22.618038+0000","last_peered":"2024-05-09T08:02:22.618038+0000","last_clean":"2024-05-09T08:02:22.618038+0000","last_became_active":"2024-05-09T08:02:14.883135+0000","last_became_peered":"2024-05-09T08:02:14.883135+0000","last_unstale":"2024-05-09T08:02:22.618038+0000","last_undegraded":"2024-05-09T08:02:22.618038+0000","last_fullsized":"2024-05-09T08:02:22.618038+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:32:48.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:20.935375+0000","last_change":"2024-05-09T08:02:20.050888+0000","last_active":"2024-05-09T08:02:20.935375+0000","last_peered":"2024-05-09T08:02:20.935375+0000","last_clean":"2024-05-09T08:02:20.935375+0000","last_became_active":"2024-05-09T08:02:20.050390+0000","last_became_peered":"2024-05-09T08:02:20.050390+0000","last_unstale":"2024-05-09T08:02:20.935375+0000","last_undegraded":"2024-05-09T08:02:20.935375+0000","last_fullsized":"2024-05-09T08:02:20.935375+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:10:24.797526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:21.677358+0000","last_change":"2024-05-09T08:02:15.899096+0000","last_active":"2024-05-09T08:02:21.677358+0000","last_peered":"2024-05-09T08:02:21.677358+0000","last_clean":"2024-05-09T08:02:21.677358+0000","last_became_active":"2024-05-09T08:02:15.898811+0000","last_became_peered":"2024-05-09T08:02:15.898811+0000","last_unstale":"2024-05-09T08:02:21.677358+0000","last_undegraded":"2024-05-09T08:02:21.677358+0000","last_fullsized":"2024-05-09T08:02:21.677358+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:28:51.229262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"24'2","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:21.677657+0000","last_change":"2024-05-09T08:02:14.868657+0000","last_active":"2024-05-09T08:02:21.677657+0000","last_peered":"2024-05-09T08:02:21.677657+0000","last_clean":"2024-05-09T08:02:21.677657+0000","last_became_active":"2024-05-09T08:02:14.868385+0000","last_became_peered":"2024-05-09T08:02:14.868385+0000","last_unstale":"2024-05-09T08:02:21.677657+0000","last_undegraded":"2024-05-09T08:02:21.677657+0000","last_fullsized":"2024-05-09T08:02:21.677657+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:46:11.160815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.618792+0000","last_change":"2024-05-09T08:02:20.053155+0000","last_active":"2024-05-09T08:02:22.618792+0000","last_peered":"2024-05-09T08:02:22.618792+0000","last_clean":"2024-05-09T08:02:22.618792+0000","last_became_active":"2024-05-09T08:02:20.052629+0000","last_became_peered":"2024-05-09T08:02:20.052629+0000","last_unstale":"2024-05-09T08:02:22.618792+0000","last_undegraded":"2024-05-09T08:02:22.618792+0000","last_fullsized":"2024-05-09T08:02:22.618792+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:35:54.628874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:21.076864+0000","last_change":"2024-05-09T08:02:15.905406+0000","last_active":"2024-05-09T08:02:21.076864+0000","last_peered":"2024-05-09T08:02:21.076864+0000","last_clean":"2024-05-09T08:02:21.076864+0000","last_became_active":"2024-05-09T08:02:15.905076+0000","last_became_peered":"2024-05-09T08:02:15.905076+0000","last_unstale":"2024-05-09T08:02:21.076864+0000","last_undegraded":"2024-05-09T08:02:21.076864+0000","last_fullsized":"2024-05-09T08:02:21.076864+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:25:01.238816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"25'10","reported_seq":32,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:39.803593+0000","last_change":"2024-05-09T08:02:14.880410+0000","last_active":"2024-05-09T08:02:39.803593+0000","last_peered":"2024-05-09T08:02:39.803593+0000","last_clean":"2024-05-09T08:02:39.803593+0000","last_became_active":"2024-05-09T08:02:14.880045+0000","last_became_peered":"2024-05-09T08:02:14.880045+0000","last_unstale":"2024-05-09T08:02:39.803593+0000","last_undegraded":"2024-05-09T08:02:39.803593+0000","last_fullsized":"2024-05-09T08:02:39.803593+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:19:56.315972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:20.934006+0000","last_change":"2024-05-09T08:02:19.963378+0000","last_active":"2024-05-09T08:02:20.934006+0000","last_peered":"2024-05-09T08:02:20.934006+0000","last_clean":"2024-05-09T08:02:20.934006+0000","last_became_active":"2024-05-09T08:02:19.958964+0000","last_became_peered":"2024-05-09T08:02:19.958964+0000","last_unstale":"2024-05-09T08:02:20.934006+0000","last_undegraded":"2024-05-09T08:02:20.934006+0000","last_fullsized":"2024-05-09T08:02:20.934006+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:41:31.470546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.618268+0000","last_change":"2024-05-09T08:02:15.888428+0000","last_active":"2024-05-09T08:02:22.618268+0000","last_peered":"2024-05-09T08:02:22.618268+0000","last_clean":"2024-05-09T08:02:22.618268+0000","last_became_active":"2024-05-09T08:02:15.888071+0000","last_became_peered":"2024-05-09T08:02:15.888071+0000","last_unstale":"2024-05-09T08:02:22.618268+0000","last_undegraded":"2024-05-09T08:02:22.618268+0000","last_fullsized":"2024-05-09T08:02:22.618268+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:36:50.643632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"24'3","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:20.935171+0000","last_change":"2024-05-09T08:02:14.872697+0000","last_active":"2024-05-09T08:02:20.935171+0000","last_peered":"2024-05-09T08:02:20.935171+0000","last_clean":"2024-05-09T08:02:20.935171+0000","last_became_active":"2024-05-09T08:02:14.871945+0000","last_became_peered":"2024-05-09T08:02:14.871945+0000","last_unstale":"2024-05-09T08:02:20.935171+0000","last_undegraded":"2024-05-09T08:02:20.935171+0000","last_fullsized":"2024-05-09T08:02:20.935171+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:19:56.315972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.617683+0000","last_change":"2024-05-09T08:02:19.953635+0000","last_active":"2024-05-09T08:02:22.617683+0000","last_peered":"2024-05-09T08:02:22.617683+0000","last_clean":"2024-05-09T08:02:22.617683+0000","last_became_active":"2024-05-09T08:02:19.953166+0000","last_became_peered":"2024-05-09T08:02:19.953166+0000","last_unstale":"2024-05-09T08:02:22.617683+0000","last_undegraded":"2024-05-09T08:02:22.617683+0000","last_fullsized":"2024-05-09T08:02:22.617683+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:18:40.377352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:21.677885+0000","last_change":"2024-05-09T08:02:15.905285+0000","last_active":"2024-05-09T08:02:21.677885+0000","last_peered":"2024-05-09T08:02:21.677885+0000","last_clean":"2024-05-09T08:02:21.677885+0000","last_became_active":"2024-05-09T08:02:15.904898+0000","last_became_peered":"2024-05-09T08:02:15.904898+0000","last_unstale":"2024-05-09T08:02:21.677885+0000","last_undegraded":"2024-05-09T08:02:21.677885+0000","last_fullsized":"2024-05-09T08:02:21.677885+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:47:32.605288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.619004+0000","last_change":"2024-05-09T08:02:14.868682+0000","last_active":"2024-05-09T08:02:22.619004+0000","last_peered":"2024-05-09T08:02:22.619004+0000","last_clean":"2024-05-09T08:02:22.619004+0000","last_became_active":"2024-05-09T08:02:14.868294+0000","last_became_peered":"2024-05-09T08:02:14.868294+0000","last_unstale":"2024-05-09T08:02:22.619004+0000","last_undegraded":"2024-05-09T08:02:22.619004+0000","last_fullsized":"2024-05-09T08:02:22.619004+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:57:06.093718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:20.933284+0000","last_change":"2024-05-09T08:02:19.956115+0000","last_active":"2024-05-09T08:02:20.933284+0000","last_peered":"2024-05-09T08:02:20.933284+0000","last_clean":"2024-05-09T08:02:20.933284+0000","last_became_active":"2024-05-09T08:02:19.955662+0000","last_became_peered":"2024-05-09T08:02:19.955662+0000","last_unstale":"2024-05-09T08:02:20.933284+0000","last_undegraded":"2024-05-09T08:02:20.933284+0000","last_fullsized":"2024-05-09T08:02:20.933284+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:15:28.286330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.618890+0000","last_change":"2024-05-09T08:02:15.888406+0000","last_active":"2024-05-09T08:02:22.618890+0000","last_peered":"2024-05-09T08:02:22.618890+0000","last_clean":"2024-05-09T08:02:22.618890+0000","last_became_active":"2024-05-09T08:02:15.888090+0000","last_became_peered":"2024-05-09T08:02:15.888090+0000","last_unstale":"2024-05-09T08:02:22.618890+0000","last_undegraded":"2024-05-09T08:02:22.618890+0000","last_fullsized":"2024-05-09T08:02:22.618890+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:53:23.178610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.619656+0000","last_change":"2024-05-09T08:02:14.862463+0000","last_active":"2024-05-09T08:02:22.619656+0000","last_peered":"2024-05-09T08:02:22.619656+0000","last_clean":"2024-05-09T08:02:22.619656+0000","last_became_active":"2024-05-09T08:02:14.862104+0000","last_became_peered":"2024-05-09T08:02:14.862104+0000","last_unstale":"2024-05-09T08:02:22.619656+0000","last_undegraded":"2024-05-09T08:02:22.619656+0000","last_fullsized":"2024-05-09T08:02:22.619656+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:32:48.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:20.932952+0000","last_change":"2024-05-09T08:02:20.053600+0000","last_active":"2024-05-09T08:02:20.932952+0000","last_peered":"2024-05-09T08:02:20.932952+0000","last_clean":"2024-05-09T08:02:20.932952+0000","last_became_active":"2024-05-09T08:02:20.052485+0000","last_became_peered":"2024-05-09T08:02:20.052485+0000","last_unstale":"2024-05-09T08:02:20.932952+0000","last_undegraded":"2024-05-09T08:02:20.932952+0000","last_fullsized":"2024-05-09T08:02:20.932952+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:35:54.628874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:21.677902+0000","last_change":"2024-05-09T08:02:20.054016+0000","last_active":"2024-05-09T08:02:21.677902+0000","last_peered":"2024-05-09T08:02:21.677902+0000","last_clean":"2024-05-09T08:02:21.677902+0000","last_became_active":"2024-05-09T08:02:20.053250+0000","last_became_peered":"2024-05-09T08:02:20.053250+0000","last_unstale":"2024-05-09T08:02:21.677902+0000","last_undegraded":"2024-05-09T08:02:21.677902+0000","last_fullsized":"2024-05-09T08:02:21.677902+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:52:40.529638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:20.932409+0000","last_change":"2024-05-09T08:02:15.907080+0000","last_active":"2024-05-09T08:02:20.932409+0000","last_peered":"2024-05-09T08:02:20.932409+0000","last_clean":"2024-05-09T08:02:20.932409+0000","last_became_active":"2024-05-09T08:02:15.906600+0000","last_became_peered":"2024-05-09T08:02:15.906600+0000","last_unstale":"2024-05-09T08:02:20.932409+0000","last_undegraded":"2024-05-09T08:02:20.932409+0000","last_fullsized":"2024-05-09T08:02:20.932409+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:43:06.938904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.617668+0000","last_change":"2024-05-09T08:02:14.870990+0000","last_active":"2024-05-09T08:02:22.617668+0000","last_peered":"2024-05-09T08:02:22.617668+0000","last_clean":"2024-05-09T08:02:22.617668+0000","last_became_active":"2024-05-09T08:02:14.869818+0000","last_became_peered":"2024-05-09T08:02:14.869818+0000","last_unstale":"2024-05-09T08:02:22.617668+0000","last_undegraded":"2024-05-09T08:02:22.617668+0000","last_fullsized":"2024-05-09T08:02:22.617668+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:25:23.306827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.619498+0000","last_change":"2024-05-09T08:02:19.952582+0000","last_active":"2024-05-09T08:02:22.619498+0000","last_peered":"2024-05-09T08:02:22.619498+0000","last_clean":"2024-05-09T08:02:22.619498+0000","last_became_active":"2024-05-09T08:02:19.952117+0000","last_became_peered":"2024-05-09T08:02:19.952117+0000","last_unstale":"2024-05-09T08:02:22.619498+0000","last_undegraded":"2024-05-09T08:02:22.619498+0000","last_fullsized":"2024-05-09T08:02:22.619498+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:42:00.939587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:20.933300+0000","last_change":"2024-05-09T08:02:15.913109+0000","last_active":"2024-05-09T08:02:20.933300+0000","last_peered":"2024-05-09T08:02:20.933300+0000","last_clean":"2024-05-09T08:02:20.933300+0000","last_became_active":"2024-05-09T08:02:15.912883+0000","last_became_peered":"2024-05-09T08:02:15.912883+0000","last_unstale":"2024-05-09T08:02:20.933300+0000","last_undegraded":"2024-05-09T08:02:20.933300+0000","last_fullsized":"2024-05-09T08:02:20.933300+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:29:59.441314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:20.933233+0000","last_change":"2024-05-09T08:02:14.885256+0000","last_active":"2024-05-09T08:02:20.933233+0000","last_peered":"2024-05-09T08:02:20.933233+0000","last_clean":"2024-05-09T08:02:20.933233+0000","last_became_active":"2024-05-09T08:02:14.884965+0000","last_became_peered":"2024-05-09T08:02:14.884965+0000","last_unstale":"2024-05-09T08:02:20.933233+0000","last_undegraded":"2024-05-09T08:02:20.933233+0000","last_fullsized":"2024-05-09T08:02:20.933233+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:14:52.920972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"24'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.618840+0000","last_change":"2024-05-09T08:02:14.860339+0000","last_active":"2024-05-09T08:02:22.618840+0000","last_peered":"2024-05-09T08:02:22.618840+0000","last_clean":"2024-05-09T08:02:22.618840+0000","last_became_active":"2024-05-09T08:02:14.859917+0000","last_became_peered":"2024-05-09T08:02:14.859917+0000","last_unstale":"2024-05-09T08:02:22.618840+0000","last_undegraded":"2024-05-09T08:02:22.618840+0000","last_fullsized":"2024-05-09T08:02:22.618840+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:14:52.920972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:21.677460+0000","last_change":"2024-05-09T08:02:15.897001+0000","last_active":"2024-05-09T08:02:21.677460+0000","last_peered":"2024-05-09T08:02:21.677460+0000","last_clean":"2024-05-09T08:02:21.677460+0000","last_became_active":"2024-05-09T08:02:15.896708+0000","last_became_peered":"2024-05-09T08:02:15.896708+0000","last_unstale":"2024-05-09T08:02:21.677460+0000","last_undegraded":"2024-05-09T08:02:21.677460+0000","last_fullsized":"2024-05-09T08:02:21.677460+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:29:59.441314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.619069+0000","last_change":"2024-05-09T08:02:19.955369+0000","last_active":"2024-05-09T08:02:22.619069+0000","last_peered":"2024-05-09T08:02:22.619069+0000","last_clean":"2024-05-09T08:02:22.619069+0000","last_became_active":"2024-05-09T08:02:19.954829+0000","last_became_peered":"2024-05-09T08:02:19.954829+0000","last_unstale":"2024-05-09T08:02:22.619069+0000","last_undegraded":"2024-05-09T08:02:22.619069+0000","last_fullsized":"2024-05-09T08:02:22.619069+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:49:58.963879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"24'1","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:20.934324+0000","last_change":"2024-05-09T08:02:14.883856+0000","last_active":"2024-05-09T08:02:20.934324+0000","last_peered":"2024-05-09T08:02:20.934324+0000","last_clean":"2024-05-09T08:02:20.934324+0000","last_became_active":"2024-05-09T08:02:14.882610+0000","last_became_peered":"2024-05-09T08:02:14.882610+0000","last_unstale":"2024-05-09T08:02:20.934324+0000","last_undegraded":"2024-05-09T08:02:20.934324+0000","last_fullsized":"2024-05-09T08:02:20.934324+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:19:31.913236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.617629+0000","last_change":"2024-05-09T08:02:15.889974+0000","last_active":"2024-05-09T08:02:22.617629+0000","last_peered":"2024-05-09T08:02:22.617629+0000","last_clean":"2024-05-09T08:02:22.617629+0000","last_became_active":"2024-05-09T08:02:15.889650+0000","last_became_peered":"2024-05-09T08:02:15.889650+0000","last_unstale":"2024-05-09T08:02:22.617629+0000","last_undegraded":"2024-05-09T08:02:22.617629+0000","last_fullsized":"2024-05-09T08:02:22.617629+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:00:41.613116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:20.934556+0000","last_change":"2024-05-09T08:02:19.955362+0000","last_active":"2024-05-09T08:02:20.934556+0000","last_peered":"2024-05-09T08:02:20.934556+0000","last_clean":"2024-05-09T08:02:20.934556+0000","last_became_active":"2024-05-09T08:02:19.954947+0000","last_became_peered":"2024-05-09T08:02:19.954947+0000","last_unstale":"2024-05-09T08:02:20.934556+0000","last_undegraded":"2024-05-09T08:02:20.934556+0000","last_fullsized":"2024-05-09T08:02:20.934556+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:43:30.549966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.618982+0000","last_change":"2024-05-09T08:02:15.909379+0000","last_active":"2024-05-09T08:02:22.618982+0000","last_peered":"2024-05-09T08:02:22.618982+0000","last_clean":"2024-05-09T08:02:22.618982+0000","last_became_active":"2024-05-09T08:02:15.909086+0000","last_became_peered":"2024-05-09T08:02:15.909086+0000","last_unstale":"2024-05-09T08:02:22.618982+0000","last_undegraded":"2024-05-09T08:02:22.618982+0000","last_fullsized":"2024-05-09T08:02:22.618982+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:05:02.181367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.618963+0000","last_change":"2024-05-09T08:02:15.891242+0000","last_active":"2024-05-09T08:02:22.618963+0000","last_peered":"2024-05-09T08:02:22.618963+0000","last_clean":"2024-05-09T08:02:22.618963+0000","last_became_active":"2024-05-09T08:02:15.890640+0000","last_became_peered":"2024-05-09T08:02:15.890640+0000","last_unstale":"2024-05-09T08:02:22.618963+0000","last_undegraded":"2024-05-09T08:02:22.618963+0000","last_fullsized":"2024-05-09T08:02:22.618963+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:57:07.107359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.617950+0000","last_change":"2024-05-09T08:02:15.899001+0000","last_active":"2024-05-09T08:02:22.617950+0000","last_peered":"2024-05-09T08:02:22.617950+0000","last_clean":"2024-05-09T08:02:22.617950+0000","last_became_active":"2024-05-09T08:02:15.898695+0000","last_became_peered":"2024-05-09T08:02:15.898695+0000","last_unstale":"2024-05-09T08:02:22.617950+0000","last_undegraded":"2024-05-09T08:02:22.617950+0000","last_fullsized":"2024-05-09T08:02:22.617950+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:41:27.402821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:21.677395+0000","last_change":"2024-05-09T08:02:15.907374+0000","last_active":"2024-05-09T08:02:21.677395+0000","last_peered":"2024-05-09T08:02:21.677395+0000","last_clean":"2024-05-09T08:02:21.677395+0000","last_became_active":"2024-05-09T08:02:15.906767+0000","last_became_peered":"2024-05-09T08:02:15.906767+0000","last_unstale":"2024-05-09T08:02:21.677395+0000","last_undegraded":"2024-05-09T08:02:21.677395+0000","last_fullsized":"2024-05-09T08:02:21.677395+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:05:02.181367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:20.935527+0000","last_change":"2024-05-09T08:02:15.905326+0000","last_active":"2024-05-09T08:02:20.935527+0000","last_peered":"2024-05-09T08:02:20.935527+0000","last_clean":"2024-05-09T08:02:20.935527+0000","last_became_active":"2024-05-09T08:02:15.905083+0000","last_became_peered":"2024-05-09T08:02:15.905083+0000","last_unstale":"2024-05-09T08:02:20.935527+0000","last_undegraded":"2024-05-09T08:02:20.935527+0000","last_fullsized":"2024-05-09T08:02:20.935527+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:23:40.248821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:20.934866+0000","last_change":"2024-05-09T08:02:15.907753+0000","last_active":"2024-05-09T08:02:20.934866+0000","last_peered":"2024-05-09T08:02:20.934866+0000","last_clean":"2024-05-09T08:02:20.934866+0000","last_became_active":"2024-05-09T08:02:15.907480+0000","last_became_peered":"2024-05-09T08:02:15.907480+0000","last_unstale":"2024-05-09T08:02:20.934866+0000","last_undegraded":"2024-05-09T08:02:20.934866+0000","last_fullsized":"2024-05-09T08:02:20.934866+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:20:07.368327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.619474+0000","last_change":"2024-05-09T08:02:15.888605+0000","last_active":"2024-05-09T08:02:22.619474+0000","last_peered":"2024-05-09T08:02:22.619474+0000","last_clean":"2024-05-09T08:02:22.619474+0000","last_became_active":"2024-05-09T08:02:15.887981+0000","last_became_peered":"2024-05-09T08:02:15.887981+0000","last_unstale":"2024-05-09T08:02:22.619474+0000","last_undegraded":"2024-05-09T08:02:22.619474+0000","last_fullsized":"2024-05-09T08:02:22.619474+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:47:32.605288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:20.935282+0000","last_change":"2024-05-09T08:02:15.904744+0000","last_active":"2024-05-09T08:02:20.935282+0000","last_peered":"2024-05-09T08:02:20.935282+0000","last_clean":"2024-05-09T08:02:20.935282+0000","last_became_active":"2024-05-09T08:02:15.904334+0000","last_became_peered":"2024-05-09T08:02:15.904334+0000","last_unstale":"2024-05-09T08:02:20.935282+0000","last_undegraded":"2024-05-09T08:02:20.935282+0000","last_fullsized":"2024-05-09T08:02:20.935282+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:30:33.351158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.617634+0000","last_change":"2024-05-09T08:02:15.904835+0000","last_active":"2024-05-09T08:02:22.617634+0000","last_peered":"2024-05-09T08:02:22.617634+0000","last_clean":"2024-05-09T08:02:22.617634+0000","last_became_active":"2024-05-09T08:02:15.904416+0000","last_became_peered":"2024-05-09T08:02:15.904416+0000","last_unstale":"2024-05-09T08:02:22.617634+0000","last_undegraded":"2024-05-09T08:02:22.617634+0000","last_fullsized":"2024-05-09T08:02:22.617634+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:47:32.605288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:20.933091+0000","last_change":"2024-05-09T08:02:15.898324+0000","last_active":"2024-05-09T08:02:20.933091+0000","last_peered":"2024-05-09T08:02:20.933091+0000","last_clean":"2024-05-09T08:02:20.933091+0000","last_became_active":"2024-05-09T08:02:15.897981+0000","last_became_peered":"2024-05-09T08:02:15.897981+0000","last_unstale":"2024-05-09T08:02:20.933091+0000","last_undegraded":"2024-05-09T08:02:20.933091+0000","last_fullsized":"2024-05-09T08:02:20.933091+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:01:50.341133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.617909+0000","last_change":"2024-05-09T08:02:15.890945+0000","last_active":"2024-05-09T08:02:22.617909+0000","last_peered":"2024-05-09T08:02:22.617909+0000","last_clean":"2024-05-09T08:02:22.617909+0000","last_became_active":"2024-05-09T08:02:15.890647+0000","last_became_peered":"2024-05-09T08:02:15.890647+0000","last_unstale":"2024-05-09T08:02:22.617909+0000","last_undegraded":"2024-05-09T08:02:22.617909+0000","last_fullsized":"2024-05-09T08:02:22.617909+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:41:56.871863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:21.677105+0000","last_change":"2024-05-09T08:02:15.913728+0000","last_active":"2024-05-09T08:02:21.677105+0000","last_peered":"2024-05-09T08:02:21.677105+0000","last_clean":"2024-05-09T08:02:21.677105+0000","last_became_active":"2024-05-09T08:02:15.909818+0000","last_became_peered":"2024-05-09T08:02:15.909818+0000","last_unstale":"2024-05-09T08:02:21.677105+0000","last_undegraded":"2024-05-09T08:02:21.677105+0000","last_fullsized":"2024-05-09T08:02:21.677105+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:41:55.788033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.619452+0000","last_change":"2024-05-09T08:02:15.885025+0000","last_active":"2024-05-09T08:02:22.619452+0000","last_peered":"2024-05-09T08:02:22.619452+0000","last_clean":"2024-05-09T08:02:22.619452+0000","last_became_active":"2024-05-09T08:02:15.884517+0000","last_became_peered":"2024-05-09T08:02:15.884517+0000","last_unstale":"2024-05-09T08:02:22.619452+0000","last_undegraded":"2024-05-09T08:02:22.619452+0000","last_fullsized":"2024-05-09T08:02:22.619452+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:01:50.341133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:21.077234+0000","last_change":"2024-05-09T08:02:15.890306+0000","last_active":"2024-05-09T08:02:21.077234+0000","last_peered":"2024-05-09T08:02:21.077234+0000","last_clean":"2024-05-09T08:02:21.077234+0000","last_became_active":"2024-05-09T08:02:15.889998+0000","last_became_peered":"2024-05-09T08:02:15.889998+0000","last_unstale":"2024-05-09T08:02:21.077234+0000","last_undegraded":"2024-05-09T08:02:21.077234+0000","last_fullsized":"2024-05-09T08:02:21.077234+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:16:57.446444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.619318+0000","last_change":"2024-05-09T08:02:15.886417+0000","last_active":"2024-05-09T08:02:22.619318+0000","last_peered":"2024-05-09T08:02:22.619318+0000","last_clean":"2024-05-09T08:02:22.619318+0000","last_became_active":"2024-05-09T08:02:15.885525+0000","last_became_peered":"2024-05-09T08:02:15.885525+0000","last_unstale":"2024-05-09T08:02:22.619318+0000","last_undegraded":"2024-05-09T08:02:22.619318+0000","last_fullsized":"2024-05-09T08:02:22.619318+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:43:02.075442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.618616+0000","last_change":"2024-05-09T08:02:15.893172+0000","last_active":"2024-05-09T08:02:22.618616+0000","last_peered":"2024-05-09T08:02:22.618616+0000","last_clean":"2024-05-09T08:02:22.618616+0000","last_became_active":"2024-05-09T08:02:15.892886+0000","last_became_peered":"2024-05-09T08:02:15.892886+0000","last_unstale":"2024-05-09T08:02:22.618616+0000","last_undegraded":"2024-05-09T08:02:22.618616+0000","last_fullsized":"2024-05-09T08:02:22.618616+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:59:14.957806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.617513+0000","last_change":"2024-05-09T08:02:15.908659+0000","last_active":"2024-05-09T08:02:22.617513+0000","last_peered":"2024-05-09T08:02:22.617513+0000","last_clean":"2024-05-09T08:02:22.617513+0000","last_became_active":"2024-05-09T08:02:15.908307+0000","last_became_peered":"2024-05-09T08:02:15.908307+0000","last_unstale":"2024-05-09T08:02:22.617513+0000","last_undegraded":"2024-05-09T08:02:22.617513+0000","last_fullsized":"2024-05-09T08:02:22.617513+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:20:07.368327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:20.933418+0000","last_change":"2024-05-09T08:02:15.885800+0000","last_active":"2024-05-09T08:02:20.933418+0000","last_peered":"2024-05-09T08:02:20.933418+0000","last_clean":"2024-05-09T08:02:20.933418+0000","last_became_active":"2024-05-09T08:02:15.885185+0000","last_became_peered":"2024-05-09T08:02:15.885185+0000","last_unstale":"2024-05-09T08:02:20.933418+0000","last_undegraded":"2024-05-09T08:02:20.933418+0000","last_fullsized":"2024-05-09T08:02:20.933418+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:01:50.341133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:21.677190+0000","last_change":"2024-05-09T08:02:15.911496+0000","last_active":"2024-05-09T08:02:21.677190+0000","last_peered":"2024-05-09T08:02:21.677190+0000","last_clean":"2024-05-09T08:02:21.677190+0000","last_became_active":"2024-05-09T08:02:15.911142+0000","last_became_peered":"2024-05-09T08:02:15.911142+0000","last_unstale":"2024-05-09T08:02:21.677190+0000","last_undegraded":"2024-05-09T08:02:21.677190+0000","last_fullsized":"2024-05-09T08:02:21.677190+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:27:59.315750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:20.933605+0000","last_change":"2024-05-09T08:02:15.898847+0000","last_active":"2024-05-09T08:02:20.933605+0000","last_peered":"2024-05-09T08:02:20.933605+0000","last_clean":"2024-05-09T08:02:20.933605+0000","last_became_active":"2024-05-09T08:02:15.898480+0000","last_became_peered":"2024-05-09T08:02:15.898480+0000","last_unstale":"2024-05-09T08:02:20.933605+0000","last_undegraded":"2024-05-09T08:02:20.933605+0000","last_fullsized":"2024-05-09T08:02:20.933605+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:20:07.368327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.618261+0000","last_change":"2024-05-09T08:02:15.883117+0000","last_active":"2024-05-09T08:02:22.618261+0000","last_peered":"2024-05-09T08:02:22.618261+0000","last_clean":"2024-05-09T08:02:22.618261+0000","last_became_active":"2024-05-09T08:02:15.882141+0000","last_became_peered":"2024-05-09T08:02:15.882141+0000","last_unstale":"2024-05-09T08:02:22.618261+0000","last_undegraded":"2024-05-09T08:02:22.618261+0000","last_fullsized":"2024-05-09T08:02:22.618261+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:57:11.352151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.618965+0000","last_change":"2024-05-09T08:02:15.893318+0000","last_active":"2024-05-09T08:02:22.618965+0000","last_peered":"2024-05-09T08:02:22.618965+0000","last_clean":"2024-05-09T08:02:22.618965+0000","last_became_active":"2024-05-09T08:02:15.892964+0000","last_became_peered":"2024-05-09T08:02:15.892964+0000","last_unstale":"2024-05-09T08:02:22.618965+0000","last_undegraded":"2024-05-09T08:02:22.618965+0000","last_fullsized":"2024-05-09T08:02:22.618965+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:20:07.368327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.619428+0000","last_change":"2024-05-09T08:02:15.886482+0000","last_active":"2024-05-09T08:02:22.619428+0000","last_peered":"2024-05-09T08:02:22.619428+0000","last_clean":"2024-05-09T08:02:22.619428+0000","last_became_active":"2024-05-09T08:02:15.886183+0000","last_became_peered":"2024-05-09T08:02:15.886183+0000","last_unstale":"2024-05-09T08:02:22.619428+0000","last_undegraded":"2024-05-09T08:02:22.619428+0000","last_fullsized":"2024-05-09T08:02:22.619428+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:23:40.248821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:21.077365+0000","last_change":"2024-05-09T08:02:15.887506+0000","last_active":"2024-05-09T08:02:21.077365+0000","last_peered":"2024-05-09T08:02:21.077365+0000","last_clean":"2024-05-09T08:02:21.077365+0000","last_became_active":"2024-05-09T08:02:15.887225+0000","last_became_peered":"2024-05-09T08:02:15.887225+0000","last_unstale":"2024-05-09T08:02:21.077365+0000","last_undegraded":"2024-05-09T08:02:21.077365+0000","last_fullsized":"2024-05-09T08:02:21.077365+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:25:24.736323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:20.933062+0000","last_change":"2024-05-09T08:02:15.902065+0000","last_active":"2024-05-09T08:02:20.933062+0000","last_peered":"2024-05-09T08:02:20.933062+0000","last_clean":"2024-05-09T08:02:20.933062+0000","last_became_active":"2024-05-09T08:02:15.901766+0000","last_became_peered":"2024-05-09T08:02:15.901766+0000","last_unstale":"2024-05-09T08:02:20.933062+0000","last_undegraded":"2024-05-09T08:02:20.933062+0000","last_fullsized":"2024-05-09T08:02:20.933062+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:05:02.181367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:21.076616+0000","last_change":"2024-05-09T08:02:15.911058+0000","last_active":"2024-05-09T08:02:21.076616+0000","last_peered":"2024-05-09T08:02:21.076616+0000","last_clean":"2024-05-09T08:02:21.076616+0000","last_became_active":"2024-05-09T08:02:15.910716+0000","last_became_peered":"2024-05-09T08:02:15.910716+0000","last_unstale":"2024-05-09T08:02:21.076616+0000","last_undegraded":"2024-05-09T08:02:21.076616+0000","last_fullsized":"2024-05-09T08:02:21.076616+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:41:56.871863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.617832+0000","last_change":"2024-05-09T08:02:15.908653+0000","last_active":"2024-05-09T08:02:22.617832+0000","last_peered":"2024-05-09T08:02:22.617832+0000","last_clean":"2024-05-09T08:02:22.617832+0000","last_became_active":"2024-05-09T08:02:15.908290+0000","last_became_peered":"2024-05-09T08:02:15.908290+0000","last_unstale":"2024-05-09T08:02:22.617832+0000","last_undegraded":"2024-05-09T08:02:22.617832+0000","last_fullsized":"2024-05-09T08:02:22.617832+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:18:36.309627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:21.076688+0000","last_change":"2024-05-09T08:02:15.896652+0000","last_active":"2024-05-09T08:02:21.076688+0000","last_peered":"2024-05-09T08:02:21.076688+0000","last_clean":"2024-05-09T08:02:21.076688+0000","last_became_active":"2024-05-09T08:02:15.896384+0000","last_became_peered":"2024-05-09T08:02:15.896384+0000","last_unstale":"2024-05-09T08:02:21.076688+0000","last_undegraded":"2024-05-09T08:02:21.076688+0000","last_fullsized":"2024-05-09T08:02:21.076688+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:18:36.309627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.617988+0000","last_change":"2024-05-09T08:02:15.898998+0000","last_active":"2024-05-09T08:02:22.617988+0000","last_peered":"2024-05-09T08:02:22.617988+0000","last_clean":"2024-05-09T08:02:22.617988+0000","last_became_active":"2024-05-09T08:02:15.898670+0000","last_became_peered":"2024-05-09T08:02:15.898670+0000","last_unstale":"2024-05-09T08:02:22.617988+0000","last_undegraded":"2024-05-09T08:02:22.617988+0000","last_fullsized":"2024-05-09T08:02:22.617988+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:23:40.248821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:21.677605+0000","last_change":"2024-05-09T08:02:15.905078+0000","last_active":"2024-05-09T08:02:21.677605+0000","last_peered":"2024-05-09T08:02:21.677605+0000","last_clean":"2024-05-09T08:02:21.677605+0000","last_became_active":"2024-05-09T08:02:15.904736+0000","last_became_peered":"2024-05-09T08:02:15.904736+0000","last_unstale":"2024-05-09T08:02:21.677605+0000","last_undegraded":"2024-05-09T08:02:21.677605+0000","last_fullsized":"2024-05-09T08:02:21.677605+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:01:50.341133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:20.936449+0000","last_change":"2024-05-09T08:02:15.897743+0000","last_active":"2024-05-09T08:02:20.936449+0000","last_peered":"2024-05-09T08:02:20.936449+0000","last_clean":"2024-05-09T08:02:20.936449+0000","last_became_active":"2024-05-09T08:02:15.897367+0000","last_became_peered":"2024-05-09T08:02:15.897367+0000","last_unstale":"2024-05-09T08:02:20.936449+0000","last_undegraded":"2024-05-09T08:02:20.936449+0000","last_fullsized":"2024-05-09T08:02:20.936449+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:47:32.605288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:21.077273+0000","last_change":"2024-05-09T08:02:15.884163+0000","last_active":"2024-05-09T08:02:21.077273+0000","last_peered":"2024-05-09T08:02:21.077273+0000","last_clean":"2024-05-09T08:02:21.077273+0000","last_became_active":"2024-05-09T08:02:15.883430+0000","last_became_peered":"2024-05-09T08:02:15.883430+0000","last_unstale":"2024-05-09T08:02:21.077273+0000","last_undegraded":"2024-05-09T08:02:21.077273+0000","last_fullsized":"2024-05-09T08:02:21.077273+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:05:15.555592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574857,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32024,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339816,"statfs":{"total":96636764160,"available":96603971584,"internally_reserved":0,"allocated":864256,"data_stored":695877,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574857,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31448,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340392,"statfs":{"total":96636764160,"available":96604561408,"internally_reserved":0,"allocated":278528,"data_stored":108769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574857,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31444,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340396,"statfs":{"total":96636764160,"available":96604565504,"internally_reserved":0,"allocated":270336,"data_stored":105017,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574858,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31452,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340388,"statfs":{"total":96636764160,"available":96604557312,"internally_reserved":0,"allocated":278528,"data_stored":105053,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574858,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":108769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574858,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":266240,"data_stored":104927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574859,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":274432,"data_stored":105087,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574858,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32032,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339808,"statfs":{"total":96636764160,"available":96603963392,"internally_reserved":0,"allocated":872448,"data_stored":695933,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T08:02:51.713 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-09T08:02:51.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:51.952+0000 7f8e30000640 1 Processor -- start 2024-05-09T08:02:51.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:51.953+0000 7f8e30000640 1 -- start start 2024-05-09T08:02:51.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:51.955+0000 7f8e30000640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f8e28151660 0x7f8e280c6c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:51.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:51.955+0000 7f8e30000640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f8e280c1ba0 0x7f8e28072d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:51.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:51.955+0000 7f8e30000640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8e280c72f0 con 0x7f8e280b1d30 2024-05-09T08:02:51.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:51.955+0000 7f8e30000640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f8e28151a40 con 0x7f8e280c1ba0 2024-05-09T08:02:51.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:51.955+0000 7f8e30000640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f8e28157620 con 0x7f8e28151660 2024-05-09T08:02:51.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:51.955+0000 7f8e2d600640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f8e28151660 0x7f8e280c6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:51.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:51.955+0000 7f8e2d600640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f8e28151660 0x7f8e280c6c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3301/0 says I am v2:172.21.15.1:41560/0 (socket says 172.21.15.1:41560) 2024-05-09T08:02:51.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:51.955+0000 7f8e2d600640 1 -- 172.21.15.1:0/4151630263 learned_addr learned my addr 172.21.15.1:0/4151630263 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:02:51.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:51.955+0000 7f8e2cc00640 1 --2- 172.21.15.1:0/4151630263 >> v2:172.21.15.1:3300/0 conn(0x7f8e280c1ba0 0x7f8e28072d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:51.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:51.956+0000 7f8e2d600640 1 -- 172.21.15.1:0/4151630263 >> v2:172.21.15.1:3300/0 conn(0x7f8e280c1ba0 msgr2=0x7f8e28072d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:02:51.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:51.956+0000 7f8e2d600640 1 --2- 172.21.15.1:0/4151630263 >> v2:172.21.15.1:3300/0 conn(0x7f8e280c1ba0 0x7f8e28072d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:51.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:51.956+0000 7f8e2d600640 1 -- 172.21.15.1:0/4151630263 >> v1:172.21.15.7:6789/0 conn(0x7f8e280b1d30 legacy=0x7f8e280c7a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:51.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:51.956+0000 7f8e2d600640 1 -- 172.21.15.1:0/4151630263 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e28157900 con 0x7f8e28151660 2024-05-09T08:02:51.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:51.956+0000 7f8e27e00640 1 -- 172.21.15.1:0/4151630263 <== mon.1 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4266214372 0 0) 0x7f8e280c72f0 con 0x7f8e280b1d30 2024-05-09T08:02:51.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:51.958+0000 7f8e2d600640 1 --2- 172.21.15.1:0/4151630263 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f8e28151660 0x7f8e280c6c90 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f8e1c009870 tx=0x7f8e1c02a250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7fc725e2d7f5e7ee server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:51.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:51.958+0000 7f8e27e00640 1 -- 172.21.15.1:0/4151630263 <== mon.2 v2:172.21.15.7:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e1c037070 con 0x7f8e28151660 2024-05-09T08:02:51.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:51.958+0000 7f8e27e00640 1 -- 172.21.15.1:0/4151630263 <== mon.2 v2:172.21.15.7:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e1c032440 con 0x7f8e28151660 2024-05-09T08:02:51.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:51.958+0000 7f8e27e00640 1 -- 172.21.15.1:0/4151630263 <== mon.2 v2:172.21.15.7:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e1c032810 con 0x7f8e28151660 2024-05-09T08:02:51.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:51.959+0000 7f8e30000640 1 -- 172.21.15.1:0/4151630263 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f8e28151660 msgr2=0x7f8e280c6c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:51.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:51.959+0000 7f8e30000640 1 --2- 172.21.15.1:0/4151630263 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f8e28151660 0x7f8e280c6c90 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f8e1c009870 tx=0x7f8e1c02a250 comp rx=0 tx=0).stop 2024-05-09T08:02:51.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:51.959+0000 7f8e30000640 1 -- 172.21.15.1:0/4151630263 shutdown_connections 2024-05-09T08:02:51.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:51.959+0000 7f8e30000640 1 --2- 172.21.15.1:0/4151630263 >> v2:172.21.15.1:3300/0 conn(0x7f8e280c1ba0 0x7f8e28072d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:51.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:51.959+0000 7f8e30000640 1 --2- 172.21.15.1:0/4151630263 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f8e28151660 0x7f8e280c6c90 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:51.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:51.959+0000 7f8e30000640 1 -- 172.21.15.1:0/4151630263 >> 172.21.15.1:0/4151630263 conn(0x7f8e280cc070 msgr2=0x7f8e280cc480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:02:51.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:51.959+0000 7f8e30000640 1 -- 172.21.15.1:0/4151630263 shutdown_connections 2024-05-09T08:02:51.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:51.959+0000 7f8e30000640 1 -- 172.21.15.1:0/4151630263 wait complete. 2024-05-09T08:02:51.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:51.961+0000 7f8e30000640 1 Processor -- start 2024-05-09T08:02:51.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:51.961+0000 7f8e30000640 1 -- start start 2024-05-09T08:02:51.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:51.962+0000 7f8e30000640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f8e280b1d30 0x7f8e28162940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:51.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:51.962+0000 7f8e30000640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8e280c1ba0 0x7f8e281f08e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:51.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:51.962+0000 7f8e30000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e28151660 0x7f8e2813fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:51.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:51.962+0000 7f8e30000640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f8e281581a0 con 0x7f8e280c1ba0 2024-05-09T08:02:51.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:51.962+0000 7f8e30000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f8e28157ec0 con 0x7f8e28151660 2024-05-09T08:02:51.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:51.962+0000 7f8e30000640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f8e281572d0 con 0x7f8e280b1d30 2024-05-09T08:02:51.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:51.962+0000 7f8e2cc00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8e280c1ba0 0x7f8e281f08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:51.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:51.962+0000 7f8e2cc00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8e280c1ba0 0x7f8e281f08e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53016/0 (socket says 172.21.15.1:53016) 2024-05-09T08:02:51.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:51.962+0000 7f8e2cc00640 1 -- 172.21.15.1:0/2522618336 learned_addr learned my addr 172.21.15.1:0/2522618336 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:02:51.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:51.962+0000 7f8e2d600640 1 --2- 172.21.15.1:0/2522618336 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f8e280b1d30 0x7f8e28162940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:51.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:51.962+0000 7f8e2e000640 1 --2- 172.21.15.1:0/2522618336 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e28151660 0x7f8e2813fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:51.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:51.963+0000 7f8e2cc00640 1 -- 172.21.15.1:0/2522618336 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f8e280b1d30 msgr2=0x7f8e28162940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:51.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:51.963+0000 7f8e2cc00640 1 --2- 172.21.15.1:0/2522618336 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f8e280b1d30 0x7f8e28162940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:51.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:51.963+0000 7f8e2cc00640 1 -- 172.21.15.1:0/2522618336 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e28151660 msgr2=0x7f8e2813fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:51.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:51.963+0000 7f8e2cc00640 1 --2- 172.21.15.1:0/2522618336 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e28151660 0x7f8e2813fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:51.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:51.963+0000 7f8e2cc00640 1 -- 172.21.15.1:0/2522618336 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e28163150 con 0x7f8e280c1ba0 2024-05-09T08:02:51.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:51.963+0000 7f8e2cc00640 1 --2- 172.21.15.1:0/2522618336 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8e280c1ba0 0x7f8e281f08e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f8e1400d7d0 tx=0x7f8e1400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:51.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:51.963+0000 7f8e25600640 1 -- 172.21.15.1:0/2522618336 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e1400b6c0 con 0x7f8e280c1ba0 2024-05-09T08:02:51.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:51.963+0000 7f8e25600640 1 -- 172.21.15.1:0/2522618336 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e14019070 con 0x7f8e280c1ba0 2024-05-09T08:02:51.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:51.963+0000 7f8e25600640 1 -- 172.21.15.1:0/2522618336 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e1400ba80 con 0x7f8e280c1ba0 2024-05-09T08:02:51.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:51.963+0000 7f8e30000640 1 -- 172.21.15.1:0/2522618336 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e28163430 con 0x7f8e280c1ba0 2024-05-09T08:02:51.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:51.964+0000 7f8e30000640 1 -- 172.21.15.1:0/2522618336 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e281639f0 con 0x7f8e280c1ba0 2024-05-09T08:02:51.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:51.965+0000 7f8e30000640 1 -- 172.21.15.1:0/2522618336 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e281f01b0 con 0x7f8e280c1ba0 2024-05-09T08:02:51.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:51.966+0000 7f8e25600640 1 -- 172.21.15.1:0/2522618336 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8e1400bc20 con 0x7f8e280c1ba0 2024-05-09T08:02:51.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:51.967+0000 7f8e25600640 1 --2- 172.21.15.1:0/2522618336 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f8e0806be30 0x7f8e0806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:51.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:51.967+0000 7f8e25600640 1 -- 172.21.15.1:0/2522618336 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f8e1408b580 con 0x7f8e280c1ba0 2024-05-09T08:02:51.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:51.967+0000 7f8e2d600640 1 --2- 172.21.15.1:0/2522618336 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f8e0806be30 0x7f8e0806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:51.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:51.968+0000 7f8e2d600640 1 --2- 172.21.15.1:0/2522618336 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f8e0806be30 0x7f8e0806e2f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f8e1c034000 tx=0x7f8e1c02e000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:51.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:51.972+0000 7f8e25600640 1 -- 172.21.15.1:0/2522618336 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f8e14059500 con 0x7f8e280c1ba0 2024-05-09T08:02:52.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:52.172+0000 7f8e30000640 1 -- 172.21.15.1:0/2522618336 --> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8e28074d70 con 0x7f8e0806be30 2024-05-09T08:02:52.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:52.182+0000 7f8e25600640 1 -- 172.21.15.1:0/2522618336 <== mgr.4114 v2:172.21.15.7:6832/3562612464 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332537 (secure 0 0 0) 0x7f8e28074d70 con 0x7f8e0806be30 2024-05-09T08:02:52.204 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T08:02:52.205 INFO:teuthology.orchestra.run.smithi001.stderr:dumped all 2024-05-09T08:02:52.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:52.188+0000 7f8e30000640 1 -- 172.21.15.1:0/2522618336 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f8e0806be30 msgr2=0x7f8e0806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:52.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:52.188+0000 7f8e30000640 1 --2- 172.21.15.1:0/2522618336 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f8e0806be30 0x7f8e0806e2f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f8e1c034000 tx=0x7f8e1c02e000 comp rx=0 tx=0).stop 2024-05-09T08:02:52.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:52.188+0000 7f8e30000640 1 -- 172.21.15.1:0/2522618336 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8e280c1ba0 msgr2=0x7f8e281f08e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:52.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:52.188+0000 7f8e30000640 1 --2- 172.21.15.1:0/2522618336 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8e280c1ba0 0x7f8e281f08e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f8e1400d7d0 tx=0x7f8e1400dc90 comp rx=0 tx=0).stop 2024-05-09T08:02:52.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:52.188+0000 7f8e30000640 1 -- 172.21.15.1:0/2522618336 shutdown_connections 2024-05-09T08:02:52.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:52.188+0000 7f8e30000640 1 --2- 172.21.15.1:0/2522618336 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f8e0806be30 0x7f8e0806e2f0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:52.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:52.188+0000 7f8e30000640 1 --2- 172.21.15.1:0/2522618336 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e28151660 0x7f8e2813fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:52.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:52.188+0000 7f8e30000640 1 --2- 172.21.15.1:0/2522618336 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8e280c1ba0 0x7f8e281f08e0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:52.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:52.188+0000 7f8e30000640 1 --2- 172.21.15.1:0/2522618336 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f8e280b1d30 0x7f8e28162940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:52.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:52.188+0000 7f8e30000640 1 -- 172.21.15.1:0/2522618336 >> 172.21.15.1:0/2522618336 conn(0x7f8e280cc070 msgr2=0x7f8e281566f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:02:52.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:52.188+0000 7f8e30000640 1 -- 172.21.15.1:0/2522618336 shutdown_connections 2024-05-09T08:02:52.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:52.188+0000 7f8e30000640 1 -- 172.21.15.1:0/2522618336 wait complete. 2024-05-09T08:02:52.222 INFO:teuthology.orchestra.run.smithi001.stdout:{"pg_ready":true,"pg_map":{"version":40,"stamp":"2024-05-09T08:02:51.105163+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":208,"num_write_kb":2435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160,"ondisk_log_size":160,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":240256,"kb_used_data":3304,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754734464,"statfs":{"total":773094113280,"available":772848091136,"internally_reserved":0,"allocated":3383296,"data_stored":2029432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":242535986},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":141,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003616"},"pg_stats":[{"pgid":"2.1f","version":"24'4","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:21.677957+0000","last_change":"2024-05-09T08:02:14.875108+0000","last_active":"2024-05-09T08:02:21.677957+0000","last_peered":"2024-05-09T08:02:21.677957+0000","last_clean":"2024-05-09T08:02:21.677957+0000","last_became_active":"2024-05-09T08:02:14.874841+0000","last_became_peered":"2024-05-09T08:02:14.874841+0000","last_unstale":"2024-05-09T08:02:21.677957+0000","last_undegraded":"2024-05-09T08:02:21.677957+0000","last_fullsized":"2024-05-09T08:02:21.677957+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:00:40.599475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:20.934410+0000","last_change":"2024-05-09T08:02:15.899818+0000","last_active":"2024-05-09T08:02:20.934410+0000","last_peered":"2024-05-09T08:02:20.934410+0000","last_clean":"2024-05-09T08:02:20.934410+0000","last_became_active":"2024-05-09T08:02:15.899479+0000","last_became_peered":"2024-05-09T08:02:15.899479+0000","last_unstale":"2024-05-09T08:02:20.934410+0000","last_undegraded":"2024-05-09T08:02:20.934410+0000","last_fullsized":"2024-05-09T08:02:20.934410+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:18:36.309627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:20.932807+0000","last_change":"2024-05-09T08:02:20.057944+0000","last_active":"2024-05-09T08:02:20.932807+0000","last_peered":"2024-05-09T08:02:20.932807+0000","last_clean":"2024-05-09T08:02:20.932807+0000","last_became_active":"2024-05-09T08:02:20.057016+0000","last_became_peered":"2024-05-09T08:02:20.057016+0000","last_unstale":"2024-05-09T08:02:20.932807+0000","last_undegraded":"2024-05-09T08:02:20.932807+0000","last_fullsized":"2024-05-09T08:02:20.932807+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:17:01.514168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"24'1","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:20.934743+0000","last_change":"2024-05-09T08:02:14.886137+0000","last_active":"2024-05-09T08:02:20.934743+0000","last_peered":"2024-05-09T08:02:20.934743+0000","last_clean":"2024-05-09T08:02:20.934743+0000","last_became_active":"2024-05-09T08:02:14.885830+0000","last_became_peered":"2024-05-09T08:02:14.885830+0000","last_unstale":"2024-05-09T08:02:20.934743+0000","last_undegraded":"2024-05-09T08:02:20.934743+0000","last_fullsized":"2024-05-09T08:02:20.934743+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:46:11.160815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:21.677645+0000","last_change":"2024-05-09T08:02:15.906280+0000","last_active":"2024-05-09T08:02:21.677645+0000","last_peered":"2024-05-09T08:02:21.677645+0000","last_clean":"2024-05-09T08:02:21.677645+0000","last_became_active":"2024-05-09T08:02:15.905590+0000","last_became_peered":"2024-05-09T08:02:15.905590+0000","last_unstale":"2024-05-09T08:02:21.677645+0000","last_undegraded":"2024-05-09T08:02:21.677645+0000","last_fullsized":"2024-05-09T08:02:21.677645+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:20:07.368327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.617387+0000","last_change":"2024-05-09T08:02:19.954548+0000","last_active":"2024-05-09T08:02:22.617387+0000","last_peered":"2024-05-09T08:02:22.617387+0000","last_clean":"2024-05-09T08:02:22.617387+0000","last_became_active":"2024-05-09T08:02:19.954161+0000","last_became_peered":"2024-05-09T08:02:19.954161+0000","last_unstale":"2024-05-09T08:02:22.617387+0000","last_undegraded":"2024-05-09T08:02:22.617387+0000","last_fullsized":"2024-05-09T08:02:22.617387+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:53:27.246334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"24'2","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:21.077047+0000","last_change":"2024-05-09T08:02:14.867128+0000","last_active":"2024-05-09T08:02:21.077047+0000","last_peered":"2024-05-09T08:02:21.077047+0000","last_clean":"2024-05-09T08:02:21.077047+0000","last_became_active":"2024-05-09T08:02:14.866809+0000","last_became_peered":"2024-05-09T08:02:14.866809+0000","last_unstale":"2024-05-09T08:02:21.077047+0000","last_undegraded":"2024-05-09T08:02:21.077047+0000","last_fullsized":"2024-05-09T08:02:21.077047+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:57:06.093718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.618173+0000","last_change":"2024-05-09T08:02:15.888431+0000","last_active":"2024-05-09T08:02:22.618173+0000","last_peered":"2024-05-09T08:02:22.618173+0000","last_clean":"2024-05-09T08:02:22.618173+0000","last_became_active":"2024-05-09T08:02:15.887509+0000","last_became_peered":"2024-05-09T08:02:15.887509+0000","last_unstale":"2024-05-09T08:02:22.618173+0000","last_undegraded":"2024-05-09T08:02:22.618173+0000","last_fullsized":"2024-05-09T08:02:22.618173+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:25:01.238816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.617803+0000","last_change":"2024-05-09T08:02:19.953600+0000","last_active":"2024-05-09T08:02:22.617803+0000","last_peered":"2024-05-09T08:02:22.617803+0000","last_clean":"2024-05-09T08:02:22.617803+0000","last_became_active":"2024-05-09T08:02:19.952275+0000","last_became_peered":"2024-05-09T08:02:19.952275+0000","last_unstale":"2024-05-09T08:02:22.617803+0000","last_undegraded":"2024-05-09T08:02:22.617803+0000","last_fullsized":"2024-05-09T08:02:22.617803+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:25:05.306541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"24'2","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:21.076799+0000","last_change":"2024-05-09T08:02:14.870819+0000","last_active":"2024-05-09T08:02:21.076799+0000","last_peered":"2024-05-09T08:02:21.076799+0000","last_clean":"2024-05-09T08:02:21.076799+0000","last_became_active":"2024-05-09T08:02:14.870476+0000","last_became_peered":"2024-05-09T08:02:14.870476+0000","last_unstale":"2024-05-09T08:02:21.076799+0000","last_undegraded":"2024-05-09T08:02:21.076799+0000","last_fullsized":"2024-05-09T08:02:21.076799+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:43:01.061801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.618291+0000","last_change":"2024-05-09T08:02:15.886471+0000","last_active":"2024-05-09T08:02:22.618291+0000","last_peered":"2024-05-09T08:02:22.618291+0000","last_clean":"2024-05-09T08:02:22.618291+0000","last_became_active":"2024-05-09T08:02:15.886168+0000","last_became_peered":"2024-05-09T08:02:15.886168+0000","last_unstale":"2024-05-09T08:02:22.618291+0000","last_undegraded":"2024-05-09T08:02:22.618291+0000","last_fullsized":"2024-05-09T08:02:22.618291+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:30:33.351158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.618044+0000","last_change":"2024-05-09T08:02:19.947347+0000","last_active":"2024-05-09T08:02:22.618044+0000","last_peered":"2024-05-09T08:02:22.618044+0000","last_clean":"2024-05-09T08:02:22.618044+0000","last_became_active":"2024-05-09T08:02:19.946906+0000","last_became_peered":"2024-05-09T08:02:19.946906+0000","last_unstale":"2024-05-09T08:02:22.618044+0000","last_undegraded":"2024-05-09T08:02:22.618044+0000","last_fullsized":"2024-05-09T08:02:22.618044+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:30:37.418883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.618712+0000","last_change":"2024-05-09T08:02:15.880765+0000","last_active":"2024-05-09T08:02:22.618712+0000","last_peered":"2024-05-09T08:02:22.618712+0000","last_clean":"2024-05-09T08:02:22.618712+0000","last_became_active":"2024-05-09T08:02:15.880439+0000","last_became_peered":"2024-05-09T08:02:15.880439+0000","last_unstale":"2024-05-09T08:02:22.618712+0000","last_undegraded":"2024-05-09T08:02:22.618712+0000","last_fullsized":"2024-05-09T08:02:22.618712+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:29:59.441314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"24'2","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:21.076978+0000","last_change":"2024-05-09T08:02:14.867587+0000","last_active":"2024-05-09T08:02:21.076978+0000","last_peered":"2024-05-09T08:02:21.076978+0000","last_clean":"2024-05-09T08:02:21.076978+0000","last_became_active":"2024-05-09T08:02:14.867291+0000","last_became_peered":"2024-05-09T08:02:14.867291+0000","last_unstale":"2024-05-09T08:02:21.076978+0000","last_undegraded":"2024-05-09T08:02:21.076978+0000","last_fullsized":"2024-05-09T08:02:21.076978+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:20:08.921752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.619129+0000","last_change":"2024-05-09T08:02:19.954902+0000","last_active":"2024-05-09T08:02:22.619129+0000","last_peered":"2024-05-09T08:02:22.619129+0000","last_clean":"2024-05-09T08:02:22.619129+0000","last_became_active":"2024-05-09T08:02:19.954007+0000","last_became_peered":"2024-05-09T08:02:19.954007+0000","last_unstale":"2024-05-09T08:02:22.619129+0000","last_undegraded":"2024-05-09T08:02:22.619129+0000","last_fullsized":"2024-05-09T08:02:22.619129+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:41:31.470546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:20.934123+0000","last_change":"2024-05-09T08:02:15.915155+0000","last_active":"2024-05-09T08:02:20.934123+0000","last_peered":"2024-05-09T08:02:20.934123+0000","last_clean":"2024-05-09T08:02:20.934123+0000","last_became_active":"2024-05-09T08:02:15.914852+0000","last_became_peered":"2024-05-09T08:02:15.914852+0000","last_unstale":"2024-05-09T08:02:20.934123+0000","last_undegraded":"2024-05-09T08:02:20.934123+0000","last_fullsized":"2024-05-09T08:02:20.934123+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:53:23.178610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:20.934852+0000","last_change":"2024-05-09T08:02:14.881770+0000","last_active":"2024-05-09T08:02:20.934852+0000","last_peered":"2024-05-09T08:02:20.934852+0000","last_clean":"2024-05-09T08:02:20.934852+0000","last_became_active":"2024-05-09T08:02:14.881437+0000","last_became_peered":"2024-05-09T08:02:14.881437+0000","last_unstale":"2024-05-09T08:02:20.934852+0000","last_undegraded":"2024-05-09T08:02:20.934852+0000","last_fullsized":"2024-05-09T08:02:20.934852+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:25:23.306827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.618007+0000","last_change":"2024-05-09T08:02:19.955928+0000","last_active":"2024-05-09T08:02:22.618007+0000","last_peered":"2024-05-09T08:02:22.618007+0000","last_clean":"2024-05-09T08:02:22.618007+0000","last_became_active":"2024-05-09T08:02:19.955432+0000","last_became_peered":"2024-05-09T08:02:19.955432+0000","last_unstale":"2024-05-09T08:02:22.618007+0000","last_undegraded":"2024-05-09T08:02:22.618007+0000","last_fullsized":"2024-05-09T08:02:22.618007+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:53:15.458120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:20.933351+0000","last_change":"2024-05-09T08:02:15.913311+0000","last_active":"2024-05-09T08:02:20.933351+0000","last_peered":"2024-05-09T08:02:20.933351+0000","last_clean":"2024-05-09T08:02:20.933351+0000","last_became_active":"2024-05-09T08:02:15.912484+0000","last_became_peered":"2024-05-09T08:02:15.912484+0000","last_unstale":"2024-05-09T08:02:20.933351+0000","last_undegraded":"2024-05-09T08:02:20.933351+0000","last_fullsized":"2024-05-09T08:02:20.933351+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:43:06.938904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:20.933542+0000","last_change":"2024-05-09T08:02:14.868884+0000","last_active":"2024-05-09T08:02:20.933542+0000","last_peered":"2024-05-09T08:02:20.933542+0000","last_clean":"2024-05-09T08:02:20.933542+0000","last_became_active":"2024-05-09T08:02:14.868553+0000","last_became_peered":"2024-05-09T08:02:14.868553+0000","last_unstale":"2024-05-09T08:02:20.933542+0000","last_undegraded":"2024-05-09T08:02:20.933542+0000","last_fullsized":"2024-05-09T08:02:20.933542+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:32:48.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:20.935948+0000","last_change":"2024-05-09T08:02:20.053648+0000","last_active":"2024-05-09T08:02:20.935948+0000","last_peered":"2024-05-09T08:02:20.935948+0000","last_clean":"2024-05-09T08:02:20.935948+0000","last_became_active":"2024-05-09T08:02:20.053152+0000","last_became_peered":"2024-05-09T08:02:20.053152+0000","last_unstale":"2024-05-09T08:02:20.935948+0000","last_undegraded":"2024-05-09T08:02:20.935948+0000","last_fullsized":"2024-05-09T08:02:20.935948+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:59:19.025530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.618049+0000","last_change":"2024-05-09T08:02:15.891115+0000","last_active":"2024-05-09T08:02:22.618049+0000","last_peered":"2024-05-09T08:02:22.618049+0000","last_clean":"2024-05-09T08:02:22.618049+0000","last_became_active":"2024-05-09T08:02:15.890806+0000","last_became_peered":"2024-05-09T08:02:15.890806+0000","last_unstale":"2024-05-09T08:02:22.618049+0000","last_undegraded":"2024-05-09T08:02:22.618049+0000","last_fullsized":"2024-05-09T08:02:22.618049+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:57:07.107359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:21.076482+0000","last_change":"2024-05-09T08:02:14.874987+0000","last_active":"2024-05-09T08:02:21.076482+0000","last_peered":"2024-05-09T08:02:21.076482+0000","last_clean":"2024-05-09T08:02:21.076482+0000","last_became_active":"2024-05-09T08:02:14.874549+0000","last_became_peered":"2024-05-09T08:02:14.874549+0000","last_unstale":"2024-05-09T08:02:21.076482+0000","last_undegraded":"2024-05-09T08:02:21.076482+0000","last_fullsized":"2024-05-09T08:02:21.076482+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:57:10.338510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.618392+0000","last_change":"2024-05-09T08:02:19.952790+0000","last_active":"2024-05-09T08:02:22.618392+0000","last_peered":"2024-05-09T08:02:22.618392+0000","last_clean":"2024-05-09T08:02:22.618392+0000","last_became_active":"2024-05-09T08:02:19.952378+0000","last_became_peered":"2024-05-09T08:02:19.952378+0000","last_unstale":"2024-05-09T08:02:22.618392+0000","last_undegraded":"2024-05-09T08:02:22.618392+0000","last_fullsized":"2024-05-09T08:02:22.618392+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:42:00.939587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:21.678285+0000","last_change":"2024-05-09T08:02:15.913011+0000","last_active":"2024-05-09T08:02:21.678285+0000","last_peered":"2024-05-09T08:02:21.678285+0000","last_clean":"2024-05-09T08:02:21.678285+0000","last_became_active":"2024-05-09T08:02:15.912590+0000","last_became_peered":"2024-05-09T08:02:15.912590+0000","last_unstale":"2024-05-09T08:02:21.678285+0000","last_undegraded":"2024-05-09T08:02:21.678285+0000","last_fullsized":"2024-05-09T08:02:21.678285+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:43:06.938904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"24'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.619767+0000","last_change":"2024-05-09T08:02:14.864763+0000","last_active":"2024-05-09T08:02:22.619767+0000","last_peered":"2024-05-09T08:02:22.619767+0000","last_clean":"2024-05-09T08:02:22.619767+0000","last_became_active":"2024-05-09T08:02:14.864470+0000","last_became_peered":"2024-05-09T08:02:14.864470+0000","last_unstale":"2024-05-09T08:02:22.619767+0000","last_undegraded":"2024-05-09T08:02:22.619767+0000","last_fullsized":"2024-05-09T08:02:22.619767+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:00:40.599475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.617800+0000","last_change":"2024-05-09T08:02:20.054896+0000","last_active":"2024-05-09T08:02:22.617800+0000","last_peered":"2024-05-09T08:02:22.617800+0000","last_clean":"2024-05-09T08:02:22.617800+0000","last_became_active":"2024-05-09T08:02:20.054424+0000","last_became_peered":"2024-05-09T08:02:20.054424+0000","last_unstale":"2024-05-09T08:02:22.617800+0000","last_undegraded":"2024-05-09T08:02:22.617800+0000","last_fullsized":"2024-05-09T08:02:22.617800+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:23:44.316545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:20.935333+0000","last_change":"2024-05-09T08:02:15.905448+0000","last_active":"2024-05-09T08:02:20.935333+0000","last_peered":"2024-05-09T08:02:20.935333+0000","last_clean":"2024-05-09T08:02:20.935333+0000","last_became_active":"2024-05-09T08:02:15.905131+0000","last_became_peered":"2024-05-09T08:02:15.905131+0000","last_unstale":"2024-05-09T08:02:20.935333+0000","last_undegraded":"2024-05-09T08:02:20.935333+0000","last_fullsized":"2024-05-09T08:02:20.935333+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:25:01.238816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"25'1","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:20.936562+0000","last_change":"2024-05-09T08:02:14.868410+0000","last_active":"2024-05-09T08:02:20.936562+0000","last_peered":"2024-05-09T08:02:20.936562+0000","last_clean":"2024-05-09T08:02:20.936562+0000","last_became_active":"2024-05-09T08:02:14.865621+0000","last_became_peered":"2024-05-09T08:02:14.865621+0000","last_unstale":"2024-05-09T08:02:20.936562+0000","last_undegraded":"2024-05-09T08:02:20.936562+0000","last_fullsized":"2024-05-09T08:02:20.936562+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:43:01.061801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:20.932915+0000","last_change":"2024-05-09T08:02:20.057603+0000","last_active":"2024-05-09T08:02:20.932915+0000","last_peered":"2024-05-09T08:02:20.932915+0000","last_clean":"2024-05-09T08:02:20.932915+0000","last_became_active":"2024-05-09T08:02:20.055365+0000","last_became_peered":"2024-05-09T08:02:20.055365+0000","last_unstale":"2024-05-09T08:02:20.932915+0000","last_undegraded":"2024-05-09T08:02:20.932915+0000","last_fullsized":"2024-05-09T08:02:20.932915+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:48:38.283562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:20.934935+0000","last_change":"2024-05-09T08:02:15.890980+0000","last_active":"2024-05-09T08:02:20.934935+0000","last_peered":"2024-05-09T08:02:20.934935+0000","last_clean":"2024-05-09T08:02:20.934935+0000","last_became_active":"2024-05-09T08:02:15.890346+0000","last_became_peered":"2024-05-09T08:02:15.890346+0000","last_unstale":"2024-05-09T08:02:20.934935+0000","last_undegraded":"2024-05-09T08:02:20.934935+0000","last_fullsized":"2024-05-09T08:02:20.934935+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:29:59.441314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:21.077241+0000","last_change":"2024-05-09T08:02:14.876206+0000","last_active":"2024-05-09T08:02:21.077241+0000","last_peered":"2024-05-09T08:02:21.077241+0000","last_clean":"2024-05-09T08:02:21.077241+0000","last_became_active":"2024-05-09T08:02:14.875894+0000","last_became_peered":"2024-05-09T08:02:14.875894+0000","last_unstale":"2024-05-09T08:02:21.077241+0000","last_undegraded":"2024-05-09T08:02:21.077241+0000","last_fullsized":"2024-05-09T08:02:21.077241+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:49:53.882513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:21.677241+0000","last_change":"2024-05-09T08:02:20.055222+0000","last_active":"2024-05-09T08:02:21.677241+0000","last_peered":"2024-05-09T08:02:21.677241+0000","last_clean":"2024-05-09T08:02:21.677241+0000","last_became_active":"2024-05-09T08:02:20.053802+0000","last_became_peered":"2024-05-09T08:02:20.053802+0000","last_unstale":"2024-05-09T08:02:21.677241+0000","last_undegraded":"2024-05-09T08:02:21.677241+0000","last_fullsized":"2024-05-09T08:02:21.677241+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:42:00.939587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.618451+0000","last_change":"2024-05-09T08:02:15.904528+0000","last_active":"2024-05-09T08:02:22.618451+0000","last_peered":"2024-05-09T08:02:22.618451+0000","last_clean":"2024-05-09T08:02:22.618451+0000","last_became_active":"2024-05-09T08:02:15.904049+0000","last_became_peered":"2024-05-09T08:02:15.904049+0000","last_unstale":"2024-05-09T08:02:22.618451+0000","last_undegraded":"2024-05-09T08:02:22.618451+0000","last_fullsized":"2024-05-09T08:02:22.618451+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:53:11.390395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:20.935040+0000","last_change":"2024-05-09T08:02:14.872897+0000","last_active":"2024-05-09T08:02:20.935040+0000","last_peered":"2024-05-09T08:02:20.935040+0000","last_clean":"2024-05-09T08:02:20.935040+0000","last_became_active":"2024-05-09T08:02:14.872096+0000","last_became_peered":"2024-05-09T08:02:14.872096+0000","last_unstale":"2024-05-09T08:02:20.935040+0000","last_undegraded":"2024-05-09T08:02:20.935040+0000","last_fullsized":"2024-05-09T08:02:20.935040+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:00:40.599475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:20.933696+0000","last_change":"2024-05-09T08:02:19.963716+0000","last_active":"2024-05-09T08:02:20.933696+0000","last_peered":"2024-05-09T08:02:20.933696+0000","last_clean":"2024-05-09T08:02:20.933696+0000","last_became_active":"2024-05-09T08:02:19.962922+0000","last_became_peered":"2024-05-09T08:02:19.962922+0000","last_unstale":"2024-05-09T08:02:20.933696+0000","last_undegraded":"2024-05-09T08:02:20.933696+0000","last_fullsized":"2024-05-09T08:02:20.933696+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:42:00.939587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.617918+0000","last_change":"2024-05-09T08:02:15.901450+0000","last_active":"2024-05-09T08:02:22.617918+0000","last_peered":"2024-05-09T08:02:22.617918+0000","last_clean":"2024-05-09T08:02:22.617918+0000","last_became_active":"2024-05-09T08:02:15.901180+0000","last_became_peered":"2024-05-09T08:02:15.901180+0000","last_unstale":"2024-05-09T08:02:22.617918+0000","last_undegraded":"2024-05-09T08:02:22.617918+0000","last_fullsized":"2024-05-09T08:02:22.617918+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:43:02.075442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.618076+0000","last_change":"2024-05-09T08:02:14.876518+0000","last_active":"2024-05-09T08:02:22.618076+0000","last_peered":"2024-05-09T08:02:22.618076+0000","last_clean":"2024-05-09T08:02:22.618076+0000","last_became_active":"2024-05-09T08:02:14.876175+0000","last_became_peered":"2024-05-09T08:02:14.876175+0000","last_unstale":"2024-05-09T08:02:22.618076+0000","last_undegraded":"2024-05-09T08:02:22.618076+0000","last_fullsized":"2024-05-09T08:02:22.618076+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:00:40.599475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:20.934897+0000","last_change":"2024-05-09T08:02:19.960940+0000","last_active":"2024-05-09T08:02:20.934897+0000","last_peered":"2024-05-09T08:02:20.934897+0000","last_clean":"2024-05-09T08:02:20.934897+0000","last_became_active":"2024-05-09T08:02:19.960496+0000","last_became_peered":"2024-05-09T08:02:19.960496+0000","last_unstale":"2024-05-09T08:02:20.934897+0000","last_undegraded":"2024-05-09T08:02:20.934897+0000","last_fullsized":"2024-05-09T08:02:20.934897+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:52:40.529638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:20.932615+0000","last_change":"2024-05-09T08:02:15.891774+0000","last_active":"2024-05-09T08:02:20.932615+0000","last_peered":"2024-05-09T08:02:20.932615+0000","last_clean":"2024-05-09T08:02:20.932615+0000","last_became_active":"2024-05-09T08:02:15.891407+0000","last_became_peered":"2024-05-09T08:02:15.891407+0000","last_unstale":"2024-05-09T08:02:20.932615+0000","last_undegraded":"2024-05-09T08:02:20.932615+0000","last_fullsized":"2024-05-09T08:02:20.932615+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:05:02.181367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.618768+0000","last_change":"2024-05-09T08:02:14.883655+0000","last_active":"2024-05-09T08:02:22.618768+0000","last_peered":"2024-05-09T08:02:22.618768+0000","last_clean":"2024-05-09T08:02:22.618768+0000","last_became_active":"2024-05-09T08:02:14.882359+0000","last_became_peered":"2024-05-09T08:02:14.882359+0000","last_unstale":"2024-05-09T08:02:22.618768+0000","last_undegraded":"2024-05-09T08:02:22.618768+0000","last_fullsized":"2024-05-09T08:02:22.618768+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:46:11.160815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:20.936081+0000","last_change":"2024-05-09T08:02:19.953319+0000","last_active":"2024-05-09T08:02:20.936081+0000","last_peered":"2024-05-09T08:02:20.936081+0000","last_clean":"2024-05-09T08:02:20.936081+0000","last_became_active":"2024-05-09T08:02:19.952740+0000","last_became_peered":"2024-05-09T08:02:19.952740+0000","last_unstale":"2024-05-09T08:02:20.936081+0000","last_undegraded":"2024-05-09T08:02:20.936081+0000","last_fullsized":"2024-05-09T08:02:20.936081+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:25:28.804048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:21.676635+0000","last_change":"2024-05-09T08:02:15.913828+0000","last_active":"2024-05-09T08:02:21.676635+0000","last_peered":"2024-05-09T08:02:21.676635+0000","last_clean":"2024-05-09T08:02:21.676635+0000","last_became_active":"2024-05-09T08:02:15.913313+0000","last_became_peered":"2024-05-09T08:02:15.913313+0000","last_unstale":"2024-05-09T08:02:21.676635+0000","last_undegraded":"2024-05-09T08:02:21.676635+0000","last_fullsized":"2024-05-09T08:02:21.676635+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:20:09.935394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:21.076590+0000","last_change":"2024-05-09T08:02:14.876438+0000","last_active":"2024-05-09T08:02:21.076590+0000","last_peered":"2024-05-09T08:02:21.076590+0000","last_clean":"2024-05-09T08:02:21.076590+0000","last_became_active":"2024-05-09T08:02:14.876146+0000","last_became_peered":"2024-05-09T08:02:14.876146+0000","last_unstale":"2024-05-09T08:02:21.076590+0000","last_undegraded":"2024-05-09T08:02:21.076590+0000","last_fullsized":"2024-05-09T08:02:21.076590+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:36:49.629991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:21.678130+0000","last_change":"2024-05-09T08:02:20.051067+0000","last_active":"2024-05-09T08:02:21.678130+0000","last_peered":"2024-05-09T08:02:21.678130+0000","last_clean":"2024-05-09T08:02:21.678130+0000","last_became_active":"2024-05-09T08:02:20.050593+0000","last_became_peered":"2024-05-09T08:02:20.050593+0000","last_unstale":"2024-05-09T08:02:21.678130+0000","last_undegraded":"2024-05-09T08:02:21.678130+0000","last_fullsized":"2024-05-09T08:02:21.678130+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:25:28.804048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:20.933046+0000","last_change":"2024-05-09T08:02:15.899137+0000","last_active":"2024-05-09T08:02:20.933046+0000","last_peered":"2024-05-09T08:02:20.933046+0000","last_clean":"2024-05-09T08:02:20.933046+0000","last_became_active":"2024-05-09T08:02:15.898766+0000","last_became_peered":"2024-05-09T08:02:15.898766+0000","last_unstale":"2024-05-09T08:02:20.933046+0000","last_undegraded":"2024-05-09T08:02:20.933046+0000","last_fullsized":"2024-05-09T08:02:20.933046+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:47:32.605288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.617934+0000","last_change":"2024-05-09T08:02:14.868850+0000","last_active":"2024-05-09T08:02:22.617934+0000","last_peered":"2024-05-09T08:02:22.617934+0000","last_clean":"2024-05-09T08:02:22.617934+0000","last_became_active":"2024-05-09T08:02:14.868384+0000","last_became_peered":"2024-05-09T08:02:14.868384+0000","last_unstale":"2024-05-09T08:02:22.617934+0000","last_undegraded":"2024-05-09T08:02:22.617934+0000","last_fullsized":"2024-05-09T08:02:22.617934+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:14:52.920972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":177,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:21.076748+0000","last_change":"2024-05-09T08:02:11.559804+0000","last_active":"2024-05-09T08:02:21.076748+0000","last_peered":"2024-05-09T08:02:21.076748+0000","last_clean":"2024-05-09T08:02:21.076748+0000","last_became_active":"2024-05-09T08:02:11.558761+0000","last_became_peered":"2024-05-09T08:02:11.558761+0000","last_unstale":"2024-05-09T08:02:21.076748+0000","last_undegraded":"2024-05-09T08:02:21.076748+0000","last_fullsized":"2024-05-09T08:02:21.076748+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:09.800143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:09.800143+0000","last_clean_scrub_stamp":"2024-05-09T08:02:09.800143+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:18:35.964149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.618755+0000","last_change":"2024-05-09T08:02:20.054746+0000","last_active":"2024-05-09T08:02:22.618755+0000","last_peered":"2024-05-09T08:02:22.618755+0000","last_clean":"2024-05-09T08:02:22.618755+0000","last_became_active":"2024-05-09T08:02:20.054186+0000","last_became_peered":"2024-05-09T08:02:20.054186+0000","last_unstale":"2024-05-09T08:02:22.618755+0000","last_undegraded":"2024-05-09T08:02:22.618755+0000","last_fullsized":"2024-05-09T08:02:22.618755+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:53:15.458120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:21.076861+0000","last_change":"2024-05-09T08:02:15.889657+0000","last_active":"2024-05-09T08:02:21.076861+0000","last_peered":"2024-05-09T08:02:21.076861+0000","last_clean":"2024-05-09T08:02:21.076861+0000","last_became_active":"2024-05-09T08:02:15.889381+0000","last_became_peered":"2024-05-09T08:02:15.889381+0000","last_unstale":"2024-05-09T08:02:21.076861+0000","last_undegraded":"2024-05-09T08:02:21.076861+0000","last_fullsized":"2024-05-09T08:02:21.076861+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:59:14.957806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.619305+0000","last_change":"2024-05-09T08:02:14.868892+0000","last_active":"2024-05-09T08:02:22.619305+0000","last_peered":"2024-05-09T08:02:22.619305+0000","last_clean":"2024-05-09T08:02:22.619305+0000","last_became_active":"2024-05-09T08:02:14.868513+0000","last_became_peered":"2024-05-09T08:02:14.868513+0000","last_unstale":"2024-05-09T08:02:22.619305+0000","last_undegraded":"2024-05-09T08:02:22.619305+0000","last_fullsized":"2024-05-09T08:02:22.619305+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:19:31.913236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.618654+0000","last_change":"2024-05-09T08:02:20.053097+0000","last_active":"2024-05-09T08:02:22.618654+0000","last_peered":"2024-05-09T08:02:22.618654+0000","last_clean":"2024-05-09T08:02:22.618654+0000","last_became_active":"2024-05-09T08:02:20.052653+0000","last_became_peered":"2024-05-09T08:02:20.052653+0000","last_unstale":"2024-05-09T08:02:22.618654+0000","last_undegraded":"2024-05-09T08:02:22.618654+0000","last_fullsized":"2024-05-09T08:02:22.618654+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:25:05.306541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.618299+0000","last_change":"2024-05-09T08:02:15.899059+0000","last_active":"2024-05-09T08:02:22.618299+0000","last_peered":"2024-05-09T08:02:22.618299+0000","last_clean":"2024-05-09T08:02:22.618299+0000","last_became_active":"2024-05-09T08:02:15.898692+0000","last_became_peered":"2024-05-09T08:02:15.898692+0000","last_unstale":"2024-05-09T08:02:22.618299+0000","last_undegraded":"2024-05-09T08:02:22.618299+0000","last_fullsized":"2024-05-09T08:02:22.618299+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:52:36.461913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"24'1","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:20.935331+0000","last_change":"2024-05-09T08:02:14.873071+0000","last_active":"2024-05-09T08:02:20.935331+0000","last_peered":"2024-05-09T08:02:20.935331+0000","last_clean":"2024-05-09T08:02:20.935331+0000","last_became_active":"2024-05-09T08:02:14.872556+0000","last_became_peered":"2024-05-09T08:02:14.872556+0000","last_unstale":"2024-05-09T08:02:20.935331+0000","last_undegraded":"2024-05-09T08:02:20.935331+0000","last_fullsized":"2024-05-09T08:02:20.935331+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:57:06.093718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:21.677388+0000","last_change":"2024-05-09T08:02:19.960414+0000","last_active":"2024-05-09T08:02:21.677388+0000","last_peered":"2024-05-09T08:02:21.677388+0000","last_clean":"2024-05-09T08:02:21.677388+0000","last_became_active":"2024-05-09T08:02:19.957941+0000","last_became_peered":"2024-05-09T08:02:19.957941+0000","last_unstale":"2024-05-09T08:02:21.677388+0000","last_undegraded":"2024-05-09T08:02:21.677388+0000","last_fullsized":"2024-05-09T08:02:21.677388+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:41:31.470546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:21.077406+0000","last_change":"2024-05-09T08:02:15.884395+0000","last_active":"2024-05-09T08:02:21.077406+0000","last_peered":"2024-05-09T08:02:21.077406+0000","last_clean":"2024-05-09T08:02:21.077406+0000","last_became_active":"2024-05-09T08:02:15.883788+0000","last_became_peered":"2024-05-09T08:02:15.883788+0000","last_unstale":"2024-05-09T08:02:21.077406+0000","last_undegraded":"2024-05-09T08:02:21.077406+0000","last_fullsized":"2024-05-09T08:02:21.077406+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:48:34.215837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"24'2","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.619345+0000","last_change":"2024-05-09T08:02:14.885044+0000","last_active":"2024-05-09T08:02:22.619345+0000","last_peered":"2024-05-09T08:02:22.619345+0000","last_clean":"2024-05-09T08:02:22.619345+0000","last_became_active":"2024-05-09T08:02:14.884706+0000","last_became_peered":"2024-05-09T08:02:14.884706+0000","last_unstale":"2024-05-09T08:02:22.619345+0000","last_undegraded":"2024-05-09T08:02:22.619345+0000","last_fullsized":"2024-05-09T08:02:22.619345+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:14:52.920972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.617383+0000","last_change":"2024-05-09T08:02:20.050865+0000","last_active":"2024-05-09T08:02:22.617383+0000","last_peered":"2024-05-09T08:02:22.617383+0000","last_clean":"2024-05-09T08:02:22.617383+0000","last_became_active":"2024-05-09T08:02:20.050270+0000","last_became_peered":"2024-05-09T08:02:20.050270+0000","last_unstale":"2024-05-09T08:02:22.617383+0000","last_undegraded":"2024-05-09T08:02:22.617383+0000","last_fullsized":"2024-05-09T08:02:22.617383+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:48:38.283562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:21.077190+0000","last_change":"2024-05-09T08:02:15.884737+0000","last_active":"2024-05-09T08:02:21.077190+0000","last_peered":"2024-05-09T08:02:21.077190+0000","last_clean":"2024-05-09T08:02:21.077190+0000","last_became_active":"2024-05-09T08:02:15.884465+0000","last_became_peered":"2024-05-09T08:02:15.884465+0000","last_unstale":"2024-05-09T08:02:21.077190+0000","last_undegraded":"2024-05-09T08:02:21.077190+0000","last_fullsized":"2024-05-09T08:02:21.077190+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:52:36.461913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"24'1","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.618658+0000","last_change":"2024-05-09T08:02:14.875003+0000","last_active":"2024-05-09T08:02:22.618658+0000","last_peered":"2024-05-09T08:02:22.618658+0000","last_clean":"2024-05-09T08:02:22.618658+0000","last_became_active":"2024-05-09T08:02:14.874665+0000","last_became_peered":"2024-05-09T08:02:14.874665+0000","last_unstale":"2024-05-09T08:02:22.618658+0000","last_undegraded":"2024-05-09T08:02:22.618658+0000","last_fullsized":"2024-05-09T08:02:22.618658+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:18:39.991319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.617254+0000","last_change":"2024-05-09T08:02:19.952692+0000","last_active":"2024-05-09T08:02:22.617254+0000","last_peered":"2024-05-09T08:02:22.617254+0000","last_clean":"2024-05-09T08:02:22.617254+0000","last_became_active":"2024-05-09T08:02:19.952224+0000","last_became_peered":"2024-05-09T08:02:19.952224+0000","last_unstale":"2024-05-09T08:02:22.617254+0000","last_undegraded":"2024-05-09T08:02:22.617254+0000","last_fullsized":"2024-05-09T08:02:22.617254+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:47:36.673013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:21.076666+0000","last_change":"2024-05-09T08:02:15.883408+0000","last_active":"2024-05-09T08:02:21.076666+0000","last_peered":"2024-05-09T08:02:21.076666+0000","last_clean":"2024-05-09T08:02:21.076666+0000","last_became_active":"2024-05-09T08:02:15.883103+0000","last_became_peered":"2024-05-09T08:02:15.883103+0000","last_unstale":"2024-05-09T08:02:21.076666+0000","last_undegraded":"2024-05-09T08:02:21.076666+0000","last_fullsized":"2024-05-09T08:02:21.076666+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:53:11.390395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"24'8","reported_seq":29,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.618529+0000","last_change":"2024-05-09T08:02:14.873184+0000","last_active":"2024-05-09T08:02:22.618529+0000","last_peered":"2024-05-09T08:02:22.618529+0000","last_clean":"2024-05-09T08:02:22.618529+0000","last_became_active":"2024-05-09T08:02:14.871055+0000","last_became_peered":"2024-05-09T08:02:14.871055+0000","last_unstale":"2024-05-09T08:02:22.618529+0000","last_undegraded":"2024-05-09T08:02:22.618529+0000","last_fullsized":"2024-05-09T08:02:22.618529+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:19:56.315972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:21.678092+0000","last_change":"2024-05-09T08:02:19.950823+0000","last_active":"2024-05-09T08:02:21.678092+0000","last_peered":"2024-05-09T08:02:21.678092+0000","last_clean":"2024-05-09T08:02:21.678092+0000","last_became_active":"2024-05-09T08:02:19.950352+0000","last_became_peered":"2024-05-09T08:02:19.950352+0000","last_unstale":"2024-05-09T08:02:21.678092+0000","last_undegraded":"2024-05-09T08:02:21.678092+0000","last_fullsized":"2024-05-09T08:02:21.678092+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:59:19.025530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.619335+0000","last_change":"2024-05-09T08:02:15.899115+0000","last_active":"2024-05-09T08:02:22.619335+0000","last_peered":"2024-05-09T08:02:22.619335+0000","last_clean":"2024-05-09T08:02:22.619335+0000","last_became_active":"2024-05-09T08:02:15.898811+0000","last_became_peered":"2024-05-09T08:02:15.898811+0000","last_unstale":"2024-05-09T08:02:22.619335+0000","last_undegraded":"2024-05-09T08:02:22.619335+0000","last_fullsized":"2024-05-09T08:02:22.619335+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:43:06.938904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.619023+0000","last_change":"2024-05-09T08:02:14.860818+0000","last_active":"2024-05-09T08:02:22.619023+0000","last_peered":"2024-05-09T08:02:22.619023+0000","last_clean":"2024-05-09T08:02:22.619023+0000","last_became_active":"2024-05-09T08:02:14.860518+0000","last_became_peered":"2024-05-09T08:02:14.860518+0000","last_unstale":"2024-05-09T08:02:22.619023+0000","last_undegraded":"2024-05-09T08:02:22.619023+0000","last_fullsized":"2024-05-09T08:02:22.619023+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:25:23.306827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:20.935214+0000","last_change":"2024-05-09T08:02:20.053251+0000","last_active":"2024-05-09T08:02:20.935214+0000","last_peered":"2024-05-09T08:02:20.935214+0000","last_clean":"2024-05-09T08:02:20.935214+0000","last_became_active":"2024-05-09T08:02:20.052783+0000","last_became_peered":"2024-05-09T08:02:20.052783+0000","last_unstale":"2024-05-09T08:02:20.935214+0000","last_undegraded":"2024-05-09T08:02:20.935214+0000","last_fullsized":"2024-05-09T08:02:20.935214+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:16:41.863341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:21.076854+0000","last_change":"2024-05-09T08:02:15.883972+0000","last_active":"2024-05-09T08:02:21.076854+0000","last_peered":"2024-05-09T08:02:21.076854+0000","last_clean":"2024-05-09T08:02:21.076854+0000","last_became_active":"2024-05-09T08:02:15.883613+0000","last_became_peered":"2024-05-09T08:02:15.883613+0000","last_unstale":"2024-05-09T08:02:21.076854+0000","last_undegraded":"2024-05-09T08:02:21.076854+0000","last_fullsized":"2024-05-09T08:02:21.076854+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:41:27.402821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.618038+0000","last_change":"2024-05-09T08:02:14.883366+0000","last_active":"2024-05-09T08:02:22.618038+0000","last_peered":"2024-05-09T08:02:22.618038+0000","last_clean":"2024-05-09T08:02:22.618038+0000","last_became_active":"2024-05-09T08:02:14.883135+0000","last_became_peered":"2024-05-09T08:02:14.883135+0000","last_unstale":"2024-05-09T08:02:22.618038+0000","last_undegraded":"2024-05-09T08:02:22.618038+0000","last_fullsized":"2024-05-09T08:02:22.618038+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:32:48.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:20.935375+0000","last_change":"2024-05-09T08:02:20.050888+0000","last_active":"2024-05-09T08:02:20.935375+0000","last_peered":"2024-05-09T08:02:20.935375+0000","last_clean":"2024-05-09T08:02:20.935375+0000","last_became_active":"2024-05-09T08:02:20.050390+0000","last_became_peered":"2024-05-09T08:02:20.050390+0000","last_unstale":"2024-05-09T08:02:20.935375+0000","last_undegraded":"2024-05-09T08:02:20.935375+0000","last_fullsized":"2024-05-09T08:02:20.935375+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:10:24.797526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:21.677358+0000","last_change":"2024-05-09T08:02:15.899096+0000","last_active":"2024-05-09T08:02:21.677358+0000","last_peered":"2024-05-09T08:02:21.677358+0000","last_clean":"2024-05-09T08:02:21.677358+0000","last_became_active":"2024-05-09T08:02:15.898811+0000","last_became_peered":"2024-05-09T08:02:15.898811+0000","last_unstale":"2024-05-09T08:02:21.677358+0000","last_undegraded":"2024-05-09T08:02:21.677358+0000","last_fullsized":"2024-05-09T08:02:21.677358+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:28:51.229262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"24'2","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:21.677657+0000","last_change":"2024-05-09T08:02:14.868657+0000","last_active":"2024-05-09T08:02:21.677657+0000","last_peered":"2024-05-09T08:02:21.677657+0000","last_clean":"2024-05-09T08:02:21.677657+0000","last_became_active":"2024-05-09T08:02:14.868385+0000","last_became_peered":"2024-05-09T08:02:14.868385+0000","last_unstale":"2024-05-09T08:02:21.677657+0000","last_undegraded":"2024-05-09T08:02:21.677657+0000","last_fullsized":"2024-05-09T08:02:21.677657+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:46:11.160815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.618792+0000","last_change":"2024-05-09T08:02:20.053155+0000","last_active":"2024-05-09T08:02:22.618792+0000","last_peered":"2024-05-09T08:02:22.618792+0000","last_clean":"2024-05-09T08:02:22.618792+0000","last_became_active":"2024-05-09T08:02:20.052629+0000","last_became_peered":"2024-05-09T08:02:20.052629+0000","last_unstale":"2024-05-09T08:02:22.618792+0000","last_undegraded":"2024-05-09T08:02:22.618792+0000","last_fullsized":"2024-05-09T08:02:22.618792+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:35:54.628874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:21.076864+0000","last_change":"2024-05-09T08:02:15.905406+0000","last_active":"2024-05-09T08:02:21.076864+0000","last_peered":"2024-05-09T08:02:21.076864+0000","last_clean":"2024-05-09T08:02:21.076864+0000","last_became_active":"2024-05-09T08:02:15.905076+0000","last_became_peered":"2024-05-09T08:02:15.905076+0000","last_unstale":"2024-05-09T08:02:21.076864+0000","last_undegraded":"2024-05-09T08:02:21.076864+0000","last_fullsized":"2024-05-09T08:02:21.076864+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:25:01.238816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"25'10","reported_seq":32,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:39.803593+0000","last_change":"2024-05-09T08:02:14.880410+0000","last_active":"2024-05-09T08:02:39.803593+0000","last_peered":"2024-05-09T08:02:39.803593+0000","last_clean":"2024-05-09T08:02:39.803593+0000","last_became_active":"2024-05-09T08:02:14.880045+0000","last_became_peered":"2024-05-09T08:02:14.880045+0000","last_unstale":"2024-05-09T08:02:39.803593+0000","last_undegraded":"2024-05-09T08:02:39.803593+0000","last_fullsized":"2024-05-09T08:02:39.803593+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:19:56.315972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:20.934006+0000","last_change":"2024-05-09T08:02:19.963378+0000","last_active":"2024-05-09T08:02:20.934006+0000","last_peered":"2024-05-09T08:02:20.934006+0000","last_clean":"2024-05-09T08:02:20.934006+0000","last_became_active":"2024-05-09T08:02:19.958964+0000","last_became_peered":"2024-05-09T08:02:19.958964+0000","last_unstale":"2024-05-09T08:02:20.934006+0000","last_undegraded":"2024-05-09T08:02:20.934006+0000","last_fullsized":"2024-05-09T08:02:20.934006+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:41:31.470546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.618268+0000","last_change":"2024-05-09T08:02:15.888428+0000","last_active":"2024-05-09T08:02:22.618268+0000","last_peered":"2024-05-09T08:02:22.618268+0000","last_clean":"2024-05-09T08:02:22.618268+0000","last_became_active":"2024-05-09T08:02:15.888071+0000","last_became_peered":"2024-05-09T08:02:15.888071+0000","last_unstale":"2024-05-09T08:02:22.618268+0000","last_undegraded":"2024-05-09T08:02:22.618268+0000","last_fullsized":"2024-05-09T08:02:22.618268+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:36:50.643632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"24'3","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:20.935171+0000","last_change":"2024-05-09T08:02:14.872697+0000","last_active":"2024-05-09T08:02:20.935171+0000","last_peered":"2024-05-09T08:02:20.935171+0000","last_clean":"2024-05-09T08:02:20.935171+0000","last_became_active":"2024-05-09T08:02:14.871945+0000","last_became_peered":"2024-05-09T08:02:14.871945+0000","last_unstale":"2024-05-09T08:02:20.935171+0000","last_undegraded":"2024-05-09T08:02:20.935171+0000","last_fullsized":"2024-05-09T08:02:20.935171+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:19:56.315972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.617683+0000","last_change":"2024-05-09T08:02:19.953635+0000","last_active":"2024-05-09T08:02:22.617683+0000","last_peered":"2024-05-09T08:02:22.617683+0000","last_clean":"2024-05-09T08:02:22.617683+0000","last_became_active":"2024-05-09T08:02:19.953166+0000","last_became_peered":"2024-05-09T08:02:19.953166+0000","last_unstale":"2024-05-09T08:02:22.617683+0000","last_undegraded":"2024-05-09T08:02:22.617683+0000","last_fullsized":"2024-05-09T08:02:22.617683+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:18:40.377352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:21.677885+0000","last_change":"2024-05-09T08:02:15.905285+0000","last_active":"2024-05-09T08:02:21.677885+0000","last_peered":"2024-05-09T08:02:21.677885+0000","last_clean":"2024-05-09T08:02:21.677885+0000","last_became_active":"2024-05-09T08:02:15.904898+0000","last_became_peered":"2024-05-09T08:02:15.904898+0000","last_unstale":"2024-05-09T08:02:21.677885+0000","last_undegraded":"2024-05-09T08:02:21.677885+0000","last_fullsized":"2024-05-09T08:02:21.677885+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:47:32.605288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.619004+0000","last_change":"2024-05-09T08:02:14.868682+0000","last_active":"2024-05-09T08:02:22.619004+0000","last_peered":"2024-05-09T08:02:22.619004+0000","last_clean":"2024-05-09T08:02:22.619004+0000","last_became_active":"2024-05-09T08:02:14.868294+0000","last_became_peered":"2024-05-09T08:02:14.868294+0000","last_unstale":"2024-05-09T08:02:22.619004+0000","last_undegraded":"2024-05-09T08:02:22.619004+0000","last_fullsized":"2024-05-09T08:02:22.619004+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:57:06.093718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:20.933284+0000","last_change":"2024-05-09T08:02:19.956115+0000","last_active":"2024-05-09T08:02:20.933284+0000","last_peered":"2024-05-09T08:02:20.933284+0000","last_clean":"2024-05-09T08:02:20.933284+0000","last_became_active":"2024-05-09T08:02:19.955662+0000","last_became_peered":"2024-05-09T08:02:19.955662+0000","last_unstale":"2024-05-09T08:02:20.933284+0000","last_undegraded":"2024-05-09T08:02:20.933284+0000","last_fullsized":"2024-05-09T08:02:20.933284+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:15:28.286330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.618890+0000","last_change":"2024-05-09T08:02:15.888406+0000","last_active":"2024-05-09T08:02:22.618890+0000","last_peered":"2024-05-09T08:02:22.618890+0000","last_clean":"2024-05-09T08:02:22.618890+0000","last_became_active":"2024-05-09T08:02:15.888090+0000","last_became_peered":"2024-05-09T08:02:15.888090+0000","last_unstale":"2024-05-09T08:02:22.618890+0000","last_undegraded":"2024-05-09T08:02:22.618890+0000","last_fullsized":"2024-05-09T08:02:22.618890+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:53:23.178610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.619656+0000","last_change":"2024-05-09T08:02:14.862463+0000","last_active":"2024-05-09T08:02:22.619656+0000","last_peered":"2024-05-09T08:02:22.619656+0000","last_clean":"2024-05-09T08:02:22.619656+0000","last_became_active":"2024-05-09T08:02:14.862104+0000","last_became_peered":"2024-05-09T08:02:14.862104+0000","last_unstale":"2024-05-09T08:02:22.619656+0000","last_undegraded":"2024-05-09T08:02:22.619656+0000","last_fullsized":"2024-05-09T08:02:22.619656+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:32:48.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:20.932952+0000","last_change":"2024-05-09T08:02:20.053600+0000","last_active":"2024-05-09T08:02:20.932952+0000","last_peered":"2024-05-09T08:02:20.932952+0000","last_clean":"2024-05-09T08:02:20.932952+0000","last_became_active":"2024-05-09T08:02:20.052485+0000","last_became_peered":"2024-05-09T08:02:20.052485+0000","last_unstale":"2024-05-09T08:02:20.932952+0000","last_undegraded":"2024-05-09T08:02:20.932952+0000","last_fullsized":"2024-05-09T08:02:20.932952+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:35:54.628874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:21.677902+0000","last_change":"2024-05-09T08:02:20.054016+0000","last_active":"2024-05-09T08:02:21.677902+0000","last_peered":"2024-05-09T08:02:21.677902+0000","last_clean":"2024-05-09T08:02:21.677902+0000","last_became_active":"2024-05-09T08:02:20.053250+0000","last_became_peered":"2024-05-09T08:02:20.053250+0000","last_unstale":"2024-05-09T08:02:21.677902+0000","last_undegraded":"2024-05-09T08:02:21.677902+0000","last_fullsized":"2024-05-09T08:02:21.677902+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:52:40.529638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:20.932409+0000","last_change":"2024-05-09T08:02:15.907080+0000","last_active":"2024-05-09T08:02:20.932409+0000","last_peered":"2024-05-09T08:02:20.932409+0000","last_clean":"2024-05-09T08:02:20.932409+0000","last_became_active":"2024-05-09T08:02:15.906600+0000","last_became_peered":"2024-05-09T08:02:15.906600+0000","last_unstale":"2024-05-09T08:02:20.932409+0000","last_undegraded":"2024-05-09T08:02:20.932409+0000","last_fullsized":"2024-05-09T08:02:20.932409+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:43:06.938904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.617668+0000","last_change":"2024-05-09T08:02:14.870990+0000","last_active":"2024-05-09T08:02:22.617668+0000","last_peered":"2024-05-09T08:02:22.617668+0000","last_clean":"2024-05-09T08:02:22.617668+0000","last_became_active":"2024-05-09T08:02:14.869818+0000","last_became_peered":"2024-05-09T08:02:14.869818+0000","last_unstale":"2024-05-09T08:02:22.617668+0000","last_undegraded":"2024-05-09T08:02:22.617668+0000","last_fullsized":"2024-05-09T08:02:22.617668+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:25:23.306827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.619498+0000","last_change":"2024-05-09T08:02:19.952582+0000","last_active":"2024-05-09T08:02:22.619498+0000","last_peered":"2024-05-09T08:02:22.619498+0000","last_clean":"2024-05-09T08:02:22.619498+0000","last_became_active":"2024-05-09T08:02:19.952117+0000","last_became_peered":"2024-05-09T08:02:19.952117+0000","last_unstale":"2024-05-09T08:02:22.619498+0000","last_undegraded":"2024-05-09T08:02:22.619498+0000","last_fullsized":"2024-05-09T08:02:22.619498+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:42:00.939587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:20.933300+0000","last_change":"2024-05-09T08:02:15.913109+0000","last_active":"2024-05-09T08:02:20.933300+0000","last_peered":"2024-05-09T08:02:20.933300+0000","last_clean":"2024-05-09T08:02:20.933300+0000","last_became_active":"2024-05-09T08:02:15.912883+0000","last_became_peered":"2024-05-09T08:02:15.912883+0000","last_unstale":"2024-05-09T08:02:20.933300+0000","last_undegraded":"2024-05-09T08:02:20.933300+0000","last_fullsized":"2024-05-09T08:02:20.933300+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:29:59.441314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:20.933233+0000","last_change":"2024-05-09T08:02:14.885256+0000","last_active":"2024-05-09T08:02:20.933233+0000","last_peered":"2024-05-09T08:02:20.933233+0000","last_clean":"2024-05-09T08:02:20.933233+0000","last_became_active":"2024-05-09T08:02:14.884965+0000","last_became_peered":"2024-05-09T08:02:14.884965+0000","last_unstale":"2024-05-09T08:02:20.933233+0000","last_undegraded":"2024-05-09T08:02:20.933233+0000","last_fullsized":"2024-05-09T08:02:20.933233+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:14:52.920972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"24'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.618840+0000","last_change":"2024-05-09T08:02:14.860339+0000","last_active":"2024-05-09T08:02:22.618840+0000","last_peered":"2024-05-09T08:02:22.618840+0000","last_clean":"2024-05-09T08:02:22.618840+0000","last_became_active":"2024-05-09T08:02:14.859917+0000","last_became_peered":"2024-05-09T08:02:14.859917+0000","last_unstale":"2024-05-09T08:02:22.618840+0000","last_undegraded":"2024-05-09T08:02:22.618840+0000","last_fullsized":"2024-05-09T08:02:22.618840+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:14:52.920972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:21.677460+0000","last_change":"2024-05-09T08:02:15.897001+0000","last_active":"2024-05-09T08:02:21.677460+0000","last_peered":"2024-05-09T08:02:21.677460+0000","last_clean":"2024-05-09T08:02:21.677460+0000","last_became_active":"2024-05-09T08:02:15.896708+0000","last_became_peered":"2024-05-09T08:02:15.896708+0000","last_unstale":"2024-05-09T08:02:21.677460+0000","last_undegraded":"2024-05-09T08:02:21.677460+0000","last_fullsized":"2024-05-09T08:02:21.677460+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:29:59.441314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.619069+0000","last_change":"2024-05-09T08:02:19.955369+0000","last_active":"2024-05-09T08:02:22.619069+0000","last_peered":"2024-05-09T08:02:22.619069+0000","last_clean":"2024-05-09T08:02:22.619069+0000","last_became_active":"2024-05-09T08:02:19.954829+0000","last_became_peered":"2024-05-09T08:02:19.954829+0000","last_unstale":"2024-05-09T08:02:22.619069+0000","last_undegraded":"2024-05-09T08:02:22.619069+0000","last_fullsized":"2024-05-09T08:02:22.619069+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:49:58.963879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"24'1","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:20.934324+0000","last_change":"2024-05-09T08:02:14.883856+0000","last_active":"2024-05-09T08:02:20.934324+0000","last_peered":"2024-05-09T08:02:20.934324+0000","last_clean":"2024-05-09T08:02:20.934324+0000","last_became_active":"2024-05-09T08:02:14.882610+0000","last_became_peered":"2024-05-09T08:02:14.882610+0000","last_unstale":"2024-05-09T08:02:20.934324+0000","last_undegraded":"2024-05-09T08:02:20.934324+0000","last_fullsized":"2024-05-09T08:02:20.934324+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:19:31.913236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.617629+0000","last_change":"2024-05-09T08:02:15.889974+0000","last_active":"2024-05-09T08:02:22.617629+0000","last_peered":"2024-05-09T08:02:22.617629+0000","last_clean":"2024-05-09T08:02:22.617629+0000","last_became_active":"2024-05-09T08:02:15.889650+0000","last_became_peered":"2024-05-09T08:02:15.889650+0000","last_unstale":"2024-05-09T08:02:22.617629+0000","last_undegraded":"2024-05-09T08:02:22.617629+0000","last_fullsized":"2024-05-09T08:02:22.617629+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:00:41.613116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:20.934556+0000","last_change":"2024-05-09T08:02:19.955362+0000","last_active":"2024-05-09T08:02:20.934556+0000","last_peered":"2024-05-09T08:02:20.934556+0000","last_clean":"2024-05-09T08:02:20.934556+0000","last_became_active":"2024-05-09T08:02:19.954947+0000","last_became_peered":"2024-05-09T08:02:19.954947+0000","last_unstale":"2024-05-09T08:02:20.934556+0000","last_undegraded":"2024-05-09T08:02:20.934556+0000","last_fullsized":"2024-05-09T08:02:20.934556+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:43:30.549966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.618982+0000","last_change":"2024-05-09T08:02:15.909379+0000","last_active":"2024-05-09T08:02:22.618982+0000","last_peered":"2024-05-09T08:02:22.618982+0000","last_clean":"2024-05-09T08:02:22.618982+0000","last_became_active":"2024-05-09T08:02:15.909086+0000","last_became_peered":"2024-05-09T08:02:15.909086+0000","last_unstale":"2024-05-09T08:02:22.618982+0000","last_undegraded":"2024-05-09T08:02:22.618982+0000","last_fullsized":"2024-05-09T08:02:22.618982+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:05:02.181367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.618963+0000","last_change":"2024-05-09T08:02:15.891242+0000","last_active":"2024-05-09T08:02:22.618963+0000","last_peered":"2024-05-09T08:02:22.618963+0000","last_clean":"2024-05-09T08:02:22.618963+0000","last_became_active":"2024-05-09T08:02:15.890640+0000","last_became_peered":"2024-05-09T08:02:15.890640+0000","last_unstale":"2024-05-09T08:02:22.618963+0000","last_undegraded":"2024-05-09T08:02:22.618963+0000","last_fullsized":"2024-05-09T08:02:22.618963+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:57:07.107359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.617950+0000","last_change":"2024-05-09T08:02:15.899001+0000","last_active":"2024-05-09T08:02:22.617950+0000","last_peered":"2024-05-09T08:02:22.617950+0000","last_clean":"2024-05-09T08:02:22.617950+0000","last_became_active":"2024-05-09T08:02:15.898695+0000","last_became_peered":"2024-05-09T08:02:15.898695+0000","last_unstale":"2024-05-09T08:02:22.617950+0000","last_undegraded":"2024-05-09T08:02:22.617950+0000","last_fullsized":"2024-05-09T08:02:22.617950+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:41:27.402821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:21.677395+0000","last_change":"2024-05-09T08:02:15.907374+0000","last_active":"2024-05-09T08:02:21.677395+0000","last_peered":"2024-05-09T08:02:21.677395+0000","last_clean":"2024-05-09T08:02:21.677395+0000","last_became_active":"2024-05-09T08:02:15.906767+0000","last_became_peered":"2024-05-09T08:02:15.906767+0000","last_unstale":"2024-05-09T08:02:21.677395+0000","last_undegraded":"2024-05-09T08:02:21.677395+0000","last_fullsized":"2024-05-09T08:02:21.677395+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:05:02.181367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:20.935527+0000","last_change":"2024-05-09T08:02:15.905326+0000","last_active":"2024-05-09T08:02:20.935527+0000","last_peered":"2024-05-09T08:02:20.935527+0000","last_clean":"2024-05-09T08:02:20.935527+0000","last_became_active":"2024-05-09T08:02:15.905083+0000","last_became_peered":"2024-05-09T08:02:15.905083+0000","last_unstale":"2024-05-09T08:02:20.935527+0000","last_undegraded":"2024-05-09T08:02:20.935527+0000","last_fullsized":"2024-05-09T08:02:20.935527+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:23:40.248821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:20.934866+0000","last_change":"2024-05-09T08:02:15.907753+0000","last_active":"2024-05-09T08:02:20.934866+0000","last_peered":"2024-05-09T08:02:20.934866+0000","last_clean":"2024-05-09T08:02:20.934866+0000","last_became_active":"2024-05-09T08:02:15.907480+0000","last_became_peered":"2024-05-09T08:02:15.907480+0000","last_unstale":"2024-05-09T08:02:20.934866+0000","last_undegraded":"2024-05-09T08:02:20.934866+0000","last_fullsized":"2024-05-09T08:02:20.934866+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:20:07.368327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.619474+0000","last_change":"2024-05-09T08:02:15.888605+0000","last_active":"2024-05-09T08:02:22.619474+0000","last_peered":"2024-05-09T08:02:22.619474+0000","last_clean":"2024-05-09T08:02:22.619474+0000","last_became_active":"2024-05-09T08:02:15.887981+0000","last_became_peered":"2024-05-09T08:02:15.887981+0000","last_unstale":"2024-05-09T08:02:22.619474+0000","last_undegraded":"2024-05-09T08:02:22.619474+0000","last_fullsized":"2024-05-09T08:02:22.619474+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:47:32.605288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:20.935282+0000","last_change":"2024-05-09T08:02:15.904744+0000","last_active":"2024-05-09T08:02:20.935282+0000","last_peered":"2024-05-09T08:02:20.935282+0000","last_clean":"2024-05-09T08:02:20.935282+0000","last_became_active":"2024-05-09T08:02:15.904334+0000","last_became_peered":"2024-05-09T08:02:15.904334+0000","last_unstale":"2024-05-09T08:02:20.935282+0000","last_undegraded":"2024-05-09T08:02:20.935282+0000","last_fullsized":"2024-05-09T08:02:20.935282+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:30:33.351158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.617634+0000","last_change":"2024-05-09T08:02:15.904835+0000","last_active":"2024-05-09T08:02:22.617634+0000","last_peered":"2024-05-09T08:02:22.617634+0000","last_clean":"2024-05-09T08:02:22.617634+0000","last_became_active":"2024-05-09T08:02:15.904416+0000","last_became_peered":"2024-05-09T08:02:15.904416+0000","last_unstale":"2024-05-09T08:02:22.617634+0000","last_undegraded":"2024-05-09T08:02:22.617634+0000","last_fullsized":"2024-05-09T08:02:22.617634+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:47:32.605288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:20.933091+0000","last_change":"2024-05-09T08:02:15.898324+0000","last_active":"2024-05-09T08:02:20.933091+0000","last_peered":"2024-05-09T08:02:20.933091+0000","last_clean":"2024-05-09T08:02:20.933091+0000","last_became_active":"2024-05-09T08:02:15.897981+0000","last_became_peered":"2024-05-09T08:02:15.897981+0000","last_unstale":"2024-05-09T08:02:20.933091+0000","last_undegraded":"2024-05-09T08:02:20.933091+0000","last_fullsized":"2024-05-09T08:02:20.933091+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:01:50.341133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.617909+0000","last_change":"2024-05-09T08:02:15.890945+0000","last_active":"2024-05-09T08:02:22.617909+0000","last_peered":"2024-05-09T08:02:22.617909+0000","last_clean":"2024-05-09T08:02:22.617909+0000","last_became_active":"2024-05-09T08:02:15.890647+0000","last_became_peered":"2024-05-09T08:02:15.890647+0000","last_unstale":"2024-05-09T08:02:22.617909+0000","last_undegraded":"2024-05-09T08:02:22.617909+0000","last_fullsized":"2024-05-09T08:02:22.617909+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:41:56.871863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:21.677105+0000","last_change":"2024-05-09T08:02:15.913728+0000","last_active":"2024-05-09T08:02:21.677105+0000","last_peered":"2024-05-09T08:02:21.677105+0000","last_clean":"2024-05-09T08:02:21.677105+0000","last_became_active":"2024-05-09T08:02:15.909818+0000","last_became_peered":"2024-05-09T08:02:15.909818+0000","last_unstale":"2024-05-09T08:02:21.677105+0000","last_undegraded":"2024-05-09T08:02:21.677105+0000","last_fullsized":"2024-05-09T08:02:21.677105+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:41:55.788033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.619452+0000","last_change":"2024-05-09T08:02:15.885025+0000","last_active":"2024-05-09T08:02:22.619452+0000","last_peered":"2024-05-09T08:02:22.619452+0000","last_clean":"2024-05-09T08:02:22.619452+0000","last_became_active":"2024-05-09T08:02:15.884517+0000","last_became_peered":"2024-05-09T08:02:15.884517+0000","last_unstale":"2024-05-09T08:02:22.619452+0000","last_undegraded":"2024-05-09T08:02:22.619452+0000","last_fullsized":"2024-05-09T08:02:22.619452+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:01:50.341133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:21.077234+0000","last_change":"2024-05-09T08:02:15.890306+0000","last_active":"2024-05-09T08:02:21.077234+0000","last_peered":"2024-05-09T08:02:21.077234+0000","last_clean":"2024-05-09T08:02:21.077234+0000","last_became_active":"2024-05-09T08:02:15.889998+0000","last_became_peered":"2024-05-09T08:02:15.889998+0000","last_unstale":"2024-05-09T08:02:21.077234+0000","last_undegraded":"2024-05-09T08:02:21.077234+0000","last_fullsized":"2024-05-09T08:02:21.077234+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:16:57.446444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.619318+0000","last_change":"2024-05-09T08:02:15.886417+0000","last_active":"2024-05-09T08:02:22.619318+0000","last_peered":"2024-05-09T08:02:22.619318+0000","last_clean":"2024-05-09T08:02:22.619318+0000","last_became_active":"2024-05-09T08:02:15.885525+0000","last_became_peered":"2024-05-09T08:02:15.885525+0000","last_unstale":"2024-05-09T08:02:22.619318+0000","last_undegraded":"2024-05-09T08:02:22.619318+0000","last_fullsized":"2024-05-09T08:02:22.619318+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:43:02.075442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.618616+0000","last_change":"2024-05-09T08:02:15.893172+0000","last_active":"2024-05-09T08:02:22.618616+0000","last_peered":"2024-05-09T08:02:22.618616+0000","last_clean":"2024-05-09T08:02:22.618616+0000","last_became_active":"2024-05-09T08:02:15.892886+0000","last_became_peered":"2024-05-09T08:02:15.892886+0000","last_unstale":"2024-05-09T08:02:22.618616+0000","last_undegraded":"2024-05-09T08:02:22.618616+0000","last_fullsized":"2024-05-09T08:02:22.618616+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:59:14.957806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.617513+0000","last_change":"2024-05-09T08:02:15.908659+0000","last_active":"2024-05-09T08:02:22.617513+0000","last_peered":"2024-05-09T08:02:22.617513+0000","last_clean":"2024-05-09T08:02:22.617513+0000","last_became_active":"2024-05-09T08:02:15.908307+0000","last_became_peered":"2024-05-09T08:02:15.908307+0000","last_unstale":"2024-05-09T08:02:22.617513+0000","last_undegraded":"2024-05-09T08:02:22.617513+0000","last_fullsized":"2024-05-09T08:02:22.617513+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:20:07.368327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:20.933418+0000","last_change":"2024-05-09T08:02:15.885800+0000","last_active":"2024-05-09T08:02:20.933418+0000","last_peered":"2024-05-09T08:02:20.933418+0000","last_clean":"2024-05-09T08:02:20.933418+0000","last_became_active":"2024-05-09T08:02:15.885185+0000","last_became_peered":"2024-05-09T08:02:15.885185+0000","last_unstale":"2024-05-09T08:02:20.933418+0000","last_undegraded":"2024-05-09T08:02:20.933418+0000","last_fullsized":"2024-05-09T08:02:20.933418+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:01:50.341133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:21.677190+0000","last_change":"2024-05-09T08:02:15.911496+0000","last_active":"2024-05-09T08:02:21.677190+0000","last_peered":"2024-05-09T08:02:21.677190+0000","last_clean":"2024-05-09T08:02:21.677190+0000","last_became_active":"2024-05-09T08:02:15.911142+0000","last_became_peered":"2024-05-09T08:02:15.911142+0000","last_unstale":"2024-05-09T08:02:21.677190+0000","last_undegraded":"2024-05-09T08:02:21.677190+0000","last_fullsized":"2024-05-09T08:02:21.677190+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:27:59.315750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:20.933605+0000","last_change":"2024-05-09T08:02:15.898847+0000","last_active":"2024-05-09T08:02:20.933605+0000","last_peered":"2024-05-09T08:02:20.933605+0000","last_clean":"2024-05-09T08:02:20.933605+0000","last_became_active":"2024-05-09T08:02:15.898480+0000","last_became_peered":"2024-05-09T08:02:15.898480+0000","last_unstale":"2024-05-09T08:02:20.933605+0000","last_undegraded":"2024-05-09T08:02:20.933605+0000","last_fullsized":"2024-05-09T08:02:20.933605+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:20:07.368327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.618261+0000","last_change":"2024-05-09T08:02:15.883117+0000","last_active":"2024-05-09T08:02:22.618261+0000","last_peered":"2024-05-09T08:02:22.618261+0000","last_clean":"2024-05-09T08:02:22.618261+0000","last_became_active":"2024-05-09T08:02:15.882141+0000","last_became_peered":"2024-05-09T08:02:15.882141+0000","last_unstale":"2024-05-09T08:02:22.618261+0000","last_undegraded":"2024-05-09T08:02:22.618261+0000","last_fullsized":"2024-05-09T08:02:22.618261+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:57:11.352151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.618965+0000","last_change":"2024-05-09T08:02:15.893318+0000","last_active":"2024-05-09T08:02:22.618965+0000","last_peered":"2024-05-09T08:02:22.618965+0000","last_clean":"2024-05-09T08:02:22.618965+0000","last_became_active":"2024-05-09T08:02:15.892964+0000","last_became_peered":"2024-05-09T08:02:15.892964+0000","last_unstale":"2024-05-09T08:02:22.618965+0000","last_undegraded":"2024-05-09T08:02:22.618965+0000","last_fullsized":"2024-05-09T08:02:22.618965+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:20:07.368327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.619428+0000","last_change":"2024-05-09T08:02:15.886482+0000","last_active":"2024-05-09T08:02:22.619428+0000","last_peered":"2024-05-09T08:02:22.619428+0000","last_clean":"2024-05-09T08:02:22.619428+0000","last_became_active":"2024-05-09T08:02:15.886183+0000","last_became_peered":"2024-05-09T08:02:15.886183+0000","last_unstale":"2024-05-09T08:02:22.619428+0000","last_undegraded":"2024-05-09T08:02:22.619428+0000","last_fullsized":"2024-05-09T08:02:22.619428+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:23:40.248821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:21.077365+0000","last_change":"2024-05-09T08:02:15.887506+0000","last_active":"2024-05-09T08:02:21.077365+0000","last_peered":"2024-05-09T08:02:21.077365+0000","last_clean":"2024-05-09T08:02:21.077365+0000","last_became_active":"2024-05-09T08:02:15.887225+0000","last_became_peered":"2024-05-09T08:02:15.887225+0000","last_unstale":"2024-05-09T08:02:21.077365+0000","last_undegraded":"2024-05-09T08:02:21.077365+0000","last_fullsized":"2024-05-09T08:02:21.077365+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:25:24.736323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:20.933062+0000","last_change":"2024-05-09T08:02:15.902065+0000","last_active":"2024-05-09T08:02:20.933062+0000","last_peered":"2024-05-09T08:02:20.933062+0000","last_clean":"2024-05-09T08:02:20.933062+0000","last_became_active":"2024-05-09T08:02:15.901766+0000","last_became_peered":"2024-05-09T08:02:15.901766+0000","last_unstale":"2024-05-09T08:02:20.933062+0000","last_undegraded":"2024-05-09T08:02:20.933062+0000","last_fullsized":"2024-05-09T08:02:20.933062+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:05:02.181367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:21.076616+0000","last_change":"2024-05-09T08:02:15.911058+0000","last_active":"2024-05-09T08:02:21.076616+0000","last_peered":"2024-05-09T08:02:21.076616+0000","last_clean":"2024-05-09T08:02:21.076616+0000","last_became_active":"2024-05-09T08:02:15.910716+0000","last_became_peered":"2024-05-09T08:02:15.910716+0000","last_unstale":"2024-05-09T08:02:21.076616+0000","last_undegraded":"2024-05-09T08:02:21.076616+0000","last_fullsized":"2024-05-09T08:02:21.076616+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:41:56.871863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.617832+0000","last_change":"2024-05-09T08:02:15.908653+0000","last_active":"2024-05-09T08:02:22.617832+0000","last_peered":"2024-05-09T08:02:22.617832+0000","last_clean":"2024-05-09T08:02:22.617832+0000","last_became_active":"2024-05-09T08:02:15.908290+0000","last_became_peered":"2024-05-09T08:02:15.908290+0000","last_unstale":"2024-05-09T08:02:22.617832+0000","last_undegraded":"2024-05-09T08:02:22.617832+0000","last_fullsized":"2024-05-09T08:02:22.617832+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:18:36.309627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:21.076688+0000","last_change":"2024-05-09T08:02:15.896652+0000","last_active":"2024-05-09T08:02:21.076688+0000","last_peered":"2024-05-09T08:02:21.076688+0000","last_clean":"2024-05-09T08:02:21.076688+0000","last_became_active":"2024-05-09T08:02:15.896384+0000","last_became_peered":"2024-05-09T08:02:15.896384+0000","last_unstale":"2024-05-09T08:02:21.076688+0000","last_undegraded":"2024-05-09T08:02:21.076688+0000","last_fullsized":"2024-05-09T08:02:21.076688+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:18:36.309627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.617988+0000","last_change":"2024-05-09T08:02:15.898998+0000","last_active":"2024-05-09T08:02:22.617988+0000","last_peered":"2024-05-09T08:02:22.617988+0000","last_clean":"2024-05-09T08:02:22.617988+0000","last_became_active":"2024-05-09T08:02:15.898670+0000","last_became_peered":"2024-05-09T08:02:15.898670+0000","last_unstale":"2024-05-09T08:02:22.617988+0000","last_undegraded":"2024-05-09T08:02:22.617988+0000","last_fullsized":"2024-05-09T08:02:22.617988+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:23:40.248821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:21.677605+0000","last_change":"2024-05-09T08:02:15.905078+0000","last_active":"2024-05-09T08:02:21.677605+0000","last_peered":"2024-05-09T08:02:21.677605+0000","last_clean":"2024-05-09T08:02:21.677605+0000","last_became_active":"2024-05-09T08:02:15.904736+0000","last_became_peered":"2024-05-09T08:02:15.904736+0000","last_unstale":"2024-05-09T08:02:21.677605+0000","last_undegraded":"2024-05-09T08:02:21.677605+0000","last_fullsized":"2024-05-09T08:02:21.677605+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:01:50.341133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:20.936449+0000","last_change":"2024-05-09T08:02:15.897743+0000","last_active":"2024-05-09T08:02:20.936449+0000","last_peered":"2024-05-09T08:02:20.936449+0000","last_clean":"2024-05-09T08:02:20.936449+0000","last_became_active":"2024-05-09T08:02:15.897367+0000","last_became_peered":"2024-05-09T08:02:15.897367+0000","last_unstale":"2024-05-09T08:02:20.936449+0000","last_undegraded":"2024-05-09T08:02:20.936449+0000","last_fullsized":"2024-05-09T08:02:20.936449+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:47:32.605288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:21.077273+0000","last_change":"2024-05-09T08:02:15.884163+0000","last_active":"2024-05-09T08:02:21.077273+0000","last_peered":"2024-05-09T08:02:21.077273+0000","last_clean":"2024-05-09T08:02:21.077273+0000","last_became_active":"2024-05-09T08:02:15.883430+0000","last_became_peered":"2024-05-09T08:02:15.883430+0000","last_unstale":"2024-05-09T08:02:21.077273+0000","last_undegraded":"2024-05-09T08:02:21.077273+0000","last_fullsized":"2024-05-09T08:02:21.077273+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:05:15.555592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574857,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32024,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339816,"statfs":{"total":96636764160,"available":96603971584,"internally_reserved":0,"allocated":864256,"data_stored":695877,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574857,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31448,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340392,"statfs":{"total":96636764160,"available":96604561408,"internally_reserved":0,"allocated":278528,"data_stored":108769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574857,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31444,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340396,"statfs":{"total":96636764160,"available":96604565504,"internally_reserved":0,"allocated":270336,"data_stored":105017,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574858,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31452,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340388,"statfs":{"total":96636764160,"available":96604557312,"internally_reserved":0,"allocated":278528,"data_stored":105053,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574858,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":108769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574858,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":266240,"data_stored":104927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574859,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":274432,"data_stored":105087,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574858,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32032,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339808,"statfs":{"total":96636764160,"available":96603963392,"internally_reserved":0,"allocated":872448,"data_stored":695933,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T08:02:52.228 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-05-09T08:02:52.228 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-05-09T08:02:52.228 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-05-09T08:02:52.228 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-05-09T08:02:52.229 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-05-09T08:02:52.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:52.452+0000 7f72e5000640 1 Processor -- start 2024-05-09T08:02:52.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:52.452+0000 7f72e5000640 1 -- start start 2024-05-09T08:02:52.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:52.454+0000 7f72e5000640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f72e00c8180 0x7f72e00cc730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:52.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:52.454+0000 7f72e5000640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f72e00c7660 0x7f72e00c7ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:52.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:52.454+0000 7f72e5000640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f72e015c790 con 0x7f72e0152500 2024-05-09T08:02:52.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:52.454+0000 7f72e5000640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f72e00c5e50 con 0x7f72e00c8180 2024-05-09T08:02:52.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:52.454+0000 7f72e5000640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f72e00c22d0 con 0x7f72e00c7660 2024-05-09T08:02:52.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:52.454+0000 7f72df400640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f72e0152500 0x7f72e015c080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:38406/0 (socket says 172.21.15.1:38406) 2024-05-09T08:02:52.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:52.454+0000 7f72df400640 1 -- 172.21.15.1:0/65986406 learned_addr learned my addr 172.21.15.1:0/65986406 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T08:02:52.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:52.454+0000 7f72dea00640 1 --2- 172.21.15.1:0/65986406 >> v2:172.21.15.1:3300/0 conn(0x7f72e00c8180 0x7f72e00cc730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:52.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:52.454+0000 7f72dfe00640 1 --2- 172.21.15.1:0/65986406 >> v2:172.21.15.7:3300/0 conn(0x7f72e00c7660 0x7f72e00c7ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:52.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:52.455+0000 7f72de000640 1 -- 172.21.15.1:0/65986406 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3710826976 0 0) 0x7f72d0001490 con 0x7f72e0152500 2024-05-09T08:02:52.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:52.455+0000 7f72de000640 1 -- 172.21.15.1:0/65986406 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f72c4003520 con 0x7f72e0152500 2024-05-09T08:02:52.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:52.455+0000 7f72dea00640 1 -- 172.21.15.1:0/65986406 >> v2:172.21.15.7:3300/0 conn(0x7f72e00c7660 msgr2=0x7f72e00c7ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:02:52.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:52.455+0000 7f72dea00640 1 --2- 172.21.15.1:0/65986406 >> v2:172.21.15.7:3300/0 conn(0x7f72e00c7660 0x7f72e00c7ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:52.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:52.455+0000 7f72dea00640 1 -- 172.21.15.1:0/65986406 >> v1:172.21.15.1:6789/0 conn(0x7f72e0152500 legacy=0x7f72e015c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:52.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:52.455+0000 7f72dea00640 1 -- 172.21.15.1:0/65986406 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72e00c6a30 con 0x7f72e00c8180 2024-05-09T08:02:52.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:52.456+0000 7f72dea00640 1 --2- 172.21.15.1:0/65986406 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f72e00c8180 0x7f72e00cc730 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f72e006eaf0 tx=0x7f72cc02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8074c5e16305a558 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:52.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:52.456+0000 7f72de000640 1 -- 172.21.15.1:0/65986406 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72cc037070 con 0x7f72e00c8180 2024-05-09T08:02:52.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:52.456+0000 7f72de000640 1 -- 172.21.15.1:0/65986406 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72cc032440 con 0x7f72e00c8180 2024-05-09T08:02:52.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:52.456+0000 7f72de000640 1 -- 172.21.15.1:0/65986406 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72cc0327f0 con 0x7f72e00c8180 2024-05-09T08:02:52.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:52.457+0000 7f72e5000640 1 -- 172.21.15.1:0/65986406 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f72e00c8180 msgr2=0x7f72e00cc730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:52.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:52.457+0000 7f72e5000640 1 --2- 172.21.15.1:0/65986406 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f72e00c8180 0x7f72e00cc730 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f72e006eaf0 tx=0x7f72cc02a1a0 comp rx=0 tx=0).stop 2024-05-09T08:02:52.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:52.457+0000 7f72e5000640 1 -- 172.21.15.1:0/65986406 shutdown_connections 2024-05-09T08:02:52.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:52.457+0000 7f72e5000640 1 --2- 172.21.15.1:0/65986406 >> v2:172.21.15.7:3300/0 conn(0x7f72e00c7660 0x7f72e00c7ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:52.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:52.457+0000 7f72e5000640 1 --2- 172.21.15.1:0/65986406 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f72e00c8180 0x7f72e00cc730 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:52.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:52.457+0000 7f72e5000640 1 -- 172.21.15.1:0/65986406 >> 172.21.15.1:0/65986406 conn(0x7f72e00cbc10 msgr2=0x7f72e00cc020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:02:52.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:52.457+0000 7f72e5000640 1 -- 172.21.15.1:0/65986406 shutdown_connections 2024-05-09T08:02:52.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:52.457+0000 7f72e5000640 1 -- 172.21.15.1:0/65986406 wait complete. 2024-05-09T08:02:52.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:52.459+0000 7f72e5000640 1 Processor -- start 2024-05-09T08:02:52.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:52.459+0000 7f72e5000640 1 -- start start 2024-05-09T08:02:52.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:52.459+0000 7f72e5000640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f72e00c7660 0x7f72e015a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:52.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:52.460+0000 7f72e5000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f72e00c8180 0x7f72e01551c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:52.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:52.460+0000 7f72e5000640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f72e0152500 0x7f72e014cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:52.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:52.460+0000 7f72e5000640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f72e00c72b0 con 0x7f72e00c7660 2024-05-09T08:02:52.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:52.460+0000 7f72e5000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f72e00c6fd0 con 0x7f72e00c8180 2024-05-09T08:02:52.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:52.460+0000 7f72e5000640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f72e015b880 con 0x7f72e0152500 2024-05-09T08:02:52.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:52.460+0000 7f72df400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f72e00c7660 0x7f72e015a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:52.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:52.460+0000 7f72df400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f72e00c7660 0x7f72e015a940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53040/0 (socket says 172.21.15.1:53040) 2024-05-09T08:02:52.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:52.460+0000 7f72df400640 1 -- 172.21.15.1:0/69772489 learned_addr learned my addr 172.21.15.1:0/69772489 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:02:52.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:52.460+0000 7f72dea00640 1 --2- 172.21.15.1:0/69772489 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f72e00c8180 0x7f72e01551c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:52.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:52.460+0000 7f72dfe00640 1 --2- 172.21.15.1:0/69772489 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f72e0152500 0x7f72e014cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:52.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:52.460+0000 7f72df400640 1 -- 172.21.15.1:0/69772489 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f72e0152500 msgr2=0x7f72e014cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:52.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:52.460+0000 7f72df400640 1 --2- 172.21.15.1:0/69772489 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f72e0152500 0x7f72e014cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:52.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:52.460+0000 7f72df400640 1 -- 172.21.15.1:0/69772489 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f72e00c8180 msgr2=0x7f72e01551c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:52.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:52.460+0000 7f72df400640 1 --2- 172.21.15.1:0/69772489 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f72e00c8180 0x7f72e01551c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:52.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:52.460+0000 7f72df400640 1 -- 172.21.15.1:0/69772489 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72e0153eb0 con 0x7f72e00c7660 2024-05-09T08:02:52.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:52.461+0000 7f72df400640 1 --2- 172.21.15.1:0/69772489 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f72e00c7660 0x7f72e015a940 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f72d000cb40 tx=0x7f72d000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:52.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:52.461+0000 7f72d7400640 1 -- 172.21.15.1:0/69772489 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72d0017070 con 0x7f72e00c7660 2024-05-09T08:02:52.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:52.461+0000 7f72d7400640 1 -- 172.21.15.1:0/69772489 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72d000ac40 con 0x7f72e00c7660 2024-05-09T08:02:52.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:52.461+0000 7f72d7400640 1 -- 172.21.15.1:0/69772489 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72d0012400 con 0x7f72e00c7660 2024-05-09T08:02:52.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:52.461+0000 7f72e5000640 1 -- 172.21.15.1:0/69772489 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72e0154120 con 0x7f72e00c7660 2024-05-09T08:02:52.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:52.461+0000 7f72e5000640 1 -- 172.21.15.1:0/69772489 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72e01546e0 con 0x7f72e00c7660 2024-05-09T08:02:52.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:52.462+0000 7f72e5000640 1 -- 172.21.15.1:0/69772489 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72e004f660 con 0x7f72e00c7660 2024-05-09T08:02:52.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:52.463+0000 7f72d7400640 1 -- 172.21.15.1:0/69772489 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f72d001a070 con 0x7f72e00c7660 2024-05-09T08:02:52.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:52.464+0000 7f72d7400640 1 --2- 172.21.15.1:0/69772489 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f72c006be30 0x7f72c006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:52.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:52.464+0000 7f72d7400640 1 -- 172.21.15.1:0/69772489 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f72d008afe0 con 0x7f72e00c7660 2024-05-09T08:02:52.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:52.465+0000 7f72dea00640 1 --2- 172.21.15.1:0/69772489 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f72c006be30 0x7f72c006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:52.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:52.465+0000 7f72dea00640 1 --2- 172.21.15.1:0/69772489 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f72c006be30 0x7f72c006e2f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f72cc034000 tx=0x7f72cc02e000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:52.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:52.469+0000 7f72d7400640 1 -- 172.21.15.1:0/69772489 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f72e015c790 con 0x7f72e00c7660 2024-05-09T08:02:52.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:52.724+0000 7f72e5000640 1 -- 172.21.15.1:0/69772489 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f72e00c2a40 con 0x7f72e00c7660 2024-05-09T08:02:52.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:52.733+0000 7f72d7400640 1 -- 172.21.15.1:0/69772489 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) ==== 74+0+172931 (secure 0 0 0) 0x7f72d0058ef0 con 0x7f72e00c7660 2024-05-09T08:02:52.752 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T08:02:52.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:52.737+0000 7f72e5000640 1 -- 172.21.15.1:0/69772489 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f72c006be30 msgr2=0x7f72c006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:52.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:52.737+0000 7f72e5000640 1 --2- 172.21.15.1:0/69772489 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f72c006be30 0x7f72c006e2f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f72cc034000 tx=0x7f72cc02e000 comp rx=0 tx=0).stop 2024-05-09T08:02:52.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:52.737+0000 7f72e5000640 1 -- 172.21.15.1:0/69772489 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f72e00c7660 msgr2=0x7f72e015a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:52.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:52.737+0000 7f72e5000640 1 --2- 172.21.15.1:0/69772489 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f72e00c7660 0x7f72e015a940 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f72d000cb40 tx=0x7f72d000a060 comp rx=0 tx=0).stop 2024-05-09T08:02:52.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:52.738+0000 7f72e5000640 1 -- 172.21.15.1:0/69772489 shutdown_connections 2024-05-09T08:02:52.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:52.738+0000 7f72e5000640 1 --2- 172.21.15.1:0/69772489 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f72c006be30 0x7f72c006e2f0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:52.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:52.738+0000 7f72e5000640 1 --2- 172.21.15.1:0/69772489 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f72e00c8180 0x7f72e01551c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:52.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:52.738+0000 7f72e5000640 1 --2- 172.21.15.1:0/69772489 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f72e0152500 0x7f72e014cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:52.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:52.738+0000 7f72e5000640 1 --2- 172.21.15.1:0/69772489 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f72e00c7660 0x7f72e015a940 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:52.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:52.738+0000 7f72e5000640 1 -- 172.21.15.1:0/69772489 >> 172.21.15.1:0/69772489 conn(0x7f72e00cbc10 msgr2=0x7f72e00cbff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:02:52.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:52.738+0000 7f72e5000640 1 -- 172.21.15.1:0/69772489 shutdown_connections 2024-05-09T08:02:52.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:52.738+0000 7f72e5000640 1 -- 172.21.15.1:0/69772489 wait complete. 2024-05-09T08:02:52.774 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":5,"flags":0,"active_gid":4114,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":3562612464},{"type":"v1","addr":"172.21.15.7:6833","nonce":3562612464}]},"active_addr":"172.21.15.7:6833/3562612464","active_change":"2024-05-09T08:02:01.083941+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4102,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.7:0","nonce":1625564439}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.7:0","nonce":546157630}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.7:0","nonce":4001035158}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.7:0","nonce":3370439472}]}]} 2024-05-09T08:02:52.778 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-05-09T08:02:52.778 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-05-09T08:02:52.779 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T08:02:52.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:52.965+0000 7f13e8c00640 1 Processor -- start 2024-05-09T08:02:52.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:52.966+0000 7f13e8c00640 1 -- start start 2024-05-09T08:02:52.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:52.967+0000 7f13e8c00640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f13e40cd040 0x7f13e415ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:52.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:52.967+0000 7f13e8c00640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f13e415b550 con 0x7f13e415a5e0 2024-05-09T08:02:52.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:52.967+0000 7f13e8c00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f13e406c8b0 con 0x7f13e406d350 2024-05-09T08:02:52.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:52.967+0000 7f13e8c00640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f13e406cec0 con 0x7f13e40cd040 2024-05-09T08:02:52.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:52.968+0000 7f13e3e00640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f13e415a5e0 0x7f13e40ca5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:38416/0 (socket says 172.21.15.1:38416) 2024-05-09T08:02:52.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:52.968+0000 7f13e3e00640 1 -- 172.21.15.1:0/3847957887 learned_addr learned my addr 172.21.15.1:0/3847957887 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T08:02:52.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:52.968+0000 7f13e3400640 1 --2- 172.21.15.1:0/3847957887 >> v2:172.21.15.1:3300/0 conn(0x7f13e40cd040 0x7f13e415ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:52.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:52.968+0000 7f13e2000640 1 -- 172.21.15.1:0/3847957887 <== mon.1 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1184406471 0 0) 0x7f13e406c8b0 con 0x7f13e406d350 2024-05-09T08:02:52.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:52.968+0000 7f13e2000640 1 -- 172.21.15.1:0/3847957887 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f13c8003520 con 0x7f13e406d350 2024-05-09T08:02:52.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:52.968+0000 7f13e2000640 1 -- 172.21.15.1:0/3847957887 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2550062103 0 0) 0x7f13e415b550 con 0x7f13e415a5e0 2024-05-09T08:02:52.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:52.968+0000 7f13e2000640 1 -- 172.21.15.1:0/3847957887 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f13e406c8b0 con 0x7f13e415a5e0 2024-05-09T08:02:52.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:52.968+0000 7f13e3400640 1 -- 172.21.15.1:0/3847957887 >> v1:172.21.15.7:6789/0 conn(0x7f13e406d350 legacy=0x7f13e406da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:52.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:52.968+0000 7f13e3400640 1 -- 172.21.15.1:0/3847957887 >> v1:172.21.15.1:6789/0 conn(0x7f13e415a5e0 legacy=0x7f13e40ca5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:52.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:52.968+0000 7f13e3400640 1 -- 172.21.15.1:0/3847957887 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13e415af90 con 0x7f13e40cd040 2024-05-09T08:02:52.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:52.969+0000 7f13e3400640 1 --2- 172.21.15.1:0/3847957887 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f13e40cd040 0x7f13e415ba50 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f13d40097a0 tx=0x7f13d402a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f33e191216cb40d8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:52.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:52.970+0000 7f13e2000640 1 -- 172.21.15.1:0/3847957887 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13d4037070 con 0x7f13e40cd040 2024-05-09T08:02:52.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:52.970+0000 7f13e2000640 1 -- 172.21.15.1:0/3847957887 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13d402ae50 con 0x7f13e40cd040 2024-05-09T08:02:52.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:52.970+0000 7f13e2000640 1 -- 172.21.15.1:0/3847957887 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13d4032640 con 0x7f13e40cd040 2024-05-09T08:02:52.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:52.970+0000 7f13e8c00640 1 -- 172.21.15.1:0/3847957887 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f13e40cd040 msgr2=0x7f13e415ba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:52.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:52.970+0000 7f13e8c00640 1 --2- 172.21.15.1:0/3847957887 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f13e40cd040 0x7f13e415ba50 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f13d40097a0 tx=0x7f13d402a100 comp rx=0 tx=0).stop 2024-05-09T08:02:52.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:52.970+0000 7f13e8c00640 1 -- 172.21.15.1:0/3847957887 shutdown_connections 2024-05-09T08:02:52.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:52.970+0000 7f13e8c00640 1 --2- 172.21.15.1:0/3847957887 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f13e40cd040 0x7f13e415ba50 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:52.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:52.970+0000 7f13e8c00640 1 -- 172.21.15.1:0/3847957887 >> 172.21.15.1:0/3847957887 conn(0x7f13e40b1780 msgr2=0x7f13e40b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:02:52.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:52.971+0000 7f13e8c00640 1 -- 172.21.15.1:0/3847957887 shutdown_connections 2024-05-09T08:02:52.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:52.971+0000 7f13e8c00640 1 -- 172.21.15.1:0/3847957887 wait complete. 2024-05-09T08:02:52.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:52.972+0000 7f13e8c00640 1 Processor -- start 2024-05-09T08:02:52.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:52.972+0000 7f13e8c00640 1 -- start start 2024-05-09T08:02:52.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:52.973+0000 7f13e8c00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f13e406d350 0x7f13e41e73a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:52.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:52.973+0000 7f13e8c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13e40cd040 0x7f13e40b39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:52.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:52.973+0000 7f13e8c00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f13e415a5e0 0x7f13e40b2660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:52.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:52.973+0000 7f13e8c00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f13e406cb50 con 0x7f13e406d350 2024-05-09T08:02:52.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:52.973+0000 7f13e8c00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f13e406d7b0 con 0x7f13e40cd040 2024-05-09T08:02:52.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:52.973+0000 7f13e8c00640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f13e40cad00 con 0x7f13e415a5e0 2024-05-09T08:02:52.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:52.973+0000 7f13e3400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f13e406d350 0x7f13e41e73a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:52.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:52.974+0000 7f13e3400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f13e406d350 0x7f13e41e73a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53050/0 (socket says 172.21.15.1:53050) 2024-05-09T08:02:52.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:52.974+0000 7f13e3400640 1 -- 172.21.15.1:0/4149562026 learned_addr learned my addr 172.21.15.1:0/4149562026 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:02:52.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:52.974+0000 7f13e3e00640 1 --2- 172.21.15.1:0/4149562026 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f13e415a5e0 0x7f13e40b2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:52.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:52.974+0000 7f13e2a00640 1 --2- 172.21.15.1:0/4149562026 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13e40cd040 0x7f13e40b39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:52.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:52.974+0000 7f13e3400640 1 -- 172.21.15.1:0/4149562026 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f13e415a5e0 msgr2=0x7f13e40b2660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:52.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:52.974+0000 7f13e3400640 1 --2- 172.21.15.1:0/4149562026 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f13e415a5e0 0x7f13e40b2660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:52.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:52.974+0000 7f13e3400640 1 -- 172.21.15.1:0/4149562026 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13e40cd040 msgr2=0x7f13e40b39d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:52.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:52.974+0000 7f13e3400640 1 --2- 172.21.15.1:0/4149562026 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13e40cd040 0x7f13e40b39d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:52.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:52.974+0000 7f13e3400640 1 -- 172.21.15.1:0/4149562026 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13e41f0320 con 0x7f13e406d350 2024-05-09T08:02:52.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:52.975+0000 7f13e3400640 1 --2- 172.21.15.1:0/4149562026 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f13e406d350 0x7f13e41e73a0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f13d4000c00 tx=0x7f13d40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:52.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:52.975+0000 7f13db400640 1 -- 172.21.15.1:0/4149562026 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13d4041030 con 0x7f13e406d350 2024-05-09T08:02:52.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:52.975+0000 7f13db400640 1 -- 172.21.15.1:0/4149562026 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13d402ac50 con 0x7f13e406d350 2024-05-09T08:02:52.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:52.975+0000 7f13db400640 1 -- 172.21.15.1:0/4149562026 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13d4037040 con 0x7f13e406d350 2024-05-09T08:02:52.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:52.975+0000 7f13e8c00640 1 -- 172.21.15.1:0/4149562026 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13e40c8060 con 0x7f13e406d350 2024-05-09T08:02:52.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:52.975+0000 7f13e8c00640 1 -- 172.21.15.1:0/4149562026 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13e40c8900 con 0x7f13e406d350 2024-05-09T08:02:52.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:52.976+0000 7f13e8c00640 1 -- 172.21.15.1:0/4149562026 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13e404f660 con 0x7f13e406d350 2024-05-09T08:02:52.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:52.977+0000 7f13db400640 1 -- 172.21.15.1:0/4149562026 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f13d4044070 con 0x7f13e406d350 2024-05-09T08:02:52.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:52.978+0000 7f13db400640 1 --2- 172.21.15.1:0/4149562026 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f13c006be30 0x7f13c006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:52.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:52.978+0000 7f13db400640 1 -- 172.21.15.1:0/4149562026 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f13d40ab6f0 con 0x7f13e406d350 2024-05-09T08:02:52.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:52.978+0000 7f13e2a00640 1 --2- 172.21.15.1:0/4149562026 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f13c006be30 0x7f13c006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:52.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:52.979+0000 7f13e2a00640 1 --2- 172.21.15.1:0/4149562026 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f13c006be30 0x7f13c006e2f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f13e4074ef0 tx=0x7f13cc009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:52.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:52.983+0000 7f13db400640 1 -- 172.21.15.1:0/4149562026 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f13e41f0320 con 0x7f13e406d350 2024-05-09T08:02:53.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.191+0000 7f13e8c00640 1 -- 172.21.15.1:0/4149562026 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f13e40cb260 con 0x7f13e406d350 2024-05-09T08:02:53.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.194+0000 7f13db400640 1 -- 172.21.15.1:0/4149562026 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18382 (secure 0 0 0) 0x7f13d4079670 con 0x7f13e406d350 2024-05-09T08:02:53.211 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T08:02:53.211 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":25,"fsid":"54329ec8-3270-4043-9a82-c6cd7420aa7f","created":"2024-05-09T08:01:56.666970+0000","modified":"2024-05-09T08:02:20.403456+0000","last_up_change":"2024-05-09T08:02:07.146082+0000","last_in_change":"2024-05-09T08:01:59.008225+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T08:02:09.165511+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T08:02:12.925485+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T08:02:14.367455+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-09T08:02:17.967064+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c064454e-a9b5-4a24-9a77-17f0a4085952","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6800","nonce":1513349547},{"type":"v1","addr":"172.21.15.1:6801","nonce":1513349547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":1513349547},{"type":"v1","addr":"172.21.15.1:6803","nonce":1513349547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":1513349547},{"type":"v1","addr":"172.21.15.1:6807","nonce":1513349547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":1513349547},{"type":"v1","addr":"172.21.15.1:6805","nonce":1513349547}]},"public_addr":"172.21.15.1:6801/1513349547","cluster_addr":"172.21.15.1:6803/1513349547","heartbeat_back_addr":"172.21.15.1:6807/1513349547","heartbeat_front_addr":"172.21.15.1:6805/1513349547","state":["exists","up"]},{"osd":1,"uuid":"1a9ea687-2ff6-407a-949b-0e7e1083af21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":2942255259},{"type":"v1","addr":"172.21.15.1:6817","nonce":2942255259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":2942255259},{"type":"v1","addr":"172.21.15.1:6819","nonce":2942255259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":2942255259},{"type":"v1","addr":"172.21.15.1:6823","nonce":2942255259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6820","nonce":2942255259},{"type":"v1","addr":"172.21.15.1:6821","nonce":2942255259}]},"public_addr":"172.21.15.1:6817/2942255259","cluster_addr":"172.21.15.1:6819/2942255259","heartbeat_back_addr":"172.21.15.1:6823/2942255259","heartbeat_front_addr":"172.21.15.1:6821/2942255259","state":["exists","up"]},{"osd":2,"uuid":"c88af08b-7c9a-4dd8-9640-44138239addb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":2925031811},{"type":"v1","addr":"172.21.15.1:6809","nonce":2925031811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":2925031811},{"type":"v1","addr":"172.21.15.1:6811","nonce":2925031811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":2925031811},{"type":"v1","addr":"172.21.15.1:6815","nonce":2925031811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":2925031811},{"type":"v1","addr":"172.21.15.1:6813","nonce":2925031811}]},"public_addr":"172.21.15.1:6809/2925031811","cluster_addr":"172.21.15.1:6811/2925031811","heartbeat_back_addr":"172.21.15.1:6815/2925031811","heartbeat_front_addr":"172.21.15.1:6813/2925031811","state":["exists","up"]},{"osd":3,"uuid":"c087b7e3-a41d-4333-9365-32e3bd8c8e6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6824","nonce":2497776654},{"type":"v1","addr":"172.21.15.1:6825","nonce":2497776654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":2497776654},{"type":"v1","addr":"172.21.15.1:6827","nonce":2497776654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":2497776654},{"type":"v1","addr":"172.21.15.1:6831","nonce":2497776654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6828","nonce":2497776654},{"type":"v1","addr":"172.21.15.1:6829","nonce":2497776654}]},"public_addr":"172.21.15.1:6825/2497776654","cluster_addr":"172.21.15.1:6827/2497776654","heartbeat_back_addr":"172.21.15.1:6831/2497776654","heartbeat_front_addr":"172.21.15.1:6829/2497776654","state":["exists","up"]},{"osd":4,"uuid":"b4576b9c-871d-47a8-8d36-65af525d0200","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":2957026670},{"type":"v1","addr":"172.21.15.7:6801","nonce":2957026670}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2957026670},{"type":"v1","addr":"172.21.15.7:6803","nonce":2957026670}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":2957026670},{"type":"v1","addr":"172.21.15.7:6807","nonce":2957026670}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2957026670},{"type":"v1","addr":"172.21.15.7:6805","nonce":2957026670}]},"public_addr":"172.21.15.7:6801/2957026670","cluster_addr":"172.21.15.7:6803/2957026670","heartbeat_back_addr":"172.21.15.7:6807/2957026670","heartbeat_front_addr":"172.21.15.7:6805/2957026670","state":["exists","up"]},{"osd":5,"uuid":"902b3d94-72f3-4499-b21a-e32453ef5144","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1670295242},{"type":"v1","addr":"172.21.15.7:6809","nonce":1670295242}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1670295242},{"type":"v1","addr":"172.21.15.7:6811","nonce":1670295242}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1670295242},{"type":"v1","addr":"172.21.15.7:6815","nonce":1670295242}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1670295242},{"type":"v1","addr":"172.21.15.7:6813","nonce":1670295242}]},"public_addr":"172.21.15.7:6809/1670295242","cluster_addr":"172.21.15.7:6811/1670295242","heartbeat_back_addr":"172.21.15.7:6815/1670295242","heartbeat_front_addr":"172.21.15.7:6813/1670295242","state":["exists","up"]},{"osd":6,"uuid":"ead1d771-1728-4f85-adb4-26be21cedff3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":2852086880},{"type":"v1","addr":"172.21.15.7:6817","nonce":2852086880}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":2852086880},{"type":"v1","addr":"172.21.15.7:6819","nonce":2852086880}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":2852086880},{"type":"v1","addr":"172.21.15.7:6823","nonce":2852086880}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":2852086880},{"type":"v1","addr":"172.21.15.7:6821","nonce":2852086880}]},"public_addr":"172.21.15.7:6817/2852086880","cluster_addr":"172.21.15.7:6819/2852086880","heartbeat_back_addr":"172.21.15.7:6823/2852086880","heartbeat_front_addr":"172.21.15.7:6821/2852086880","state":["exists","up"]},{"osd":7,"uuid":"3ab5a201-9394-48a8-8918-e95197893669","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":1442884138},{"type":"v1","addr":"172.21.15.7:6825","nonce":1442884138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1442884138},{"type":"v1","addr":"172.21.15.7:6827","nonce":1442884138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1442884138},{"type":"v1","addr":"172.21.15.7:6831","nonce":1442884138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1442884138},{"type":"v1","addr":"172.21.15.7:6829","nonce":1442884138}]},"public_addr":"172.21.15.7:6825/1442884138","cluster_addr":"172.21.15.7:6827/1442884138","heartbeat_back_addr":"172.21.15.7:6831/1442884138","heartbeat_front_addr":"172.21.15.7:6829/1442884138","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:02:04.615675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:02:04.385974+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:02:04.619217+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:02:04.401681+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:02:04.097081+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:02:04.129981+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:02:04.225327+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:02:04.193351+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T08:02:53.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.197+0000 7f13e8c00640 1 -- 172.21.15.1:0/4149562026 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f13c006be30 msgr2=0x7f13c006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:53.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.197+0000 7f13e8c00640 1 --2- 172.21.15.1:0/4149562026 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f13c006be30 0x7f13c006e2f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f13e4074ef0 tx=0x7f13cc009040 comp rx=0 tx=0).stop 2024-05-09T08:02:53.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.197+0000 7f13e8c00640 1 -- 172.21.15.1:0/4149562026 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f13e406d350 msgr2=0x7f13e41e73a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:53.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.197+0000 7f13e8c00640 1 --2- 172.21.15.1:0/4149562026 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f13e406d350 0x7f13e41e73a0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f13d4000c00 tx=0x7f13d40046c0 comp rx=0 tx=0).stop 2024-05-09T08:02:53.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.197+0000 7f13e8c00640 1 -- 172.21.15.1:0/4149562026 shutdown_connections 2024-05-09T08:02:53.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.197+0000 7f13e8c00640 1 --2- 172.21.15.1:0/4149562026 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f13c006be30 0x7f13c006e2f0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:53.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.197+0000 7f13e8c00640 1 --2- 172.21.15.1:0/4149562026 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13e40cd040 0x7f13e40b39d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:53.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.197+0000 7f13e8c00640 1 --2- 172.21.15.1:0/4149562026 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f13e415a5e0 0x7f13e40b2660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:53.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.197+0000 7f13e8c00640 1 --2- 172.21.15.1:0/4149562026 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f13e406d350 0x7f13e41e73a0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:53.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.197+0000 7f13e8c00640 1 -- 172.21.15.1:0/4149562026 >> 172.21.15.1:0/4149562026 conn(0x7f13e40b1780 msgr2=0x7f13e40744e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:02:53.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.197+0000 7f13e8c00640 1 -- 172.21.15.1:0/4149562026 shutdown_connections 2024-05-09T08:02:53.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.197+0000 7f13e8c00640 1 -- 172.21.15.1:0/4149562026 wait complete. 2024-05-09T08:02:53.234 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-05-09T08:02:53.234 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T08:02:53.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.424+0000 7ff7b5c00640 1 Processor -- start 2024-05-09T08:02:53.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.424+0000 7ff7b5c00640 1 -- start start 2024-05-09T08:02:53.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.426+0000 7ff7b5c00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ff7b00c6d10 0x7ff7b00c7640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:53.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.426+0000 7ff7b5c00640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff7b015f4e0 con 0x7ff7b006ed50 2024-05-09T08:02:53.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.426+0000 7ff7b5c00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff7b00c3230 con 0x7ff7b0158cb0 2024-05-09T08:02:53.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.426+0000 7ff7b5c00640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7ff7b00c2da0 con 0x7ff7b00c6d10 2024-05-09T08:02:53.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.426+0000 7ff7ae000640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7ff7b006ed50 0x7ff7b0167320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:38418/0 (socket says 172.21.15.1:38418) 2024-05-09T08:02:53.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.426+0000 7ff7ae000640 1 -- 172.21.15.1:0/2226944399 learned_addr learned my addr 172.21.15.1:0/2226944399 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T08:02:53.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.426+0000 7ff7af400640 1 --2- 172.21.15.1:0/2226944399 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ff7b00c6d10 0x7ff7b00c7640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:53.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.426+0000 7ff7ad600640 1 -- 172.21.15.1:0/2226944399 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 712817536 0 0) 0x7ff7b015f4e0 con 0x7ff7b006ed50 2024-05-09T08:02:53.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.427+0000 7ff7ad600640 1 -- 172.21.15.1:0/2226944399 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff794003520 con 0x7ff7b006ed50 2024-05-09T08:02:53.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.427+0000 7ff7ad600640 1 -- 172.21.15.1:0/2226944399 <== mon.1 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 504414300 0 0) 0x7ff7b00c3230 con 0x7ff7b0158cb0 2024-05-09T08:02:53.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.427+0000 7ff7ad600640 1 -- 172.21.15.1:0/2226944399 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff7b015f4e0 con 0x7ff7b0158cb0 2024-05-09T08:02:53.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.427+0000 7ff7af400640 1 -- 172.21.15.1:0/2226944399 >> v1:172.21.15.7:6789/0 conn(0x7ff7b0158cb0 legacy=0x7ff7b006d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:53.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.427+0000 7ff7af400640 1 -- 172.21.15.1:0/2226944399 >> v1:172.21.15.1:6789/0 conn(0x7ff7b006ed50 legacy=0x7ff7b0167320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:53.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.427+0000 7ff7af400640 1 -- 172.21.15.1:0/2226944399 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7b015f7b0 con 0x7ff7b00c6d10 2024-05-09T08:02:53.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.428+0000 7ff7af400640 1 --2- 172.21.15.1:0/2226944399 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ff7b00c6d10 0x7ff7b00c7640 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ff7a80097d0 tx=0x7ff7a802a290 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a028dac8b3c688ee server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:53.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.428+0000 7ff7ad600640 1 -- 172.21.15.1:0/2226944399 <== mon.2 v2:172.21.15.7:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7a8037070 con 0x7ff7b00c6d10 2024-05-09T08:02:53.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.428+0000 7ff7ad600640 1 -- 172.21.15.1:0/2226944399 <== mon.2 v2:172.21.15.7:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7a8032440 con 0x7ff7b00c6d10 2024-05-09T08:02:53.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.428+0000 7ff7ad600640 1 -- 172.21.15.1:0/2226944399 <== mon.2 v2:172.21.15.7:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7a80327f0 con 0x7ff7b00c6d10 2024-05-09T08:02:53.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.429+0000 7ff7b5c00640 1 -- 172.21.15.1:0/2226944399 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ff7b00c6d10 msgr2=0x7ff7b00c7640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:53.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.429+0000 7ff7b5c00640 1 --2- 172.21.15.1:0/2226944399 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ff7b00c6d10 0x7ff7b00c7640 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ff7a80097d0 tx=0x7ff7a802a290 comp rx=0 tx=0).stop 2024-05-09T08:02:53.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.429+0000 7ff7b5c00640 1 -- 172.21.15.1:0/2226944399 shutdown_connections 2024-05-09T08:02:53.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.429+0000 7ff7b5c00640 1 --2- 172.21.15.1:0/2226944399 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ff7b00c6d10 0x7ff7b00c7640 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:53.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.429+0000 7ff7b5c00640 1 -- 172.21.15.1:0/2226944399 >> 172.21.15.1:0/2226944399 conn(0x7ff7b00cc070 msgr2=0x7ff7b00cc480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:02:53.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.429+0000 7ff7b5c00640 1 -- 172.21.15.1:0/2226944399 shutdown_connections 2024-05-09T08:02:53.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.429+0000 7ff7b5c00640 1 -- 172.21.15.1:0/2226944399 wait complete. 2024-05-09T08:02:53.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.431+0000 7ff7b5c00640 1 Processor -- start 2024-05-09T08:02:53.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.432+0000 7ff7b5c00640 1 -- start start 2024-05-09T08:02:53.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.433+0000 7ff7b5c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff7b006ed50 0x7ff7b01f5330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:53.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.433+0000 7ff7b5c00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff7b00c6d10 0x7ff7b015a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:53.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.433+0000 7ff7b5c00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ff7b0158cb0 0x7ff7b015fe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:53.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.433+0000 7ff7b5c00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ff7b00c2a30 con 0x7ff7b00c6d10 2024-05-09T08:02:53.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.433+0000 7ff7b5c00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff7b00c2750 con 0x7ff7b006ed50 2024-05-09T08:02:53.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.433+0000 7ff7b5c00640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7ff7b00c7d50 con 0x7ff7b0158cb0 2024-05-09T08:02:53.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.433+0000 7ff7ae000640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff7b00c6d10 0x7ff7b015a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:53.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.433+0000 7ff7ae000640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff7b00c6d10 0x7ff7b015a520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53064/0 (socket says 172.21.15.1:53064) 2024-05-09T08:02:53.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.433+0000 7ff7ae000640 1 -- 172.21.15.1:0/3709631519 learned_addr learned my addr 172.21.15.1:0/3709631519 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:02:53.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.433+0000 7ff7aea00640 1 --2- 172.21.15.1:0/3709631519 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff7b006ed50 0x7ff7b01f5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:53.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.433+0000 7ff7af400640 1 --2- 172.21.15.1:0/3709631519 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ff7b0158cb0 0x7ff7b015fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:53.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.434+0000 7ff7ae000640 1 -- 172.21.15.1:0/3709631519 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ff7b0158cb0 msgr2=0x7ff7b015fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:53.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.434+0000 7ff7ae000640 1 --2- 172.21.15.1:0/3709631519 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ff7b0158cb0 0x7ff7b015fe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:53.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.434+0000 7ff7ae000640 1 -- 172.21.15.1:0/3709631519 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff7b006ed50 msgr2=0x7ff7b01f5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:53.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.434+0000 7ff7ae000640 1 --2- 172.21.15.1:0/3709631519 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff7b006ed50 0x7ff7b01f5330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:53.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.434+0000 7ff7ae000640 1 -- 172.21.15.1:0/3709631519 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7b015c400 con 0x7ff7b00c6d10 2024-05-09T08:02:53.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.434+0000 7ff7aea00640 1 --2- 172.21.15.1:0/3709631519 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff7b006ed50 0x7ff7b01f5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:02:53.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.434+0000 7ff7ae000640 1 --2- 172.21.15.1:0/3709631519 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff7b00c6d10 0x7ff7b015a520 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7ff79c00c440 tx=0x7ff79c00c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:53.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.434+0000 7ff7a6a00640 1 -- 172.21.15.1:0/3709631519 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff79c017070 con 0x7ff7b00c6d10 2024-05-09T08:02:53.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.434+0000 7ff7a6a00640 1 -- 172.21.15.1:0/3709631519 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff79c005d00 con 0x7ff7b00c6d10 2024-05-09T08:02:53.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.434+0000 7ff7a6a00640 1 -- 172.21.15.1:0/3709631519 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff79c0072b0 con 0x7ff7b00c6d10 2024-05-09T08:02:53.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.434+0000 7ff7b5c00640 1 -- 172.21.15.1:0/3709631519 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7b0159480 con 0x7ff7b00c6d10 2024-05-09T08:02:53.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.435+0000 7ff7b5c00640 1 -- 172.21.15.1:0/3709631519 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7b0159a40 con 0x7ff7b00c6d10 2024-05-09T08:02:53.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.436+0000 7ff7b5c00640 1 -- 172.21.15.1:0/3709631519 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7b0074d70 con 0x7ff7b00c6d10 2024-05-09T08:02:53.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.437+0000 7ff7a6a00640 1 -- 172.21.15.1:0/3709631519 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff79c01a070 con 0x7ff7b00c6d10 2024-05-09T08:02:53.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.437+0000 7ff7a6a00640 1 --2- 172.21.15.1:0/3709631519 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7ff79006be30 0x7ff79006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:53.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.438+0000 7ff7a6a00640 1 -- 172.21.15.1:0/3709631519 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7ff79c08a4f0 con 0x7ff7b00c6d10 2024-05-09T08:02:53.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.438+0000 7ff7aea00640 1 --2- 172.21.15.1:0/3709631519 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7ff79006be30 0x7ff79006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:53.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.439+0000 7ff7aea00640 1 --2- 172.21.15.1:0/3709631519 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7ff79006be30 0x7ff79006e2f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7ff7a000f290 tx=0x7ff7a0009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:53.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.442+0000 7ff7a6a00640 1 -- 172.21.15.1:0/3709631519 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ff79c058540 con 0x7ff7b00c6d10 2024-05-09T08:02:53.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.644+0000 7ff7b5c00640 1 -- 172.21.15.1:0/3709631519 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff7b00cb3e0 con 0x7ff7b00c6d10 2024-05-09T08:02:53.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.645+0000 7ff7a6a00640 1 -- 172.21.15.1:0/3709631519 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18382 (secure 0 0 0) 0x7ff79c05c840 con 0x7ff7b00c6d10 2024-05-09T08:02:53.663 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T08:02:53.663 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":25,"fsid":"54329ec8-3270-4043-9a82-c6cd7420aa7f","created":"2024-05-09T08:01:56.666970+0000","modified":"2024-05-09T08:02:20.403456+0000","last_up_change":"2024-05-09T08:02:07.146082+0000","last_in_change":"2024-05-09T08:01:59.008225+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T08:02:09.165511+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T08:02:12.925485+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T08:02:14.367455+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-09T08:02:17.967064+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c064454e-a9b5-4a24-9a77-17f0a4085952","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6800","nonce":1513349547},{"type":"v1","addr":"172.21.15.1:6801","nonce":1513349547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":1513349547},{"type":"v1","addr":"172.21.15.1:6803","nonce":1513349547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":1513349547},{"type":"v1","addr":"172.21.15.1:6807","nonce":1513349547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":1513349547},{"type":"v1","addr":"172.21.15.1:6805","nonce":1513349547}]},"public_addr":"172.21.15.1:6801/1513349547","cluster_addr":"172.21.15.1:6803/1513349547","heartbeat_back_addr":"172.21.15.1:6807/1513349547","heartbeat_front_addr":"172.21.15.1:6805/1513349547","state":["exists","up"]},{"osd":1,"uuid":"1a9ea687-2ff6-407a-949b-0e7e1083af21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":2942255259},{"type":"v1","addr":"172.21.15.1:6817","nonce":2942255259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":2942255259},{"type":"v1","addr":"172.21.15.1:6819","nonce":2942255259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":2942255259},{"type":"v1","addr":"172.21.15.1:6823","nonce":2942255259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6820","nonce":2942255259},{"type":"v1","addr":"172.21.15.1:6821","nonce":2942255259}]},"public_addr":"172.21.15.1:6817/2942255259","cluster_addr":"172.21.15.1:6819/2942255259","heartbeat_back_addr":"172.21.15.1:6823/2942255259","heartbeat_front_addr":"172.21.15.1:6821/2942255259","state":["exists","up"]},{"osd":2,"uuid":"c88af08b-7c9a-4dd8-9640-44138239addb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":2925031811},{"type":"v1","addr":"172.21.15.1:6809","nonce":2925031811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":2925031811},{"type":"v1","addr":"172.21.15.1:6811","nonce":2925031811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":2925031811},{"type":"v1","addr":"172.21.15.1:6815","nonce":2925031811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":2925031811},{"type":"v1","addr":"172.21.15.1:6813","nonce":2925031811}]},"public_addr":"172.21.15.1:6809/2925031811","cluster_addr":"172.21.15.1:6811/2925031811","heartbeat_back_addr":"172.21.15.1:6815/2925031811","heartbeat_front_addr":"172.21.15.1:6813/2925031811","state":["exists","up"]},{"osd":3,"uuid":"c087b7e3-a41d-4333-9365-32e3bd8c8e6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6824","nonce":2497776654},{"type":"v1","addr":"172.21.15.1:6825","nonce":2497776654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":2497776654},{"type":"v1","addr":"172.21.15.1:6827","nonce":2497776654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":2497776654},{"type":"v1","addr":"172.21.15.1:6831","nonce":2497776654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6828","nonce":2497776654},{"type":"v1","addr":"172.21.15.1:6829","nonce":2497776654}]},"public_addr":"172.21.15.1:6825/2497776654","cluster_addr":"172.21.15.1:6827/2497776654","heartbeat_back_addr":"172.21.15.1:6831/2497776654","heartbeat_front_addr":"172.21.15.1:6829/2497776654","state":["exists","up"]},{"osd":4,"uuid":"b4576b9c-871d-47a8-8d36-65af525d0200","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":2957026670},{"type":"v1","addr":"172.21.15.7:6801","nonce":2957026670}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2957026670},{"type":"v1","addr":"172.21.15.7:6803","nonce":2957026670}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":2957026670},{"type":"v1","addr":"172.21.15.7:6807","nonce":2957026670}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2957026670},{"type":"v1","addr":"172.21.15.7:6805","nonce":2957026670}]},"public_addr":"172.21.15.7:6801/2957026670","cluster_addr":"172.21.15.7:6803/2957026670","heartbeat_back_addr":"172.21.15.7:6807/2957026670","heartbeat_front_addr":"172.21.15.7:6805/2957026670","state":["exists","up"]},{"osd":5,"uuid":"902b3d94-72f3-4499-b21a-e32453ef5144","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1670295242},{"type":"v1","addr":"172.21.15.7:6809","nonce":1670295242}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1670295242},{"type":"v1","addr":"172.21.15.7:6811","nonce":1670295242}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1670295242},{"type":"v1","addr":"172.21.15.7:6815","nonce":1670295242}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1670295242},{"type":"v1","addr":"172.21.15.7:6813","nonce":1670295242}]},"public_addr":"172.21.15.7:6809/1670295242","cluster_addr":"172.21.15.7:6811/1670295242","heartbeat_back_addr":"172.21.15.7:6815/1670295242","heartbeat_front_addr":"172.21.15.7:6813/1670295242","state":["exists","up"]},{"osd":6,"uuid":"ead1d771-1728-4f85-adb4-26be21cedff3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":2852086880},{"type":"v1","addr":"172.21.15.7:6817","nonce":2852086880}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":2852086880},{"type":"v1","addr":"172.21.15.7:6819","nonce":2852086880}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":2852086880},{"type":"v1","addr":"172.21.15.7:6823","nonce":2852086880}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":2852086880},{"type":"v1","addr":"172.21.15.7:6821","nonce":2852086880}]},"public_addr":"172.21.15.7:6817/2852086880","cluster_addr":"172.21.15.7:6819/2852086880","heartbeat_back_addr":"172.21.15.7:6823/2852086880","heartbeat_front_addr":"172.21.15.7:6821/2852086880","state":["exists","up"]},{"osd":7,"uuid":"3ab5a201-9394-48a8-8918-e95197893669","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":1442884138},{"type":"v1","addr":"172.21.15.7:6825","nonce":1442884138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1442884138},{"type":"v1","addr":"172.21.15.7:6827","nonce":1442884138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1442884138},{"type":"v1","addr":"172.21.15.7:6831","nonce":1442884138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1442884138},{"type":"v1","addr":"172.21.15.7:6829","nonce":1442884138}]},"public_addr":"172.21.15.7:6825/1442884138","cluster_addr":"172.21.15.7:6827/1442884138","heartbeat_back_addr":"172.21.15.7:6831/1442884138","heartbeat_front_addr":"172.21.15.7:6829/1442884138","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:02:04.615675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:02:04.385974+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:02:04.619217+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:02:04.401681+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:02:04.097081+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:02:04.129981+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:02:04.225327+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:02:04.193351+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T08:02:53.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.649+0000 7ff7b5c00640 1 -- 172.21.15.1:0/3709631519 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7ff79006be30 msgr2=0x7ff79006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:53.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.649+0000 7ff7b5c00640 1 --2- 172.21.15.1:0/3709631519 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7ff79006be30 0x7ff79006e2f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7ff7a000f290 tx=0x7ff7a0009040 comp rx=0 tx=0).stop 2024-05-09T08:02:53.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.649+0000 7ff7b5c00640 1 -- 172.21.15.1:0/3709631519 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff7b00c6d10 msgr2=0x7ff7b015a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:53.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.649+0000 7ff7b5c00640 1 --2- 172.21.15.1:0/3709631519 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff7b00c6d10 0x7ff7b015a520 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7ff79c00c440 tx=0x7ff79c00c900 comp rx=0 tx=0).stop 2024-05-09T08:02:53.667 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.649+0000 7ff7b5c00640 1 -- 172.21.15.1:0/3709631519 shutdown_connections 2024-05-09T08:02:53.667 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.649+0000 7ff7b5c00640 1 --2- 172.21.15.1:0/3709631519 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7ff79006be30 0x7ff79006e2f0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:53.667 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.649+0000 7ff7b5c00640 1 --2- 172.21.15.1:0/3709631519 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ff7b0158cb0 0x7ff7b015fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:53.667 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.649+0000 7ff7b5c00640 1 --2- 172.21.15.1:0/3709631519 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff7b00c6d10 0x7ff7b015a520 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:53.667 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.649+0000 7ff7b5c00640 1 --2- 172.21.15.1:0/3709631519 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff7b006ed50 0x7ff7b01f5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:53.667 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.649+0000 7ff7b5c00640 1 -- 172.21.15.1:0/3709631519 >> 172.21.15.1:0/3709631519 conn(0x7ff7b00cc070 msgr2=0x7ff7b00cac40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:02:53.667 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.649+0000 7ff7b5c00640 1 -- 172.21.15.1:0/3709631519 shutdown_connections 2024-05-09T08:02:53.667 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.649+0000 7ff7b5c00640 1 -- 172.21.15.1:0/3709631519 wait complete. 2024-05-09T08:02:53.686 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-05-09T08:02:53.687 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-05-09T08:02:53.687 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-05-09T08:02:53.688 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-05-09T08:02:53.688 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-05-09T08:02:53.688 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-05-09T08:02:53.689 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-05-09T08:02:53.689 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-05-09T08:02:53.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.969+0000 7f9075400640 1 Processor -- start 2024-05-09T08:02:53.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.969+0000 7f9075400640 1 -- start start 2024-05-09T08:02:53.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.971+0000 7f9075400640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f9070153be0 0x7f9070159a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:53.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.972+0000 7fa62c600640 1 Processor -- start 2024-05-09T08:02:53.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.972+0000 7fa62c600640 1 -- start start 2024-05-09T08:02:53.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.973+0000 7f7c0e000640 1 Processor -- start 2024-05-09T08:02:53.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.973+0000 7fbbcee00640 1 Processor -- start 2024-05-09T08:02:53.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.974+0000 7f7c0e000640 1 -- start start 2024-05-09T08:02:53.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.974+0000 7fbbcee00640 1 -- start start 2024-05-09T08:02:53.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.971+0000 7f9075400640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f9070157200 0x7f9070160d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:53.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.972+0000 7f9075400640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f907015a8e0 con 0x7f9070156de0 2024-05-09T08:02:53.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.972+0000 7f9075400640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f907015a040 con 0x7f9070153be0 2024-05-09T08:02:53.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.972+0000 7f9075400640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f907015a320 con 0x7f9070157200 2024-05-09T08:02:53.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.972+0000 7f906de00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f9070156de0 0x7f90700730d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.1:41992/0 (socket says 172.21.15.1:41992) 2024-05-09T08:02:53.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.972+0000 7f906de00640 1 -- 172.21.15.1:0/1517525595 learned_addr learned my addr 172.21.15.1:0/1517525595 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T08:02:53.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.973+0000 7f906d400640 1 -- 172.21.15.1:0/1517525595 <== mon.1 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 272990936 0 0) 0x7f907015a8e0 con 0x7f9070156de0 2024-05-09T08:02:53.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.973+0000 7f906d400640 1 -- 172.21.15.1:0/1517525595 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f905400bd30 con 0x7f9070156de0 2024-05-09T08:02:53.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.973+0000 7f906f200640 1 --2- 172.21.15.1:0/1517525595 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f9070157200 0x7f9070160d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:53.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.974+0000 7f906f200640 1 -- 172.21.15.1:0/1517525595 >> v2:172.21.15.1:3300/0 conn(0x7f9070153be0 msgr2=0x7f9070159a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:02:53.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.974+0000 7f906f200640 1 --2- 172.21.15.1:0/1517525595 >> v2:172.21.15.1:3300/0 conn(0x7f9070153be0 0x7f9070159a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:53.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.974+0000 7f906f200640 1 -- 172.21.15.1:0/1517525595 >> v1:172.21.15.7:6789/0 conn(0x7f9070156de0 legacy=0x7f90700730d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:53.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.974+0000 7f906f200640 1 -- 172.21.15.1:0/1517525595 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f907015a600 con 0x7f9070157200 2024-05-09T08:02:53.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.975+0000 7fa62c600640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fa62406cce0 0x7fa6240c81f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:53.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.975+0000 7fa62c600640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa62415b860 con 0x7fa62406c900 2024-05-09T08:02:53.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.975+0000 7fa62c600640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa6240b5de0 con 0x7fa6240c8730 2024-05-09T08:02:53.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.975+0000 7fa62c600640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7fa6240c7f30 con 0x7fa62406cce0 2024-05-09T08:02:53.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.975+0000 7fa62a600640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fa6240c8730 0x7fa6241547c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.1:42004/0 (socket says 172.21.15.1:42004) 2024-05-09T08:02:53.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.976+0000 7f88f9600640 1 Processor -- start 2024-05-09T08:02:53.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.976+0000 7fa629200640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fa62406cce0 0x7fa6240c81f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:53.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.976+0000 7fa629200640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fa62406cce0 0x7fa6240c81f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3301/0 says I am v2:172.21.15.1:44468/0 (socket says 172.21.15.1:44468) 2024-05-09T08:02:53.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.976+0000 7fa629200640 1 -- 172.21.15.1:0/4268577264 learned_addr learned my addr 172.21.15.1:0/4268577264 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:02:53.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.976+0000 7f88f9600640 1 -- start start 2024-05-09T08:02:53.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.976+0000 7f7c0e000640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f7c080cc5f0 0x7f7c0815a570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:53.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.976+0000 7f7c0e000640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f7c080ce4e0 0x7f7c0815aab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:53.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.976+0000 7f7c0e000640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c080ccb20 con 0x7f7c080cac60 2024-05-09T08:02:53.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.976+0000 7f7c0e000640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f7c080cce00 con 0x7f7c080ce4e0 2024-05-09T08:02:53.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.976+0000 7f906f200640 1 --2- 172.21.15.1:0/1517525595 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f9070157200 0x7f9070160d70 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f90580097a0 tx=0x7f905802a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=af15adedb4478fef server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:53.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.976+0000 7f7c0e000640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f7c080cf120 con 0x7f7c080cc5f0 2024-05-09T08:02:53.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.976+0000 7f7c07e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f7c080cc5f0 0x7f7c0815a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:53.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.976+0000 7f7c07e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f7c080cc5f0 0x7f7c0815a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.1:58740/0 (socket says 172.21.15.1:58740) 2024-05-09T08:02:53.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.976+0000 7f7c07e00640 1 -- 172.21.15.1:0/1899812427 learned_addr learned my addr 172.21.15.1:0/1899812427 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:02:53.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.976+0000 7fa623e00640 1 -- 172.21.15.1:0/4268577264 <== mon.1 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 884947070 0 0) 0x7fa6240b5de0 con 0x7fa6240c8730 2024-05-09T08:02:53.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.976+0000 7fa623e00640 1 -- 172.21.15.1:0/4268577264 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa60c003520 con 0x7fa6240c8730 2024-05-09T08:02:53.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.977+0000 7fa629200640 1 -- 172.21.15.1:0/4268577264 >> v1:172.21.15.7:6789/0 conn(0x7fa6240c8730 legacy=0x7fa6241547c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:53.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.977+0000 7f7c0d600640 1 --2- 172.21.15.1:0/1899812427 >> v2:172.21.15.1:3300/0 conn(0x7f7c080ce4e0 0x7f7c0815aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:53.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.977+0000 7f906d400640 1 -- 172.21.15.1:0/1517525595 <== mon.2 v2:172.21.15.7:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9058037070 con 0x7f9070157200 2024-05-09T08:02:53.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.977+0000 7f906d400640 1 -- 172.21.15.1:0/1517525595 <== mon.2 v2:172.21.15.7:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f905802ac10 con 0x7f9070157200 2024-05-09T08:02:53.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.977+0000 7f906d400640 1 -- 172.21.15.1:0/1517525595 <== mon.2 v2:172.21.15.7:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9060001670 con 0x7f9070157200 2024-05-09T08:02:53.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.978+0000 7fa623e00640 1 -- 172.21.15.1:0/4268577264 <== mon.1 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 388990168 0 0) 0x7fa60c003520 con 0x7fa6240c8730 2024-05-09T08:02:53.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.978+0000 7f88f9600640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f88f40c88a0 0x7f88f40c80e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:53.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.978+0000 7f88f9600640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f88f40c69a0 0x7f88f415a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:53.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.978+0000 7f88f9600640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f88f40c8c80 0x7f88f415e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:53.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.978+0000 7f88f9600640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f88f415b460 con 0x7f88f40c69a0 2024-05-09T08:02:53.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.978+0000 7f88f9600640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f88f40c9110 con 0x7f88f40c88a0 2024-05-09T08:02:53.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.978+0000 7f88f9600640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f88f415abc0 con 0x7f88f40c8c80 2024-05-09T08:02:53.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.978+0000 7fa629200640 1 -- 172.21.15.1:0/4268577264 >> v1:172.21.15.1:6789/0 conn(0x7fa62406c900 legacy=0x7fa6240c7280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:53.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.978+0000 7fa629200640 1 -- 172.21.15.1:0/4268577264 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6240c7990 con 0x7fa62406cce0 2024-05-09T08:02:53.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.978+0000 7f9075400640 1 -- 172.21.15.1:0/1517525595 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f9070157200 msgr2=0x7f9070160d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:54.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.978+0000 7f9075400640 1 --2- 172.21.15.1:0/1517525595 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f9070157200 0x7f9070160d70 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f90580097a0 tx=0x7f905802a180 comp rx=0 tx=0).stop 2024-05-09T08:02:54.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.977+0000 7fbbcee00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fbbc00dac60 0x7fbbc00d4d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:54.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.978+0000 7fbbcee00640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fbbc004b500 0x7fbbc004b950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:54.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.978+0000 7fbbcee00640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbbc0006900 con 0x7fbbc0050ca0 2024-05-09T08:02:54.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.978+0000 7fbbcee00640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7fbbc0006be0 con 0x7fbbc004b500 2024-05-09T08:02:54.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.978+0000 7fbbcee00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fbbc0006ec0 con 0x7fbbc00dac60 2024-05-09T08:02:54.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.979+0000 7f88f3e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f88f40c88a0 0x7f88f40c80e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:54.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.979+0000 7f88f3e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f88f40c88a0 0x7f88f40c80e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.1:58752/0 (socket says 172.21.15.1:58752) 2024-05-09T08:02:54.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.979+0000 7f88f3e00640 1 -- 172.21.15.1:0/3036592946 learned_addr learned my addr 172.21.15.1:0/3036592946 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:02:54.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.978+0000 7fbbcd000640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fbbc00dac60 0x7fbbc00d4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:54.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.979+0000 7fbbcd000640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fbbc00dac60 0x7fbbc00d4d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.1:58742/0 (socket says 172.21.15.1:58742) 2024-05-09T08:02:54.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.979+0000 7fbbcd000640 1 -- 172.21.15.1:0/509447966 learned_addr learned my addr 172.21.15.1:0/509447966 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:02:54.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.979+0000 7f88f8c00640 1 --2- 172.21.15.1:0/3036592946 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f88f40c8c80 0x7f88f415e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:54.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.979+0000 7f9075400640 1 -- 172.21.15.1:0/1517525595 shutdown_connections 2024-05-09T08:02:54.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.979+0000 7f9075400640 1 --2- 172.21.15.1:0/1517525595 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f9070157200 0x7f9070160d70 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:54.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.979+0000 7f9075400640 1 --2- 172.21.15.1:0/1517525595 >> v2:172.21.15.1:3300/0 conn(0x7f9070153be0 0x7f9070159a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:54.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.979+0000 7f9075400640 1 -- 172.21.15.1:0/1517525595 >> 172.21.15.1:0/1517525595 conn(0x7f907006f440 msgr2=0x7f907006f820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:02:54.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.979+0000 7f7c07400640 1 -- 172.21.15.1:0/1899812427 <== mon.1 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4123932623 0 0) 0x7f7c080ccb20 con 0x7f7c080cac60 2024-05-09T08:02:54.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.979+0000 7f7c07e00640 1 -- 172.21.15.1:0/1899812427 >> v2:172.21.15.1:3300/0 conn(0x7f7c080ce4e0 msgr2=0x7f7c0815aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:02:54.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.979+0000 7f7c07e00640 1 --2- 172.21.15.1:0/1899812427 >> v2:172.21.15.1:3300/0 conn(0x7f7c080ce4e0 0x7f7c0815aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:54.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.979+0000 7f7c07e00640 1 -- 172.21.15.1:0/1899812427 >> v1:172.21.15.7:6789/0 conn(0x7f7c080cac60 legacy=0x7f7c080cda00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:54.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.979+0000 7f7c07e00640 1 -- 172.21.15.1:0/1899812427 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c080cf400 con 0x7f7c080cc5f0 2024-05-09T08:02:54.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.979+0000 7fbbcd000640 1 -- 172.21.15.1:0/509447966 >> v2:172.21.15.1:3300/0 conn(0x7fbbc004b500 msgr2=0x7fbbc004b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:02:54.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.979+0000 7fbbcd000640 1 --2- 172.21.15.1:0/509447966 >> v2:172.21.15.1:3300/0 conn(0x7fbbc004b500 0x7fbbc004b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:54.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.979+0000 7f88f3e00640 1 -- 172.21.15.1:0/3036592946 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f88f40c8c80 msgr2=0x7f88f415e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:02:54.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.979+0000 7f88f3e00640 1 --2- 172.21.15.1:0/3036592946 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f88f40c8c80 0x7f88f415e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:54.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.979+0000 7f88f3e00640 1 -- 172.21.15.1:0/3036592946 >> v2:172.21.15.1:3300/0 conn(0x7f88f40c69a0 msgr2=0x7f88f415a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:02:54.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.979+0000 7f88f3e00640 1 --2- 172.21.15.1:0/3036592946 >> v2:172.21.15.1:3300/0 conn(0x7f88f40c69a0 0x7f88f415a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:54.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.979+0000 7f88f3e00640 1 -- 172.21.15.1:0/3036592946 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88f415aea0 con 0x7f88f40c88a0 2024-05-09T08:02:54.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.980+0000 7f9075400640 1 -- 172.21.15.1:0/1517525595 shutdown_connections 2024-05-09T08:02:54.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.979+0000 7fbbcd000640 1 -- 172.21.15.1:0/509447966 >> v1:172.21.15.1:6789/0 conn(0x7fbbc0050ca0 legacy=0x7fbbc0005ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:02:54.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.979+0000 7fbbcd000640 1 -- 172.21.15.1:0/509447966 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbc00071a0 con 0x7fbbc00dac60 2024-05-09T08:02:54.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.980+0000 7f7c0d600640 1 --2- 172.21.15.1:0/1899812427 >> v2:172.21.15.1:3300/0 conn(0x7f7c080ce4e0 0x7f7c0815aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:02:54.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.980+0000 7f9075400640 1 -- 172.21.15.1:0/1517525595 wait complete. 2024-05-09T08:02:54.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.980+0000 7fa629200640 1 --2- 172.21.15.1:0/4268577264 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fa62406cce0 0x7fa6240c81f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fa610009820 tx=0x7fa61002a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=18ab4ed6cadd2a1b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:54.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.980+0000 7ffba5600640 1 Processor -- start 2024-05-09T08:02:54.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.981+0000 7fa623e00640 1 -- 172.21.15.1:0/4268577264 <== mon.2 v2:172.21.15.7:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa610037070 con 0x7fa62406cce0 2024-05-09T08:02:54.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.981+0000 7fa623e00640 1 -- 172.21.15.1:0/4268577264 <== mon.2 v2:172.21.15.7:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa61002ad20 con 0x7fa62406cce0 2024-05-09T08:02:54.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.981+0000 7fa623e00640 1 -- 172.21.15.1:0/4268577264 <== mon.2 v2:172.21.15.7:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa61c001650 con 0x7fa62406cce0 2024-05-09T08:02:54.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.981+0000 7fbbcd000640 1 --2- 172.21.15.1:0/509447966 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbbc00dac60 0x7fbbc00d4d90 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fbbc00d5520 tx=0x7fbbb402a2c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cf9c8fe7a0e874dd server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:54.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.981+0000 7fbbc7e00640 1 -- 172.21.15.1:0/509447966 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbb4037070 con 0x7fbbc00dac60 2024-05-09T08:02:54.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.982+0000 7fbbc7e00640 1 -- 172.21.15.1:0/509447966 <== mon.1 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbbb4032440 con 0x7fbbc00dac60 2024-05-09T08:02:54.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.981+0000 7ffba5600640 1 -- start start 2024-05-09T08:02:54.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.982+0000 7f7c07e00640 1 --2- 172.21.15.1:0/1899812427 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c080cc5f0 0x7f7c0815a570 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f7c08154dd0 tx=0x7f7bf402a730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4689cbcbe71f9bec server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:54.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.982+0000 7fa62c600640 1 -- 172.21.15.1:0/4268577264 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fa62406cce0 msgr2=0x7fa6240c81f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:54.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.982+0000 7fa62c600640 1 --2- 172.21.15.1:0/4268577264 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fa62406cce0 0x7fa6240c81f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fa610009820 tx=0x7fa61002a210 comp rx=0 tx=0).stop 2024-05-09T08:02:54.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.982+0000 7fa62c600640 1 -- 172.21.15.1:0/4268577264 shutdown_connections 2024-05-09T08:02:54.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.982+0000 7fa62c600640 1 --2- 172.21.15.1:0/4268577264 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fa62406cce0 0x7fa6240c81f0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:54.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.982+0000 7fa62c600640 1 -- 172.21.15.1:0/4268577264 >> 172.21.15.1:0/4268577264 conn(0x7fa6240cfe80 msgr2=0x7fa6240d0290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:02:54.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.982+0000 7fa62c600640 1 -- 172.21.15.1:0/4268577264 shutdown_connections 2024-05-09T08:02:54.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.982+0000 7fa62c600640 1 -- 172.21.15.1:0/4268577264 wait complete. 2024-05-09T08:02:54.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.982+0000 7fbbc7e00640 1 -- 172.21.15.1:0/509447966 <== mon.1 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbb4032810 con 0x7fbbc00dac60 2024-05-09T08:02:54.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.983+0000 7f7c07400640 1 -- 172.21.15.1:0/1899812427 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bf4036070 con 0x7f7c080cc5f0 2024-05-09T08:02:54.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.983+0000 7f7c07400640 1 -- 172.21.15.1:0/1899812427 <== mon.1 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7bf4031600 con 0x7f7c080cc5f0 2024-05-09T08:02:54.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.983+0000 7fbbcee00640 1 -- 172.21.15.1:0/509447966 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbbc00dac60 msgr2=0x7fbbc00d4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:54.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.983+0000 7fbbcee00640 1 --2- 172.21.15.1:0/509447966 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbbc00dac60 0x7fbbc00d4d90 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fbbc00d5520 tx=0x7fbbb402a2c0 comp rx=0 tx=0).stop 2024-05-09T08:02:54.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.983+0000 7f7c07400640 1 -- 172.21.15.1:0/1899812427 <== mon.1 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bf40319d0 con 0x7f7c080cc5f0 2024-05-09T08:02:54.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.983+0000 7fbbcee00640 1 -- 172.21.15.1:0/509447966 shutdown_connections 2024-05-09T08:02:54.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.983+0000 7ffba5600640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7ffba0158b90 0x7ffba01625e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:54.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.983+0000 7f9075400640 1 Processor -- start 2024-05-09T08:02:54.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.984+0000 7fbbcee00640 1 --2- 172.21.15.1:0/509447966 >> v2:172.21.15.1:3300/0 conn(0x7fbbc004b500 0x7fbbc004b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:54.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.984+0000 7f9075400640 1 -- start start 2024-05-09T08:02:54.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.984+0000 7f7c0e000640 1 -- 172.21.15.1:0/1899812427 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c080cc5f0 msgr2=0x7f7c0815a570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:54.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.984+0000 7fbbcee00640 1 --2- 172.21.15.1:0/509447966 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbbc00dac60 0x7fbbc00d4d90 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:54.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.984+0000 7fbbcee00640 1 -- 172.21.15.1:0/509447966 >> 172.21.15.1:0/509447966 conn(0x7fbbc0050180 msgr2=0x7fbbc0050590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:02:54.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.984+0000 7f7c0e000640 1 --2- 172.21.15.1:0/1899812427 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c080cc5f0 0x7f7c0815a570 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f7c08154dd0 tx=0x7f7bf402a730 comp rx=0 tx=0).stop 2024-05-09T08:02:54.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.984+0000 7f7c0e000640 1 -- 172.21.15.1:0/1899812427 shutdown_connections 2024-05-09T08:02:54.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.984+0000 7f7c0e000640 1 --2- 172.21.15.1:0/1899812427 >> v2:172.21.15.1:3300/0 conn(0x7f7c080ce4e0 0x7f7c0815aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:54.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.984+0000 7f7c0e000640 1 --2- 172.21.15.1:0/1899812427 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c080cc5f0 0x7f7c0815a570 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:54.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.984+0000 7f7c0e000640 1 -- 172.21.15.1:0/1899812427 >> 172.21.15.1:0/1899812427 conn(0x7f7c080cfc20 msgr2=0x7f7c080d0030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:02:54.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.984+0000 7fbbcee00640 1 -- 172.21.15.1:0/509447966 shutdown_connections 2024-05-09T08:02:54.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.984+0000 7fbbcee00640 1 -- 172.21.15.1:0/509447966 wait complete. 2024-05-09T08:02:54.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.984+0000 7f7c0e000640 1 -- 172.21.15.1:0/1899812427 shutdown_connections 2024-05-09T08:02:54.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.984+0000 7ffb9ea00640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7ffba0158b90 0x7ffba01625e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:54.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.984+0000 7f7c0e000640 1 -- 172.21.15.1:0/1899812427 wait complete. 2024-05-09T08:02:54.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.984+0000 7ffb9f400640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7ffba00c87f0 0x7ffba01692b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.1:42032/0 (socket says 172.21.15.1:42032) 2024-05-09T08:02:54.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.985+0000 7ffb9f400640 1 -- 172.21.15.1:0/1679279904 learned_addr learned my addr 172.21.15.1:0/1679279904 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T08:02:54.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.985+0000 7ffb9ea00640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7ffba0158b90 0x7ffba01625e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:37998/0 (socket says 172.21.15.1:37998) 2024-05-09T08:02:54.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.985+0000 7ffba5600640 1 -- 172.21.15.1:0/1679279904 --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffba015ee40 con 0x7ffba015e9b0 2024-05-09T08:02:54.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.985+0000 7ffba5600640 1 -- 172.21.15.1:0/1679279904 --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffba015f120 con 0x7ffba00c87f0 2024-05-09T08:02:54.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.985+0000 7f6260a00640 1 Processor -- start 2024-05-09T08:02:54.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.985+0000 7fa62c600640 1 Processor -- start 2024-05-09T08:02:54.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.985+0000 7ffba5600640 1 -- 172.21.15.1:0/1679279904 --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7ffba0162150 con 0x7ffba0158b90 2024-05-09T08:02:54.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.986+0000 7fa62c600640 1 -- start start 2024-05-09T08:02:54.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.986+0000 7f6260a00640 1 -- start start 2024-05-09T08:02:54.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.986+0000 7f9075400640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f9070153be0 0x7f90701ef9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:54.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.986+0000 7ffb9d600640 1 -- 172.21.15.1:0/1679279904 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3144240014 0 0) 0x7ffba015ee40 con 0x7ffba015e9b0 2024-05-09T08:02:54.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.986+0000 7f9075400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9070156de0 0x7f907014bf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:54.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.986+0000 7f9075400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9070157200 0x7f90701593f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:54.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.986+0000 7f88f3e00640 1 --2- 172.21.15.1:0/3036592946 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f88f40c88a0 0x7f88f40c80e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f88f4070490 tx=0x7f88e402a1d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4939daeec5c2a626 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:54.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.986+0000 7ffb9d600640 1 -- 172.21.15.1:0/1679279904 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffb84003520 con 0x7ffba015e9b0 2024-05-09T08:02:54.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.986+0000 7ffb9d600640 1 -- 172.21.15.1:0/1679279904 <== mon.1 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2402446734 0 0) 0x7ffba015f120 con 0x7ffba00c87f0 2024-05-09T08:02:54.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.986+0000 7ffb9d600640 1 -- 172.21.15.1:0/1679279904 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffba015ee40 con 0x7ffba00c87f0 2024-05-09T08:02:54.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.987+0000 7f9075400640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f90700c3010 con 0x7f9070156de0 2024-05-09T08:02:54.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.987+0000 7f88f2a00640 1 -- 172.21.15.1:0/3036592946 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88e4037070 con 0x7f88f40c88a0 2024-05-09T08:02:54.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.987+0000 7f88f2a00640 1 -- 172.21.15.1:0/3036592946 <== mon.1 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88e4032440 con 0x7f88f40c88a0 2024-05-09T08:02:54.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.987+0000 7f88f2a00640 1 -- 172.21.15.1:0/3036592946 <== mon.1 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88e4032810 con 0x7f88f40c88a0 2024-05-09T08:02:54.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.986+0000 7ffb9ea00640 1 -- 172.21.15.1:0/1679279904 >> v1:172.21.15.7:6789/0 conn(0x7ffba00c87f0 legacy=0x7ffba01692b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:54.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.986+0000 7ffb9ea00640 1 -- 172.21.15.1:0/1679279904 >> v1:172.21.15.1:6789/0 conn(0x7ffba015e9b0 legacy=0x7ffba006f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:54.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.986+0000 7ffb9ea00640 1 -- 172.21.15.1:0/1679279904 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffba015f400 con 0x7ffba0158b90 2024-05-09T08:02:54.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.987+0000 7f6260a00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f625c06cce0 0x7f625c0c81f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:54.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.987+0000 7fbbcee00640 1 Processor -- start 2024-05-09T08:02:54.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.987+0000 7f6260a00640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f625c15b860 con 0x7f625c06c900 2024-05-09T08:02:54.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.988+0000 7f6260a00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f625c0b5de0 con 0x7f625c0c8730 2024-05-09T08:02:54.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.988+0000 7f6260a00640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f625c0c7f30 con 0x7f625c06cce0 2024-05-09T08:02:54.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.987+0000 7f9075400640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f90701568c0 con 0x7f9070157200 2024-05-09T08:02:54.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.987+0000 7f906e800640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f9070153be0 0x7f90701ef9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:54.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.987+0000 7f906e800640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f9070153be0 0x7f90701ef9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3301/0 says I am v2:172.21.15.1:44472/0 (socket says 172.21.15.1:44472) 2024-05-09T08:02:54.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.987+0000 7f906e800640 1 -- 172.21.15.1:0/2896389809 learned_addr learned my addr 172.21.15.1:0/2896389809 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:02:54.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.988+0000 7f9075400640 1 -- 172.21.15.1:0/2896389809 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f90700737e0 con 0x7f9070153be0 2024-05-09T08:02:54.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.988+0000 7f906f200640 1 --2- 172.21.15.1:0/2896389809 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9070157200 0x7f90701593f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:54.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.988+0000 7fa62c600640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa62406c900 0x7fa6241508f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:54.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.988+0000 7f7c0e000640 1 Processor -- start 2024-05-09T08:02:54.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.988+0000 7f88f9600640 1 -- 172.21.15.1:0/3036592946 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f88f40c88a0 msgr2=0x7f88f40c80e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:54.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.988+0000 7f6259400640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f625c06cce0 0x7f625c0c81f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:54.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.988+0000 7f6259400640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f625c06cce0 0x7f625c0c81f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3301/0 says I am v2:172.21.15.1:44474/0 (socket says 172.21.15.1:44474) 2024-05-09T08:02:54.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.988+0000 7f6259400640 1 -- 172.21.15.1:0/3405581470 learned_addr learned my addr 172.21.15.1:0/3405581470 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:02:54.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.988+0000 7fbbcee00640 1 -- start start 2024-05-09T08:02:54.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.988+0000 7f7c0e000640 1 -- start start 2024-05-09T08:02:54.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.988+0000 7ffb9ea00640 1 --2- 172.21.15.1:0/1679279904 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffba0158b90 0x7ffba01625e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ffb900097a0 tx=0x7ffb9002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b5d5fc59101137e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:54.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.988+0000 7f88f9600640 1 --2- 172.21.15.1:0/3036592946 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f88f40c88a0 0x7f88f40c80e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f88f4070490 tx=0x7f88e402a1d0 comp rx=0 tx=0).stop 2024-05-09T08:02:54.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.989+0000 7f6259400640 1 -- 172.21.15.1:0/3405581470 >> v1:172.21.15.7:6789/0 conn(0x7f625c0c8730 legacy=0x7f625c1547c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:02:54.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.988+0000 7f906de00640 1 --2- 172.21.15.1:0/2896389809 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9070156de0 0x7f907014bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:54.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.989+0000 7f88f9600640 1 -- 172.21.15.1:0/3036592946 shutdown_connections 2024-05-09T08:02:54.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.989+0000 7f88f9600640 1 --2- 172.21.15.1:0/3036592946 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f88f40c8c80 0x7f88f415e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:54.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.989+0000 7f6259400640 1 -- 172.21.15.1:0/3405581470 >> v1:172.21.15.1:6789/0 conn(0x7f625c06c900 legacy=0x7f625c0c7280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:02:54.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.989+0000 7f88f9600640 1 --2- 172.21.15.1:0/3036592946 >> v2:172.21.15.1:3300/0 conn(0x7f88f40c69a0 0x7f88f415a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:54.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.989+0000 7f88f9600640 1 --2- 172.21.15.1:0/3036592946 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f88f40c88a0 0x7f88f40c80e0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:54.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.989+0000 7ffb9d600640 1 -- 172.21.15.1:0/1679279904 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb90037070 con 0x7ffba0158b90 2024-05-09T08:02:54.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.989+0000 7ffb9d600640 1 -- 172.21.15.1:0/1679279904 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb90032440 con 0x7ffba0158b90 2024-05-09T08:02:54.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.989+0000 7ffb9d600640 1 -- 172.21.15.1:0/1679279904 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb90032810 con 0x7ffba0158b90 2024-05-09T08:02:54.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.989+0000 7f88f9600640 1 -- 172.21.15.1:0/3036592946 >> 172.21.15.1:0/3036592946 conn(0x7f88f406b250 msgr2=0x7f88f406b630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:02:54.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.989+0000 7fa629c00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa62406c900 0x7fa6241508f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:54.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.989+0000 7fa629c00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa62406c900 0x7fa6241508f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38008/0 (socket says 172.21.15.1:38008) 2024-05-09T08:02:54.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.989+0000 7fa629c00640 1 -- 172.21.15.1:0/1791582975 learned_addr learned my addr 172.21.15.1:0/1791582975 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:02:54.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.989+0000 7fa3a6200640 1 Processor -- start 2024-05-09T08:02:54.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.989+0000 7f906f200640 1 -- 172.21.15.1:0/2896389809 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f9070153be0 msgr2=0x7f90701ef9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:54.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.989+0000 7f88f9600640 1 -- 172.21.15.1:0/3036592946 shutdown_connections 2024-05-09T08:02:54.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.989+0000 7f906f200640 1 --2- 172.21.15.1:0/2896389809 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f9070153be0 0x7f90701ef9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:54.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.989+0000 7f906f200640 1 -- 172.21.15.1:0/2896389809 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9070156de0 msgr2=0x7f907014bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:54.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.989+0000 7f906f200640 1 --2- 172.21.15.1:0/2896389809 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9070156de0 0x7f907014bf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:54.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.989+0000 7f906f200640 1 -- 172.21.15.1:0/2896389809 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f907014a900 con 0x7f9070157200 2024-05-09T08:02:54.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.990+0000 7f906de00640 1 --2- 172.21.15.1:0/2896389809 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9070156de0 0x7f907014bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T08:02:54.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.990+0000 7f906e800640 1 --2- 172.21.15.1:0/2896389809 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f9070153be0 0x7f90701ef9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:02:54.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.990+0000 7f7c0e000640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7c080cac60 0x7f7c0814d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:54.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.990+0000 7fa62c600640 1 --2- 172.21.15.1:0/1791582975 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fa62406cce0 0x7fa62414f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:54.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.990+0000 7fa62c600640 1 --2- 172.21.15.1:0/1791582975 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6240c8730 0x7fa62414e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:54.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.990+0000 7fa629200640 1 --2- 172.21.15.1:0/1791582975 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fa62406cce0 0x7fa62414f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:54.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.990+0000 7f88f9600640 1 -- 172.21.15.1:0/3036592946 wait complete. 2024-05-09T08:02:54.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.990+0000 7fbbcee00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbbc004b500 0x7fbbc0048130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:54.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.990+0000 7ffba5600640 1 -- 172.21.15.1:0/1679279904 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffba0158b90 msgr2=0x7ffba01625e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:54.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.990+0000 7ffba5600640 1 --2- 172.21.15.1:0/1679279904 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffba0158b90 0x7ffba01625e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ffb900097a0 tx=0x7ffb9002a210 comp rx=0 tx=0).stop 2024-05-09T08:02:54.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.990+0000 7fa3a6200640 1 -- start start 2024-05-09T08:02:54.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.989+0000 7f6259400640 1 -- 172.21.15.1:0/3405581470 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f625c0c7990 con 0x7f625c06cce0 2024-05-09T08:02:54.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.990+0000 7fa62a600640 1 --2- 172.21.15.1:0/1791582975 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6240c8730 0x7fa62414e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:54.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.990+0000 7fbbcee00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fbbc0050ca0 0x7fbbc00466f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:54.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.990+0000 7f906f200640 1 --2- 172.21.15.1:0/2896389809 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9070157200 0x7f90701593f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f907006e9e0 tx=0x7f90580046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:54.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.990+0000 7f7c0e000640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f7c080cc5f0 0x7f7c0814b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:54.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.990+0000 7f7c0e000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c080ce4e0 0x7f7c0814bc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:54.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.990+0000 7f7c0e000640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f7c080ce110 con 0x7f7c080cac60 2024-05-09T08:02:54.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.990+0000 7f7c0e000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7c080cd550 con 0x7f7c080ce4e0 2024-05-09T08:02:54.033 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.991+0000 7fa62c600640 1 -- 172.21.15.1:0/1791582975 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fa62415ad80 con 0x7fa62406c900 2024-05-09T08:02:54.033 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.990+0000 7f7c0e000640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f7c080ce290 con 0x7f7c080cc5f0 2024-05-09T08:02:54.033 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.990+0000 7f7c07e00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f7c080cc5f0 0x7f7c0814b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:54.033 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.990+0000 7f7c07e00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f7c080cc5f0 0x7f7c0814b1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3301/0 says I am v2:172.21.15.1:44488/0 (socket says 172.21.15.1:44488) 2024-05-09T08:02:54.033 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.990+0000 7f7c07e00640 1 -- 172.21.15.1:0/934130762 learned_addr learned my addr 172.21.15.1:0/934130762 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:02:54.033 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.990+0000 7f7c0d600640 1 --2- 172.21.15.1:0/934130762 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c080ce4e0 0x7f7c0814bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:54.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.991+0000 7fa62c600640 1 -- 172.21.15.1:0/1791582975 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa62415aaa0 con 0x7fa6240c8730 2024-05-09T08:02:54.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.991+0000 7fa62c600640 1 -- 172.21.15.1:0/1791582975 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7fa62415b060 con 0x7fa62406cce0 2024-05-09T08:02:54.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.991+0000 7fbbcee00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbbc00dac60 0x7fbbc017c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:54.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.991+0000 7ffba5600640 1 -- 172.21.15.1:0/1679279904 shutdown_connections 2024-05-09T08:02:54.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.991+0000 7ffba5600640 1 --2- 172.21.15.1:0/1679279904 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffba0158b90 0x7ffba01625e0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:54.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.991+0000 7ffba5600640 1 -- 172.21.15.1:0/1679279904 >> 172.21.15.1:0/1679279904 conn(0x7ffba006dad0 msgr2=0x7ffba006deb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:02:54.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.991+0000 7f9066a00640 1 -- 172.21.15.1:0/2896389809 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9058041070 con 0x7f9070157200 2024-05-09T08:02:54.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.991+0000 7f9066a00640 1 -- 172.21.15.1:0/2896389809 <== mon.1 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9058004030 con 0x7f9070157200 2024-05-09T08:02:54.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.991+0000 7f9066a00640 1 -- 172.21.15.1:0/2896389809 <== mon.1 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9058037040 con 0x7f9070157200 2024-05-09T08:02:54.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.991+0000 7fbbcd000640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fbbc0050ca0 0x7fbbc00466f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:54.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.991+0000 7fbbcd000640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fbbc0050ca0 0x7fbbc00466f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3301/0 says I am v2:172.21.15.1:44504/0 (socket says 172.21.15.1:44504) 2024-05-09T08:02:54.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.991+0000 7fbbcda00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbbc004b500 0x7fbbc0048130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:54.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.991+0000 7f9075400640 1 -- 172.21.15.1:0/2896389809 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f907014e020 con 0x7f9070157200 2024-05-09T08:02:54.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.991+0000 7fbbcda00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbbc004b500 0x7fbbc0048130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.1:58796/0 (socket says 172.21.15.1:58796) 2024-05-09T08:02:54.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.991+0000 7ffba5600640 1 -- 172.21.15.1:0/1679279904 shutdown_connections 2024-05-09T08:02:54.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.991+0000 7f6259400640 1 --2- 172.21.15.1:0/3405581470 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f625c06cce0 0x7f625c0c81f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f624c009e80 tx=0x7f624c02a6f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a56fa65674670f64 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:54.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.991+0000 7f7c07e00640 1 -- 172.21.15.1:0/934130762 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c080ce4e0 msgr2=0x7f7c0814bc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:54.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.991+0000 7f9075400640 1 -- 172.21.15.1:0/2896389809 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90701ebc50 con 0x7f9070157200 2024-05-09T08:02:54.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.992+0000 7fa629c00640 1 -- 172.21.15.1:0/1791582975 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fa62406cce0 msgr2=0x7fa62414f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:54.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.992+0000 7fa629c00640 1 --2- 172.21.15.1:0/1791582975 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fa62406cce0 0x7fa62414f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:54.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.992+0000 7fa629c00640 1 -- 172.21.15.1:0/1791582975 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6240c8730 msgr2=0x7fa62414e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:54.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.992+0000 7fa629c00640 1 --2- 172.21.15.1:0/1791582975 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6240c8730 0x7fa62414e270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:54.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.992+0000 7f7c0cc00640 1 --2- 172.21.15.1:0/934130762 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7c080cac60 0x7f7c0814d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:54.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.992+0000 7fa629c00640 1 -- 172.21.15.1:0/1791582975 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6241f03b0 con 0x7fa62406c900 2024-05-09T08:02:54.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.992+0000 7fa629200640 1 --2- 172.21.15.1:0/1791582975 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fa62406cce0 0x7fa62414f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:02:54.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.992+0000 7f6258a00640 1 -- 172.21.15.1:0/3405581470 <== mon.2 v2:172.21.15.7:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f624c036070 con 0x7f625c06cce0 2024-05-09T08:02:54.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.992+0000 7f9075400640 1 -- 172.21.15.1:0/2896389809 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9048001030 con 0x7f9070157200 2024-05-09T08:02:54.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.992+0000 7f7c07e00640 1 --2- 172.21.15.1:0/934130762 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c080ce4e0 0x7f7c0814bc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:54.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.992+0000 7f7c07e00640 1 -- 172.21.15.1:0/934130762 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7c080cac60 msgr2=0x7f7c0814d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:54.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.992+0000 7f7c07e00640 1 --2- 172.21.15.1:0/934130762 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7c080cac60 0x7f7c0814d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:54.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.992+0000 7f7c07e00640 1 -- 172.21.15.1:0/934130762 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c081514c0 con 0x7f7c080cc5f0 2024-05-09T08:02:54.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.993+0000 7f7c0cc00640 1 --2- 172.21.15.1:0/934130762 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7c080cac60 0x7f7c0814d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T08:02:54.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.993+0000 7f6258a00640 1 -- 172.21.15.1:0/3405581470 <== mon.2 v2:172.21.15.7:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f624c031600 con 0x7f625c06cce0 2024-05-09T08:02:54.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.993+0000 7f7c0d600640 1 --2- 172.21.15.1:0/934130762 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c080ce4e0 0x7f7c0814bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:02:54.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.993+0000 7f6258a00640 1 -- 172.21.15.1:0/3405581470 <== mon.2 v2:172.21.15.7:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f624c0319d0 con 0x7f625c06cce0 2024-05-09T08:02:54.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.993+0000 7fa629c00640 1 --2- 172.21.15.1:0/1791582975 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa62406c900 0x7fa6241508f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fa61800caf0 tx=0x7fa618002100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:54.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.991+0000 7ffba5600640 1 -- 172.21.15.1:0/1679279904 wait complete. 2024-05-09T08:02:54.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.991+0000 7fbbce400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbbc00dac60 0x7fbbc017c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:54.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.991+0000 7fbbce400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbbc00dac60 0x7fbbc017c530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38012/0 (socket says 172.21.15.1:38012) 2024-05-09T08:02:54.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.991+0000 7fbbcee00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fbbc0034a40 con 0x7fbbc00dac60 2024-05-09T08:02:54.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.991+0000 7fbbcee00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fbbc00065f0 con 0x7fbbc004b500 2024-05-09T08:02:54.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.992+0000 7fbbcee00640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7fbbc0034bc0 con 0x7fbbc0050ca0 2024-05-09T08:02:54.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.993+0000 7fa621600640 1 -- 172.21.15.1:0/1791582975 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa618003b70 con 0x7fa62406c900 2024-05-09T08:02:54.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.993+0000 7fa621600640 1 -- 172.21.15.1:0/1791582975 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa618003d30 con 0x7fa62406c900 2024-05-09T08:02:54.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.993+0000 7fa621600640 1 -- 172.21.15.1:0/1791582975 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa618005310 con 0x7fa62406c900 2024-05-09T08:02:54.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.993+0000 7fa62c600640 1 -- 172.21.15.1:0/1791582975 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6241f0650 con 0x7fa62406c900 2024-05-09T08:02:54.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.994+0000 7fa62c600640 1 -- 172.21.15.1:0/1791582975 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6241f0bf0 con 0x7fa62406c900 2024-05-09T08:02:54.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.994+0000 7fa3a6200640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa3a01507b0 0x7fa3a00728e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:54.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.994+0000 7f6260a00640 1 -- 172.21.15.1:0/3405581470 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f625c06cce0 msgr2=0x7f625c0c81f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:54.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.994+0000 7f6260a00640 1 --2- 172.21.15.1:0/3405581470 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f625c06cce0 0x7f625c0c81f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f624c009e80 tx=0x7f624c02a6f0 comp rx=0 tx=0).stop 2024-05-09T08:02:54.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.994+0000 7f6260a00640 1 -- 172.21.15.1:0/3405581470 shutdown_connections 2024-05-09T08:02:54.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.994+0000 7f7c07e00640 1 --2- 172.21.15.1:0/934130762 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f7c080cc5f0 0x7f7c0814b1f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f7bf4009f40 tx=0x7f7bf4004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:54.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.994+0000 7f88f9600640 1 Processor -- start 2024-05-09T08:02:54.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.994+0000 7f9066a00640 1 -- 172.21.15.1:0/2896389809 <== mon.1 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9058044070 con 0x7f9070157200 2024-05-09T08:02:54.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.994+0000 7fbbcd000640 1 -- 172.21.15.1:0/546460052 learned_addr learned my addr 172.21.15.1:0/546460052 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:02:54.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.994+0000 7fa3a6200640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fa3a0072e20 0x7fa3a0156d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:54.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.994+0000 7fa3a6200640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fa3a01572c0 0x7fa3a006f820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:54.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.994+0000 7f6260a00640 1 --2- 172.21.15.1:0/3405581470 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f625c06cce0 0x7f625c0c81f0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:54.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.994+0000 7f6260a00640 1 -- 172.21.15.1:0/3405581470 >> 172.21.15.1:0/3405581470 conn(0x7f625c0cfe80 msgr2=0x7f625c0d0290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:02:54.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.995+0000 7fa62c600640 1 -- 172.21.15.1:0/1791582975 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa624074d70 con 0x7fa62406c900 2024-05-09T08:02:54.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.995+0000 7f6260a00640 1 -- 172.21.15.1:0/3405581470 shutdown_connections 2024-05-09T08:02:54.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.995+0000 7fa3a6200640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7fa3a01568f0 con 0x7fa3a01572c0 2024-05-09T08:02:54.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.995+0000 7fa3a6200640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fa3a01565a0 con 0x7fa3a01507b0 2024-05-09T08:02:54.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.995+0000 7fa3a6200640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7fa3a0073520 con 0x7fa3a0072e20 2024-05-09T08:02:54.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.995+0000 7fbbcd000640 1 -- 172.21.15.1:0/546460052 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbbc004b500 msgr2=0x7fbbc0048130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:54.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.995+0000 7fbbcd000640 1 --2- 172.21.15.1:0/546460052 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbbc004b500 0x7fbbc0048130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:54.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.995+0000 7fa39f400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa3a01507b0 0x7fa3a00728e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:54.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.995+0000 7fa39f400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa3a01507b0 0x7fa3a00728e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.1:58800/0 (socket says 172.21.15.1:58800) 2024-05-09T08:02:54.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.995+0000 7fa39f400640 1 -- 172.21.15.1:0/2247256599 learned_addr learned my addr 172.21.15.1:0/2247256599 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:02:54.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.995+0000 7fbbcd000640 1 -- 172.21.15.1:0/546460052 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbbc00dac60 msgr2=0x7fbbc017c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:54.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.995+0000 7f9066a00640 1 --2- 172.21.15.1:0/2896389809 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f904c06be30 0x7f904c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:54.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.996+0000 7f9066a00640 1 -- 172.21.15.1:0/2896389809 <== mon.1 v2:172.21.15.7:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f90580ab2f0 con 0x7f9070157200 2024-05-09T08:02:54.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.995+0000 7fbbcd000640 1 --2- 172.21.15.1:0/546460052 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbbc00dac60 0x7fbbc017c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:54.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.995+0000 7fbbcd000640 1 -- 172.21.15.1:0/546460052 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbc00c6e90 con 0x7fbbc0050ca0 2024-05-09T08:02:54.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.995+0000 7fa39fe00640 1 --2- 172.21.15.1:0/2247256599 >> v2:172.21.15.1:3300/0 conn(0x7fa3a01572c0 0x7fa3a006f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:54.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.996+0000 7fa39f400640 1 -- 172.21.15.1:0/2247256599 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fa3a0072e20 msgr2=0x7fa3a0156d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:02:54.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.996+0000 7fa39ea00640 1 --2- 172.21.15.1:0/2247256599 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fa3a0072e20 0x7fa3a0156d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:54.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.996+0000 7fa39f400640 1 --2- 172.21.15.1:0/2247256599 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fa3a0072e20 0x7fa3a0156d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:54.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.996+0000 7fa39f400640 1 -- 172.21.15.1:0/2247256599 >> v2:172.21.15.1:3300/0 conn(0x7fa3a01572c0 msgr2=0x7fa3a006f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:02:54.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.996+0000 7fa39f400640 1 --2- 172.21.15.1:0/2247256599 >> v2:172.21.15.1:3300/0 conn(0x7fa3a01572c0 0x7fa3a006f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:54.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.996+0000 7fa39f400640 1 -- 172.21.15.1:0/2247256599 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3a00c2080 con 0x7fa3a01507b0 2024-05-09T08:02:54.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.995+0000 7f7c04c00640 1 -- 172.21.15.1:0/934130762 <== mon.2 v2:172.21.15.7:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bf403f020 con 0x7f7c080cc5f0 2024-05-09T08:02:54.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.996+0000 7f9066a00640 1 --2- 172.21.15.1:0/2896389809 >> [v2:172.21.15.1:6800/1513349547,v1:172.21.15.1:6801/1513349547] conn(0x7f904c075520 0x7f904c077960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:54.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.995+0000 7f88f9600640 1 -- start start 2024-05-09T08:02:54.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.996+0000 7f9066a00640 1 -- 172.21.15.1:0/2896389809 --> [v2:172.21.15.1:6800/1513349547,v1:172.21.15.1:6801/1513349547] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9058008eb0 con 0x7f904c075520 2024-05-09T08:02:54.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.996+0000 7f9066a00640 1 -- 172.21.15.1:0/2896389809 <== mon.1 v2:172.21.15.7:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f90580b2050 con 0x7f9070157200 2024-05-09T08:02:54.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.996+0000 7ffba5600640 1 Processor -- start 2024-05-09T08:02:54.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.996+0000 7fa39ea00640 1 --2- 172.21.15.1:0/2247256599 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fa3a0072e20 0x7fa3a0156d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T08:02:54.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.996+0000 7fbbcda00640 1 --2- 172.21.15.1:0/546460052 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbbc004b500 0x7fbbc0048130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T08:02:54.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.996+0000 7fbbce400640 1 --2- 172.21.15.1:0/546460052 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbbc00dac60 0x7fbbc017c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:02:54.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.996+0000 7f7c0e000640 1 -- 172.21.15.1:0/934130762 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c081517a0 con 0x7f7c080cc5f0 2024-05-09T08:02:54.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.996+0000 7f7c0e000640 1 -- 172.21.15.1:0/934130762 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c081527b0 con 0x7f7c080cc5f0 2024-05-09T08:02:54.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.996+0000 7f7c04c00640 1 -- 172.21.15.1:0/934130762 <== mon.2 v2:172.21.15.7:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7bf4037040 con 0x7f7c080cc5f0 2024-05-09T08:02:54.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.997+0000 7f906e800640 1 --2- 172.21.15.1:0/2896389809 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f904c06be30 0x7f904c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:54.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.997+0000 7ffba5600640 1 -- start start 2024-05-09T08:02:54.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.997+0000 7f88f9600640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f88f40c69a0 0x7f88f41589c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:54.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.997+0000 7f88f9600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f88f40c8c80 0x7f88f41539a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:54.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.997+0000 7fbbcd000640 1 --2- 172.21.15.1:0/546460052 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fbbc0050ca0 0x7fbbc00466f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fbbb4000c00 tx=0x7fbbb40046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:54.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.997+0000 7f88f9600640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f88f40b17f0 0x7f88f41e4960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:54.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.998+0000 7fa39f400640 1 --2- 172.21.15.1:0/2247256599 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3a01507b0 0x7fa3a00728e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fa3a00c8da0 tx=0x7fa39802c3e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8056b46177601639 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:54.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.998+0000 7f7c04c00640 1 -- 172.21.15.1:0/934130762 <== mon.2 v2:172.21.15.7:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bf403f020 con 0x7f7c080cc5f0 2024-05-09T08:02:54.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.997+0000 7fbbc5600640 1 -- 172.21.15.1:0/546460052 <== mon.2 v2:172.21.15.7:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbb4041070 con 0x7fbbc0050ca0 2024-05-09T08:02:54.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.998+0000 7fbbc5600640 1 -- 172.21.15.1:0/546460052 <== mon.2 v2:172.21.15.7:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbbb403c950 con 0x7fbbc0050ca0 2024-05-09T08:02:54.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.998+0000 7fbbc5600640 1 -- 172.21.15.1:0/546460052 <== mon.2 v2:172.21.15.7:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbb4037040 con 0x7fbbc0050ca0 2024-05-09T08:02:54.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.998+0000 7fbbcee00640 1 -- 172.21.15.1:0/546460052 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbc00c7150 con 0x7fbbc0050ca0 2024-05-09T08:02:54.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.998+0000 7f906de00640 1 --2- 172.21.15.1:0/2896389809 >> [v2:172.21.15.1:6800/1513349547,v1:172.21.15.1:6801/1513349547] conn(0x7f904c075520 0x7f904c077960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:54.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.998+0000 7fbbcee00640 1 -- 172.21.15.1:0/546460052 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbc0045ef0 con 0x7fbbc0050ca0 2024-05-09T08:02:54.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.998+0000 7fa39e000640 1 -- 172.21.15.1:0/2247256599 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa398038070 con 0x7fa3a01507b0 2024-05-09T08:02:54.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.998+0000 7fa39e000640 1 -- 172.21.15.1:0/2247256599 <== mon.1 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa398031710 con 0x7fa3a01507b0 2024-05-09T08:02:54.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.998+0000 7fa39e000640 1 -- 172.21.15.1:0/2247256599 <== mon.1 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa398031ae0 con 0x7fa3a01507b0 2024-05-09T08:02:54.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.996+0000 7f6260a00640 1 -- 172.21.15.1:0/3405581470 wait complete. 2024-05-09T08:02:54.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.998+0000 7f6260a00640 1 Processor -- start 2024-05-09T08:02:54.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.999+0000 7f6260a00640 1 -- start start 2024-05-09T08:02:54.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.998+0000 7f88f3e00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f88f40c69a0 0x7f88f41589c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:54.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.999+0000 7f906e800640 1 --2- 172.21.15.1:0/2896389809 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f904c06be30 0x7f904c06e2f0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f906800c530 tx=0x7f906800b040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:54.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.997+0000 7fa621600640 1 -- 172.21.15.1:0/1791582975 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa6180054b0 con 0x7fa62406c900 2024-05-09T08:02:54.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.999+0000 7f88f9600640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f88f415dec0 con 0x7f88f40b17f0 2024-05-09T08:02:54.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.999+0000 7f88f9600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f88f415dbe0 con 0x7f88f40c8c80 2024-05-09T08:02:54.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.999+0000 7f88f9600640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f88f415e920 con 0x7f88f40c69a0 2024-05-09T08:02:54.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.999+0000 7f88f3e00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f88f40c69a0 0x7f88f41589c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3301/0 says I am v2:172.21.15.1:44510/0 (socket says 172.21.15.1:44510) 2024-05-09T08:02:54.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.999+0000 7f88f3e00640 1 -- 172.21.15.1:0/2735335039 learned_addr learned my addr 172.21.15.1:0/2735335039 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:02:54.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.999+0000 7f88f3400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f88f40c8c80 0x7f88f41539a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:54.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.999+0000 7f88f8c00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f88f40b17f0 0x7f88f41e4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:54.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.998+0000 7ffba5600640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ffba00c87f0 0x7ffba01e8350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:54.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.998+0000 7ffba5600640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffba0158b90 0x7ffba00cb7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:54.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.998+0000 7ffba5600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffba00caf10 0x7ffba00cbd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:54.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.998+0000 7ffba5600640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ffba006ffb0 con 0x7ffba0158b90 2024-05-09T08:02:54.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.998+0000 7ffba5600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ffba006fe30 con 0x7ffba00caf10 2024-05-09T08:02:54.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.998+0000 7ffba5600640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7ffba00c7e70 con 0x7ffba00c87f0 2024-05-09T08:02:54.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.999+0000 7ffb9ea00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ffba00c87f0 0x7ffba01e8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:54.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.999+0000 7ffb9ea00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ffba00c87f0 0x7ffba01e8350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3301/0 says I am v2:172.21.15.1:44520/0 (socket says 172.21.15.1:44520) 2024-05-09T08:02:54.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.999+0000 7ffb9ea00640 1 -- 172.21.15.1:0/3082526225 learned_addr learned my addr 172.21.15.1:0/3082526225 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:02:54.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.999+0000 7f906de00640 1 --2- 172.21.15.1:0/2896389809 >> [v2:172.21.15.1:6800/1513349547,v1:172.21.15.1:6801/1513349547] conn(0x7f904c075520 0x7f904c077960 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:54.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:53.999+0000 7f7c0e000640 1 -- 172.21.15.1:0/934130762 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7be0001030 con 0x7f7c080cc5f0 2024-05-09T08:02:54.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.000+0000 7fa621600640 1 --2- 172.21.15.1:0/1791582975 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fa60406bf00 0x7fa60406e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:54.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.000+0000 7fa621600640 1 -- 172.21.15.1:0/1791582975 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fa61808afb0 con 0x7fa62406c900 2024-05-09T08:02:54.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.000+0000 7fa629200640 1 --2- 172.21.15.1:0/1791582975 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fa60406bf00 0x7fa60406e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:54.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.000+0000 7fa621600640 1 --2- 172.21.15.1:0/1791582975 >> [v2:172.21.15.1:6808/2925031811,v1:172.21.15.1:6809/2925031811] conn(0x7fa604075670 0x7fa604077ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:54.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.000+0000 7fa621600640 1 -- 172.21.15.1:0/1791582975 --> [v2:172.21.15.1:6808/2925031811,v1:172.21.15.1:6809/2925031811] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa618010e50 con 0x7fa604075670 2024-05-09T08:02:54.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.000+0000 7fa621600640 1 -- 172.21.15.1:0/1791582975 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa618090580 con 0x7fa62406c900 2024-05-09T08:02:54.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.000+0000 7fbbcee00640 1 -- 172.21.15.1:0/546460052 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fbbc00052c0 con 0x7fbbc0050ca0 2024-05-09T08:02:54.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.000+0000 7ffb9f400640 1 --2- 172.21.15.1:0/3082526225 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffba00caf10 0x7ffba00cbd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:54.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.001+0000 7ffb9f400640 1 -- 172.21.15.1:0/3082526225 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ffba00c87f0 msgr2=0x7ffba01e8350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:54.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.000+0000 7fa3a6200640 1 -- 172.21.15.1:0/2247256599 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3a01507b0 msgr2=0x7fa3a00728e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:54.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.000+0000 7fa3a6200640 1 --2- 172.21.15.1:0/2247256599 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3a01507b0 0x7fa3a00728e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fa3a00c8da0 tx=0x7fa39802c3e0 comp rx=0 tx=0).stop 2024-05-09T08:02:54.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.001+0000 7f6260a00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f625c06c900 0x7f625c155250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:54.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.001+0000 7f6260a00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f625c06cce0 0x7f625c150050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:54.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.000+0000 7f88f3e00640 1 -- 172.21.15.1:0/2735335039 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f88f40c8c80 msgr2=0x7f88f41539a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:54.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.001+0000 7ffb9f400640 1 --2- 172.21.15.1:0/3082526225 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ffba00c87f0 0x7ffba01e8350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:54.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.001+0000 7ffb9f400640 1 -- 172.21.15.1:0/3082526225 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffba0158b90 msgr2=0x7ffba00cb7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:54.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.000+0000 7f88f3e00640 1 --2- 172.21.15.1:0/2735335039 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f88f40c8c80 0x7f88f41539a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:54.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.000+0000 7f88f3e00640 1 -- 172.21.15.1:0/2735335039 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f88f40b17f0 msgr2=0x7f88f41e4960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:54.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.000+0000 7f88f3e00640 1 --2- 172.21.15.1:0/2735335039 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f88f40b17f0 0x7f88f41e4960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:54.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.000+0000 7f88f3e00640 1 -- 172.21.15.1:0/2735335039 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88f41e57d0 con 0x7f88f40c69a0 2024-05-09T08:02:54.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.001+0000 7fa629200640 1 --2- 172.21.15.1:0/1791582975 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fa60406bf00 0x7fa60406e3c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa610034000 tx=0x7fa61002e000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:54.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.002+0000 7fa62a600640 1 --2- 172.21.15.1:0/1791582975 >> [v2:172.21.15.1:6808/2925031811,v1:172.21.15.1:6809/2925031811] conn(0x7fa604075670 0x7fa604077ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:54.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.001+0000 7ffb9f400640 1 --2- 172.21.15.1:0/3082526225 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffba0158b90 0x7ffba00cb7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:54.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.001+0000 7ffb9f400640 1 -- 172.21.15.1:0/3082526225 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffba01ea180 con 0x7ffba00caf10 2024-05-09T08:02:54.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.001+0000 7ffb9f400640 1 --2- 172.21.15.1:0/3082526225 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffba00caf10 0x7ffba00cbd00 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ffba00c8c60 tx=0x7ffb9800cbe0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:54.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.001+0000 7f6260a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f625c0c8730 0x7f625c0c6560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:54.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.001+0000 7f6259e00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f625c06c900 0x7f625c155250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:54.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.001+0000 7f6259e00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f625c06c900 0x7f625c155250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3301/0 says I am v2:172.21.15.1:44532/0 (socket says 172.21.15.1:44532) 2024-05-09T08:02:54.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.001+0000 7f6259e00640 1 -- 172.21.15.1:0/3364473488 learned_addr learned my addr 172.21.15.1:0/3364473488 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:02:54.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.001+0000 7f6260a00640 1 -- 172.21.15.1:0/3364473488 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f625c15ad80 con 0x7f625c06cce0 2024-05-09T08:02:54.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.001+0000 7f6260a00640 1 -- 172.21.15.1:0/3364473488 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f625c15aaa0 con 0x7f625c0c8730 2024-05-09T08:02:54.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.001+0000 7f625a800640 1 --2- 172.21.15.1:0/3364473488 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f625c0c8730 0x7f625c0c6560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:54.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.001+0000 7f6260a00640 1 -- 172.21.15.1:0/3364473488 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f625c15b060 con 0x7f625c06c900 2024-05-09T08:02:54.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.002+0000 7f6259400640 1 --2- 172.21.15.1:0/3364473488 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f625c06cce0 0x7f625c150050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:54.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.002+0000 7f9066a00640 1 -- 172.21.15.1:0/2896389809 <== osd.0 v2:172.21.15.1:6800/1513349547 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9058008eb0 con 0x7f904c075520 2024-05-09T08:02:54.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.002+0000 7ffb96a00640 1 -- 172.21.15.1:0/3082526225 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb98018070 con 0x7ffba00caf10 2024-05-09T08:02:54.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.002+0000 7ffb96a00640 1 -- 172.21.15.1:0/3082526225 <== mon.1 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb9800ce30 con 0x7ffba00caf10 2024-05-09T08:02:54.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.002+0000 7fa3a6200640 1 -- 172.21.15.1:0/2247256599 shutdown_connections 2024-05-09T08:02:54.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.002+0000 7fa3a6200640 1 --2- 172.21.15.1:0/2247256599 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fa3a0072e20 0x7fa3a0156d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:54.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.002+0000 7fa3a6200640 1 --2- 172.21.15.1:0/2247256599 >> v2:172.21.15.1:3300/0 conn(0x7fa3a01572c0 0x7fa3a006f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:54.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.002+0000 7fa3a6200640 1 --2- 172.21.15.1:0/2247256599 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3a01507b0 0x7fa3a00728e0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:54.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.002+0000 7fa3a6200640 1 -- 172.21.15.1:0/2247256599 >> 172.21.15.1:0/2247256599 conn(0x7fa3a006dad0 msgr2=0x7fa3a006deb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:02:54.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.002+0000 7f88f3e00640 1 --2- 172.21.15.1:0/2735335039 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f88f40c69a0 0x7f88f41589c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f88e402a740 tx=0x7f88e40046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:54.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.002+0000 7ffb96a00640 1 -- 172.21.15.1:0/3082526225 <== mon.1 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb9800a830 con 0x7ffba00caf10 2024-05-09T08:02:54.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.002+0000 7ffba5600640 1 -- 172.21.15.1:0/3082526225 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffba01e9ea0 con 0x7ffba00caf10 2024-05-09T08:02:54.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.001+0000 7f7c04c00640 1 -- 172.21.15.1:0/934130762 <== mon.2 v2:172.21.15.7:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7bf4042020 con 0x7f7c080cc5f0 2024-05-09T08:02:54.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.003+0000 7fa62a600640 1 --2- 172.21.15.1:0/1791582975 >> [v2:172.21.15.1:6808/2925031811,v1:172.21.15.1:6809/2925031811] conn(0x7fa604075670 0x7fa604077ab0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:54.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.002+0000 7f88ebe00640 1 -- 172.21.15.1:0/2735335039 <== mon.2 v2:172.21.15.7:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88e4041020 con 0x7f88f40c69a0 2024-05-09T08:02:54.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.003+0000 7fbbc5600640 1 -- 172.21.15.1:0/546460052 <== mon.2 v2:172.21.15.7:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fbbb4044070 con 0x7fbbc0050ca0 2024-05-09T08:02:54.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.002+0000 7ffba5600640 1 -- 172.21.15.1:0/3082526225 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffba00c3810 con 0x7ffba00caf10 2024-05-09T08:02:54.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.002+0000 7fa3a6200640 1 -- 172.21.15.1:0/2247256599 shutdown_connections 2024-05-09T08:02:54.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.003+0000 7f88f9600640 1 -- 172.21.15.1:0/2735335039 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88f4158f00 con 0x7f88f40c69a0 2024-05-09T08:02:54.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.002+0000 7f6259e00640 1 -- 172.21.15.1:0/3364473488 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f625c0c8730 msgr2=0x7f625c0c6560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:54.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.003+0000 7f6259e00640 1 --2- 172.21.15.1:0/3364473488 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f625c0c8730 0x7f625c0c6560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:54.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.003+0000 7f6259e00640 1 -- 172.21.15.1:0/3364473488 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f625c06cce0 msgr2=0x7f625c150050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:54.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.003+0000 7f6259e00640 1 --2- 172.21.15.1:0/3364473488 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f625c06cce0 0x7f625c150050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:54.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.003+0000 7f6259e00640 1 -- 172.21.15.1:0/3364473488 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f625c14f2d0 con 0x7f625c06c900 2024-05-09T08:02:54.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.003+0000 7f6259400640 1 --2- 172.21.15.1:0/3364473488 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f625c06cce0 0x7f625c150050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T08:02:54.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.003+0000 7f625a800640 1 --2- 172.21.15.1:0/3364473488 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f625c0c8730 0x7f625c0c6560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T08:02:54.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.003+0000 7f6259e00640 1 --2- 172.21.15.1:0/3364473488 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f625c06c900 0x7f625c155250 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f624400b410 tx=0x7f624400b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:54.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.003+0000 7f88f9600640 1 -- 172.21.15.1:0/2735335039 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f88f4149d60 con 0x7f88f40c69a0 2024-05-09T08:02:54.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.004+0000 7f6252000640 1 -- 172.21.15.1:0/3364473488 <== mon.2 v2:172.21.15.7:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6244015020 con 0x7f625c06c900 2024-05-09T08:02:54.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.004+0000 7fa621600640 1 -- 172.21.15.1:0/1791582975 <== osd.2 v2:172.21.15.1:6808/2925031811 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa618010e50 con 0x7fa604075670 2024-05-09T08:02:54.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.002+0000 7fa3a6200640 1 -- 172.21.15.1:0/2247256599 wait complete. 2024-05-09T08:02:54.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.005+0000 7f88f9600640 1 -- 172.21.15.1:0/2735335039 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f88f404f660 con 0x7f88f40c69a0 2024-05-09T08:02:54.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.005+0000 7fbbc5600640 1 --2- 172.21.15.1:0/546460052 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fbba806be30 0x7fbba806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:54.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.005+0000 7ffba5600640 1 -- 172.21.15.1:0/3082526225 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ffba015c610 con 0x7ffba00caf10 2024-05-09T08:02:54.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.004+0000 7f6260a00640 1 -- 172.21.15.1:0/3364473488 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f625c1fe210 con 0x7f625c06c900 2024-05-09T08:02:54.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.004+0000 7f6252000640 1 -- 172.21.15.1:0/3364473488 <== mon.2 v2:172.21.15.7:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6244011830 con 0x7f625c06c900 2024-05-09T08:02:54.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.004+0000 7f6252000640 1 -- 172.21.15.1:0/3364473488 <== mon.2 v2:172.21.15.7:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6244015020 con 0x7f625c06c900 2024-05-09T08:02:54.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.004+0000 7f6260a00640 1 -- 172.21.15.1:0/3364473488 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f625c14f850 con 0x7f625c06c900 2024-05-09T08:02:54.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.005+0000 7f88ebe00640 1 -- 172.21.15.1:0/2735335039 <== mon.2 v2:172.21.15.7:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88e4032440 con 0x7f88f40c69a0 2024-05-09T08:02:54.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.005+0000 7f88ebe00640 1 -- 172.21.15.1:0/2735335039 <== mon.2 v2:172.21.15.7:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88e4041020 con 0x7f88f40c69a0 2024-05-09T08:02:54.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.005+0000 7f7c04c00640 1 --2- 172.21.15.1:0/934130762 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f7be806c030 0x7f7be806e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:54.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.005+0000 7f7c04c00640 1 -- 172.21.15.1:0/934130762 <== mon.2 v2:172.21.15.7:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f7bf40ab190 con 0x7f7c080cc5f0 2024-05-09T08:02:54.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.005+0000 7fbbc5600640 1 -- 172.21.15.1:0/546460052 <== mon.2 v2:172.21.15.7:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fbbb40abd50 con 0x7fbbc0050ca0 2024-05-09T08:02:54.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.005+0000 7fbbc5600640 1 --2- 172.21.15.1:0/546460052 >> [v2:172.21.15.7:6800/2957026670,v1:172.21.15.7:6801/2957026670] conn(0x7fbba80755a0 0x7fbba80779e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:54.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.005+0000 7fbbc5600640 1 -- 172.21.15.1:0/546460052 --> [v2:172.21.15.7:6800/2957026670,v1:172.21.15.7:6801/2957026670] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fbbb4034ec0 con 0x7fbba80755a0 2024-05-09T08:02:54.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.005+0000 7fbbc5600640 1 -- 172.21.15.1:0/546460052 <== mon.2 v2:172.21.15.7:3301/0 6 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fbbb40ae3b0 con 0x7fbbc0050ca0 2024-05-09T08:02:54.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.005+0000 7fa3a6200640 1 Processor -- start 2024-05-09T08:02:54.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.006+0000 7f624b400640 1 -- 172.21.15.1:0/3364473488 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f625c14fc30 con 0x7f625c06c900 2024-05-09T08:02:54.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.006+0000 7f7c04c00640 1 --2- 172.21.15.1:0/934130762 >> [v2:172.21.15.1:6816/2942255259,v1:172.21.15.1:6817/2942255259] conn(0x7f7be80756a0 0x7f7be8077ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:54.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.006+0000 7f7c04c00640 1 -- 172.21.15.1:0/934130762 --> [v2:172.21.15.1:6816/2942255259,v1:172.21.15.1:6817/2942255259] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7bf4034e70 con 0x7f7be80756a0 2024-05-09T08:02:54.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.006+0000 7f7c04c00640 1 -- 172.21.15.1:0/934130762 <== mon.2 v2:172.21.15.7:3301/0 6 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f7bf40ab540 con 0x7f7c080cc5f0 2024-05-09T08:02:54.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.007+0000 7f7c0d600640 1 --2- 172.21.15.1:0/934130762 >> [v2:172.21.15.1:6816/2942255259,v1:172.21.15.1:6817/2942255259] conn(0x7f7be80756a0 0x7f7be8077ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:54.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.007+0000 7f7c0cc00640 1 --2- 172.21.15.1:0/934130762 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f7be806c030 0x7f7be806e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:54.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.008+0000 7f7c0d600640 1 --2- 172.21.15.1:0/934130762 >> [v2:172.21.15.1:6816/2942255259,v1:172.21.15.1:6817/2942255259] conn(0x7f7be80756a0 0x7f7be8077ae0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:54.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.009+0000 7f7c0cc00640 1 --2- 172.21.15.1:0/934130762 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f7be806c030 0x7f7be806e4f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f7bf8000f60 tx=0x7f7bf8009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:54.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.006+0000 7fbbcda00640 1 --2- 172.21.15.1:0/546460052 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fbba806be30 0x7fbba806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:54.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.006+0000 7fbbce400640 1 --2- 172.21.15.1:0/546460052 >> [v2:172.21.15.7:6800/2957026670,v1:172.21.15.7:6801/2957026670] conn(0x7fbba80755a0 0x7fbba80779e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:54.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.007+0000 7fbbcda00640 1 --2- 172.21.15.1:0/546460052 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fbba806be30 0x7fbba806e2f0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fbbc8050860 tx=0x7fbbc8072040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:54.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.009+0000 7fbbce400640 1 --2- 172.21.15.1:0/546460052 >> [v2:172.21.15.7:6800/2957026670,v1:172.21.15.7:6801/2957026670] conn(0x7fbba80755a0 0x7fbba80779e0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:54.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.007+0000 7ffb96a00640 1 -- 172.21.15.1:0/3082526225 <== mon.1 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ffb9801b070 con 0x7ffba00caf10 2024-05-09T08:02:54.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.006+0000 7fa3a6200640 1 -- start start 2024-05-09T08:02:54.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.008+0000 7fa3a6200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3a0072e20 0x7fa3a00c4a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:54.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.008+0000 7fa3a6200640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa3a01507b0 0x7fa3a00c5740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:54.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.008+0000 7fa3a6200640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fa3a01572c0 0x7fa3a014e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:54.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.008+0000 7fa3a6200640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fa3a015a690 con 0x7fa3a01507b0 2024-05-09T08:02:54.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.008+0000 7fa3a6200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa3a015a3b0 con 0x7fa3a0072e20 2024-05-09T08:02:54.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.008+0000 7fa3a6200640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7fa3a015a970 con 0x7fa3a01572c0 2024-05-09T08:02:54.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.009+0000 7fa39fe00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fa3a01572c0 0x7fa3a014e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:54.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.009+0000 7fa39f400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3a0072e20 0x7fa3a00c4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:54.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.009+0000 7fa39fe00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fa3a01572c0 0x7fa3a014e7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3301/0 says I am v2:172.21.15.1:44534/0 (socket says 172.21.15.1:44534) 2024-05-09T08:02:54.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.009+0000 7fa39fe00640 1 -- 172.21.15.1:0/30039871 learned_addr learned my addr 172.21.15.1:0/30039871 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:02:54.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.009+0000 7fa39ea00640 1 --2- 172.21.15.1:0/30039871 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa3a01507b0 0x7fa3a00c5740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:54.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.009+0000 7ffb96a00640 1 --2- 172.21.15.1:0/3082526225 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7ffb8006bf30 0x7ffb8006e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:54.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.009+0000 7ffb96a00640 1 -- 172.21.15.1:0/3082526225 <== mon.1 v2:172.21.15.7:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7ffb9808a730 con 0x7ffba00caf10 2024-05-09T08:02:54.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.010+0000 7ffb9ea00640 1 --2- 172.21.15.1:0/3082526225 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7ffb8006bf30 0x7ffb8006e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:54.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.010+0000 7ffb96a00640 1 --2- 172.21.15.1:0/3082526225 >> [v2:172.21.15.7:6808/1670295242,v1:172.21.15.7:6809/1670295242] conn(0x7ffb800756a0 0x7ffb80077ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:54.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.010+0000 7ffb96a00640 1 -- 172.21.15.1:0/3082526225 --> [v2:172.21.15.7:6808/1670295242,v1:172.21.15.7:6809/1670295242] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ffb98015ec0 con 0x7ffb800756a0 2024-05-09T08:02:54.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.010+0000 7ffb96a00640 1 -- 172.21.15.1:0/3082526225 <== mon.1 v2:172.21.15.7:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ffb98091050 con 0x7ffba00caf10 2024-05-09T08:02:54.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.007+0000 7f6252000640 1 -- 172.21.15.1:0/3364473488 <== mon.2 v2:172.21.15.7:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6244018020 con 0x7f625c06c900 2024-05-09T08:02:54.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.008+0000 7f6252000640 1 --2- 172.21.15.1:0/3364473488 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f623806c030 0x7f623806e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:54.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.010+0000 7fa39f400640 1 -- 172.21.15.1:0/30039871 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fa3a01572c0 msgr2=0x7fa3a014e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:54.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.010+0000 7fa39f400640 1 --2- 172.21.15.1:0/30039871 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fa3a01572c0 0x7fa3a014e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:54.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.010+0000 7fa39f400640 1 -- 172.21.15.1:0/30039871 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa3a01507b0 msgr2=0x7fa3a00c5740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:54.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.010+0000 7fa39f400640 1 --2- 172.21.15.1:0/30039871 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa3a01507b0 0x7fa3a00c5740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:54.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.010+0000 7fbbc5600640 1 -- 172.21.15.1:0/546460052 <== osd.4 v2:172.21.15.7:6800/2957026670 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbbb4034ec0 con 0x7fbba80755a0 2024-05-09T08:02:54.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.008+0000 7f6252000640 1 -- 172.21.15.1:0/3364473488 <== mon.2 v2:172.21.15.7:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f624408ab80 con 0x7f625c06c900 2024-05-09T08:02:54.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.008+0000 7f6259400640 1 --2- 172.21.15.1:0/3364473488 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f623806c030 0x7f623806e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:54.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.010+0000 7ffb9e000640 1 --2- 172.21.15.1:0/3082526225 >> [v2:172.21.15.7:6808/1670295242,v1:172.21.15.7:6809/1670295242] conn(0x7ffb800756a0 0x7ffb80077ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:54.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.010+0000 7ffb9e000640 1 --2- 172.21.15.1:0/3082526225 >> [v2:172.21.15.7:6808/1670295242,v1:172.21.15.7:6809/1670295242] conn(0x7ffb800756a0 0x7ffb80077ae0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:54.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.009+0000 7f6252000640 1 --2- 172.21.15.1:0/3364473488 >> [v2:172.21.15.7:6824/1442884138,v1:172.21.15.7:6825/1442884138] conn(0x7f6238075720 0x7f6238077b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:54.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.009+0000 7f6252000640 1 -- 172.21.15.1:0/3364473488 --> [v2:172.21.15.7:6824/1442884138,v1:172.21.15.7:6825/1442884138] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f624400de50 con 0x7f6238075720 2024-05-09T08:02:54.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.009+0000 7f6252000640 1 -- 172.21.15.1:0/3364473488 <== mon.2 v2:172.21.15.7:3301/0 6 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f624408f2a0 con 0x7f625c06c900 2024-05-09T08:02:54.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.009+0000 7f625a800640 1 --2- 172.21.15.1:0/3364473488 >> [v2:172.21.15.7:6824/1442884138,v1:172.21.15.7:6825/1442884138] conn(0x7f6238075720 0x7f6238077b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:54.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.010+0000 7f6259400640 1 --2- 172.21.15.1:0/3364473488 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f623806c030 0x7f623806e4f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f624c033000 tx=0x7f624c009000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:54.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.009+0000 7f88ebe00640 1 -- 172.21.15.1:0/2735335039 <== mon.2 v2:172.21.15.7:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f88e4044020 con 0x7f88f40c69a0 2024-05-09T08:02:54.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.011+0000 7f625a800640 1 --2- 172.21.15.1:0/3364473488 >> [v2:172.21.15.7:6824/1442884138,v1:172.21.15.7:6825/1442884138] conn(0x7f6238075720 0x7f6238077b60 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:54.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.011+0000 7f7c04c00640 1 -- 172.21.15.1:0/934130762 <== osd.1 v2:172.21.15.1:6816/2942255259 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7bf4034e70 con 0x7f7be80756a0 2024-05-09T08:02:54.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.011+0000 7fa39f400640 1 -- 172.21.15.1:0/30039871 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3a014dcf0 con 0x7fa3a0072e20 2024-05-09T08:02:54.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.011+0000 7ffb9ea00640 1 --2- 172.21.15.1:0/3082526225 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7ffb8006bf30 0x7ffb8006e3f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ffb90009560 tx=0x7ffb9002e000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:54.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.011+0000 7fa39f400640 1 --2- 172.21.15.1:0/30039871 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3a0072e20 0x7fa3a00c4a60 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fa3a0156a70 tx=0x7fa3980046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:54.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.011+0000 7f88ebe00640 1 --2- 172.21.15.1:0/2735335039 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f88d406c060 0x7f88d406e520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:54.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.011+0000 7f88ebe00640 1 -- 172.21.15.1:0/2735335039 <== mon.2 v2:172.21.15.7:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f88e40ab9b0 con 0x7f88f40c69a0 2024-05-09T08:02:54.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.011+0000 7f88f3400640 1 --2- 172.21.15.1:0/2735335039 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f88d406c060 0x7f88d406e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:54.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.011+0000 7ffb96a00640 1 -- 172.21.15.1:0/3082526225 <== osd.5 v2:172.21.15.7:6808/1670295242 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ffb98015ec0 con 0x7ffb800756a0 2024-05-09T08:02:54.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.012+0000 7fa397400640 1 -- 172.21.15.1:0/30039871 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa398038040 con 0x7fa3a0072e20 2024-05-09T08:02:54.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.012+0000 7f88ebe00640 1 --2- 172.21.15.1:0/2735335039 >> [v2:172.21.15.1:6824/2497776654,v1:172.21.15.1:6825/2497776654] conn(0x7f88d40757d0 0x7f88d4077c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:54.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.012+0000 7f88ebe00640 1 -- 172.21.15.1:0/2735335039 --> [v2:172.21.15.1:6824/2497776654,v1:172.21.15.1:6825/2497776654] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f88d40782e0 con 0x7f88d40757d0 2024-05-09T08:02:54.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.012+0000 7f88ebe00640 1 -- 172.21.15.1:0/2735335039 <== mon.2 v2:172.21.15.7:3301/0 6 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f88e40abda0 con 0x7f88f40c69a0 2024-05-09T08:02:54.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.012+0000 7fa397400640 1 -- 172.21.15.1:0/30039871 <== mon.1 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa398033830 con 0x7fa3a0072e20 2024-05-09T08:02:54.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.012+0000 7f88f3400640 1 --2- 172.21.15.1:0/2735335039 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f88d406c060 0x7f88d406e520 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f88f4149a70 tx=0x7f88ec0094f0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:54.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.012+0000 7fa3a6200640 1 -- 172.21.15.1:0/30039871 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3a014dfb0 con 0x7fa3a0072e20 2024-05-09T08:02:54.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.012+0000 7fa397400640 1 -- 172.21.15.1:0/30039871 <== mon.1 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa398033c20 con 0x7fa3a0072e20 2024-05-09T08:02:54.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.012+0000 7fa3a6200640 1 -- 172.21.15.1:0/30039871 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3a01f49f0 con 0x7fa3a0072e20 2024-05-09T08:02:54.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.013+0000 7fa3a6200640 1 -- 172.21.15.1:0/30039871 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa3a0074d70 con 0x7fa3a0072e20 2024-05-09T08:02:54.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.012+0000 7f6252000640 1 -- 172.21.15.1:0/3364473488 <== osd.7 v2:172.21.15.7:6824/1442884138 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f624400de50 con 0x7f6238075720 2024-05-09T08:02:54.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.014+0000 7f88f8c00640 1 --2- 172.21.15.1:0/2735335039 >> [v2:172.21.15.1:6824/2497776654,v1:172.21.15.1:6825/2497776654] conn(0x7f88d40757d0 0x7f88d4077c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:54.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.015+0000 7f88f8c00640 1 --2- 172.21.15.1:0/2735335039 >> [v2:172.21.15.1:6824/2497776654,v1:172.21.15.1:6825/2497776654] conn(0x7f88d40757d0 0x7f88d4077c10 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:54.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.017+0000 7f88ebe00640 1 -- 172.21.15.1:0/2735335039 <== osd.3 v2:172.21.15.1:6824/2497776654 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f88d40782e0 con 0x7f88d40757d0 2024-05-09T08:02:54.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.015+0000 7fa397400640 1 -- 172.21.15.1:0/30039871 <== mon.1 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa39803e430 con 0x7fa3a0072e20 2024-05-09T08:02:54.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.017+0000 7fa397400640 1 --2- 172.21.15.1:0/30039871 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fa37c06bf00 0x7fa37c06e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:54.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.017+0000 7fa397400640 1 -- 172.21.15.1:0/30039871 <== mon.1 v2:172.21.15.7:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fa3980aba60 con 0x7fa3a0072e20 2024-05-09T08:02:54.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.017+0000 7fa397400640 1 --2- 172.21.15.1:0/30039871 >> [v2:172.21.15.7:6816/2852086880,v1:172.21.15.7:6817/2852086880] conn(0x7fa37c0755f0 0x7fa37c077a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:54.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.017+0000 7fa397400640 1 -- 172.21.15.1:0/30039871 --> [v2:172.21.15.7:6816/2852086880,v1:172.21.15.7:6817/2852086880] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa398049e70 con 0x7fa37c0755f0 2024-05-09T08:02:54.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.018+0000 7fa397400640 1 -- 172.21.15.1:0/30039871 <== mon.1 v2:172.21.15.7:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa398075dc0 con 0x7fa3a0072e20 2024-05-09T08:02:54.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.018+0000 7fa39ea00640 1 --2- 172.21.15.1:0/30039871 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fa37c06bf00 0x7fa37c06e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:54.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.019+0000 7fa39fe00640 1 --2- 172.21.15.1:0/30039871 >> [v2:172.21.15.7:6816/2852086880,v1:172.21.15.7:6817/2852086880] conn(0x7fa37c0755f0 0x7fa37c077a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:54.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.019+0000 7fa39ea00640 1 --2- 172.21.15.1:0/30039871 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fa37c06bf00 0x7fa37c06e3c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fa3a00c8bc0 tx=0x7fa390008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:54.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.019+0000 7fa39fe00640 1 --2- 172.21.15.1:0/30039871 >> [v2:172.21.15.7:6816/2852086880,v1:172.21.15.7:6817/2852086880] conn(0x7fa37c0755f0 0x7fa37c077a30 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:54.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.020+0000 7fa397400640 1 -- 172.21.15.1:0/30039871 <== osd.6 v2:172.21.15.7:6816/2852086880 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa398049e70 con 0x7fa37c0755f0 2024-05-09T08:02:54.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.038+0000 7f9075400640 1 -- 172.21.15.1:0/2896389809 --> [v2:172.21.15.1:6800/1513349547,v1:172.21.15.1:6801/1513349547] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9048002dd0 con 0x7f904c075520 2024-05-09T08:02:54.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.039+0000 7f9066a00640 1 -- 172.21.15.1:0/2896389809 <== osd.0 v2:172.21.15.1:6800/1513349547 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f9048002dd0 con 0x7f904c075520 2024-05-09T08:02:54.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.041+0000 7fa62c600640 1 -- 172.21.15.1:0/1791582975 --> [v2:172.21.15.1:6808/2925031811,v1:172.21.15.1:6809/2925031811] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa6240d0780 con 0x7fa604075670 2024-05-09T08:02:54.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.041+0000 7f9075400640 1 -- 172.21.15.1:0/2896389809 >> [v2:172.21.15.1:6800/1513349547,v1:172.21.15.1:6801/1513349547] conn(0x7f904c075520 msgr2=0x7f904c077960 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:54.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.041+0000 7f9075400640 1 --2- 172.21.15.1:0/2896389809 >> [v2:172.21.15.1:6800/1513349547,v1:172.21.15.1:6801/1513349547] conn(0x7f904c075520 0x7f904c077960 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:54.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.041+0000 7fbbcee00640 1 -- 172.21.15.1:0/546460052 --> [v2:172.21.15.7:6800/2957026670,v1:172.21.15.7:6801/2957026670] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fbbc0032d40 con 0x7fbba80755a0 2024-05-09T08:02:54.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.041+0000 7f9075400640 1 -- 172.21.15.1:0/2896389809 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f904c06be30 msgr2=0x7f904c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:54.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.041+0000 7f9075400640 1 --2- 172.21.15.1:0/2896389809 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f904c06be30 0x7f904c06e2f0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f906800c530 tx=0x7f906800b040 comp rx=0 tx=0).stop 2024-05-09T08:02:54.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.041+0000 7f9075400640 1 -- 172.21.15.1:0/2896389809 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9070157200 msgr2=0x7f90701593f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:54.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.041+0000 7f9075400640 1 --2- 172.21.15.1:0/2896389809 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9070157200 0x7f90701593f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f907006e9e0 tx=0x7f90580046c0 comp rx=0 tx=0).stop 2024-05-09T08:02:54.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.041+0000 7f906f200640 1 -- 172.21.15.1:0/2896389809 reap_dead start 2024-05-09T08:02:54.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.041+0000 7fa621600640 1 -- 172.21.15.1:0/1791582975 <== osd.2 v2:172.21.15.1:6808/2925031811 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fa6240d0780 con 0x7fa604075670 2024-05-09T08:02:54.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.041+0000 7f9075400640 1 -- 172.21.15.1:0/2896389809 shutdown_connections 2024-05-09T08:02:54.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.041+0000 7f9075400640 1 -- 172.21.15.1:0/2896389809 >> 172.21.15.1:0/2896389809 conn(0x7f907006f440 msgr2=0x7f90700ca350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:02:54.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.042+0000 7fbbc5600640 1 -- 172.21.15.1:0/546460052 <== osd.4 v2:172.21.15.7:6800/2957026670 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fbbc0032d40 con 0x7fbba80755a0 2024-05-09T08:02:54.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.042+0000 7f9075400640 1 -- 172.21.15.1:0/2896389809 shutdown_connections 2024-05-09T08:02:54.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.042+0000 7f9075400640 1 -- 172.21.15.1:0/2896389809 wait complete. 2024-05-09T08:02:54.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.042+0000 7fbbcee00640 1 -- 172.21.15.1:0/546460052 >> [v2:172.21.15.7:6800/2957026670,v1:172.21.15.7:6801/2957026670] conn(0x7fbba80755a0 msgr2=0x7fbba80779e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:54.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.042+0000 7fbbcee00640 1 --2- 172.21.15.1:0/546460052 >> [v2:172.21.15.7:6800/2957026670,v1:172.21.15.7:6801/2957026670] conn(0x7fbba80755a0 0x7fbba80779e0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:54.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.043+0000 7fa62c600640 1 -- 172.21.15.1:0/1791582975 >> [v2:172.21.15.1:6808/2925031811,v1:172.21.15.1:6809/2925031811] conn(0x7fa604075670 msgr2=0x7fa604077ab0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:54.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.043+0000 7fa62c600640 1 --2- 172.21.15.1:0/1791582975 >> [v2:172.21.15.1:6808/2925031811,v1:172.21.15.1:6809/2925031811] conn(0x7fa604075670 0x7fa604077ab0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:54.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.043+0000 7fbbcee00640 1 -- 172.21.15.1:0/546460052 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fbba806be30 msgr2=0x7fbba806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:54.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.043+0000 7fbbcee00640 1 --2- 172.21.15.1:0/546460052 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fbba806be30 0x7fbba806e2f0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fbbc8050860 tx=0x7fbbc8072040 comp rx=0 tx=0).stop 2024-05-09T08:02:54.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.043+0000 7fa62c600640 1 -- 172.21.15.1:0/1791582975 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fa60406bf00 msgr2=0x7fa60406e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:54.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.043+0000 7fa62c600640 1 --2- 172.21.15.1:0/1791582975 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fa60406bf00 0x7fa60406e3c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa610034000 tx=0x7fa61002e000 comp rx=0 tx=0).stop 2024-05-09T08:02:54.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.043+0000 7fa62c600640 1 -- 172.21.15.1:0/1791582975 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa62406c900 msgr2=0x7fa6241508f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:54.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.043+0000 7fa62c600640 1 --2- 172.21.15.1:0/1791582975 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa62406c900 0x7fa6241508f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fa61800caf0 tx=0x7fa618002100 comp rx=0 tx=0).stop 2024-05-09T08:02:54.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.043+0000 7fa62a600640 1 -- 172.21.15.1:0/1791582975 reap_dead start 2024-05-09T08:02:54.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.043+0000 7fbbcee00640 1 -- 172.21.15.1:0/546460052 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fbbc0050ca0 msgr2=0x7fbbc00466f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:54.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.043+0000 7fbbcee00640 1 --2- 172.21.15.1:0/546460052 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fbbc0050ca0 0x7fbbc00466f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fbbb4000c00 tx=0x7fbbb40046c0 comp rx=0 tx=0).stop 2024-05-09T08:02:54.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.043+0000 7fbbce400640 1 -- 172.21.15.1:0/546460052 reap_dead start 2024-05-09T08:02:54.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.043+0000 7fbbcee00640 1 -- 172.21.15.1:0/546460052 shutdown_connections 2024-05-09T08:02:54.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.043+0000 7fbbcee00640 1 -- 172.21.15.1:0/546460052 >> 172.21.15.1:0/546460052 conn(0x7fbbc0050180 msgr2=0x7fbbc004cb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:02:54.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.044+0000 7fa62c600640 1 -- 172.21.15.1:0/1791582975 shutdown_connections 2024-05-09T08:02:54.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.044+0000 7fa62c600640 1 -- 172.21.15.1:0/1791582975 >> 172.21.15.1:0/1791582975 conn(0x7fa6240cfe80 msgr2=0x7fa624154590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:02:54.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.043+0000 7fbbcee00640 1 -- 172.21.15.1:0/546460052 shutdown_connections 2024-05-09T08:02:54.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.044+0000 7fa62c600640 1 -- 172.21.15.1:0/1791582975 shutdown_connections 2024-05-09T08:02:54.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.044+0000 7fa62c600640 1 -- 172.21.15.1:0/1791582975 wait complete. 2024-05-09T08:02:54.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.043+0000 7fbbcee00640 1 -- 172.21.15.1:0/546460052 wait complete. 2024-05-09T08:02:54.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.047+0000 7f624b400640 1 -- 172.21.15.1:0/3364473488 --> [v2:172.21.15.7:6824/1442884138,v1:172.21.15.7:6825/1442884138] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f625c074d70 con 0x7f6238075720 2024-05-09T08:02:54.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.047+0000 7f6252000640 1 -- 172.21.15.1:0/3364473488 <== osd.7 v2:172.21.15.7:6824/1442884138 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f625c074d70 con 0x7f6238075720 2024-05-09T08:02:54.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.048+0000 7f7c0e000640 1 -- 172.21.15.1:0/934130762 --> [v2:172.21.15.1:6816/2942255259,v1:172.21.15.1:6817/2942255259] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f7be0002dd0 con 0x7f7be80756a0 2024-05-09T08:02:54.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.048+0000 7f624b400640 1 -- 172.21.15.1:0/3364473488 >> [v2:172.21.15.7:6824/1442884138,v1:172.21.15.7:6825/1442884138] conn(0x7f6238075720 msgr2=0x7f6238077b60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:54.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.048+0000 7f624b400640 1 --2- 172.21.15.1:0/3364473488 >> [v2:172.21.15.7:6824/1442884138,v1:172.21.15.7:6825/1442884138] conn(0x7f6238075720 0x7f6238077b60 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:54.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.048+0000 7f624b400640 1 -- 172.21.15.1:0/3364473488 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f623806c030 msgr2=0x7f623806e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:54.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.048+0000 7f624b400640 1 --2- 172.21.15.1:0/3364473488 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f623806c030 0x7f623806e4f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f624c033000 tx=0x7f624c009000 comp rx=0 tx=0).stop 2024-05-09T08:02:54.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.048+0000 7f624b400640 1 -- 172.21.15.1:0/3364473488 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f625c06c900 msgr2=0x7f625c155250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:54.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.048+0000 7f624b400640 1 --2- 172.21.15.1:0/3364473488 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f625c06c900 0x7f625c155250 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f624400b410 tx=0x7f624400b8d0 comp rx=0 tx=0).stop 2024-05-09T08:02:54.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.049+0000 7f625a800640 1 -- 172.21.15.1:0/3364473488 reap_dead start 2024-05-09T08:02:54.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.049+0000 7f624b400640 1 -- 172.21.15.1:0/3364473488 shutdown_connections 2024-05-09T08:02:54.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.049+0000 7f624b400640 1 -- 172.21.15.1:0/3364473488 >> 172.21.15.1:0/3364473488 conn(0x7f625c0cfe80 msgr2=0x7f625c0ca700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:02:54.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.049+0000 7f7c04c00640 1 -- 172.21.15.1:0/934130762 <== osd.1 v2:172.21.15.1:6816/2942255259 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f7be0002dd0 con 0x7f7be80756a0 2024-05-09T08:02:54.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.049+0000 7ffba5600640 1 -- 172.21.15.1:0/3082526225 --> [v2:172.21.15.7:6808/1670295242,v1:172.21.15.7:6809/1670295242] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ffba0074d70 con 0x7ffb800756a0 2024-05-09T08:02:54.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.049+0000 7f88f9600640 1 -- 172.21.15.1:0/2735335039 --> [v2:172.21.15.1:6824/2497776654,v1:172.21.15.1:6825/2497776654] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f88f40b2030 con 0x7f88d40757d0 2024-05-09T08:02:54.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.050+0000 7f624b400640 1 -- 172.21.15.1:0/3364473488 shutdown_connections 2024-05-09T08:02:54.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.050+0000 7f624b400640 1 -- 172.21.15.1:0/3364473488 wait complete. 2024-05-09T08:02:54.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.050+0000 7ffb96a00640 1 -- 172.21.15.1:0/3082526225 <== osd.5 v2:172.21.15.7:6808/1670295242 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7ffba0074d70 con 0x7ffb800756a0 2024-05-09T08:02:54.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.050+0000 7f7c0e000640 1 -- 172.21.15.1:0/934130762 >> [v2:172.21.15.1:6816/2942255259,v1:172.21.15.1:6817/2942255259] conn(0x7f7be80756a0 msgr2=0x7f7be8077ae0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:54.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.050+0000 7f7c0e000640 1 --2- 172.21.15.1:0/934130762 >> [v2:172.21.15.1:6816/2942255259,v1:172.21.15.1:6817/2942255259] conn(0x7f7be80756a0 0x7f7be8077ae0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:54.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.050+0000 7f88ebe00640 1 -- 172.21.15.1:0/2735335039 <== osd.3 v2:172.21.15.1:6824/2497776654 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f88f40b2030 con 0x7f88d40757d0 2024-05-09T08:02:54.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.051+0000 7f7c0e000640 1 -- 172.21.15.1:0/934130762 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f7be806c030 msgr2=0x7f7be806e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:54.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.051+0000 7f7c0e000640 1 --2- 172.21.15.1:0/934130762 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f7be806c030 0x7f7be806e4f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f7bf8000f60 tx=0x7f7bf8009040 comp rx=0 tx=0).stop 2024-05-09T08:02:54.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.051+0000 7f7c0e000640 1 -- 172.21.15.1:0/934130762 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f7c080cc5f0 msgr2=0x7f7c0814b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:54.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.051+0000 7f7c0e000640 1 --2- 172.21.15.1:0/934130762 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f7c080cc5f0 0x7f7c0814b1f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f7bf4009f40 tx=0x7f7bf4004a40 comp rx=0 tx=0).stop 2024-05-09T08:02:54.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.051+0000 7f7c0d600640 1 -- 172.21.15.1:0/934130762 reap_dead start 2024-05-09T08:02:54.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.051+0000 7f7c0e000640 1 -- 172.21.15.1:0/934130762 shutdown_connections 2024-05-09T08:02:54.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.051+0000 7f88f9600640 1 -- 172.21.15.1:0/2735335039 >> [v2:172.21.15.1:6824/2497776654,v1:172.21.15.1:6825/2497776654] conn(0x7f88d40757d0 msgr2=0x7f88d4077c10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:54.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.051+0000 7f88f9600640 1 --2- 172.21.15.1:0/2735335039 >> [v2:172.21.15.1:6824/2497776654,v1:172.21.15.1:6825/2497776654] conn(0x7f88d40757d0 0x7f88d4077c10 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:54.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.051+0000 7f7c0e000640 1 -- 172.21.15.1:0/934130762 >> 172.21.15.1:0/934130762 conn(0x7f7c080cfc20 msgr2=0x7f7c080b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:02:54.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.052+0000 7f7c0e000640 1 -- 172.21.15.1:0/934130762 shutdown_connections 2024-05-09T08:02:54.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.052+0000 7f88f9600640 1 -- 172.21.15.1:0/2735335039 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f88d406c060 msgr2=0x7f88d406e520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:54.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.052+0000 7f88f9600640 1 --2- 172.21.15.1:0/2735335039 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f88d406c060 0x7f88d406e520 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f88f4149a70 tx=0x7f88ec0094f0 comp rx=0 tx=0).stop 2024-05-09T08:02:54.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.052+0000 7ffba5600640 1 -- 172.21.15.1:0/3082526225 >> [v2:172.21.15.7:6808/1670295242,v1:172.21.15.7:6809/1670295242] conn(0x7ffb800756a0 msgr2=0x7ffb80077ae0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:54.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.052+0000 7ffba5600640 1 --2- 172.21.15.1:0/3082526225 >> [v2:172.21.15.7:6808/1670295242,v1:172.21.15.7:6809/1670295242] conn(0x7ffb800756a0 0x7ffb80077ae0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:54.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.052+0000 7f88f9600640 1 -- 172.21.15.1:0/2735335039 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f88f40c69a0 msgr2=0x7f88f41589c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:54.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.052+0000 7f7c0e000640 1 -- 172.21.15.1:0/934130762 wait complete. 2024-05-09T08:02:54.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.052+0000 7ffba5600640 1 -- 172.21.15.1:0/3082526225 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7ffb8006bf30 msgr2=0x7ffb8006e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:54.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.052+0000 7f88f9600640 1 --2- 172.21.15.1:0/2735335039 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f88f40c69a0 0x7f88f41589c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f88e402a740 tx=0x7f88e40046c0 comp rx=0 tx=0).stop 2024-05-09T08:02:54.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.052+0000 7f88f8c00640 1 -- 172.21.15.1:0/2735335039 reap_dead start 2024-05-09T08:02:54.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.052+0000 7f88f9600640 1 -- 172.21.15.1:0/2735335039 shutdown_connections 2024-05-09T08:02:54.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.052+0000 7ffba5600640 1 --2- 172.21.15.1:0/3082526225 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7ffb8006bf30 0x7ffb8006e3f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ffb90009560 tx=0x7ffb9002e000 comp rx=0 tx=0).stop 2024-05-09T08:02:54.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.052+0000 7f88f9600640 1 -- 172.21.15.1:0/2735335039 >> 172.21.15.1:0/2735335039 conn(0x7f88f406b250 msgr2=0x7f88f40c4a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:02:54.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.052+0000 7f88f9600640 1 -- 172.21.15.1:0/2735335039 shutdown_connections 2024-05-09T08:02:54.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.053+0000 7f88f9600640 1 -- 172.21.15.1:0/2735335039 wait complete. 2024-05-09T08:02:54.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.052+0000 7ffba5600640 1 -- 172.21.15.1:0/3082526225 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffba00caf10 msgr2=0x7ffba00cbd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:54.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.052+0000 7ffba5600640 1 --2- 172.21.15.1:0/3082526225 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffba00caf10 0x7ffba00cbd00 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ffba00c8c60 tx=0x7ffb9800cbe0 comp rx=0 tx=0).stop 2024-05-09T08:02:54.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.052+0000 7ffb9f400640 1 -- 172.21.15.1:0/3082526225 reap_dead start 2024-05-09T08:02:54.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.052+0000 7ffba5600640 1 -- 172.21.15.1:0/3082526225 shutdown_connections 2024-05-09T08:02:54.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.052+0000 7ffba5600640 1 -- 172.21.15.1:0/3082526225 >> 172.21.15.1:0/3082526225 conn(0x7ffba006dad0 msgr2=0x7ffba0152450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:02:54.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.053+0000 7ffba5600640 1 -- 172.21.15.1:0/3082526225 shutdown_connections 2024-05-09T08:02:54.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.053+0000 7ffba5600640 1 -- 172.21.15.1:0/3082526225 wait complete. 2024-05-09T08:02:54.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.055+0000 7fa3a6200640 1 -- 172.21.15.1:0/30039871 --> [v2:172.21.15.7:6816/2852086880,v1:172.21.15.7:6817/2852086880] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa3a00726c0 con 0x7fa37c0755f0 2024-05-09T08:02:54.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.056+0000 7fa397400640 1 -- 172.21.15.1:0/30039871 <== osd.6 v2:172.21.15.7:6816/2852086880 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fa3a00726c0 con 0x7fa37c0755f0 2024-05-09T08:02:54.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.058+0000 7fa3a6200640 1 -- 172.21.15.1:0/30039871 >> [v2:172.21.15.7:6816/2852086880,v1:172.21.15.7:6817/2852086880] conn(0x7fa37c0755f0 msgr2=0x7fa37c077a30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:54.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.058+0000 7fa3a6200640 1 --2- 172.21.15.1:0/30039871 >> [v2:172.21.15.7:6816/2852086880,v1:172.21.15.7:6817/2852086880] conn(0x7fa37c0755f0 0x7fa37c077a30 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:54.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.058+0000 7fa3a6200640 1 -- 172.21.15.1:0/30039871 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fa37c06bf00 msgr2=0x7fa37c06e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:54.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.059+0000 7fa3a6200640 1 --2- 172.21.15.1:0/30039871 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fa37c06bf00 0x7fa37c06e3c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fa3a00c8bc0 tx=0x7fa390008040 comp rx=0 tx=0).stop 2024-05-09T08:02:54.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.059+0000 7fa3a6200640 1 -- 172.21.15.1:0/30039871 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3a0072e20 msgr2=0x7fa3a00c4a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:54.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.059+0000 7fa3a6200640 1 --2- 172.21.15.1:0/30039871 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3a0072e20 0x7fa3a00c4a60 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fa3a0156a70 tx=0x7fa3980046c0 comp rx=0 tx=0).stop 2024-05-09T08:02:54.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.059+0000 7fa39fe00640 1 -- 172.21.15.1:0/30039871 reap_dead start 2024-05-09T08:02:54.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.059+0000 7fa3a6200640 1 -- 172.21.15.1:0/30039871 shutdown_connections 2024-05-09T08:02:54.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.060+0000 7fa3a6200640 1 -- 172.21.15.1:0/30039871 >> 172.21.15.1:0/30039871 conn(0x7fa3a006dad0 msgr2=0x7fa3a006f720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:02:54.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.060+0000 7fa3a6200640 1 -- 172.21.15.1:0/30039871 shutdown_connections 2024-05-09T08:02:54.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.060+0000 7fa3a6200640 1 -- 172.21.15.1:0/30039871 wait complete. 2024-05-09T08:02:54.111 INFO:teuthology.orchestra.run.smithi001.stdout:55834574860 2024-05-09T08:02:54.112 INFO:teuthology.orchestra.run.smithi001.stdout:55834574860 2024-05-09T08:02:54.112 INFO:teuthology.orchestra.run.smithi001.stdout:55834574860 2024-05-09T08:02:54.113 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-05-09T08:02:54.113 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-05-09T08:02:54.114 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-05-09T08:02:54.115 INFO:teuthology.orchestra.run.smithi001.stdout:55834574859 2024-05-09T08:02:54.115 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-05-09T08:02:54.116 INFO:teuthology.orchestra.run.smithi001.stdout:55834574861 2024-05-09T08:02:54.116 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-05-09T08:02:54.117 INFO:teuthology.orchestra.run.smithi001.stdout:55834574860 2024-05-09T08:02:54.117 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-05-09T08:02:54.118 INFO:teuthology.orchestra.run.smithi001.stdout:55834574859 2024-05-09T08:02:54.119 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-05-09T08:02:54.119 INFO:teuthology.orchestra.run.smithi001.stdout:55834574859 2024-05-09T08:02:54.120 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-05-09T08:02:54.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.423+0000 7f58eb200640 1 Processor -- start 2024-05-09T08:02:54.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.423+0000 7f58eb200640 1 -- start start 2024-05-09T08:02:54.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.425+0000 7f58eb200640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f58e40cac60 0x7f58e40cda00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:54.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.425+0000 7f58eb200640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f58e415ac80 0x7f58e40cdf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:54.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.425+0000 7f58eb200640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58e40ccb20 con 0x7f58e40cc5f0 2024-05-09T08:02:54.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.425+0000 7f58eb200640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f58e40cce00 con 0x7f58e415ac80 2024-05-09T08:02:54.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.425+0000 7f58eb200640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f58e40cf120 con 0x7f58e40cac60 2024-05-09T08:02:54.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.426+0000 7f58ea800640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f58e415ac80 0x7f58e40cdf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:54.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.426+0000 7f58ea800640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f58e415ac80 0x7f58e40cdf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.1:58846/0 (socket says 172.21.15.1:58846) 2024-05-09T08:02:54.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.426+0000 7f58ea800640 1 -- 172.21.15.1:0/3918367885 learned_addr learned my addr 172.21.15.1:0/3918367885 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:02:54.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.427+0000 7f58e9e00640 1 --2- 172.21.15.1:0/3918367885 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f58e40cac60 0x7f58e40cda00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:54.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.427+0000 7f58ea800640 1 -- 172.21.15.1:0/3918367885 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f58e40cac60 msgr2=0x7f58e40cda00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:02:54.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.427+0000 7f58ea800640 1 --2- 172.21.15.1:0/3918367885 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f58e40cac60 0x7f58e40cda00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:54.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.427+0000 7f58ea800640 1 -- 172.21.15.1:0/3918367885 >> v1:172.21.15.1:6789/0 conn(0x7f58e40cc5f0 legacy=0x7f58e415a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:02:54.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.427+0000 7f58ea800640 1 -- 172.21.15.1:0/3918367885 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58e40cf400 con 0x7f58e415ac80 2024-05-09T08:02:54.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.429+0000 7f58e9e00640 1 --2- 172.21.15.1:0/3918367885 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f58e40cac60 0x7f58e40cda00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T08:02:54.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.429+0000 7f58ea800640 1 --2- 172.21.15.1:0/3918367885 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58e415ac80 0x7f58e40cdf40 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f58d4009870 tx=0x7f58d402a2c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e41585c671b0faf9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:54.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.430+0000 7f58e8a00640 1 -- 172.21.15.1:0/3918367885 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58d4037070 con 0x7f58e415ac80 2024-05-09T08:02:54.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.430+0000 7f58e8a00640 1 -- 172.21.15.1:0/3918367885 <== mon.1 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58d4032440 con 0x7f58e415ac80 2024-05-09T08:02:54.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.430+0000 7f58e8a00640 1 -- 172.21.15.1:0/3918367885 <== mon.1 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58d4032850 con 0x7f58e415ac80 2024-05-09T08:02:54.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.432+0000 7f02c3200640 1 Processor -- start 2024-05-09T08:02:54.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.432+0000 7f02c3200640 1 -- start start 2024-05-09T08:02:54.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.432+0000 7f58eb200640 1 -- 172.21.15.1:0/3918367885 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58e415ac80 msgr2=0x7f58e40cdf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:54.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.433+0000 7f58eb200640 1 --2- 172.21.15.1:0/3918367885 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58e415ac80 0x7f58e40cdf40 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f58d4009870 tx=0x7f58d402a2c0 comp rx=0 tx=0).stop 2024-05-09T08:02:54.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.433+0000 7f58eb200640 1 -- 172.21.15.1:0/3918367885 shutdown_connections 2024-05-09T08:02:54.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.433+0000 7f58eb200640 1 --2- 172.21.15.1:0/3918367885 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58e415ac80 0x7f58e40cdf40 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:54.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.433+0000 7f58eb200640 1 --2- 172.21.15.1:0/3918367885 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f58e40cac60 0x7f58e40cda00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:54.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.433+0000 7f58eb200640 1 -- 172.21.15.1:0/3918367885 >> 172.21.15.1:0/3918367885 conn(0x7f58e40cfc20 msgr2=0x7f58e40d0030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:02:54.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.433+0000 7fecf2e00640 1 Processor -- start 2024-05-09T08:02:54.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.433+0000 7fecf2e00640 1 -- start start 2024-05-09T08:02:54.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.434+0000 7f58eb200640 1 -- 172.21.15.1:0/3918367885 shutdown_connections 2024-05-09T08:02:54.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.434+0000 7f02c3200640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f02bc0e2e10 0x7f02bc046870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:54.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.434+0000 7f02c3200640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f02bc0e28f0 0x7f02bc005ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:54.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.434+0000 7f02c3200640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f02bc04b400 con 0x7f02bc0e2510 2024-05-09T08:02:54.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.434+0000 7f02c3200640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f02bc045860 con 0x7f02bc0e2e10 2024-05-09T08:02:54.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.434+0000 7f02c3200640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f02bc045ab0 con 0x7f02bc0e28f0 2024-05-09T08:02:54.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.434+0000 7f58eb200640 1 -- 172.21.15.1:0/3918367885 wait complete. 2024-05-09T08:02:54.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.435+0000 7f02c1e00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f02bc0e2510 0x7f02bc04c470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.1:42038/0 (socket says 172.21.15.1:42038) 2024-05-09T08:02:54.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.435+0000 7f02c1e00640 1 -- 172.21.15.1:0/3472772444 learned_addr learned my addr 172.21.15.1:0/3472772444 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T08:02:54.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.435+0000 7f02c2800640 1 --2- 172.21.15.1:0/3472772444 >> v2:172.21.15.7:3300/0 conn(0x7f02bc0e28f0 0x7f02bc005ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:54.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.436+0000 7f02c1400640 1 --2- 172.21.15.1:0/3472772444 >> v2:172.21.15.1:3300/0 conn(0x7f02bc0e2e10 0x7f02bc046870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:54.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.436+0000 7f02c2800640 1 -- 172.21.15.1:0/3472772444 >> v2:172.21.15.1:3300/0 conn(0x7f02bc0e2e10 msgr2=0x7f02bc046870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:02:54.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.436+0000 7f02c2800640 1 --2- 172.21.15.1:0/3472772444 >> v2:172.21.15.1:3300/0 conn(0x7f02bc0e2e10 0x7f02bc046870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:54.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.437+0000 7f58eb200640 1 Processor -- start 2024-05-09T08:02:54.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.437+0000 7f58eb200640 1 -- start start 2024-05-09T08:02:54.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.435+0000 7fecf2e00640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fece4037870 0x7fece4038520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:54.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.435+0000 7fecf2e00640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fece40df1e0 con 0x7fece40d9090 2024-05-09T08:02:54.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.436+0000 7fecf2e00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fece40df4a0 con 0x7fece40deda0 2024-05-09T08:02:54.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.436+0000 7fecf2e00640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7fece4038090 con 0x7fece4037870 2024-05-09T08:02:54.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.436+0000 7fecf1000640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fece40deda0 0x7fece4037160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.1:42042/0 (socket says 172.21.15.1:42042) 2024-05-09T08:02:54.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.437+0000 7fecf1000640 1 -- 172.21.15.1:0/3953462808 learned_addr learned my addr 172.21.15.1:0/3953462808 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T08:02:54.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.437+0000 7fecf2400640 1 --2- 172.21.15.1:0/3953462808 >> v2:172.21.15.1:3300/0 conn(0x7fece4037870 0x7fece4038520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:54.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.436+0000 7f02c2800640 1 -- 172.21.15.1:0/3472772444 >> v1:172.21.15.7:6789/0 conn(0x7f02bc0e2510 legacy=0x7f02bc04c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:54.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.437+0000 7f02c2800640 1 -- 172.21.15.1:0/3472772444 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02bc045d90 con 0x7f02bc0e28f0 2024-05-09T08:02:54.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.437+0000 7f02c1400640 1 --2- 172.21.15.1:0/3472772444 >> v2:172.21.15.1:3300/0 conn(0x7f02bc0e2e10 0x7f02bc046870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T08:02:54.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.439+0000 7f58eb200640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f58e40cac60 0x7f58e40b14e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:54.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.439+0000 7f58eb200640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f58e40cc5f0 0x7f58e4150550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:54.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.439+0000 7fecf2400640 1 -- 172.21.15.1:0/3953462808 >> v1:172.21.15.7:6789/0 conn(0x7fece40deda0 legacy=0x7fece4037160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:54.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.439+0000 7fecf2400640 1 -- 172.21.15.1:0/3953462808 >> v1:172.21.15.1:6789/0 conn(0x7fece40d9090 legacy=0x7fece4050180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:54.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.440+0000 7f58eb200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58e415ac80 0x7f58e4150a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:54.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.440+0000 7fecebe00640 1 -- 172.21.15.1:0/3953462808 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3026569002 0 0) 0x7fece40df1e0 con 0x7fece40d9090 2024-05-09T08:02:54.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.440+0000 7f58e9400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f58e40cc5f0 0x7f58e4150550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:54.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.440+0000 7f58ea800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58e415ac80 0x7f58e4150a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:54.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.440+0000 7f58e9400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f58e40cc5f0 0x7f58e4150550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38090/0 (socket says 172.21.15.1:38090) 2024-05-09T08:02:54.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.440+0000 7f58e9400640 1 -- 172.21.15.1:0/1189377898 learned_addr learned my addr 172.21.15.1:0/1189377898 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:02:54.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.440+0000 7f58eb200640 1 -- 172.21.15.1:0/1189377898 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f58e40ce4e0 con 0x7f58e40cc5f0 2024-05-09T08:02:54.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.440+0000 7f58eb200640 1 -- 172.21.15.1:0/1189377898 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f58e40cd550 con 0x7f58e415ac80 2024-05-09T08:02:54.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.440+0000 7f58eb200640 1 -- 172.21.15.1:0/1189377898 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f58e40ce660 con 0x7f58e40cac60 2024-05-09T08:02:54.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.440+0000 7f0c42c00640 1 Processor -- start 2024-05-09T08:02:54.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.440+0000 7fecf2400640 1 -- 172.21.15.1:0/3953462808 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fece40df780 con 0x7fece4037870 2024-05-09T08:02:54.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.441+0000 7f0c42c00640 1 -- start start 2024-05-09T08:02:54.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.441+0000 7f58e9e00640 1 --2- 172.21.15.1:0/1189377898 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f58e40cac60 0x7f58e40b14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:54.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.441+0000 7fecf2400640 1 --2- 172.21.15.1:0/3953462808 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fece4037870 0x7fece4038520 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fecec068a50 tx=0x7fecec0a5e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=910029c27f6cfa00 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:54.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.441+0000 7f02c2800640 1 --2- 172.21.15.1:0/3472772444 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02bc0e28f0 0x7f02bc005ee0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f02c4068a50 tx=0x7f02c40a5e20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a69a1b9b04af03e3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:54.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.441+0000 7f02c0a00640 1 -- 172.21.15.1:0/3472772444 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02c40a68a0 con 0x7f02bc0e28f0 2024-05-09T08:02:54.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.441+0000 7f58ea800640 1 -- 172.21.15.1:0/1189377898 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f58e40cac60 msgr2=0x7f58e40b14e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:54.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.441+0000 7f58ea800640 1 --2- 172.21.15.1:0/1189377898 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f58e40cac60 0x7f58e40b14e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:54.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.441+0000 7f58ea800640 1 -- 172.21.15.1:0/1189377898 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f58e40cc5f0 msgr2=0x7f58e4150550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:54.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.441+0000 7f58ea800640 1 --2- 172.21.15.1:0/1189377898 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f58e40cc5f0 0x7f58e4150550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:54.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.441+0000 7f58ea800640 1 -- 172.21.15.1:0/1189377898 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58e41f12a0 con 0x7f58e415ac80 2024-05-09T08:02:54.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.442+0000 7f02c0a00640 1 -- 172.21.15.1:0/3472772444 <== mon.1 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02c40a6a40 con 0x7f02bc0e28f0 2024-05-09T08:02:54.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.442+0000 7f02c0a00640 1 -- 172.21.15.1:0/3472772444 <== mon.1 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02c406e450 con 0x7f02bc0e28f0 2024-05-09T08:02:54.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.441+0000 7fecebe00640 1 -- 172.21.15.1:0/3953462808 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecec0a6990 con 0x7fece4037870 2024-05-09T08:02:54.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.442+0000 7fecebe00640 1 -- 172.21.15.1:0/3953462808 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fecec0a6b30 con 0x7fece4037870 2024-05-09T08:02:54.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.442+0000 7fecebe00640 1 -- 172.21.15.1:0/3953462808 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecec06e450 con 0x7fece4037870 2024-05-09T08:02:54.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.442+0000 7f02c3200640 1 -- 172.21.15.1:0/3472772444 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02bc0e28f0 msgr2=0x7f02bc005ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:54.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.442+0000 7f58ea800640 1 --2- 172.21.15.1:0/1189377898 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58e415ac80 0x7f58e4150a90 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f58e4068810 tx=0x7f58d40046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:54.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.443+0000 7fecf2e00640 1 -- 172.21.15.1:0/3953462808 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fece4037870 msgr2=0x7fece4038520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:54.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.443+0000 7fecf2e00640 1 --2- 172.21.15.1:0/3953462808 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fece4037870 0x7fece4038520 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fecec068a50 tx=0x7fecec0a5e90 comp rx=0 tx=0).stop 2024-05-09T08:02:54.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.443+0000 7f58de000640 1 -- 172.21.15.1:0/1189377898 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58d4041070 con 0x7f58e415ac80 2024-05-09T08:02:54.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.443+0000 7f58de000640 1 -- 172.21.15.1:0/1189377898 <== mon.1 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58d403c870 con 0x7f58e415ac80 2024-05-09T08:02:54.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.443+0000 7fecf2e00640 1 -- 172.21.15.1:0/3953462808 shutdown_connections 2024-05-09T08:02:54.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.443+0000 7fecf2e00640 1 --2- 172.21.15.1:0/3953462808 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fece4037870 0x7fece4038520 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:54.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.443+0000 7f0c42c00640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f0c3c0cc260 0x7f0c3c06ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:54.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.443+0000 7f0c42c00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f0c3c06cdc0 0x7f0c3c0cb090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:54.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.443+0000 7f0c42c00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0c3c15e630 con 0x7f0c3c06c9e0 2024-05-09T08:02:54.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.443+0000 7f0c42c00640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f0c3c0ca920 con 0x7f0c3c0cc260 2024-05-09T08:02:54.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.443+0000 7f0c42c00640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f0c3c15ebe0 con 0x7f0c3c06cdc0 2024-05-09T08:02:54.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.443+0000 7f58de000640 1 -- 172.21.15.1:0/1189377898 <== mon.1 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58d4037040 con 0x7f58e415ac80 2024-05-09T08:02:54.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.443+0000 7f58eb200640 1 -- 172.21.15.1:0/1189377898 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58e40b0c20 con 0x7f58e415ac80 2024-05-09T08:02:54.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.443+0000 7f58eb200640 1 -- 172.21.15.1:0/1189377898 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58e41e38a0 con 0x7f58e415ac80 2024-05-09T08:02:54.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.443+0000 7fecf2e00640 1 -- 172.21.15.1:0/3953462808 >> 172.21.15.1:0/3953462808 conn(0x7fece4054610 msgr2=0x7fece4054a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:02:54.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.443+0000 7fecf2e00640 1 -- 172.21.15.1:0/3953462808 shutdown_connections 2024-05-09T08:02:54.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.444+0000 7f0c3b400640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f0c3c0cc260 0x7f0c3c06ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:54.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.444+0000 7f0c3b400640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f0c3c0cc260 0x7f0c3c06ddc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38096/0 (socket says 172.21.15.1:38096) 2024-05-09T08:02:54.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.444+0000 7f0c3b400640 1 -- 172.21.15.1:0/2265804887 learned_addr learned my addr 172.21.15.1:0/2265804887 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:02:54.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.444+0000 7fecf2e00640 1 -- 172.21.15.1:0/3953462808 wait complete. 2024-05-09T08:02:54.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.444+0000 7f573e000640 1 Processor -- start 2024-05-09T08:02:54.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.444+0000 7f0c3b400640 1 -- 172.21.15.1:0/2265804887 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f0c3c06cdc0 msgr2=0x7f0c3c0cb090 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T08:02:54.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.444+0000 7f0c3b400640 1 --2- 172.21.15.1:0/2265804887 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f0c3c06cdc0 0x7f0c3c0cb090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:54.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.444+0000 7f0c3b400640 1 -- 172.21.15.1:0/2265804887 >> v1:172.21.15.7:6789/0 conn(0x7f0c3c06c9e0 legacy=0x7f0c3c1545b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T08:02:54.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.444+0000 7f0c3b400640 1 -- 172.21.15.1:0/2265804887 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c3c15e900 con 0x7f0c3c0cc260 2024-05-09T08:02:54.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.444+0000 7f573e000640 1 -- start start 2024-05-09T08:02:54.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.442+0000 7f02c3200640 1 --2- 172.21.15.1:0/3472772444 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02bc0e28f0 0x7f02bc005ee0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f02c4068a50 tx=0x7f02c40a5e20 comp rx=0 tx=0).stop 2024-05-09T08:02:54.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.443+0000 7f02c3200640 1 -- 172.21.15.1:0/3472772444 shutdown_connections 2024-05-09T08:02:54.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.443+0000 7f02c3200640 1 --2- 172.21.15.1:0/3472772444 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02bc0e28f0 0x7f02bc005ee0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:54.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.443+0000 7f02c3200640 1 --2- 172.21.15.1:0/3472772444 >> v2:172.21.15.1:3300/0 conn(0x7f02bc0e2e10 0x7f02bc046870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:54.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.443+0000 7f02c3200640 1 -- 172.21.15.1:0/3472772444 >> 172.21.15.1:0/3472772444 conn(0x7f02bc050180 msgr2=0x7f02bc050590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:02:54.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.443+0000 7f02c3200640 1 -- 172.21.15.1:0/3472772444 shutdown_connections 2024-05-09T08:02:54.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.445+0000 7f02c3200640 1 -- 172.21.15.1:0/3472772444 wait complete. 2024-05-09T08:02:54.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.446+0000 7fecf2e00640 1 Processor -- start 2024-05-09T08:02:54.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.447+0000 7fecf2e00640 1 -- start start 2024-05-09T08:02:54.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.446+0000 7f0c3b400640 1 --2- 172.21.15.1:0/2265804887 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0c3c0cc260 0x7f0c3c06ddc0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f0c28009870 tx=0x7f0c2802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec8fdb257ebd4e21 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:54.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.447+0000 7f0c3aa00640 1 -- 172.21.15.1:0/2265804887 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c28037070 con 0x7f0c3c0cc260 2024-05-09T08:02:54.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.446+0000 7f58de000640 1 -- 172.21.15.1:0/1189377898 <== mon.1 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f58d4044070 con 0x7f58e415ac80 2024-05-09T08:02:54.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.448+0000 7f0c3aa00640 1 -- 172.21.15.1:0/2265804887 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c28032440 con 0x7f0c3c0cc260 2024-05-09T08:02:54.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.448+0000 7f02c3200640 1 Processor -- start 2024-05-09T08:02:54.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.448+0000 7f58eb200640 1 -- 172.21.15.1:0/1189377898 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58c0005290 con 0x7f58e415ac80 2024-05-09T08:02:54.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.448+0000 7f02c3200640 1 -- start start 2024-05-09T08:02:54.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.449+0000 7f0c42c00640 1 -- 172.21.15.1:0/2265804887 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0c3c0cc260 msgr2=0x7f0c3c06ddc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:54.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.449+0000 7f0c42c00640 1 --2- 172.21.15.1:0/2265804887 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0c3c0cc260 0x7f0c3c06ddc0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f0c28009870 tx=0x7f0c2802a1a0 comp rx=0 tx=0).stop 2024-05-09T08:02:54.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.448+0000 7f0c3aa00640 1 -- 172.21.15.1:0/2265804887 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c28032810 con 0x7f0c3c0cc260 2024-05-09T08:02:54.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.447+0000 7f573e000640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f5738154a60 0x7f573806d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:54.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.447+0000 7f573e000640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f57380b4210 0x7f57380b4620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:54.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.447+0000 7f573e000640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f57380b4b60 con 0x7f57380cc4c0 2024-05-09T08:02:54.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.447+0000 7f573e000640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f573806dff0 con 0x7f57380b4210 2024-05-09T08:02:54.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.447+0000 7f573e000640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f573815eb60 con 0x7f5738154a60 2024-05-09T08:02:54.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.448+0000 7f572fe00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f57380cc4c0 0x7f573806c770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.1:42048/0 (socket says 172.21.15.1:42048) 2024-05-09T08:02:54.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.448+0000 7f572fe00640 1 -- 172.21.15.1:0/2180769876 learned_addr learned my addr 172.21.15.1:0/2180769876 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T08:02:54.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.449+0000 7fecf2e00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fece4037870 0x7fece40dcfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:54.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.449+0000 7f58de000640 1 --2- 172.21.15.1:0/1189377898 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f58c806bf00 0x7f58c806e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:54.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.449+0000 7f0c42c00640 1 -- 172.21.15.1:0/2265804887 shutdown_connections 2024-05-09T08:02:54.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.449+0000 7f0c42c00640 1 --2- 172.21.15.1:0/2265804887 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f0c3c06cdc0 0x7f0c3c0cb090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:54.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.449+0000 7f0c42c00640 1 --2- 172.21.15.1:0/2265804887 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0c3c0cc260 0x7f0c3c06ddc0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:54.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.449+0000 7f0c42c00640 1 -- 172.21.15.1:0/2265804887 >> 172.21.15.1:0/2265804887 conn(0x7f0c3c0cffa0 msgr2=0x7f0c3c0d03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:02:54.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.449+0000 7fecf2e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fece40d9090 0x7fece4053050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:54.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.449+0000 7fecf1a00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fece4037870 0x7fece40dcfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:54.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.449+0000 7fecf1a00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fece4037870 0x7fece40dcfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3301/0 says I am v2:172.21.15.1:44556/0 (socket says 172.21.15.1:44556) 2024-05-09T08:02:54.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.449+0000 7fecf1a00640 1 -- 172.21.15.1:0/2676723000 learned_addr learned my addr 172.21.15.1:0/2676723000 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:02:54.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.450+0000 7f0c42c00640 1 -- 172.21.15.1:0/2265804887 shutdown_connections 2024-05-09T08:02:54.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.450+0000 7f0c42c00640 1 -- 172.21.15.1:0/2265804887 wait complete. 2024-05-09T08:02:54.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.449+0000 7f5737400640 1 --2- 172.21.15.1:0/2180769876 >> v2:172.21.15.7:3300/0 conn(0x7f5738154a60 0x7f573806d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:54.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.450+0000 7f58de000640 1 -- 172.21.15.1:0/1189377898 <== mon.1 v2:172.21.15.7:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f58d40ab970 con 0x7f58e415ac80 2024-05-09T08:02:54.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.450+0000 7f58e9e00640 1 --2- 172.21.15.1:0/1189377898 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f58c806bf00 0x7f58c806e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:54.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.450+0000 7f5950600640 1 Processor -- start 2024-05-09T08:02:54.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.450+0000 7fecf2e00640 1 --2- 172.21.15.1:0/2676723000 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fece40deda0 0x7fece416ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:54.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.450+0000 7f5737e00640 1 --2- 172.21.15.1:0/2180769876 >> v2:172.21.15.1:3300/0 conn(0x7f57380b4210 0x7f57380b4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:54.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.450+0000 7fecf2e00640 1 -- 172.21.15.1:0/2676723000 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fece40d9470 con 0x7fece40d9090 2024-05-09T08:02:54.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.450+0000 7fecf2e00640 1 -- 172.21.15.1:0/2676723000 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fece4037d20 con 0x7fece40deda0 2024-05-09T08:02:54.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.450+0000 7fecf2e00640 1 -- 172.21.15.1:0/2676723000 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7fece4050890 con 0x7fece4037870 2024-05-09T08:02:54.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.450+0000 7fecf2400640 1 --2- 172.21.15.1:0/2676723000 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fece40deda0 0x7fece416ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:54.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.451+0000 7f02c3200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02bc0e2510 0x7f02bc1823b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:54.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.450+0000 7f5950600640 1 -- start start 2024-05-09T08:02:54.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.451+0000 7f02c3200640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f02bc0e28f0 0x7f02bc185780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:54.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.450+0000 7fecf1000640 1 --2- 172.21.15.1:0/2676723000 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fece40d9090 0x7fece4053050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:54.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.450+0000 7fecf2400640 1 -- 172.21.15.1:0/2676723000 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fece4037870 msgr2=0x7fece40dcfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:54.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.450+0000 7fecf2400640 1 --2- 172.21.15.1:0/2676723000 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fece4037870 0x7fece40dcfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:54.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.450+0000 7fecf2400640 1 -- 172.21.15.1:0/2676723000 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fece40d9090 msgr2=0x7fece4053050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:54.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.450+0000 7fecf2400640 1 --2- 172.21.15.1:0/2676723000 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fece40d9090 0x7fece4053050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:54.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.450+0000 7fecf2400640 1 -- 172.21.15.1:0/2676723000 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fece40dbb20 con 0x7fece40deda0 2024-05-09T08:02:54.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.451+0000 7fecf1a00640 1 --2- 172.21.15.1:0/2676723000 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fece4037870 0x7fece40dcfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:02:54.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.451+0000 7fecf1000640 1 --2- 172.21.15.1:0/2676723000 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fece40d9090 0x7fece4053050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T08:02:54.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.451+0000 7fecf2400640 1 --2- 172.21.15.1:0/2676723000 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fece40deda0 0x7fece416ea80 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fecec06a620 tx=0x7fecec069970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:54.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.451+0000 7f02c3200640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f02bc0e2e10 0x7f02bc031370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:54.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.451+0000 7f02c3200640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f02bc04aee0 con 0x7f02bc0e28f0 2024-05-09T08:02:54.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.451+0000 7f02c3200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f02bc046350 con 0x7f02bc0e2510 2024-05-09T08:02:54.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.451+0000 7f02c3200640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f02bc04b060 con 0x7f02bc0e2e10 2024-05-09T08:02:54.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.451+0000 7f5737400640 1 -- 172.21.15.1:0/2180769876 >> v2:172.21.15.1:3300/0 conn(0x7f57380b4210 msgr2=0x7f57380b4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:02:54.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.451+0000 7f5737400640 1 --2- 172.21.15.1:0/2180769876 >> v2:172.21.15.1:3300/0 conn(0x7f57380b4210 0x7f57380b4620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:54.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.451+0000 7f5737400640 1 -- 172.21.15.1:0/2180769876 >> v1:172.21.15.7:6789/0 conn(0x7f57380cc4c0 legacy=0x7f573806c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:54.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.451+0000 7f5737400640 1 -- 172.21.15.1:0/2180769876 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f573815a830 con 0x7f5738154a60 2024-05-09T08:02:54.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.452+0000 7f58e9e00640 1 --2- 172.21.15.1:0/1189377898 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f58c806bf00 0x7f58c806e3c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f58d800aad0 tx=0x7f58d8008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:54.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.452+0000 7f02c1400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f02bc0e28f0 0x7f02bc185780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:54.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.452+0000 7f02c1400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f02bc0e28f0 0x7f02bc185780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38126/0 (socket says 172.21.15.1:38126) 2024-05-09T08:02:54.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.452+0000 7f02c1400640 1 -- 172.21.15.1:0/411966865 learned_addr learned my addr 172.21.15.1:0/411966865 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:02:54.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.452+0000 7f02c1e00640 1 --2- 172.21.15.1:0/411966865 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02bc0e2510 0x7f02bc1823b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:54.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.452+0000 7f02c2800640 1 --2- 172.21.15.1:0/411966865 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f02bc0e2e10 0x7f02bc031370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:54.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.452+0000 7f02c1400640 1 -- 172.21.15.1:0/411966865 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f02bc0e2e10 msgr2=0x7f02bc031370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:54.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.453+0000 7f02c1400640 1 --2- 172.21.15.1:0/411966865 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f02bc0e2e10 0x7f02bc031370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:54.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.453+0000 7f02c1400640 1 -- 172.21.15.1:0/411966865 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02bc0e2510 msgr2=0x7f02bc1823b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:54.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.453+0000 7f02c1400640 1 --2- 172.21.15.1:0/411966865 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02bc0e2510 0x7f02bc1823b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:54.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.453+0000 7f24edc00640 1 Processor -- start 2024-05-09T08:02:54.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.453+0000 7f02c1400640 1 -- 172.21.15.1:0/411966865 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02bc0dfe60 con 0x7f02bc0e28f0 2024-05-09T08:02:54.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.453+0000 7f0c42c00640 1 Processor -- start 2024-05-09T08:02:54.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.453+0000 7f02c1e00640 1 --2- 172.21.15.1:0/411966865 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02bc0e2510 0x7f02bc1823b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T08:02:54.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.453+0000 7f24edc00640 1 -- start start 2024-05-09T08:02:54.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.453+0000 7f02c2800640 1 --2- 172.21.15.1:0/411966865 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f02bc0e2e10 0x7f02bc031370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T08:02:54.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.453+0000 7f0c42c00640 1 -- start start 2024-05-09T08:02:54.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.453+0000 7f5950600640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f594815e9b0 0x7f594806f720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:54.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.453+0000 7f5950600640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f59480c87f0 0x7f594806fc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:54.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.453+0000 7f02c1400640 1 --2- 172.21.15.1:0/411966865 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f02bc0e28f0 0x7f02bc185780 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f02b000e730 tx=0x7f02b000ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:54.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.453+0000 7f5950600640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f594815ee40 con 0x7f5948158b90 2024-05-09T08:02:54.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.453+0000 7f5950600640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f5948162150 con 0x7f594815e9b0 2024-05-09T08:02:54.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.453+0000 7f5950600640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f594815f120 con 0x7f59480c87f0 2024-05-09T08:02:54.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.453+0000 7f5737400640 1 --2- 172.21.15.1:0/2180769876 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5738154a60 0x7f573806d7e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f573808d5e0 tx=0x7f572802a110 comp rx=0 tx=0).ready entity=mon.1 client_cookie=961c83447b6c64d2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:54.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.453+0000 7f594dc00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f5948158b90 0x7f59481625e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.1:42060/0 (socket says 172.21.15.1:42060) 2024-05-09T08:02:54.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.453+0000 7f594dc00640 1 -- 172.21.15.1:0/1270860378 learned_addr learned my addr 172.21.15.1:0/1270860378 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T08:02:54.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.454+0000 7f02ba000640 1 -- 172.21.15.1:0/411966865 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02b000f040 con 0x7f02bc0e28f0 2024-05-09T08:02:54.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.454+0000 7f594e600640 1 --2- 172.21.15.1:0/1270860378 >> v2:172.21.15.7:3300/0 conn(0x7f59480c87f0 0x7f594806fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:54.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.454+0000 7f02c3200640 1 -- 172.21.15.1:0/411966865 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02bc0e0140 con 0x7f02bc0e28f0 2024-05-09T08:02:54.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.454+0000 7f5736a00640 1 -- 172.21.15.1:0/2180769876 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5728037070 con 0x7f5738154a60 2024-05-09T08:02:54.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.454+0000 7f5736a00640 1 -- 172.21.15.1:0/2180769876 <== mon.1 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5728032440 con 0x7f5738154a60 2024-05-09T08:02:54.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.454+0000 7f5736a00640 1 -- 172.21.15.1:0/2180769876 <== mon.1 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f572802ae60 con 0x7f5738154a60 2024-05-09T08:02:54.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.454+0000 7fece9600640 1 -- 172.21.15.1:0/2676723000 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecec0bb070 con 0x7fece40deda0 2024-05-09T08:02:54.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.454+0000 7fece9600640 1 -- 172.21.15.1:0/2676723000 <== mon.1 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fecec0ae640 con 0x7fece40deda0 2024-05-09T08:02:54.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.454+0000 7f02c3200640 1 -- 172.21.15.1:0/411966865 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02bc184f80 con 0x7f02bc0e28f0 2024-05-09T08:02:54.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.454+0000 7f02ba000640 1 -- 172.21.15.1:0/411966865 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02b0004230 con 0x7f02bc0e28f0 2024-05-09T08:02:54.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.454+0000 7f58de000640 1 -- 172.21.15.1:0/1189377898 <== mon.1 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f58d4079970 con 0x7f58e415ac80 2024-05-09T08:02:54.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.454+0000 7f02ba000640 1 -- 172.21.15.1:0/411966865 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02b000ee40 con 0x7f02bc0e28f0 2024-05-09T08:02:54.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.455+0000 7fecf2e00640 1 -- 172.21.15.1:0/2676723000 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fece40dbe00 con 0x7fece40deda0 2024-05-09T08:02:54.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.455+0000 7fecf2e00640 1 -- 172.21.15.1:0/2676723000 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fece4052570 con 0x7fece40deda0 2024-05-09T08:02:54.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.455+0000 7fece9600640 1 -- 172.21.15.1:0/2676723000 <== mon.1 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecec0bb070 con 0x7fece40deda0 2024-05-09T08:02:54.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.455+0000 7f594d200640 1 --2- 172.21.15.1:0/1270860378 >> v2:172.21.15.1:3300/0 conn(0x7f594815e9b0 0x7f594806f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:54.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.455+0000 7f594e600640 1 -- 172.21.15.1:0/1270860378 >> v2:172.21.15.1:3300/0 conn(0x7f594815e9b0 msgr2=0x7f594806f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:02:54.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.456+0000 7f0c42c00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f0c3c06c9e0 0x7f0c3c15b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:54.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.456+0000 7f0c42c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c3c06cdc0 0x7f0c3c1f40c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:54.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.456+0000 7f0c42c00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0c3c0cc260 0x7f0c3c0cf230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:54.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.456+0000 7f0c42c00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f0c3c0cab70 con 0x7f0c3c0cc260 2024-05-09T08:02:54.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.456+0000 7f0c42c00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0c3c06d980 con 0x7f0c3c06cdc0 2024-05-09T08:02:54.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.456+0000 7f0c42c00640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f0c3c154cc0 con 0x7f0c3c06c9e0 2024-05-09T08:02:54.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.455+0000 7f573e000640 1 -- 172.21.15.1:0/2180769876 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5738154a60 msgr2=0x7f573806d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:54.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.455+0000 7f573e000640 1 --2- 172.21.15.1:0/2180769876 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5738154a60 0x7f573806d7e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f573808d5e0 tx=0x7f572802a110 comp rx=0 tx=0).stop 2024-05-09T08:02:54.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.456+0000 7f0c3be00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f0c3c06c9e0 0x7f0c3c15b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:54.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.457+0000 7f573e000640 1 -- 172.21.15.1:0/2180769876 shutdown_connections 2024-05-09T08:02:54.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.457+0000 7f573e000640 1 --2- 172.21.15.1:0/2180769876 >> v2:172.21.15.1:3300/0 conn(0x7f57380b4210 0x7f57380b4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:54.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.457+0000 7f573e000640 1 --2- 172.21.15.1:0/2180769876 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5738154a60 0x7f573806d7e0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:54.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.457+0000 7f573e000640 1 -- 172.21.15.1:0/2180769876 >> 172.21.15.1:0/2180769876 conn(0x7f57380cffa0 msgr2=0x7f57380d03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:02:54.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.457+0000 7f02c3200640 1 -- 172.21.15.1:0/411966865 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f029c005290 con 0x7f02bc0e28f0 2024-05-09T08:02:54.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.458+0000 7fecf2e00640 1 -- 172.21.15.1:0/2676723000 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fece40052c0 con 0x7fece40deda0 2024-05-09T08:02:54.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.456+0000 7f0c3be00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f0c3c06c9e0 0x7f0c3c15b780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3301/0 says I am v2:172.21.15.1:44566/0 (socket says 172.21.15.1:44566) 2024-05-09T08:02:54.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.456+0000 7f0c3be00640 1 -- 172.21.15.1:0/202381751 learned_addr learned my addr 172.21.15.1:0/202381751 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:02:54.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.457+0000 7f0c3b400640 1 --2- 172.21.15.1:0/202381751 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c3c06cdc0 0x7f0c3c1f40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:54.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.457+0000 7f0c40c00640 1 --2- 172.21.15.1:0/202381751 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0c3c0cc260 0x7f0c3c0cf230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:54.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.455+0000 7f24edc00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f24e815e9b0 0x7f24e806f720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:54.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.455+0000 7f24edc00640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f24e815ee40 con 0x7f24e8158b90 2024-05-09T08:02:54.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.459+0000 7f0c3b400640 1 -- 172.21.15.1:0/202381751 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f0c3c06c9e0 msgr2=0x7f0c3c15b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:54.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.455+0000 7f24edc00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f24e815f120 con 0x7f24e80c87f0 2024-05-09T08:02:54.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.455+0000 7f594e600640 1 --2- 172.21.15.1:0/1270860378 >> v2:172.21.15.1:3300/0 conn(0x7f594815e9b0 0x7f594806f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:54.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.455+0000 7f594e600640 1 -- 172.21.15.1:0/1270860378 >> v1:172.21.15.7:6789/0 conn(0x7f5948158b90 legacy=0x7f59481625e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:54.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.455+0000 7f594e600640 1 -- 172.21.15.1:0/1270860378 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f594815f400 con 0x7f59480c87f0 2024-05-09T08:02:54.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.455+0000 7f24edc00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f24e8162150 con 0x7f24e815e9b0 2024-05-09T08:02:54.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.459+0000 7f594d200640 1 --2- 172.21.15.1:0/1270860378 >> v2:172.21.15.1:3300/0 conn(0x7f594815e9b0 0x7f594806f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T08:02:54.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.456+0000 7f24e6a00640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f24e8158b90 0x7f24e81625e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:56026/0 (socket says 172.21.15.1:56026) 2024-05-09T08:02:54.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.458+0000 7f573e000640 1 -- 172.21.15.1:0/2180769876 shutdown_connections 2024-05-09T08:02:54.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.458+0000 7f573e000640 1 -- 172.21.15.1:0/2180769876 wait complete. 2024-05-09T08:02:54.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.456+0000 7f24e6a00640 1 -- 172.21.15.1:0/523293361 learned_addr learned my addr 172.21.15.1:0/523293361 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T08:02:54.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.456+0000 7f24e6000640 1 --2- 172.21.15.1:0/523293361 >> v2:172.21.15.7:3300/0 conn(0x7f24e815e9b0 0x7f24e806f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:54.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.459+0000 7f0c3b400640 1 --2- 172.21.15.1:0/202381751 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f0c3c06c9e0 0x7f0c3c15b780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:54.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.459+0000 7f0c3b400640 1 -- 172.21.15.1:0/202381751 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0c3c0cc260 msgr2=0x7f0c3c0cf230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:54.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.459+0000 7f0c3b400640 1 --2- 172.21.15.1:0/202381751 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0c3c0cc260 0x7f0c3c0cf230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:54.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.459+0000 7f0c3b400640 1 -- 172.21.15.1:0/202381751 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c3c15c8e0 con 0x7f0c3c06cdc0 2024-05-09T08:02:54.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.459+0000 7f0c40c00640 1 --2- 172.21.15.1:0/202381751 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0c3c0cc260 0x7f0c3c0cf230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T08:02:54.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.459+0000 7f0c3be00640 1 --2- 172.21.15.1:0/202381751 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f0c3c06c9e0 0x7f0c3c15b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:02:54.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.459+0000 7f594e600640 1 --2- 172.21.15.1:0/1270860378 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f59480c87f0 0x7f594806fc60 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f59480724b0 tx=0x7f593c02a250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=63a4913683e28bb5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:54.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.460+0000 7fece9600640 1 -- 172.21.15.1:0/2676723000 <== mon.1 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fecec0be070 con 0x7fece40deda0 2024-05-09T08:02:54.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.457+0000 7f24e6000640 1 -- 172.21.15.1:0/523293361 >> v1:172.21.15.7:6789/0 conn(0x7f24e80c87f0 legacy=0x7f24e81692b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T08:02:54.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.460+0000 7f0c3b400640 1 --2- 172.21.15.1:0/202381751 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c3c06cdc0 0x7f0c3c1f40c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f0c28031000 tx=0x7f0c280046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:54.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.457+0000 7f24e6000640 1 -- 172.21.15.1:0/523293361 >> v1:172.21.15.1:6789/0 conn(0x7f24e8158b90 legacy=0x7f24e81625e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:54.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.457+0000 7f24e6000640 1 -- 172.21.15.1:0/523293361 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24e815f400 con 0x7f24e815e9b0 2024-05-09T08:02:54.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.459+0000 7f24e6000640 1 --2- 172.21.15.1:0/523293361 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24e815e9b0 0x7f24e806f720 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f24d80097a0 tx=0x7f24d802a290 comp rx=0 tx=0).ready entity=mon.1 client_cookie=947928aaead7a645 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:54.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.459+0000 7f24e5600640 1 -- 172.21.15.1:0/523293361 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24d8037070 con 0x7f24e815e9b0 2024-05-09T08:02:54.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.460+0000 7f24e5600640 1 -- 172.21.15.1:0/523293361 <== mon.1 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24d8032440 con 0x7f24e815e9b0 2024-05-09T08:02:54.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.460+0000 7f02ba000640 1 -- 172.21.15.1:0/411966865 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f02b000c6d0 con 0x7f02bc0e28f0 2024-05-09T08:02:54.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.460+0000 7f24e5600640 1 -- 172.21.15.1:0/523293361 <== mon.1 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24d8032850 con 0x7f24e815e9b0 2024-05-09T08:02:54.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.461+0000 7f5947e00640 1 -- 172.21.15.1:0/1270860378 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f593c037070 con 0x7f59480c87f0 2024-05-09T08:02:54.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.461+0000 7f5947e00640 1 -- 172.21.15.1:0/1270860378 <== mon.1 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f593c032440 con 0x7f59480c87f0 2024-05-09T08:02:54.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.461+0000 7f5947e00640 1 -- 172.21.15.1:0/1270860378 <== mon.1 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f593c032850 con 0x7f59480c87f0 2024-05-09T08:02:54.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.461+0000 7fece9600640 1 --2- 172.21.15.1:0/2676723000 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7feccc06c060 0x7feccc06e520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:54.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.461+0000 7f573e000640 1 Processor -- start 2024-05-09T08:02:54.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.462+0000 7fece9600640 1 -- 172.21.15.1:0/2676723000 <== mon.1 v2:172.21.15.7:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fecec1265b0 con 0x7fece40deda0 2024-05-09T08:02:54.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.461+0000 7f573e000640 1 -- start start 2024-05-09T08:02:54.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.462+0000 7fecf1a00640 1 --2- 172.21.15.1:0/2676723000 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7feccc06c060 0x7feccc06e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:54.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.462+0000 7f24edc00640 1 -- 172.21.15.1:0/523293361 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24e815e9b0 msgr2=0x7f24e806f720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:54.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.462+0000 7f24edc00640 1 --2- 172.21.15.1:0/523293361 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24e815e9b0 0x7f24e806f720 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f24d80097a0 tx=0x7f24d802a290 comp rx=0 tx=0).stop 2024-05-09T08:02:54.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.462+0000 7f24edc00640 1 -- 172.21.15.1:0/523293361 shutdown_connections 2024-05-09T08:02:54.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.462+0000 7f24edc00640 1 --2- 172.21.15.1:0/523293361 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24e815e9b0 0x7f24e806f720 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:54.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.462+0000 7f24edc00640 1 -- 172.21.15.1:0/523293361 >> 172.21.15.1:0/523293361 conn(0x7f24e806dad0 msgr2=0x7f24e806deb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:02:54.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.462+0000 7f24edc00640 1 -- 172.21.15.1:0/523293361 shutdown_connections 2024-05-09T08:02:54.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.462+0000 7f24edc00640 1 -- 172.21.15.1:0/523293361 wait complete. 2024-05-09T08:02:54.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.463+0000 7f5950600640 1 -- 172.21.15.1:0/1270860378 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f59480c87f0 msgr2=0x7f594806fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:54.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.463+0000 7f5950600640 1 --2- 172.21.15.1:0/1270860378 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f59480c87f0 0x7f594806fc60 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f59480724b0 tx=0x7f593c02a250 comp rx=0 tx=0).stop 2024-05-09T08:02:54.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.463+0000 7f02ba000640 1 --2- 172.21.15.1:0/411966865 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f02a406be30 0x7f02a406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:54.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.463+0000 7fecf1a00640 1 --2- 172.21.15.1:0/2676723000 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7feccc06c060 0x7feccc06e520 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fece40d62d0 tx=0x7fecd8009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:54.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.463+0000 7f5950600640 1 -- 172.21.15.1:0/1270860378 shutdown_connections 2024-05-09T08:02:54.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.463+0000 7f5950600640 1 --2- 172.21.15.1:0/1270860378 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f59480c87f0 0x7f594806fc60 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:54.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.463+0000 7f5950600640 1 --2- 172.21.15.1:0/1270860378 >> v2:172.21.15.1:3300/0 conn(0x7f594815e9b0 0x7f594806f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:54.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.463+0000 7f5950600640 1 -- 172.21.15.1:0/1270860378 >> 172.21.15.1:0/1270860378 conn(0x7f594806dad0 msgr2=0x7f594806deb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:02:54.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.463+0000 7f5950600640 1 -- 172.21.15.1:0/1270860378 shutdown_connections 2024-05-09T08:02:54.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.463+0000 7f02c1e00640 1 --2- 172.21.15.1:0/411966865 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f02a406be30 0x7f02a406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:54.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.464+0000 7f02ba000640 1 -- 172.21.15.1:0/411966865 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f02b008af00 con 0x7f02bc0e28f0 2024-05-09T08:02:54.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.464+0000 7f573e000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57380b4210 0x7f57380c7ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:54.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.464+0000 7f5950600640 1 -- 172.21.15.1:0/1270860378 wait complete. 2024-05-09T08:02:54.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.464+0000 7f573e000640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f57380cc4c0 0x7f57381f37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:54.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.464+0000 7f5737400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57380b4210 0x7f57380c7ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:54.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.464+0000 7f5737400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57380b4210 0x7f57380c7ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.1:58932/0 (socket says 172.21.15.1:58932) 2024-05-09T08:02:54.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.464+0000 7f5737400640 1 -- 172.21.15.1:0/450387781 learned_addr learned my addr 172.21.15.1:0/450387781 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:02:54.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.465+0000 7f02c1e00640 1 --2- 172.21.15.1:0/411966865 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f02a406be30 0x7f02a406e2f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f02b4000f60 tx=0x7f02b4007ce0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:54.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.465+0000 7f573e000640 1 --2- 172.21.15.1:0/450387781 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f5738154a60 0x7f573814dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:54.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.465+0000 7f573e000640 1 -- 172.21.15.1:0/450387781 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f573815b090 con 0x7f57380cc4c0 2024-05-09T08:02:54.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.465+0000 7f573e000640 1 -- 172.21.15.1:0/450387781 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f573815adb0 con 0x7f57380b4210 2024-05-09T08:02:54.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.465+0000 7f573e000640 1 -- 172.21.15.1:0/450387781 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f573815b370 con 0x7f5738154a60 2024-05-09T08:02:54.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.465+0000 7f5737e00640 1 --2- 172.21.15.1:0/450387781 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f5738154a60 0x7f573814dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:54.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.464+0000 7f24edc00640 1 Processor -- start 2024-05-09T08:02:54.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.464+0000 7f24edc00640 1 -- start start 2024-05-09T08:02:54.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.465+0000 7f24edc00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f24e80c87f0 0x7f24e81f8950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:54.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.465+0000 7f24edc00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24e8158b90 0x7f24e80cb7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:54.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.465+0000 7f24edc00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f24e80c9e00 0x7f24e81f5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:54.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.465+0000 7f24edc00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f24e8162e70 con 0x7f24e80c87f0 2024-05-09T08:02:54.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.465+0000 7f24edc00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f24e8162cf0 con 0x7f24e8158b90 2024-05-09T08:02:54.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.465+0000 7f24edc00640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f24e80c7e70 con 0x7f24e80c9e00 2024-05-09T08:02:54.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.466+0000 7f24e6000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24e8158b90 0x7f24e80cb7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:54.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.466+0000 7fece9600640 1 -- 172.21.15.1:0/2676723000 <== mon.1 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fecec0b4040 con 0x7fece40deda0 2024-05-09T08:02:54.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.466+0000 7f24e6000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24e8158b90 0x7f24e80cb7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.1:58942/0 (socket says 172.21.15.1:58942) 2024-05-09T08:02:54.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.466+0000 7f24e6000640 1 -- 172.21.15.1:0/4227163944 learned_addr learned my addr 172.21.15.1:0/4227163944 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:02:54.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.466+0000 7f5737400640 1 -- 172.21.15.1:0/450387781 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f5738154a60 msgr2=0x7f573814dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:54.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.466+0000 7f5737400640 1 --2- 172.21.15.1:0/450387781 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f5738154a60 0x7f573814dc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:54.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.466+0000 7f5737400640 1 -- 172.21.15.1:0/450387781 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f57380cc4c0 msgr2=0x7f57381f37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:54.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.466+0000 7f5737400640 1 --2- 172.21.15.1:0/450387781 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f57380cc4c0 0x7f57381f37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:54.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.466+0000 7f24e6a00640 1 --2- 172.21.15.1:0/4227163944 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f24e80c87f0 0x7f24e81f8950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:54.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.466+0000 7f24e6000640 1 -- 172.21.15.1:0/4227163944 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f24e80c9e00 msgr2=0x7f24e81f5250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:54.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.466+0000 7f0c33e00640 1 -- 172.21.15.1:0/202381751 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c28041070 con 0x7f0c3c06cdc0 2024-05-09T08:02:54.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.466+0000 7f0c33e00640 1 -- 172.21.15.1:0/202381751 <== mon.1 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c2803c870 con 0x7f0c3c06cdc0 2024-05-09T08:02:54.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.466+0000 7f0c33e00640 1 -- 172.21.15.1:0/202381751 <== mon.1 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c28037040 con 0x7f0c3c06cdc0 2024-05-09T08:02:54.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.466+0000 7f5737400640 1 -- 172.21.15.1:0/450387781 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57381563d0 con 0x7f57380b4210 2024-05-09T08:02:54.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.466+0000 7f24e7400640 1 --2- 172.21.15.1:0/4227163944 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f24e80c9e00 0x7f24e81f5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:54.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.466+0000 7f24e6000640 1 --2- 172.21.15.1:0/4227163944 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f24e80c9e00 0x7f24e81f5250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:54.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.466+0000 7f24e6000640 1 -- 172.21.15.1:0/4227163944 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f24e80c87f0 msgr2=0x7f24e81f8950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:54.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.466+0000 7f24e6000640 1 --2- 172.21.15.1:0/4227163944 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f24e80c87f0 0x7f24e81f8950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:54.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.466+0000 7f24e6000640 1 -- 172.21.15.1:0/4227163944 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24e81594a0 con 0x7f24e8158b90 2024-05-09T08:02:54.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.467+0000 7f24e6a00640 1 --2- 172.21.15.1:0/4227163944 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f24e80c87f0 0x7f24e81f8950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T08:02:54.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.466+0000 7f0c42c00640 1 -- 172.21.15.1:0/202381751 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c3c15cbc0 con 0x7f0c3c06cdc0 2024-05-09T08:02:54.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.467+0000 7f0c42c00640 1 -- 172.21.15.1:0/202381751 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c3c15d180 con 0x7f0c3c06cdc0 2024-05-09T08:02:54.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.467+0000 7f5950600640 1 Processor -- start 2024-05-09T08:02:54.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.467+0000 7f5737400640 1 --2- 172.21.15.1:0/450387781 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57380b4210 0x7f57380c7ee0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f5728009890 tx=0x7f57280046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:54.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.467+0000 7f24e6000640 1 --2- 172.21.15.1:0/4227163944 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24e8158b90 0x7f24e80cb7d0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f24d80098c0 tx=0x7f24d80046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:54.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.467+0000 7f5950600640 1 -- start start 2024-05-09T08:02:54.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.468+0000 7f24dea00640 1 -- 172.21.15.1:0/4227163944 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24d8037050 con 0x7f24e8158b90 2024-05-09T08:02:54.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.468+0000 7f24dea00640 1 -- 172.21.15.1:0/4227163944 <== mon.1 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24d8043070 con 0x7f24e8158b90 2024-05-09T08:02:54.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.468+0000 7f24dea00640 1 -- 172.21.15.1:0/4227163944 <== mon.1 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24d8030e20 con 0x7f24e8158b90 2024-05-09T08:02:54.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.468+0000 7f24edc00640 1 -- 172.21.15.1:0/4227163944 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24e8159780 con 0x7f24e8158b90 2024-05-09T08:02:54.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.468+0000 7f572f400640 1 -- 172.21.15.1:0/450387781 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5728037050 con 0x7f57380b4210 2024-05-09T08:02:54.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.468+0000 7f573e000640 1 -- 172.21.15.1:0/450387781 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57380b6190 con 0x7f57380b4210 2024-05-09T08:02:54.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.468+0000 7f573e000640 1 -- 172.21.15.1:0/450387781 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5738155e50 con 0x7f57380b4210 2024-05-09T08:02:54.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.468+0000 7f572f400640 1 -- 172.21.15.1:0/450387781 <== mon.1 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5728038040 con 0x7f57380b4210 2024-05-09T08:02:54.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.468+0000 7f572f400640 1 -- 172.21.15.1:0/450387781 <== mon.1 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5728037050 con 0x7f57380b4210 2024-05-09T08:02:54.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.468+0000 7f24edc00640 1 -- 172.21.15.1:0/4227163944 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24e81f7e70 con 0x7f24e8158b90 2024-05-09T08:02:54.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.471+0000 7f02ba000640 1 -- 172.21.15.1:0/411966865 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f02b0017020 con 0x7f02bc0e28f0 2024-05-09T08:02:54.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.471+0000 7f0c42c00640 1 -- 172.21.15.1:0/202381751 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c14005290 con 0x7f0c3c06cdc0 2024-05-09T08:02:54.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.471+0000 7f573e000640 1 -- 172.21.15.1:0/450387781 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5738074d70 con 0x7f57380b4210 2024-05-09T08:02:54.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.471+0000 7f5950600640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f59480c87f0 0x7f59481f4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:54.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.471+0000 7f5950600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5948158b90 0x7f594806b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:54.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.471+0000 7f5950600640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f59480cb420 0x7f59480ca760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:54.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.472+0000 7f594d200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5948158b90 0x7f594806b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:54.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.472+0000 7f24edc00640 1 -- 172.21.15.1:0/4227163944 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24e8074d70 con 0x7f24e8158b90 2024-05-09T08:02:54.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.472+0000 7f594d200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5948158b90 0x7f594806b4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.1:58954/0 (socket says 172.21.15.1:58954) 2024-05-09T08:02:54.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.472+0000 7f594e600640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f59480cb420 0x7f59480ca760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:54.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.472+0000 7f594e600640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f59480cb420 0x7f59480ca760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3301/0 says I am v2:172.21.15.1:44588/0 (socket says 172.21.15.1:44588) 2024-05-09T08:02:54.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.472+0000 7f24dea00640 1 -- 172.21.15.1:0/4227163944 <== mon.1 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f24d8048460 con 0x7f24e8158b90 2024-05-09T08:02:54.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.472+0000 7f5950600640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f5948162e70 con 0x7f59480c87f0 2024-05-09T08:02:54.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.472+0000 7f5950600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5948162cf0 con 0x7f5948158b90 2024-05-09T08:02:54.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.472+0000 7f5950600640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f59480c7e70 con 0x7f59480cb420 2024-05-09T08:02:54.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.473+0000 7f594d200640 1 -- 172.21.15.1:0/3496074172 learned_addr learned my addr 172.21.15.1:0/3496074172 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:02:54.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.473+0000 7f572f400640 1 -- 172.21.15.1:0/450387781 <== mon.1 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f572803c910 con 0x7f57380b4210 2024-05-09T08:02:54.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.473+0000 7f0c33e00640 1 -- 172.21.15.1:0/202381751 <== mon.1 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0c28044070 con 0x7f0c3c06cdc0 2024-05-09T08:02:54.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.473+0000 7f594dc00640 1 --2- 172.21.15.1:0/3496074172 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f59480c87f0 0x7f59481f4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:54.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.474+0000 7f24dea00640 1 --2- 172.21.15.1:0/4227163944 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f24c806bf30 0x7f24c806e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:54.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.474+0000 7f24dea00640 1 -- 172.21.15.1:0/4227163944 <== mon.1 v2:172.21.15.7:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f24d80abf90 con 0x7f24e8158b90 2024-05-09T08:02:54.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.474+0000 7f594e600640 1 -- 172.21.15.1:0/3496074172 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5948158b90 msgr2=0x7f594806b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:54.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.474+0000 7f594e600640 1 --2- 172.21.15.1:0/3496074172 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5948158b90 0x7f594806b4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:54.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.474+0000 7f594e600640 1 -- 172.21.15.1:0/3496074172 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f59480c87f0 msgr2=0x7f59481f4db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:54.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.474+0000 7f594e600640 1 --2- 172.21.15.1:0/3496074172 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f59480c87f0 0x7f59481f4db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:54.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.474+0000 7f594e600640 1 -- 172.21.15.1:0/3496074172 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59481f52f0 con 0x7f59480cb420 2024-05-09T08:02:54.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.474+0000 7f6492a00640 1 Processor -- start 2024-05-09T08:02:54.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.475+0000 7f0c33e00640 1 --2- 172.21.15.1:0/202381751 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f0c1c06be60 0x7f0c1c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:54.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.475+0000 7f594e600640 1 --2- 172.21.15.1:0/3496074172 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f59480cb420 0x7f59480ca760 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f593c031000 tx=0x7f593c0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:54.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.475+0000 7f6492a00640 1 -- start start 2024-05-09T08:02:54.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.475+0000 7f0c33e00640 1 -- 172.21.15.1:0/202381751 <== mon.1 v2:172.21.15.7:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f0c280abd60 con 0x7f0c3c06cdc0 2024-05-09T08:02:54.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.475+0000 7f5945600640 1 -- 172.21.15.1:0/3496074172 <== mon.2 v2:172.21.15.7:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f593c040020 con 0x7f59480cb420 2024-05-09T08:02:54.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.475+0000 7f0c3be00640 1 --2- 172.21.15.1:0/202381751 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f0c1c06be60 0x7f0c1c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:54.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.475+0000 7f24e6a00640 1 --2- 172.21.15.1:0/4227163944 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f24c806bf30 0x7f24c806e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:54.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.475+0000 7f24e6a00640 1 --2- 172.21.15.1:0/4227163944 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f24c806bf30 0x7f24c806e3f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f24e0005c00 tx=0x7f24e0009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:54.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.476+0000 7f0c3be00640 1 --2- 172.21.15.1:0/202381751 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f0c1c06be60 0x7f0c1c06e320 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f0c2c009e00 tx=0x7f0c2c009190 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:54.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.476+0000 7f5950600640 1 -- 172.21.15.1:0/3496074172 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59481f8710 con 0x7f59480cb420 2024-05-09T08:02:54.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.476+0000 7f5950600640 1 -- 172.21.15.1:0/3496074172 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f59481595e0 con 0x7f59480cb420 2024-05-09T08:02:54.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.476+0000 7f5945600640 1 -- 172.21.15.1:0/3496074172 <== mon.2 v2:172.21.15.7:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f593c038040 con 0x7f59480cb420 2024-05-09T08:02:54.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.476+0000 7f5945600640 1 -- 172.21.15.1:0/3496074172 <== mon.2 v2:172.21.15.7:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f593c037070 con 0x7f59480cb420 2024-05-09T08:02:54.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.476+0000 7f572f400640 1 --2- 172.21.15.1:0/450387781 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f571406c060 0x7f571406e520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:54.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.476+0000 7f572f400640 1 -- 172.21.15.1:0/450387781 <== mon.1 v2:172.21.15.7:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f57280ab6b0 con 0x7f57380b4210 2024-05-09T08:02:54.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.478+0000 7f572fe00640 1 --2- 172.21.15.1:0/450387781 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f571406c060 0x7f571406e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:54.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.477+0000 7f6492a00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f648c157040 0x7f648c0c3830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:54.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.478+0000 7f6492a00640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f648c0c67f0 con 0x7f648c0c59e0 2024-05-09T08:02:54.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.478+0000 7f6492a00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f648c070640 con 0x7f648c0c5600 2024-05-09T08:02:54.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.478+0000 7f6492a00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f648c0c3d70 con 0x7f648c157040 2024-05-09T08:02:54.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.478+0000 7f24dea00640 1 -- 172.21.15.1:0/4227163944 <== mon.1 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f24d807a670 con 0x7f24e8158b90 2024-05-09T08:02:54.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.478+0000 7f5950600640 1 -- 172.21.15.1:0/3496074172 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5924005290 con 0x7f59480cb420 2024-05-09T08:02:54.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.478+0000 7f6491600640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f648c0c5600 0x7f648c1507e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.1:42078/0 (socket says 172.21.15.1:42078) 2024-05-09T08:02:54.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.478+0000 7f6492000640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f648c157040 0x7f648c0c3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:54.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.478+0000 7f6491600640 1 -- 172.21.15.1:0/3925848896 learned_addr learned my addr 172.21.15.1:0/3925848896 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T08:02:54.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.479+0000 7f572fe00640 1 --2- 172.21.15.1:0/450387781 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f571406c060 0x7f571406e520 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f57380c6520 tx=0x7f5720007a40 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:54.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.479+0000 7f6492000640 1 -- 172.21.15.1:0/3925848896 >> v1:172.21.15.7:6789/0 conn(0x7f648c0c5600 legacy=0x7f648c1507e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:02:54.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.479+0000 7f6492000640 1 -- 172.21.15.1:0/3925848896 >> v1:172.21.15.1:6789/0 conn(0x7f648c0c59e0 legacy=0x7f648c156930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:02:54.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.479+0000 7f6492000640 1 -- 172.21.15.1:0/3925848896 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f648c070900 con 0x7f648c157040 2024-05-09T08:02:54.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.480+0000 7f5945600640 1 -- 172.21.15.1:0/3496074172 <== mon.2 v2:172.21.15.7:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f593c043020 con 0x7f59480cb420 2024-05-09T08:02:54.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.481+0000 7f6492000640 1 --2- 172.21.15.1:0/3925848896 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f648c157040 0x7f648c0c3830 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f648000ba40 tx=0x7f648002c6b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f86b633442d25ba1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:54.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.481+0000 7f5945600640 1 --2- 172.21.15.1:0/3496074172 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f592806bf30 0x7f592806e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:54.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.481+0000 7f5945600640 1 -- 172.21.15.1:0/3496074172 <== mon.2 v2:172.21.15.7:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f593c0ab100 con 0x7f59480cb420 2024-05-09T08:02:54.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.481+0000 7f594dc00640 1 --2- 172.21.15.1:0/3496074172 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f592806bf30 0x7f592806e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:54.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.482+0000 7f594dc00640 1 --2- 172.21.15.1:0/3496074172 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f592806bf30 0x7f592806e3f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f59400012e0 tx=0x7f5940009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:54.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.483+0000 7f0c33e00640 1 -- 172.21.15.1:0/202381751 <== mon.1 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f0c28079e00 con 0x7f0c3c06cdc0 2024-05-09T08:02:54.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.486+0000 7f5945600640 1 -- 172.21.15.1:0/3496074172 <== mon.2 v2:172.21.15.7:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f593c04a070 con 0x7f59480cb420 2024-05-09T08:02:54.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.487+0000 7f572f400640 1 -- 172.21.15.1:0/450387781 <== mon.1 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f5728079d90 con 0x7f57380b4210 2024-05-09T08:02:54.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.487+0000 7f648be00640 1 -- 172.21.15.1:0/3925848896 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f648003a070 con 0x7f648c157040 2024-05-09T08:02:54.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.487+0000 7f648be00640 1 -- 172.21.15.1:0/3925848896 <== mon.1 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6480032480 con 0x7f648c157040 2024-05-09T08:02:54.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.487+0000 7f648be00640 1 -- 172.21.15.1:0/3925848896 <== mon.1 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6480032850 con 0x7f648c157040 2024-05-09T08:02:54.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.488+0000 7f6492a00640 1 -- 172.21.15.1:0/3925848896 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f648c157040 msgr2=0x7f648c0c3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:54.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.488+0000 7f6492a00640 1 --2- 172.21.15.1:0/3925848896 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f648c157040 0x7f648c0c3830 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f648000ba40 tx=0x7f648002c6b0 comp rx=0 tx=0).stop 2024-05-09T08:02:54.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.489+0000 7f6492a00640 1 -- 172.21.15.1:0/3925848896 shutdown_connections 2024-05-09T08:02:54.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.489+0000 7f6492a00640 1 --2- 172.21.15.1:0/3925848896 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f648c157040 0x7f648c0c3830 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:54.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.489+0000 7f6492a00640 1 -- 172.21.15.1:0/3925848896 >> 172.21.15.1:0/3925848896 conn(0x7f648c0c8e30 msgr2=0x7f648c0c9210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:02:54.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.489+0000 7f6492a00640 1 -- 172.21.15.1:0/3925848896 shutdown_connections 2024-05-09T08:02:54.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.489+0000 7f6492a00640 1 -- 172.21.15.1:0/3925848896 wait complete. 2024-05-09T08:02:54.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.491+0000 7f6492a00640 1 Processor -- start 2024-05-09T08:02:54.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.492+0000 7f6492a00640 1 -- start start 2024-05-09T08:02:54.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.494+0000 7f6492a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f648c0c5600 0x7f648c1e15a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:54.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.494+0000 7f6492a00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f648c0c59e0 0x7f648c1ec0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:54.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.494+0000 7f6492a00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f648c157040 0x7f648c146a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:54.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.495+0000 7f6490c00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f648c0c59e0 0x7f648c1ec0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:54.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.495+0000 7f6490c00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f648c0c59e0 0x7f648c1ec0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3301/0 says I am v2:172.21.15.1:44592/0 (socket says 172.21.15.1:44592) 2024-05-09T08:02:54.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.495+0000 7f6492000640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f648c157040 0x7f648c146a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:54.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.495+0000 7f6492000640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f648c157040 0x7f648c146a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38166/0 (socket says 172.21.15.1:38166) 2024-05-09T08:02:54.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.495+0000 7f6490c00640 1 -- 172.21.15.1:0/3568813400 learned_addr learned my addr 172.21.15.1:0/3568813400 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:02:54.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.495+0000 7f6492a00640 1 -- 172.21.15.1:0/3568813400 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f648c06fd60 con 0x7f648c157040 2024-05-09T08:02:54.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.496+0000 7f6492a00640 1 -- 172.21.15.1:0/3568813400 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f648c06f4c0 con 0x7f648c0c5600 2024-05-09T08:02:54.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.496+0000 7f6492a00640 1 -- 172.21.15.1:0/3568813400 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f648c06f9f0 con 0x7f648c0c59e0 2024-05-09T08:02:54.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.497+0000 7f6490c00640 1 -- 172.21.15.1:0/3568813400 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f648c0c5600 msgr2=0x7f648c1e15a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T08:02:54.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.497+0000 7f6490c00640 1 --2- 172.21.15.1:0/3568813400 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f648c0c5600 0x7f648c1e15a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:54.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.497+0000 7f6490c00640 1 -- 172.21.15.1:0/3568813400 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f648c157040 msgr2=0x7f648c146a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:54.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.497+0000 7f6490c00640 1 --2- 172.21.15.1:0/3568813400 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f648c157040 0x7f648c146a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:54.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.497+0000 7f6490c00640 1 -- 172.21.15.1:0/3568813400 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f648c1ef330 con 0x7f648c0c59e0 2024-05-09T08:02:54.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.497+0000 7f6492000640 1 --2- 172.21.15.1:0/3568813400 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f648c157040 0x7f648c146a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:02:54.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.498+0000 7f6490c00640 1 --2- 172.21.15.1:0/3568813400 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f648c0c59e0 0x7f648c1ec0d0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f647800c440 tx=0x7f647800c900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:54.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.498+0000 7f6489600640 1 -- 172.21.15.1:0/3568813400 <== mon.2 v2:172.21.15.7:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6478017070 con 0x7f648c0c59e0 2024-05-09T08:02:54.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.498+0000 7f6492a00640 1 -- 172.21.15.1:0/3568813400 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f648c1ef610 con 0x7f648c0c59e0 2024-05-09T08:02:54.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.499+0000 7f6492a00640 1 -- 172.21.15.1:0/3568813400 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f648c1efbd0 con 0x7f648c0c59e0 2024-05-09T08:02:54.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.499+0000 7f6489600640 1 -- 172.21.15.1:0/3568813400 <== mon.2 v2:172.21.15.7:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64780035f0 con 0x7f648c0c59e0 2024-05-09T08:02:54.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.499+0000 7f6489600640 1 -- 172.21.15.1:0/3568813400 <== mon.2 v2:172.21.15.7:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64780039e0 con 0x7f648c0c59e0 2024-05-09T08:02:54.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.501+0000 7f6492a00640 1 -- 172.21.15.1:0/3568813400 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6468005290 con 0x7f648c0c59e0 2024-05-09T08:02:54.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.502+0000 7f6489600640 1 -- 172.21.15.1:0/3568813400 <== mon.2 v2:172.21.15.7:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f647801a070 con 0x7f648c0c59e0 2024-05-09T08:02:54.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.503+0000 7f6489600640 1 --2- 172.21.15.1:0/3568813400 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f646c06be30 0x7f646c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:54.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.504+0000 7f6489600640 1 -- 172.21.15.1:0/3568813400 <== mon.2 v2:172.21.15.7:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f647808a470 con 0x7f648c0c59e0 2024-05-09T08:02:54.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.504+0000 7f6491600640 1 --2- 172.21.15.1:0/3568813400 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f646c06be30 0x7f646c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:54.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.505+0000 7f6491600640 1 --2- 172.21.15.1:0/3568813400 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f646c06be30 0x7f646c06e2f0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f64840059a0 tx=0x7f6484004f40 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:54.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.507+0000 7f6489600640 1 -- 172.21.15.1:0/3568813400 <== mon.2 v2:172.21.15.7:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f64780010c0 con 0x7f648c0c59e0 2024-05-09T08:02:54.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.902+0000 7f58eb200640 1 -- 172.21.15.1:0/1189377898 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f58c0005580 con 0x7f58e415ac80 2024-05-09T08:02:54.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.903+0000 7f58de000640 1 -- 172.21.15.1:0/1189377898 <== mon.1 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f58d407dc70 con 0x7f58e415ac80 2024-05-09T08:02:54.920 INFO:teuthology.orchestra.run.smithi001.stdout:55834574859 2024-05-09T08:02:54.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.906+0000 7f58eb200640 1 -- 172.21.15.1:0/1189377898 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f58c806bf00 msgr2=0x7f58c806e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:54.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.906+0000 7f58eb200640 1 --2- 172.21.15.1:0/1189377898 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f58c806bf00 0x7f58c806e3c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f58d800aad0 tx=0x7f58d8008040 comp rx=0 tx=0).stop 2024-05-09T08:02:54.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.906+0000 7f58eb200640 1 -- 172.21.15.1:0/1189377898 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58e415ac80 msgr2=0x7f58e4150a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:54.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.906+0000 7f58eb200640 1 --2- 172.21.15.1:0/1189377898 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58e415ac80 0x7f58e4150a90 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f58e4068810 tx=0x7f58d40046c0 comp rx=0 tx=0).stop 2024-05-09T08:02:54.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.907+0000 7f58eb200640 1 -- 172.21.15.1:0/1189377898 shutdown_connections 2024-05-09T08:02:54.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.907+0000 7f58eb200640 1 --2- 172.21.15.1:0/1189377898 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f58c806bf00 0x7f58c806e3c0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:54.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.907+0000 7f58eb200640 1 --2- 172.21.15.1:0/1189377898 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58e415ac80 0x7f58e4150a90 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:54.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.907+0000 7f58eb200640 1 --2- 172.21.15.1:0/1189377898 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f58e40cc5f0 0x7f58e4150550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:54.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.907+0000 7f58eb200640 1 --2- 172.21.15.1:0/1189377898 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f58e40cac60 0x7f58e40b14e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:54.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.907+0000 7f58eb200640 1 -- 172.21.15.1:0/1189377898 >> 172.21.15.1:0/1189377898 conn(0x7f58e40cfc20 msgr2=0x7f58e406d910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:02:54.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.907+0000 7f58eb200640 1 -- 172.21.15.1:0/1189377898 shutdown_connections 2024-05-09T08:02:54.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.908+0000 7fecf2e00640 1 -- 172.21.15.1:0/2676723000 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fece4050ba0 con 0x7fece40deda0 2024-05-09T08:02:54.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.908+0000 7f58eb200640 1 -- 172.21.15.1:0/1189377898 wait complete. 2024-05-09T08:02:54.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.908+0000 7fece9600640 1 -- 172.21.15.1:0/2676723000 <== mon.1 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fecec0f4420 con 0x7fece40deda0 2024-05-09T08:02:54.926 INFO:teuthology.orchestra.run.smithi001.stdout:55834574859 2024-05-09T08:02:54.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.912+0000 7fecf2e00640 1 -- 172.21.15.1:0/2676723000 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7feccc06c060 msgr2=0x7feccc06e520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:54.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.912+0000 7fecf2e00640 1 --2- 172.21.15.1:0/2676723000 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7feccc06c060 0x7feccc06e520 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fece40d62d0 tx=0x7fecd8009040 comp rx=0 tx=0).stop 2024-05-09T08:02:54.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.912+0000 7fecf2e00640 1 -- 172.21.15.1:0/2676723000 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fece40deda0 msgr2=0x7fece416ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:54.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.912+0000 7fecf2e00640 1 --2- 172.21.15.1:0/2676723000 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fece40deda0 0x7fece416ea80 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fecec06a620 tx=0x7fecec069970 comp rx=0 tx=0).stop 2024-05-09T08:02:54.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.912+0000 7fecf2e00640 1 -- 172.21.15.1:0/2676723000 shutdown_connections 2024-05-09T08:02:54.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.912+0000 7fecf2e00640 1 --2- 172.21.15.1:0/2676723000 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7feccc06c060 0x7feccc06e520 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:54.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.912+0000 7fecf2e00640 1 --2- 172.21.15.1:0/2676723000 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fece40deda0 0x7fece416ea80 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:54.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.912+0000 7fecf2e00640 1 --2- 172.21.15.1:0/2676723000 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fece40d9090 0x7fece4053050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:54.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.912+0000 7fecf2e00640 1 --2- 172.21.15.1:0/2676723000 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fece4037870 0x7fece40dcfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:54.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.912+0000 7fecf2e00640 1 -- 172.21.15.1:0/2676723000 >> 172.21.15.1:0/2676723000 conn(0x7fece4054610 msgr2=0x7fece40d43b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:02:54.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.912+0000 7fecf2e00640 1 -- 172.21.15.1:0/2676723000 shutdown_connections 2024-05-09T08:02:54.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.913+0000 7fecf2e00640 1 -- 172.21.15.1:0/2676723000 wait complete. 2024-05-09T08:02:54.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.916+0000 7f24edc00640 1 -- 172.21.15.1:0/4227163944 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f24e815fdc0 con 0x7f24e8158b90 2024-05-09T08:02:54.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.917+0000 7f24dea00640 1 -- 172.21.15.1:0/4227163944 <== mon.1 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f24d8079f70 con 0x7f24e8158b90 2024-05-09T08:02:54.936 INFO:teuthology.orchestra.run.smithi001.stdout:55834574859 2024-05-09T08:02:54.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.922+0000 7f24edc00640 1 -- 172.21.15.1:0/4227163944 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f24c806bf30 msgr2=0x7f24c806e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:54.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.922+0000 7f24edc00640 1 --2- 172.21.15.1:0/4227163944 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f24c806bf30 0x7f24c806e3f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f24e0005c00 tx=0x7f24e0009040 comp rx=0 tx=0).stop 2024-05-09T08:02:54.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.922+0000 7f24edc00640 1 -- 172.21.15.1:0/4227163944 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24e8158b90 msgr2=0x7f24e80cb7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:54.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.922+0000 7f24edc00640 1 --2- 172.21.15.1:0/4227163944 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24e8158b90 0x7f24e80cb7d0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f24d80098c0 tx=0x7f24d80046c0 comp rx=0 tx=0).stop 2024-05-09T08:02:54.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.922+0000 7f24edc00640 1 -- 172.21.15.1:0/4227163944 shutdown_connections 2024-05-09T08:02:54.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.922+0000 7f24edc00640 1 --2- 172.21.15.1:0/4227163944 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f24c806bf30 0x7f24c806e3f0 secure :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f24e0005c00 tx=0x7f24e0009040 comp rx=0 tx=0).stop 2024-05-09T08:02:54.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.922+0000 7f24edc00640 1 --2- 172.21.15.1:0/4227163944 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24e8158b90 0x7f24e80cb7d0 secure :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f24d80098c0 tx=0x7f24d80046c0 comp rx=0 tx=0).stop 2024-05-09T08:02:54.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.922+0000 7f24edc00640 1 --2- 172.21.15.1:0/4227163944 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f24e80c9e00 0x7f24e81f5250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:54.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.922+0000 7f24edc00640 1 --2- 172.21.15.1:0/4227163944 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f24e80c87f0 0x7f24e81f8950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:54.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.922+0000 7f24edc00640 1 -- 172.21.15.1:0/4227163944 >> 172.21.15.1:0/4227163944 conn(0x7f24e806dad0 msgr2=0x7f24e806deb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:02:54.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.922+0000 7f24edc00640 1 -- 172.21.15.1:0/4227163944 shutdown_connections 2024-05-09T08:02:54.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.923+0000 7f02c3200640 1 -- 172.21.15.1:0/411966865 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f029c005580 con 0x7f02bc0e28f0 2024-05-09T08:02:54.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.923+0000 7f24edc00640 1 -- 172.21.15.1:0/4227163944 wait complete. 2024-05-09T08:02:54.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.924+0000 7f02ba000640 1 -- 172.21.15.1:0/411966865 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f02b00594d0 con 0x7f02bc0e28f0 2024-05-09T08:02:54.941 INFO:teuthology.orchestra.run.smithi001.stdout:55834574859 2024-05-09T08:02:54.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.927+0000 7f02c3200640 1 -- 172.21.15.1:0/411966865 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f02a406be30 msgr2=0x7f02a406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:54.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.927+0000 7f02c3200640 1 --2- 172.21.15.1:0/411966865 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f02a406be30 0x7f02a406e2f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f02b4000f60 tx=0x7f02b4007ce0 comp rx=0 tx=0).stop 2024-05-09T08:02:54.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.927+0000 7f02c3200640 1 -- 172.21.15.1:0/411966865 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f02bc0e28f0 msgr2=0x7f02bc185780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:54.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.927+0000 7f02c3200640 1 --2- 172.21.15.1:0/411966865 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f02bc0e28f0 0x7f02bc185780 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f02b000e730 tx=0x7f02b000ebf0 comp rx=0 tx=0).stop 2024-05-09T08:02:54.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.931+0000 7f02c3200640 1 -- 172.21.15.1:0/411966865 shutdown_connections 2024-05-09T08:02:54.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.931+0000 7f02c3200640 1 --2- 172.21.15.1:0/411966865 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f02a406be30 0x7f02a406e2f0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:54.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.931+0000 7f02c3200640 1 --2- 172.21.15.1:0/411966865 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f02bc0e2e10 0x7f02bc031370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:54.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.931+0000 7f02c3200640 1 --2- 172.21.15.1:0/411966865 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f02bc0e28f0 0x7f02bc185780 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:54.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.931+0000 7f02c3200640 1 --2- 172.21.15.1:0/411966865 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02bc0e2510 0x7f02bc1823b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:54.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.931+0000 7f02c3200640 1 -- 172.21.15.1:0/411966865 >> 172.21.15.1:0/411966865 conn(0x7f02bc050180 msgr2=0x7f02bc16a930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:02:54.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.932+0000 7f02c3200640 1 -- 172.21.15.1:0/411966865 shutdown_connections 2024-05-09T08:02:54.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.932+0000 7f02c3200640 1 -- 172.21.15.1:0/411966865 wait complete. 2024-05-09T08:02:54.953 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574860 got 55834574859 for osd.4 2024-05-09T08:02:54.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.939+0000 7f0c42c00640 1 -- 172.21.15.1:0/202381751 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f0c14005580 con 0x7f0c3c06cdc0 2024-05-09T08:02:54.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.939+0000 7f5950600640 1 -- 172.21.15.1:0/3496074172 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f5924005580 con 0x7f59480cb420 2024-05-09T08:02:54.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.939+0000 7f5945600640 1 -- 172.21.15.1:0/3496074172 <== mon.2 v2:172.21.15.7:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f593c0797e0 con 0x7f59480cb420 2024-05-09T08:02:54.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.939+0000 7f0c33e00640 1 -- 172.21.15.1:0/202381751 <== mon.1 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f0c2807e100 con 0x7f0c3c06cdc0 2024-05-09T08:02:54.959 INFO:teuthology.orchestra.run.smithi001.stdout:55834574858 2024-05-09T08:02:54.959 INFO:teuthology.orchestra.run.smithi001.stdout:55834574858 2024-05-09T08:02:54.960 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574860 got 55834574859 for osd.2 2024-05-09T08:02:54.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.943+0000 7f573e000640 1 -- 172.21.15.1:0/450387781 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f57380cba00 con 0x7f57380b4210 2024-05-09T08:02:54.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.943+0000 7f0c42c00640 1 -- 172.21.15.1:0/202381751 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f0c1c06be60 msgr2=0x7f0c1c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:54.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.943+0000 7f0c42c00640 1 --2- 172.21.15.1:0/202381751 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f0c1c06be60 0x7f0c1c06e320 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f0c2c009e00 tx=0x7f0c2c009190 comp rx=0 tx=0).stop 2024-05-09T08:02:54.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.943+0000 7f0c42c00640 1 -- 172.21.15.1:0/202381751 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c3c06cdc0 msgr2=0x7f0c3c1f40c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:54.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.943+0000 7f0c42c00640 1 --2- 172.21.15.1:0/202381751 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c3c06cdc0 0x7f0c3c1f40c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f0c28031000 tx=0x7f0c280046c0 comp rx=0 tx=0).stop 2024-05-09T08:02:54.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.943+0000 7f5950600640 1 -- 172.21.15.1:0/3496074172 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f592806bf30 msgr2=0x7f592806e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:54.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.943+0000 7f5950600640 1 --2- 172.21.15.1:0/3496074172 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f592806bf30 0x7f592806e3f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f59400012e0 tx=0x7f5940009040 comp rx=0 tx=0).stop 2024-05-09T08:02:54.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.943+0000 7f5950600640 1 -- 172.21.15.1:0/3496074172 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f59480cb420 msgr2=0x7f59480ca760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:54.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.943+0000 7f5950600640 1 --2- 172.21.15.1:0/3496074172 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f59480cb420 0x7f59480ca760 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f593c031000 tx=0x7f593c0046c0 comp rx=0 tx=0).stop 2024-05-09T08:02:54.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.943+0000 7f0c42c00640 1 -- 172.21.15.1:0/202381751 shutdown_connections 2024-05-09T08:02:54.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.943+0000 7f0c42c00640 1 --2- 172.21.15.1:0/202381751 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f0c1c06be60 0x7f0c1c06e320 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:54.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.943+0000 7f0c42c00640 1 --2- 172.21.15.1:0/202381751 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c3c06cdc0 0x7f0c3c1f40c0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:54.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.943+0000 7f0c42c00640 1 --2- 172.21.15.1:0/202381751 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0c3c0cc260 0x7f0c3c0cf230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:54.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.943+0000 7f0c42c00640 1 --2- 172.21.15.1:0/202381751 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f0c3c06c9e0 0x7f0c3c15b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:54.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.943+0000 7f0c42c00640 1 -- 172.21.15.1:0/202381751 >> 172.21.15.1:0/202381751 conn(0x7f0c3c0cffa0 msgr2=0x7f0c3c0cb090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:02:54.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.943+0000 7f5950600640 1 -- 172.21.15.1:0/3496074172 shutdown_connections 2024-05-09T08:02:54.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.943+0000 7f5950600640 1 --2- 172.21.15.1:0/3496074172 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f592806bf30 0x7f592806e3f0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:54.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.943+0000 7f5950600640 1 --2- 172.21.15.1:0/3496074172 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5948158b90 0x7f594806b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:54.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.943+0000 7f5950600640 1 --2- 172.21.15.1:0/3496074172 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f59480cb420 0x7f59480ca760 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:54.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.943+0000 7f5950600640 1 --2- 172.21.15.1:0/3496074172 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f59480c87f0 0x7f59481f4db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:54.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.943+0000 7f0c42c00640 1 -- 172.21.15.1:0/202381751 shutdown_connections 2024-05-09T08:02:54.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.943+0000 7f572f400640 1 -- 172.21.15.1:0/450387781 <== mon.1 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f5728079690 con 0x7f57380b4210 2024-05-09T08:02:54.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.943+0000 7f5950600640 1 -- 172.21.15.1:0/3496074172 >> 172.21.15.1:0/3496074172 conn(0x7f594806dad0 msgr2=0x7f594814f160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:02:54.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.944+0000 7f0c42c00640 1 -- 172.21.15.1:0/202381751 wait complete. 2024-05-09T08:02:54.964 INFO:teuthology.orchestra.run.smithi001.stdout:55834574860 2024-05-09T08:02:54.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.944+0000 7f5950600640 1 -- 172.21.15.1:0/3496074172 shutdown_connections 2024-05-09T08:02:54.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.944+0000 7f5950600640 1 -- 172.21.15.1:0/3496074172 wait complete. 2024-05-09T08:02:54.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.946+0000 7f573e000640 1 -- 172.21.15.1:0/450387781 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f571406c060 msgr2=0x7f571406e520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:54.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.946+0000 7f573e000640 1 --2- 172.21.15.1:0/450387781 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f571406c060 0x7f571406e520 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f57380c6520 tx=0x7f5720007a40 comp rx=0 tx=0).stop 2024-05-09T08:02:54.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.946+0000 7f573e000640 1 -- 172.21.15.1:0/450387781 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57380b4210 msgr2=0x7f57380c7ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:54.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.946+0000 7f573e000640 1 --2- 172.21.15.1:0/450387781 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57380b4210 0x7f57380c7ee0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f5728009890 tx=0x7f57280046c0 comp rx=0 tx=0).stop 2024-05-09T08:02:54.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.947+0000 7f573e000640 1 -- 172.21.15.1:0/450387781 shutdown_connections 2024-05-09T08:02:54.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.947+0000 7f573e000640 1 --2- 172.21.15.1:0/450387781 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f571406c060 0x7f571406e520 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:54.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.947+0000 7f573e000640 1 --2- 172.21.15.1:0/450387781 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f5738154a60 0x7f573814dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:54.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.947+0000 7f573e000640 1 --2- 172.21.15.1:0/450387781 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f57380cc4c0 0x7f57381f37d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:54.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.947+0000 7f573e000640 1 --2- 172.21.15.1:0/450387781 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57380b4210 0x7f57380c7ee0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:54.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.947+0000 7f573e000640 1 -- 172.21.15.1:0/450387781 >> 172.21.15.1:0/450387781 conn(0x7f57380cffa0 msgr2=0x7f57380d0380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:02:54.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.947+0000 7f573e000640 1 -- 172.21.15.1:0/450387781 shutdown_connections 2024-05-09T08:02:54.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.947+0000 7f573e000640 1 -- 172.21.15.1:0/450387781 wait complete. 2024-05-09T08:02:54.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.951+0000 7f6492a00640 1 -- 172.21.15.1:0/3568813400 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f6468005580 con 0x7f648c0c59e0 2024-05-09T08:02:54.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.952+0000 7f6489600640 1 -- 172.21.15.1:0/3568813400 <== mon.2 v2:172.21.15.7:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f64780583f0 con 0x7f648c0c59e0 2024-05-09T08:02:54.969 INFO:teuthology.orchestra.run.smithi001.stdout:55834574858 2024-05-09T08:02:54.970 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574860 got 55834574859 for osd.3 2024-05-09T08:02:54.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.955+0000 7f6492a00640 1 -- 172.21.15.1:0/3568813400 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f646c06be30 msgr2=0x7f646c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:54.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.955+0000 7f6492a00640 1 --2- 172.21.15.1:0/3568813400 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f646c06be30 0x7f646c06e2f0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f64840059a0 tx=0x7f6484004f40 comp rx=0 tx=0).stop 2024-05-09T08:02:54.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.955+0000 7f6492a00640 1 -- 172.21.15.1:0/3568813400 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f648c0c59e0 msgr2=0x7f648c1ec0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:54.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.955+0000 7f6492a00640 1 --2- 172.21.15.1:0/3568813400 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f648c0c59e0 0x7f648c1ec0d0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f647800c440 tx=0x7f647800c900 comp rx=0 tx=0).stop 2024-05-09T08:02:54.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.955+0000 7f6492a00640 1 -- 172.21.15.1:0/3568813400 shutdown_connections 2024-05-09T08:02:54.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.955+0000 7f6492a00640 1 --2- 172.21.15.1:0/3568813400 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f646c06be30 0x7f646c06e2f0 secure :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f64840059a0 tx=0x7f6484004f40 comp rx=0 tx=0).stop 2024-05-09T08:02:54.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.955+0000 7f6492a00640 1 --2- 172.21.15.1:0/3568813400 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f648c157040 0x7f648c146a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:54.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.955+0000 7f6492a00640 1 --2- 172.21.15.1:0/3568813400 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f648c0c59e0 0x7f648c1ec0d0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:54.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.955+0000 7f6492a00640 1 --2- 172.21.15.1:0/3568813400 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f648c0c5600 0x7f648c1e15a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:54.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.955+0000 7f6492a00640 1 -- 172.21.15.1:0/3568813400 >> 172.21.15.1:0/3568813400 conn(0x7f648c0c8e30 msgr2=0x7f648c06b250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:02:54.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.955+0000 7f6492a00640 1 -- 172.21.15.1:0/3568813400 shutdown_connections 2024-05-09T08:02:54.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:54.955+0000 7f6492a00640 1 -- 172.21.15.1:0/3568813400 wait complete. 2024-05-09T08:02:54.977 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574860 got 55834574859 for osd.0 2024-05-09T08:02:54.982 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574858 for osd.6 2024-05-09T08:02:54.984 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574861 got 55834574860 for osd.1 2024-05-09T08:02:54.986 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574858 for osd.7 2024-05-09T08:02:54.991 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574858 for osd.5 2024-05-09T08:02:55.953 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-05-09T08:02:55.961 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-05-09T08:02:55.971 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-05-09T08:02:55.977 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-05-09T08:02:55.984 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-05-09T08:02:55.985 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-05-09T08:02:55.987 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-05-09T08:02:55.991 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-05-09T08:02:56.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.230+0000 7f546da00640 1 Processor -- start 2024-05-09T08:02:56.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.230+0000 7f546da00640 1 -- start start 2024-05-09T08:02:56.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.232+0000 7f546da00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f5468158e20 0x7f5468161f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:56.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.233+0000 7f546da00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f54680724b0 0x7f546815f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:56.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.233+0000 7f546da00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f546815f900 con 0x7f54680c7e00 2024-05-09T08:02:56.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.233+0000 7f546da00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f54681625d0 con 0x7f5468158e20 2024-05-09T08:02:56.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.233+0000 7f546da00640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f54681628b0 con 0x7f54680724b0 2024-05-09T08:02:56.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.233+0000 7f5466a00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f5468158e20 0x7f5468161f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:56.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.233+0000 7f5466a00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f5468158e20 0x7f5468161f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.1:58974/0 (socket says 172.21.15.1:58974) 2024-05-09T08:02:56.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.233+0000 7f5466a00640 1 -- 172.21.15.1:0/814109820 learned_addr learned my addr 172.21.15.1:0/814109820 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:02:56.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.233+0000 7f5466000640 1 --2- 172.21.15.1:0/814109820 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f54680724b0 0x7f546815f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:56.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.234+0000 7f5466a00640 1 -- 172.21.15.1:0/814109820 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f54680724b0 msgr2=0x7f546815f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:02:56.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.234+0000 7f5466a00640 1 --2- 172.21.15.1:0/814109820 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f54680724b0 0x7f546815f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:56.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.234+0000 7f5466a00640 1 -- 172.21.15.1:0/814109820 >> v1:172.21.15.7:6789/0 conn(0x7f54680c7e00 legacy=0x7f546806f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:02:56.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.235+0000 7f5466a00640 1 -- 172.21.15.1:0/814109820 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5468162b90 con 0x7f5468158e20 2024-05-09T08:02:56.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.236+0000 7f5466a00640 1 --2- 172.21.15.1:0/814109820 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5468158e20 0x7f5468161f20 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f5450009870 tx=0x7f545002a2a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=765cee43c4cb6d50 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:56.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.237+0000 7f5465600640 1 -- 172.21.15.1:0/814109820 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5450037070 con 0x7f5468158e20 2024-05-09T08:02:56.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.237+0000 7f5465600640 1 -- 172.21.15.1:0/814109820 <== mon.1 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5450032440 con 0x7f5468158e20 2024-05-09T08:02:56.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.237+0000 7f5465600640 1 -- 172.21.15.1:0/814109820 <== mon.1 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5450032810 con 0x7f5468158e20 2024-05-09T08:02:56.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.238+0000 7f546da00640 1 -- 172.21.15.1:0/814109820 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5468158e20 msgr2=0x7f5468161f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:56.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.238+0000 7f546da00640 1 --2- 172.21.15.1:0/814109820 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5468158e20 0x7f5468161f20 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f5450009870 tx=0x7f545002a2a0 comp rx=0 tx=0).stop 2024-05-09T08:02:56.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.238+0000 7f546da00640 1 -- 172.21.15.1:0/814109820 shutdown_connections 2024-05-09T08:02:56.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.238+0000 7f546da00640 1 --2- 172.21.15.1:0/814109820 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f54680724b0 0x7f546815f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:56.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.238+0000 7f546da00640 1 --2- 172.21.15.1:0/814109820 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5468158e20 0x7f5468161f20 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:56.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.238+0000 7f546da00640 1 -- 172.21.15.1:0/814109820 >> 172.21.15.1:0/814109820 conn(0x7f546806dad0 msgr2=0x7f546806deb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:02:56.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.239+0000 7f546da00640 1 -- 172.21.15.1:0/814109820 shutdown_connections 2024-05-09T08:02:56.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.239+0000 7f546da00640 1 -- 172.21.15.1:0/814109820 wait complete. 2024-05-09T08:02:56.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.242+0000 7f546da00640 1 Processor -- start 2024-05-09T08:02:56.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.243+0000 7f546da00640 1 -- start start 2024-05-09T08:02:56.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.245+0000 7f546da00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54680724b0 0x7f54681ea180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:56.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.245+0000 7f546da00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f54680c7e00 0x7f54680cb130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:56.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.246+0000 7f5466a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54680724b0 0x7f54681ea180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:56.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.246+0000 7f5466a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54680724b0 0x7f54681ea180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.1:58990/0 (socket says 172.21.15.1:58990) 2024-05-09T08:02:56.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.246+0000 7f5466a00640 1 -- 172.21.15.1:0/2764213648 learned_addr learned my addr 172.21.15.1:0/2764213648 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:02:56.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.246+0000 7f546da00640 1 --2- 172.21.15.1:0/2764213648 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f5468158e20 0x7f54680cb670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:56.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.247+0000 7f5466000640 1 --2- 172.21.15.1:0/2764213648 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f54680c7e00 0x7f54680cb130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:56.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.247+0000 7f546da00640 1 -- 172.21.15.1:0/2764213648 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f546806fe30 con 0x7f54680c7e00 2024-05-09T08:02:56.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.247+0000 7f5467400640 1 --2- 172.21.15.1:0/2764213648 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f5468158e20 0x7f54680cb670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:56.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.247+0000 7f546da00640 1 -- 172.21.15.1:0/2764213648 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f546815eea0 con 0x7f54680724b0 2024-05-09T08:02:56.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.247+0000 7f546da00640 1 -- 172.21.15.1:0/2764213648 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f546806ffb0 con 0x7f5468158e20 2024-05-09T08:02:56.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.248+0000 7f5466a00640 1 -- 172.21.15.1:0/2764213648 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f5468158e20 msgr2=0x7f54680cb670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:56.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.248+0000 7f5466a00640 1 --2- 172.21.15.1:0/2764213648 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f5468158e20 0x7f54680cb670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:56.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.248+0000 7f5466a00640 1 -- 172.21.15.1:0/2764213648 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f54680c7e00 msgr2=0x7f54680cb130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:56.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.248+0000 7f5466a00640 1 --2- 172.21.15.1:0/2764213648 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f54680c7e00 0x7f54680cb130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:56.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.248+0000 7f5466a00640 1 -- 172.21.15.1:0/2764213648 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5468201970 con 0x7f54680724b0 2024-05-09T08:02:56.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.250+0000 7f5466a00640 1 --2- 172.21.15.1:0/2764213648 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54680724b0 0x7f54681ea180 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f5450009870 tx=0x7f54500046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:56.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.250+0000 7f545ea00640 1 -- 172.21.15.1:0/2764213648 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5450037040 con 0x7f54680724b0 2024-05-09T08:02:56.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.251+0000 7f546da00640 1 -- 172.21.15.1:0/2764213648 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5468159dd0 con 0x7f54680724b0 2024-05-09T08:02:56.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.251+0000 7f546da00640 1 -- 172.21.15.1:0/2764213648 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f546815a350 con 0x7f54680724b0 2024-05-09T08:02:56.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.251+0000 7f545ea00640 1 -- 172.21.15.1:0/2764213648 <== mon.1 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5450042070 con 0x7f54680724b0 2024-05-09T08:02:56.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.251+0000 7f545ea00640 1 -- 172.21.15.1:0/2764213648 <== mon.1 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5450037040 con 0x7f54680724b0 2024-05-09T08:02:56.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.254+0000 7f546da00640 1 -- 172.21.15.1:0/2764213648 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5440005290 con 0x7f54680724b0 2024-05-09T08:02:56.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.256+0000 7f545ea00640 1 -- 172.21.15.1:0/2764213648 <== mon.1 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5450030640 con 0x7f54680724b0 2024-05-09T08:02:56.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.258+0000 7f545ea00640 1 --2- 172.21.15.1:0/2764213648 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f544406c030 0x7f544406e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:56.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.259+0000 7f5466000640 1 --2- 172.21.15.1:0/2764213648 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f544406c030 0x7f544406e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:56.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.259+0000 7f545ea00640 1 -- 172.21.15.1:0/2764213648 <== mon.1 v2:172.21.15.7:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f54500ab460 con 0x7f54680724b0 2024-05-09T08:02:56.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.260+0000 7f5466000640 1 --2- 172.21.15.1:0/2764213648 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f544406c030 0x7f544406e4f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f546812d940 tx=0x7f5458009190 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:56.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.262+0000 7f545ea00640 1 -- 172.21.15.1:0/2764213648 <== mon.1 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f5450079470 con 0x7f54680724b0 2024-05-09T08:02:56.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.282+0000 7f7991600640 1 Processor -- start 2024-05-09T08:02:56.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.282+0000 7f7991600640 1 -- start start 2024-05-09T08:02:56.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.284+0000 7f7991600640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f798c15d730 0x7f798c15db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:56.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.284+0000 7f7991600640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f798c15ba80 0x7f798c15b020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:56.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.284+0000 7f7991600640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f798c15b560 con 0x7f798c0cda80 2024-05-09T08:02:56.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.284+0000 7f7991600640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f798c15ead0 con 0x7f798c15d730 2024-05-09T08:02:56.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.284+0000 7f7991600640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f798c15e230 con 0x7f798c15ba80 2024-05-09T08:02:56.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.286+0000 7f798be00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f798c15d730 0x7f798c15db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:56.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.287+0000 7f798be00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f798c15d730 0x7f798c15db10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.1:59004/0 (socket says 172.21.15.1:59004) 2024-05-09T08:02:56.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.287+0000 7f798be00640 1 -- 172.21.15.1:0/3674438263 learned_addr learned my addr 172.21.15.1:0/3674438263 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:02:56.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.287+0000 7f798b400640 1 --2- 172.21.15.1:0/3674438263 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f798c15ba80 0x7f798c15b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:56.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.287+0000 7f798be00640 1 -- 172.21.15.1:0/3674438263 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f798c15ba80 msgr2=0x7f798c15b020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:02:56.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.287+0000 7f798be00640 1 --2- 172.21.15.1:0/3674438263 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f798c15ba80 0x7f798c15b020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:56.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.287+0000 7f798be00640 1 -- 172.21.15.1:0/3674438263 >> v1:172.21.15.7:6789/0 conn(0x7f798c0cda80 legacy=0x7f798c0ca620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:56.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.287+0000 7f798aa00640 1 -- 172.21.15.1:0/3674438263 <== mon.1 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1329448672 0 0) 0x7f798c15b560 con 0x7f798c0cda80 2024-05-09T08:02:56.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.288+0000 7f798be00640 1 -- 172.21.15.1:0/3674438263 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f798c15e510 con 0x7f798c15d730 2024-05-09T08:02:56.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.289+0000 7f798be00640 1 --2- 172.21.15.1:0/3674438263 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f798c15d730 0x7f798c15db10 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f7984009870 tx=0x7f798402a300 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5915d8bf6e3c07d5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:56.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.290+0000 7f798aa00640 1 -- 172.21.15.1:0/3674438263 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7984037070 con 0x7f798c15d730 2024-05-09T08:02:56.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.290+0000 7f798aa00640 1 -- 172.21.15.1:0/3674438263 <== mon.1 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f798402ae50 con 0x7f798c15d730 2024-05-09T08:02:56.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.290+0000 7f798aa00640 1 -- 172.21.15.1:0/3674438263 <== mon.1 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7984032860 con 0x7f798c15d730 2024-05-09T08:02:56.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.290+0000 7f90d0e00640 1 Processor -- start 2024-05-09T08:02:56.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.290+0000 7f90d0e00640 1 -- start start 2024-05-09T08:02:56.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.291+0000 7f7991600640 1 -- 172.21.15.1:0/3674438263 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f798c15d730 msgr2=0x7f798c15db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:56.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.291+0000 7f7991600640 1 --2- 172.21.15.1:0/3674438263 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f798c15d730 0x7f798c15db10 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f7984009870 tx=0x7f798402a300 comp rx=0 tx=0).stop 2024-05-09T08:02:56.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.291+0000 7fd159a00640 1 Processor -- start 2024-05-09T08:02:56.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.291+0000 7f7991600640 1 -- 172.21.15.1:0/3674438263 shutdown_connections 2024-05-09T08:02:56.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.291+0000 7f7991600640 1 --2- 172.21.15.1:0/3674438263 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f798c15ba80 0x7f798c15b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:56.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.291+0000 7f7991600640 1 --2- 172.21.15.1:0/3674438263 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f798c15d730 0x7f798c15db10 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:56.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.291+0000 7f7991600640 1 -- 172.21.15.1:0/3674438263 >> 172.21.15.1:0/3674438263 conn(0x7f798c066ed0 msgr2=0x7f798c0672e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:02:56.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.292+0000 7f7991600640 1 -- 172.21.15.1:0/3674438263 shutdown_connections 2024-05-09T08:02:56.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.292+0000 7f90d0e00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f90cc15c5c0 0x7f90cc071af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:56.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.292+0000 7f90d0e00640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f90cc0c7030 0x7f90cc0c8280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:56.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.293+0000 7f90d0e00640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f90cc0c8f60 con 0x7f90cc0c6c50 2024-05-09T08:02:56.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.293+0000 7f90d0e00640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f90cc0c89a0 con 0x7f90cc0c7030 2024-05-09T08:02:56.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.293+0000 7f90d0e00640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f90cc0c8c80 con 0x7f90cc15c5c0 2024-05-09T08:02:56.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.293+0000 7f90caa00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f90cc15c5c0 0x7f90cc071af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:56.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.293+0000 7f90caa00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f90cc15c5c0 0x7f90cc071af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3301/0 says I am v2:172.21.15.1:44634/0 (socket says 172.21.15.1:44634) 2024-05-09T08:02:56.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.293+0000 7f90caa00640 1 -- 172.21.15.1:0/1433008396 learned_addr learned my addr 172.21.15.1:0/1433008396 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:02:56.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.294+0000 7f90cbe00640 1 --2- 172.21.15.1:0/1433008396 >> v2:172.21.15.1:3300/0 conn(0x7f90cc0c7030 0x7f90cc0c8280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:56.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.294+0000 7f90caa00640 1 -- 172.21.15.1:0/1433008396 >> v2:172.21.15.1:3300/0 conn(0x7f90cc0c7030 msgr2=0x7f90cc0c8280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:02:56.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.294+0000 7f32a5e00640 1 Processor -- start 2024-05-09T08:02:56.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.292+0000 7f7991600640 1 -- 172.21.15.1:0/3674438263 wait complete. 2024-05-09T08:02:56.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.294+0000 7f90caa00640 1 --2- 172.21.15.1:0/1433008396 >> v2:172.21.15.1:3300/0 conn(0x7f90cc0c7030 0x7f90cc0c8280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:56.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.294+0000 7f90caa00640 1 -- 172.21.15.1:0/1433008396 >> v1:172.21.15.1:6789/0 conn(0x7f90cc0c6c50 legacy=0x7f90cc15beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:02:56.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.294+0000 7f90caa00640 1 -- 172.21.15.1:0/1433008396 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90cc06f0f0 con 0x7f90cc15c5c0 2024-05-09T08:02:56.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.294+0000 7f7991600640 1 Processor -- start 2024-05-09T08:02:56.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.294+0000 7f32a5e00640 1 -- start start 2024-05-09T08:02:56.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.295+0000 7f7991600640 1 -- start start 2024-05-09T08:02:56.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.292+0000 7fd159a00640 1 -- start start 2024-05-09T08:02:56.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.294+0000 7fd159a00640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fd15415ac80 0x7fd154165d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:56.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.294+0000 7fd159a00640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd1540ccb20 con 0x7fd1540cac60 2024-05-09T08:02:56.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.294+0000 7fd159a00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd1540cce00 con 0x7fd1540cc5f0 2024-05-09T08:02:56.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.294+0000 7fd159a00640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7fd1540cf120 con 0x7fd15415ac80 2024-05-09T08:02:56.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.294+0000 7fd153400640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fd1540cc5f0 0x7fd15415a570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.1:42104/0 (socket says 172.21.15.1:42104) 2024-05-09T08:02:56.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.294+0000 7fd153400640 1 -- 172.21.15.1:0/2795097148 learned_addr learned my addr 172.21.15.1:0/2795097148 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T08:02:56.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.295+0000 7fd159000640 1 --2- 172.21.15.1:0/2795097148 >> v2:172.21.15.1:3300/0 conn(0x7fd15415ac80 0x7fd154165d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:56.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.295+0000 7fd152a00640 1 -- 172.21.15.1:0/2795097148 <== mon.1 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2194359011 0 0) 0x7fd1540cce00 con 0x7fd1540cc5f0 2024-05-09T08:02:56.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.296+0000 7fd152a00640 1 -- 172.21.15.1:0/2795097148 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd138003520 con 0x7fd1540cc5f0 2024-05-09T08:02:56.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.296+0000 7fd152a00640 1 -- 172.21.15.1:0/2795097148 <== mon.1 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2022568268 0 0) 0x7fd138003520 con 0x7fd1540cc5f0 2024-05-09T08:02:56.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.296+0000 7fd152a00640 1 -- 172.21.15.1:0/2795097148 >> v2:172.21.15.1:3300/0 conn(0x7fd15415ac80 msgr2=0x7fd154165d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:02:56.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.296+0000 7fd152a00640 1 --2- 172.21.15.1:0/2795097148 >> v2:172.21.15.1:3300/0 conn(0x7fd15415ac80 0x7fd154165d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:56.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.296+0000 7fd152a00640 1 -- 172.21.15.1:0/2795097148 >> v1:172.21.15.1:6789/0 conn(0x7fd1540cac60 legacy=0x7fd1540cda00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:02:56.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.296+0000 7f90caa00640 1 --2- 172.21.15.1:0/1433008396 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f90cc15c5c0 0x7f90cc071af0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f90bc0098a0 tx=0x7f90bc02a250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8087b8ae1abebbae server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:56.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.297+0000 7f32a5e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f3298037630 0x7f329804f520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:56.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.297+0000 7f32a5e00640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f3298037a10 0x7f32980e3980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:56.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.297+0000 7f90ca000640 1 -- 172.21.15.1:0/1433008396 <== mon.2 v2:172.21.15.7:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90bc037070 con 0x7f90cc15c5c0 2024-05-09T08:02:56.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.297+0000 7f90ca000640 1 -- 172.21.15.1:0/1433008396 <== mon.2 v2:172.21.15.7:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90bc032440 con 0x7f90cc15c5c0 2024-05-09T08:02:56.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.297+0000 7f90ca000640 1 -- 172.21.15.1:0/1433008396 <== mon.2 v2:172.21.15.7:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90bc032810 con 0x7f90cc15c5c0 2024-05-09T08:02:56.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.297+0000 7f32a5e00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f329804fa60 con 0x7f3298007810 2024-05-09T08:02:56.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.297+0000 7f32a5e00640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f329803a2d0 con 0x7f3298037a10 2024-05-09T08:02:56.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.297+0000 7f32a5e00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f3298039480 con 0x7f3298037630 2024-05-09T08:02:56.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.297+0000 7fc277e00640 1 Processor -- start 2024-05-09T08:02:56.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.297+0000 7f32a5400640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f3298037a10 0x7f32980e3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:56.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.297+0000 7f32a5400640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f3298037a10 0x7f32980e3980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38200/0 (socket says 172.21.15.1:38200) 2024-05-09T08:02:56.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.297+0000 7f32a5400640 1 -- 172.21.15.1:0/1770594148 learned_addr learned my addr 172.21.15.1:0/1770594148 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:02:56.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.297+0000 7f32a4a00640 1 --2- 172.21.15.1:0/1770594148 >> v2:172.21.15.7:3300/0 conn(0x7f3298037630 0x7f329804f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:56.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.297+0000 7fd152a00640 1 -- 172.21.15.1:0/2795097148 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1540cf400 con 0x7fd1540cc5f0 2024-05-09T08:02:56.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.297+0000 7fd152a00640 1 -- 172.21.15.1:0/2795097148 <== mon.1 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4179846320 0 0) 0x7fd13c001670 con 0x7fd1540cc5f0 2024-05-09T08:02:56.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.297+0000 7fd152a00640 1 -- 172.21.15.1:0/2795097148 >> v1:172.21.15.7:6789/0 conn(0x7fd1540cc5f0 legacy=0x7fd15415a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:56.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.297+0000 7fd152a00640 1 --2- 172.21.15.1:0/2795097148 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fd1380039b0 0x7fd138003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:56.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.297+0000 7fd152a00640 1 --2- 172.21.15.1:0/2795097148 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd138004800 0x7fd138004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:56.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.297+0000 7fd152a00640 1 --2- 172.21.15.1:0/2795097148 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1380052e0 0x7fd13800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:56.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.298+0000 7fd152a00640 1 -- 172.21.15.1:0/2795097148 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fd1540cf120 con 0x7fd138004800 2024-05-09T08:02:56.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.298+0000 7fd152a00640 1 -- 172.21.15.1:0/2795097148 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd1380149e0 con 0x7fd1380052e0 2024-05-09T08:02:56.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.298+0000 7fd152a00640 1 -- 172.21.15.1:0/2795097148 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7fd138014be0 con 0x7fd1380039b0 2024-05-09T08:02:56.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.298+0000 7fc277e00640 1 -- start start 2024-05-09T08:02:56.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.298+0000 7f329f400640 1 -- 172.21.15.1:0/1770594148 <== mon.1 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2576126470 0 0) 0x7f329804fa60 con 0x7f3298007810 2024-05-09T08:02:56.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.298+0000 7f90d0e00640 1 -- 172.21.15.1:0/1433008396 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f90cc15c5c0 msgr2=0x7f90cc071af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:56.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.299+0000 7f90d0e00640 1 --2- 172.21.15.1:0/1433008396 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f90cc15c5c0 0x7f90cc071af0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f90bc0098a0 tx=0x7f90bc02a250 comp rx=0 tx=0).stop 2024-05-09T08:02:56.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.298+0000 7f7991600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f798c0cda80 0x7f798c154500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:56.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.298+0000 7f7991600640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f798c15ba80 0x7f798c1e5970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:56.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.298+0000 7f7991600640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f798c15d730 0x7f798c1e5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:56.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.298+0000 7f7991600640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f798c15ab00 con 0x7f798c15ba80 2024-05-09T08:02:56.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.298+0000 7f7991600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f798c15a820 con 0x7f798c0cda80 2024-05-09T08:02:56.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.298+0000 7f7991600640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f798c0cad30 con 0x7f798c15d730 2024-05-09T08:02:56.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.298+0000 7f329f400640 1 -- 172.21.15.1:0/1770594148 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3288003520 con 0x7f3298007810 2024-05-09T08:02:56.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.298+0000 7fd159000640 1 --2- 172.21.15.1:0/2795097148 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1380052e0 0x7fd13800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:56.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.299+0000 7f90d0e00640 1 -- 172.21.15.1:0/1433008396 shutdown_connections 2024-05-09T08:02:56.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.299+0000 7f90d0e00640 1 --2- 172.21.15.1:0/1433008396 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f90cc15c5c0 0x7f90cc071af0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:56.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.299+0000 7f90d0e00640 1 --2- 172.21.15.1:0/1433008396 >> v2:172.21.15.1:3300/0 conn(0x7f90cc0c7030 0x7f90cc0c8280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:56.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.299+0000 7f90d0e00640 1 -- 172.21.15.1:0/1433008396 >> 172.21.15.1:0/1433008396 conn(0x7f90cc06b250 msgr2=0x7f90cc06b630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:02:56.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.299+0000 7f90d0e00640 1 -- 172.21.15.1:0/1433008396 shutdown_connections 2024-05-09T08:02:56.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.299+0000 7f32a5400640 1 -- 172.21.15.1:0/1770594148 >> v2:172.21.15.7:3300/0 conn(0x7f3298037630 msgr2=0x7f329804f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:02:56.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.299+0000 7f32a5400640 1 --2- 172.21.15.1:0/1770594148 >> v2:172.21.15.7:3300/0 conn(0x7f3298037630 0x7f329804f520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:56.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.299+0000 7f32a5400640 1 -- 172.21.15.1:0/1770594148 >> v1:172.21.15.7:6789/0 conn(0x7f3298007810 legacy=0x7f3298038d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:56.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.299+0000 7f32a5400640 1 -- 172.21.15.1:0/1770594148 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3298006210 con 0x7f3298037a10 2024-05-09T08:02:56.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.299+0000 7fd153e00640 1 --2- 172.21.15.1:0/2795097148 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fd1380039b0 0x7fd138003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:56.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.299+0000 7f32a4a00640 1 --2- 172.21.15.1:0/1770594148 >> v2:172.21.15.7:3300/0 conn(0x7f3298037630 0x7f329804f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:02:56.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.299+0000 7fd153400640 1 --2- 172.21.15.1:0/2795097148 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd138004800 0x7fd138004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:56.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.299+0000 7f798b400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f798c15ba80 0x7f798c1e5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:56.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.300+0000 7fd153e00640 1 -- 172.21.15.1:0/2795097148 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1380052e0 msgr2=0x7fd13800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:02:56.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.300+0000 7fd153e00640 1 --2- 172.21.15.1:0/2795097148 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1380052e0 0x7fd13800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:56.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.300+0000 7fd153e00640 1 -- 172.21.15.1:0/2795097148 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd138004800 msgr2=0x7fd138004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:02:56.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.300+0000 7fd153e00640 1 --2- 172.21.15.1:0/2795097148 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd138004800 0x7fd138004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:56.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.300+0000 7fd153e00640 1 -- 172.21.15.1:0/2795097148 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd138003520 con 0x7fd1380039b0 2024-05-09T08:02:56.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.300+0000 7fc277e00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fc2700deda0 0x7fc270037160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:56.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.300+0000 7fc277e00640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc2700df1e0 con 0x7fc2700d9090 2024-05-09T08:02:56.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.300+0000 7fc277e00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc2700df4a0 con 0x7fc2700376a0 2024-05-09T08:02:56.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.300+0000 7fc277e00640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7fc270038090 con 0x7fc2700deda0 2024-05-09T08:02:56.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.300+0000 7f94da600640 1 Processor -- start 2024-05-09T08:02:56.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.301+0000 7fd159000640 1 -- 172.21.15.1:0/2795097148 reap_dead start 2024-05-09T08:02:56.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.299+0000 7f798b400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f798c15ba80 0x7f798c1e5970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38228/0 (socket says 172.21.15.1:38228) 2024-05-09T08:02:56.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.299+0000 7f798b400640 1 -- 172.21.15.1:0/1143769927 learned_addr learned my addr 172.21.15.1:0/1143769927 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:02:56.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.299+0000 7f7990c00640 1 --2- 172.21.15.1:0/1143769927 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f798c15d730 0x7f798c1e5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:56.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.300+0000 7f798be00640 1 --2- 172.21.15.1:0/1143769927 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f798c0cda80 0x7f798c154500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:56.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.301+0000 7f798be00640 1 -- 172.21.15.1:0/1143769927 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f798c15d730 msgr2=0x7f798c1e5eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:56.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.301+0000 7f798be00640 1 --2- 172.21.15.1:0/1143769927 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f798c15d730 0x7f798c1e5eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:56.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.301+0000 7f798be00640 1 -- 172.21.15.1:0/1143769927 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f798c15ba80 msgr2=0x7f798c1e5970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:56.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.301+0000 7f798be00640 1 --2- 172.21.15.1:0/1143769927 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f798c15ba80 0x7f798c1e5970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:56.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.301+0000 7f798be00640 1 -- 172.21.15.1:0/1143769927 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f798c155740 con 0x7f798c0cda80 2024-05-09T08:02:56.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.301+0000 7f798b400640 1 --2- 172.21.15.1:0/1143769927 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f798c15ba80 0x7f798c1e5970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T08:02:56.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.301+0000 7f90d0e00640 1 -- 172.21.15.1:0/1433008396 wait complete. 2024-05-09T08:02:56.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.301+0000 7fc276a00640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7fc2700d9090 0x7fc270050180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:56074/0 (socket says 172.21.15.1:56074) 2024-05-09T08:02:56.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.301+0000 7fc276a00640 1 -- 172.21.15.1:0/258183096 learned_addr learned my addr 172.21.15.1:0/258183096 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T08:02:56.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.301+0000 7f798be00640 1 --2- 172.21.15.1:0/1143769927 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f798c0cda80 0x7f798c154500 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f7984031000 tx=0x7f79840046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:56.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.301+0000 7f94da600640 1 -- start start 2024-05-09T08:02:56.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.302+0000 7f32a5400640 1 --2- 172.21.15.1:0/1770594148 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3298037a10 0x7f32980e3980 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f3298011c80 tx=0x7f329402a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7e486fed926f0a7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:56.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.301+0000 7fc276000640 1 --2- 172.21.15.1:0/258183096 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fc2700deda0 0x7fc270037160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:56.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.302+0000 7fc275600640 1 -- 172.21.15.1:0/258183096 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 844980557 0 0) 0x7fc2700df1e0 con 0x7fc2700d9090 2024-05-09T08:02:56.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.302+0000 7fc276000640 1 -- 172.21.15.1:0/258183096 >> v1:172.21.15.7:6789/0 conn(0x7fc2700376a0 legacy=0x7fc270038520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:56.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.302+0000 7fc276000640 1 -- 172.21.15.1:0/258183096 >> v1:172.21.15.1:6789/0 conn(0x7fc2700d9090 legacy=0x7fc270050180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:56.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.302+0000 7fc276000640 1 -- 172.21.15.1:0/258183096 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2700df780 con 0x7fc2700deda0 2024-05-09T08:02:56.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.303+0000 7f329f400640 1 -- 172.21.15.1:0/1770594148 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3294037070 con 0x7f3298037a10 2024-05-09T08:02:56.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.302+0000 7f7983e00640 1 -- 172.21.15.1:0/1143769927 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7984041070 con 0x7f798c0cda80 2024-05-09T08:02:56.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.302+0000 7f7983e00640 1 -- 172.21.15.1:0/1143769927 <== mon.1 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f798403c870 con 0x7f798c0cda80 2024-05-09T08:02:56.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.303+0000 7f329f400640 1 -- 172.21.15.1:0/1770594148 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f329402ae50 con 0x7f3298037a10 2024-05-09T08:02:56.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.303+0000 7f329f400640 1 -- 172.21.15.1:0/1770594148 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3294037070 con 0x7f3298037a10 2024-05-09T08:02:56.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.303+0000 7fd153e00640 1 --2- 172.21.15.1:0/2795097148 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fd1380039b0 0x7fd138003db0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fd14c001e50 tx=0x7fd138005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a7bc75bb6cbbb3f2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:56.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.303+0000 7f94da600640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f94d4157e60 0x7f94d40c6a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:56.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.303+0000 7f94da600640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f94d4150d00 0x7f94d40c7040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:56.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.304+0000 7fd152a00640 1 -- 172.21.15.1:0/2795097148 <== mon.2 v2:172.21.15.7:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd14c001a50 con 0x7fd1380039b0 2024-05-09T08:02:56.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.302+0000 7f7983e00640 1 -- 172.21.15.1:0/1143769927 <== mon.1 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7984037040 con 0x7f798c0cda80 2024-05-09T08:02:56.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.302+0000 7f7991600640 1 -- 172.21.15.1:0/1143769927 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f798c0b4f40 con 0x7f798c0cda80 2024-05-09T08:02:56.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.302+0000 7f7991600640 1 -- 172.21.15.1:0/1143769927 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f798c0b5500 con 0x7f798c0cda80 2024-05-09T08:02:56.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.304+0000 7f90d0e00640 1 Processor -- start 2024-05-09T08:02:56.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.304+0000 7f90d0e00640 1 -- start start 2024-05-09T08:02:56.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.304+0000 7f94da600640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f94d4072280 con 0x7f94d40c2fa0 2024-05-09T08:02:56.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.304+0000 7f94da600640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f94d406ff30 con 0x7f94d4157e60 2024-05-09T08:02:56.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.304+0000 7f94da600640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f94d4072f30 con 0x7f94d4150d00 2024-05-09T08:02:56.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.304+0000 7f94d3400640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f94d40c2fa0 0x7f94d406f820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.1:42142/0 (socket says 172.21.15.1:42142) 2024-05-09T08:02:56.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.304+0000 7f94d3400640 1 -- 172.21.15.1:0/3620975756 learned_addr learned my addr 172.21.15.1:0/3620975756 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T08:02:56.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.304+0000 7f94d3e00640 1 --2- 172.21.15.1:0/3620975756 >> v2:172.21.15.7:3300/0 conn(0x7f94d4150d00 0x7f94d40c7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:56.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.304+0000 7f94cbe00640 1 --2- 172.21.15.1:0/3620975756 >> v2:172.21.15.1:3300/0 conn(0x7f94d4157e60 0x7f94d40c6a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:56.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.304+0000 7fd152a00640 1 -- 172.21.15.1:0/2795097148 <== mon.2 v2:172.21.15.7:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd14c02ae20 con 0x7fd1380039b0 2024-05-09T08:02:56.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.305+0000 7f94d3e00640 1 -- 172.21.15.1:0/3620975756 >> v2:172.21.15.1:3300/0 conn(0x7f94d4157e60 msgr2=0x7f94d40c6a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:02:56.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.305+0000 7fd159a00640 1 -- 172.21.15.1:0/2795097148 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fd1380039b0 msgr2=0x7fd138003db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:56.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.305+0000 7fd159a00640 1 --2- 172.21.15.1:0/2795097148 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fd1380039b0 0x7fd138003db0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fd14c001e50 tx=0x7fd138005830 comp rx=0 tx=0).stop 2024-05-09T08:02:56.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.305+0000 7f94d2a00640 1 -- 172.21.15.1:0/3620975756 <== mon.1 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1550771779 0 0) 0x7f94d4072280 con 0x7f94d40c2fa0 2024-05-09T08:02:56.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.305+0000 7f94d3e00640 1 --2- 172.21.15.1:0/3620975756 >> v2:172.21.15.1:3300/0 conn(0x7f94d4157e60 0x7f94d40c6a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:56.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.305+0000 7f32a5e00640 1 -- 172.21.15.1:0/1770594148 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3298037a10 msgr2=0x7f32980e3980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:56.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.305+0000 7f94d3e00640 1 -- 172.21.15.1:0/3620975756 >> v1:172.21.15.7:6789/0 conn(0x7f94d40c2fa0 legacy=0x7f94d406f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:56.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.305+0000 7f94d3e00640 1 -- 172.21.15.1:0/3620975756 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94d4072560 con 0x7f94d4150d00 2024-05-09T08:02:56.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.305+0000 7fd159a00640 1 -- 172.21.15.1:0/2795097148 shutdown_connections 2024-05-09T08:02:56.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.305+0000 7fd159a00640 1 --2- 172.21.15.1:0/2795097148 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fd1380039b0 0x7fd138003db0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:56.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.305+0000 7fd159a00640 1 -- 172.21.15.1:0/2795097148 >> 172.21.15.1:0/2795097148 conn(0x7fd1540cfc20 msgr2=0x7fd1540d0030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:02:56.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.305+0000 7fd159a00640 1 -- 172.21.15.1:0/2795097148 shutdown_connections 2024-05-09T08:02:56.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.305+0000 7f32a5e00640 1 --2- 172.21.15.1:0/1770594148 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3298037a10 0x7f32980e3980 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f3298011c80 tx=0x7f329402a300 comp rx=0 tx=0).stop 2024-05-09T08:02:56.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.305+0000 7f32a5e00640 1 -- 172.21.15.1:0/1770594148 shutdown_connections 2024-05-09T08:02:56.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.305+0000 7f32a5e00640 1 --2- 172.21.15.1:0/1770594148 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3298037a10 0x7f32980e3980 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:56.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.305+0000 7f32a5e00640 1 --2- 172.21.15.1:0/1770594148 >> v2:172.21.15.7:3300/0 conn(0x7f3298037630 0x7f329804f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:56.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.305+0000 7f32a5e00640 1 -- 172.21.15.1:0/1770594148 >> 172.21.15.1:0/1770594148 conn(0x7f3298054610 msgr2=0x7f3298054a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:02:56.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.305+0000 7fd159a00640 1 -- 172.21.15.1:0/2795097148 wait complete. 2024-05-09T08:02:56.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.305+0000 7f94cbe00640 1 --2- 172.21.15.1:0/3620975756 >> v2:172.21.15.1:3300/0 conn(0x7f94d4157e60 0x7f94d40c6a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:02:56.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.305+0000 7fc276000640 1 --2- 172.21.15.1:0/258183096 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fc2700deda0 0x7fc270037160 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc2640097d0 tx=0x7fc26402a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5d1773f3691e84f9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:56.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.306+0000 7fc275600640 1 -- 172.21.15.1:0/258183096 <== mon.2 v2:172.21.15.7:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc264037070 con 0x7fc2700deda0 2024-05-09T08:02:56.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.306+0000 7fc275600640 1 -- 172.21.15.1:0/258183096 <== mon.2 v2:172.21.15.7:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc264032440 con 0x7fc2700deda0 2024-05-09T08:02:56.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.306+0000 7fc275600640 1 -- 172.21.15.1:0/258183096 <== mon.2 v2:172.21.15.7:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc264032810 con 0x7fc2700deda0 2024-05-09T08:02:56.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.306+0000 7f7983e00640 1 -- 172.21.15.1:0/1143769927 <== mon.1 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7984044070 con 0x7f798c0cda80 2024-05-09T08:02:56.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.306+0000 7f90d0e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f90cc0c6c50 0x7f90cc15e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:56.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.306+0000 7f90d0e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90cc0c7030 0x7f90cc0c9970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:56.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.306+0000 7f90d0e00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f90cc15c5c0 0x7f90cc0c9eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:56.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.307+0000 7fc277e00640 1 -- 172.21.15.1:0/258183096 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fc2700deda0 msgr2=0x7fc270037160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:56.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.307+0000 7fc277e00640 1 --2- 172.21.15.1:0/258183096 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fc2700deda0 0x7fc270037160 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc2640097d0 tx=0x7fc26402a210 comp rx=0 tx=0).stop 2024-05-09T08:02:56.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.307+0000 7f92bfe00640 1 Processor -- start 2024-05-09T08:02:56.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.307+0000 7f94d3e00640 1 --2- 172.21.15.1:0/3620975756 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f94d4150d00 0x7f94d40c7040 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f94d40c2490 tx=0x7f94cc02a730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f0232547664025a0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:56.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.307+0000 7fc277e00640 1 -- 172.21.15.1:0/258183096 shutdown_connections 2024-05-09T08:02:56.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.307+0000 7fc277e00640 1 --2- 172.21.15.1:0/258183096 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fc2700deda0 0x7fc270037160 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:56.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.307+0000 7fc277e00640 1 -- 172.21.15.1:0/258183096 >> 172.21.15.1:0/258183096 conn(0x7fc270054610 msgr2=0x7fc270054a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:02:56.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.307+0000 7f94d2a00640 1 -- 172.21.15.1:0/3620975756 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94cc007740 con 0x7f94d4150d00 2024-05-09T08:02:56.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.307+0000 7f94d2a00640 1 -- 172.21.15.1:0/3620975756 <== mon.1 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94cc0078e0 con 0x7f94d4150d00 2024-05-09T08:02:56.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.307+0000 7f94d2a00640 1 -- 172.21.15.1:0/3620975756 <== mon.1 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94cc007cf0 con 0x7f94d4150d00 2024-05-09T08:02:56.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.308+0000 7fd159a00640 1 Processor -- start 2024-05-09T08:02:56.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.307+0000 7f92bfe00640 1 -- start start 2024-05-09T08:02:56.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.308+0000 7f90caa00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90cc0c7030 0x7f90cc0c9970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:56.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.308+0000 7f90cb400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f90cc0c6c50 0x7f90cc15e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:56.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.308+0000 7f90cb400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f90cc0c6c50 0x7f90cc15e9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38260/0 (socket says 172.21.15.1:38260) 2024-05-09T08:02:56.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.308+0000 7f90caa00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90cc0c7030 0x7f90cc0c9970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.1:59054/0 (socket says 172.21.15.1:59054) 2024-05-09T08:02:56.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.308+0000 7f94da600640 1 -- 172.21.15.1:0/3620975756 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f94d4150d00 msgr2=0x7f94d40c7040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:56.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.308+0000 7f94da600640 1 --2- 172.21.15.1:0/3620975756 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f94d4150d00 0x7f94d40c7040 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f94d40c2490 tx=0x7f94cc02a730 comp rx=0 tx=0).stop 2024-05-09T08:02:56.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.308+0000 7f90cb400640 1 -- 172.21.15.1:0/3286634966 learned_addr learned my addr 172.21.15.1:0/3286634966 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:02:56.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.308+0000 7f94da600640 1 -- 172.21.15.1:0/3620975756 shutdown_connections 2024-05-09T08:02:56.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.308+0000 7f94da600640 1 --2- 172.21.15.1:0/3620975756 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f94d4150d00 0x7f94d40c7040 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:56.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.308+0000 7f94da600640 1 --2- 172.21.15.1:0/3620975756 >> v2:172.21.15.1:3300/0 conn(0x7f94d4157e60 0x7f94d40c6a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:56.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.308+0000 7f94da600640 1 -- 172.21.15.1:0/3620975756 >> 172.21.15.1:0/3620975756 conn(0x7f94d406dad0 msgr2=0x7f94d406deb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:02:56.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.307+0000 7f32a5e00640 1 -- 172.21.15.1:0/1770594148 shutdown_connections 2024-05-09T08:02:56.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.307+0000 7f32a5e00640 1 -- 172.21.15.1:0/1770594148 wait complete. 2024-05-09T08:02:56.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.309+0000 7f90d0e00640 1 -- 172.21.15.1:0/3286634966 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f90cc0c7740 con 0x7f90cc0c6c50 2024-05-09T08:02:56.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.307+0000 7f7991600640 1 -- 172.21.15.1:0/1143769927 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7964005290 con 0x7f798c0cda80 2024-05-09T08:02:56.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.309+0000 7f7983e00640 1 --2- 172.21.15.1:0/1143769927 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f796806be80 0x7f796806e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:56.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.309+0000 7f7983e00640 1 -- 172.21.15.1:0/1143769927 <== mon.1 v2:172.21.15.7:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f79840ab9f0 con 0x7f798c0cda80 2024-05-09T08:02:56.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.309+0000 7f90d0e00640 1 -- 172.21.15.1:0/3286634966 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f90cc06f6b0 con 0x7f90cc0c7030 2024-05-09T08:02:56.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.309+0000 7f90d0e00640 1 -- 172.21.15.1:0/3286634966 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f90cc06fbd0 con 0x7f90cc15c5c0 2024-05-09T08:02:56.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.309+0000 7f92bfe00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f92c00cc5f0 0x7f92c015a570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:56.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.309+0000 7f92bfe00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f92c00ce4e0 0x7f92c015aab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:56.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.308+0000 7fd159a00640 1 -- start start 2024-05-09T08:02:56.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.309+0000 7fd159a00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd1380039b0 0x7fd15414e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:56.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.309+0000 7fd159a00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fd154150310 0x7fd1541efbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:56.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.310+0000 7fd159a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd154150790 0x7fd1541f0100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:56.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.310+0000 7fd159a00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fd1540ce4e0 con 0x7fd1380039b0 2024-05-09T08:02:56.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.310+0000 7fd159a00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd1540cd550 con 0x7fd154150790 2024-05-09T08:02:56.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.310+0000 7fd159a00640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7fd1540ce660 con 0x7fd154150310 2024-05-09T08:02:56.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.309+0000 7f798b400640 1 --2- 172.21.15.1:0/1143769927 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f796806be80 0x7f796806e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:56.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.309+0000 7f32a5e00640 1 Processor -- start 2024-05-09T08:02:56.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.308+0000 7fc277e00640 1 -- 172.21.15.1:0/258183096 shutdown_connections 2024-05-09T08:02:56.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.308+0000 7fc277e00640 1 -- 172.21.15.1:0/258183096 wait complete. 2024-05-09T08:02:56.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.309+0000 7f94da600640 1 -- 172.21.15.1:0/3620975756 shutdown_connections 2024-05-09T08:02:56.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.310+0000 7f94da600640 1 -- 172.21.15.1:0/3620975756 wait complete. 2024-05-09T08:02:56.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.309+0000 7f92bfe00640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f92c00ccb20 con 0x7f92c00cac60 2024-05-09T08:02:56.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.309+0000 7f92bfe00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f92c00cce00 con 0x7f92c00cc5f0 2024-05-09T08:02:56.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.309+0000 7f92bfe00640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f92c00cf120 con 0x7f92c00ce4e0 2024-05-09T08:02:56.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.309+0000 7f92be000640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f92c00cc5f0 0x7f92c015a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:56.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.309+0000 7f92bf400640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f92c00ce4e0 0x7f92c015aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:56.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.310+0000 7f92bf400640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f92c00ce4e0 0x7f92c015aab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3301/0 says I am v2:172.21.15.1:44692/0 (socket says 172.21.15.1:44692) 2024-05-09T08:02:56.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.310+0000 7f92bf400640 1 -- 172.21.15.1:0/2959884947 learned_addr learned my addr 172.21.15.1:0/2959884947 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:02:56.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.310+0000 7f32a5e00640 1 -- start start 2024-05-09T08:02:56.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.310+0000 7f798b400640 1 --2- 172.21.15.1:0/1143769927 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f796806be80 0x7f796806e340 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f798c157f80 tx=0x7f7974008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:56.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.310+0000 7fd153e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd1380039b0 0x7fd15414e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:56.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.310+0000 7fd153400640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fd154150310 0x7fd1541efbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:56.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.310+0000 7fd153e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd1380039b0 0x7fd15414e160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38262/0 (socket says 172.21.15.1:38262) 2024-05-09T08:02:56.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.310+0000 7fd153e00640 1 -- 172.21.15.1:0/1223524150 learned_addr learned my addr 172.21.15.1:0/1223524150 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:02:56.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.311+0000 7fd159000640 1 --2- 172.21.15.1:0/1223524150 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd154150790 0x7fd1541f0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:56.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.311+0000 7fd153e00640 1 -- 172.21.15.1:0/1223524150 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fd154150310 msgr2=0x7fd1541efbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:56.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.311+0000 7fd153e00640 1 --2- 172.21.15.1:0/1223524150 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fd154150310 0x7fd1541efbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:56.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.311+0000 7fd153e00640 1 -- 172.21.15.1:0/1223524150 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd154150790 msgr2=0x7fd1541f0100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:56.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.311+0000 7fc277e00640 1 Processor -- start 2024-05-09T08:02:56.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.311+0000 7fc277e00640 1 -- start start 2024-05-09T08:02:56.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.311+0000 7fd153e00640 1 --2- 172.21.15.1:0/1223524150 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd154150790 0x7fd1541f0100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:56.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.311+0000 7fd153e00640 1 -- 172.21.15.1:0/1223524150 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1540b2920 con 0x7fd1380039b0 2024-05-09T08:02:56.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.311+0000 7f32a5e00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f3298007810 0x7f329804dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:56.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.311+0000 7f32a5e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3298037630 0x7f329804e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:56.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.311+0000 7f32a5e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3298037a10 0x7f32980d55c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:56.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.310+0000 7f90cbe00640 1 --2- 172.21.15.1:0/3286634966 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f90cc15c5c0 0x7f90cc0c9eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:56.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.312+0000 7f32a5e00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f3298006a50 con 0x7f3298037630 2024-05-09T08:02:56.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.310+0000 7f92bf400640 1 -- 172.21.15.1:0/2959884947 >> v2:172.21.15.7:3300/0 conn(0x7f92c00cc5f0 msgr2=0x7f92c015a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:02:56.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.310+0000 7f92bf400640 1 --2- 172.21.15.1:0/2959884947 >> v2:172.21.15.7:3300/0 conn(0x7f92c00cc5f0 0x7f92c015a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:56.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.310+0000 7f92bf400640 1 -- 172.21.15.1:0/2959884947 >> v1:172.21.15.1:6789/0 conn(0x7f92c00cac60 legacy=0x7f92c00cda00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T08:02:56.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.312+0000 7f32a5e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f3298006770 con 0x7f3298037a10 2024-05-09T08:02:56.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.310+0000 7f92bf400640 1 -- 172.21.15.1:0/2959884947 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92c00cf400 con 0x7f92c00ce4e0 2024-05-09T08:02:56.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.312+0000 7f32a5e00640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f3298006d30 con 0x7f3298007810 2024-05-09T08:02:56.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.312+0000 7f32a5400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3298037a10 0x7f32980d55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:56.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.312+0000 7f32a5400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3298037a10 0x7f32980d55c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.1:59072/0 (socket says 172.21.15.1:59072) 2024-05-09T08:02:56.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.312+0000 7f32a5400640 1 -- 172.21.15.1:0/879103317 learned_addr learned my addr 172.21.15.1:0/879103317 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:02:56.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.312+0000 7f329fe00640 1 --2- 172.21.15.1:0/879103317 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3298037630 0x7f329804e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:56.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.312+0000 7f90cb400640 1 -- 172.21.15.1:0/3286634966 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f90cc15c5c0 msgr2=0x7f90cc0c9eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:56.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.312+0000 7fd153e00640 1 --2- 172.21.15.1:0/1223524150 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd1380039b0 0x7fd15414e160 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fd14c001e50 tx=0x7fd14c001440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:56.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.312+0000 7f90cb400640 1 --2- 172.21.15.1:0/3286634966 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f90cc15c5c0 0x7f90cc0c9eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:56.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.312+0000 7f90cb400640 1 -- 172.21.15.1:0/3286634966 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90cc0c7030 msgr2=0x7f90cc0c9970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:56.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.312+0000 7f90cb400640 1 --2- 172.21.15.1:0/3286634966 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90cc0c7030 0x7f90cc0c9970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:56.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.312+0000 7f90cb400640 1 -- 172.21.15.1:0/3286634966 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90cc154c50 con 0x7f90cc0c6c50 2024-05-09T08:02:56.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.312+0000 7f94da600640 1 Processor -- start 2024-05-09T08:02:56.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.312+0000 7f32a5400640 1 -- 172.21.15.1:0/879103317 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f3298007810 msgr2=0x7f329804dfa0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T08:02:56.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.313+0000 7f94da600640 1 -- start start 2024-05-09T08:02:56.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.313+0000 7f92bf400640 1 --2- 172.21.15.1:0/2959884947 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f92c00ce4e0 0x7f92c015aab0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f92b8009e50 tx=0x7f92b802a750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5f480422e4f08373 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:56.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.313+0000 7f32a5400640 1 --2- 172.21.15.1:0/879103317 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f3298007810 0x7f329804dfa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:56.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.313+0000 7f32a5400640 1 -- 172.21.15.1:0/879103317 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3298037630 msgr2=0x7f329804e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:56.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.313+0000 7f32a5400640 1 --2- 172.21.15.1:0/879103317 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3298037630 0x7f329804e4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:56.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.313+0000 7f32a5400640 1 -- 172.21.15.1:0/879103317 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f329804db80 con 0x7f3298037a10 2024-05-09T08:02:56.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.313+0000 7f329fe00640 1 --2- 172.21.15.1:0/879103317 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3298037630 0x7f329804e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T08:02:56.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.313+0000 7f90cb400640 1 --2- 172.21.15.1:0/3286634966 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f90cc0c6c50 0x7f90cc15e9e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f90b400c780 tx=0x7f90b400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:56.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.313+0000 7f32a5400640 1 --2- 172.21.15.1:0/879103317 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3298037a10 0x7f32980d55c0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f32940090b0 tx=0x7f32940046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:56.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.313+0000 7fd14be00640 1 -- 172.21.15.1:0/1223524150 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd14c007df0 con 0x7fd1380039b0 2024-05-09T08:02:56.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.313+0000 7fd14be00640 1 -- 172.21.15.1:0/1223524150 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd14c041070 con 0x7fd1380039b0 2024-05-09T08:02:56.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.313+0000 7f92bd600640 1 -- 172.21.15.1:0/2959884947 <== mon.2 v2:172.21.15.7:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92b8036070 con 0x7f92c00ce4e0 2024-05-09T08:02:56.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.313+0000 7f90c3400640 1 -- 172.21.15.1:0/3286634966 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90b4017070 con 0x7f90cc0c6c50 2024-05-09T08:02:56.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.313+0000 7f92bd600640 1 -- 172.21.15.1:0/2959884947 <== mon.2 v2:172.21.15.7:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92b8031600 con 0x7f92c00ce4e0 2024-05-09T08:02:56.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.313+0000 7f92bd600640 1 -- 172.21.15.1:0/2959884947 <== mon.2 v2:172.21.15.7:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92b80319d0 con 0x7f92c00ce4e0 2024-05-09T08:02:56.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.313+0000 7fc277e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc2700376a0 0x7fc27004fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:56.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.314+0000 7f90c3400640 1 -- 172.21.15.1:0/3286634966 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90b4005290 con 0x7f90cc0c6c50 2024-05-09T08:02:56.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.314+0000 7f90c3400640 1 -- 172.21.15.1:0/3286634966 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90b400a880 con 0x7f90cc0c6c50 2024-05-09T08:02:56.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.314+0000 7f90d0e00640 1 -- 172.21.15.1:0/3286634966 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90cc153a10 con 0x7f90cc0c6c50 2024-05-09T08:02:56.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.314+0000 7fc277e00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fc2700d9090 0x7fc2700d18f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:56.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.314+0000 7f90d0e00640 1 -- 172.21.15.1:0/3286634966 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90cc153cb0 con 0x7f90cc0c6c50 2024-05-09T08:02:56.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.314+0000 7fc277e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc2700deda0 0x7fc2700d1e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:56.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.314+0000 7fc277e00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fc2700d9470 con 0x7fc2700376a0 2024-05-09T08:02:56.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.314+0000 7f329cc00640 1 -- 172.21.15.1:0/879103317 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3294041020 con 0x7f3298037a10 2024-05-09T08:02:56.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.314+0000 7fc277e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc270037d20 con 0x7fc2700deda0 2024-05-09T08:02:56.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.314+0000 7fc277e00640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7fc270050890 con 0x7fc2700d9090 2024-05-09T08:02:56.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.315+0000 7fc276000640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fc2700d9090 0x7fc2700d18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:56.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.315+0000 7fc276000640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fc2700d9090 0x7fc2700d18f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3301/0 says I am v2:172.21.15.1:44720/0 (socket says 172.21.15.1:44720) 2024-05-09T08:02:56.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.315+0000 7fc276000640 1 -- 172.21.15.1:0/4090439372 learned_addr learned my addr 172.21.15.1:0/4090439372 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:02:56.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.315+0000 7fc276a00640 1 --2- 172.21.15.1:0/4090439372 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc2700376a0 0x7fc27004fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:56.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.315+0000 7f92bfe00640 1 -- 172.21.15.1:0/2959884947 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f92c00ce4e0 msgr2=0x7f92c015aab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:56.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.315+0000 7f92bfe00640 1 --2- 172.21.15.1:0/2959884947 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f92c00ce4e0 0x7f92c015aab0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f92b8009e50 tx=0x7f92b802a750 comp rx=0 tx=0).stop 2024-05-09T08:02:56.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.315+0000 7f7983e00640 1 -- 172.21.15.1:0/1143769927 <== mon.1 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f798404b030 con 0x7f798c0cda80 2024-05-09T08:02:56.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.313+0000 7fd14be00640 1 -- 172.21.15.1:0/1223524150 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd14c005710 con 0x7fd1380039b0 2024-05-09T08:02:56.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.314+0000 7fd159a00640 1 -- 172.21.15.1:0/1223524150 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1540b2be0 con 0x7fd1380039b0 2024-05-09T08:02:56.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.314+0000 7fd159a00640 1 -- 172.21.15.1:0/1223524150 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1540b31a0 con 0x7fd1380039b0 2024-05-09T08:02:56.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.315+0000 7f90d0e00640 1 -- 172.21.15.1:0/3286634966 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90a4005290 con 0x7f90cc0c6c50 2024-05-09T08:02:56.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.316+0000 7fc277400640 1 --2- 172.21.15.1:0/4090439372 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc2700deda0 0x7fc2700d1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:56.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.314+0000 7f94da600640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f94d40c2fa0 0x7f94d414c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:56.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.314+0000 7f94da600640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f94d4150d00 0x7f94d40c33e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:56.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.314+0000 7f94da600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f94d4157e60 0x7f94d40c3920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:56.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.315+0000 7f94da600640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f94d4156fa0 con 0x7f94d4150d00 2024-05-09T08:02:56.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.315+0000 7f94da600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f94d4156cc0 con 0x7f94d4157e60 2024-05-09T08:02:56.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.315+0000 7f94da600640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f94d4156970 con 0x7f94d40c2fa0 2024-05-09T08:02:56.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.316+0000 7f94d3e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f94d4157e60 0x7f94d40c3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:56.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.316+0000 7f94d3e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f94d4157e60 0x7f94d40c3920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.1:59090/0 (socket says 172.21.15.1:59090) 2024-05-09T08:02:56.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.316+0000 7f94d3e00640 1 -- 172.21.15.1:0/3662576646 learned_addr learned my addr 172.21.15.1:0/3662576646 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:02:56.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.315+0000 7f92bfe00640 1 -- 172.21.15.1:0/2959884947 shutdown_connections 2024-05-09T08:02:56.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.315+0000 7f92bfe00640 1 --2- 172.21.15.1:0/2959884947 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f92c00ce4e0 0x7f92c015aab0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:56.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.315+0000 7f92bfe00640 1 --2- 172.21.15.1:0/2959884947 >> v2:172.21.15.7:3300/0 conn(0x7f92c00cc5f0 0x7f92c015a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:56.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.315+0000 7f92bfe00640 1 -- 172.21.15.1:0/2959884947 >> 172.21.15.1:0/2959884947 conn(0x7f92c00cfc20 msgr2=0x7f92c00d0030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:02:56.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.316+0000 7f32a5e00640 1 -- 172.21.15.1:0/879103317 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3298178e80 con 0x7f3298037a10 2024-05-09T08:02:56.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.316+0000 7f32a5e00640 1 -- 172.21.15.1:0/879103317 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3298179430 con 0x7f3298037a10 2024-05-09T08:02:56.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.316+0000 7f329cc00640 1 -- 172.21.15.1:0/879103317 <== mon.1 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3294003c60 con 0x7f3298037a10 2024-05-09T08:02:56.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.316+0000 7f329cc00640 1 -- 172.21.15.1:0/879103317 <== mon.1 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3294037040 con 0x7f3298037a10 2024-05-09T08:02:56.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.316+0000 7f94d3400640 1 --2- 172.21.15.1:0/3662576646 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f94d40c2fa0 0x7f94d414c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:56.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.317+0000 7fc277400640 1 -- 172.21.15.1:0/4090439372 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fc2700d9090 msgr2=0x7fc2700d18f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:56.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.317+0000 7fc277400640 1 --2- 172.21.15.1:0/4090439372 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fc2700d9090 0x7fc2700d18f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:56.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.319+0000 7fd159a00640 1 -- 172.21.15.1:0/1223524150 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd154068ab0 con 0x7fd1380039b0 2024-05-09T08:02:56.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.317+0000 7f32a5e00640 1 -- 172.21.15.1:0/879103317 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f327c005290 con 0x7f3298037a10 2024-05-09T08:02:56.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.317+0000 7fc277400640 1 -- 172.21.15.1:0/4090439372 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc2700376a0 msgr2=0x7fc27004fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:56.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.317+0000 7fc277400640 1 --2- 172.21.15.1:0/4090439372 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc2700376a0 0x7fc27004fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:56.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.317+0000 7fc277400640 1 -- 172.21.15.1:0/4090439372 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2700dc6d0 con 0x7fc2700deda0 2024-05-09T08:02:56.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.317+0000 7f94cbe00640 1 --2- 172.21.15.1:0/3662576646 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f94d4150d00 0x7f94d40c33e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:56.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.317+0000 7f94d3e00640 1 -- 172.21.15.1:0/3662576646 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f94d40c2fa0 msgr2=0x7f94d414c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:56.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.317+0000 7f94d3e00640 1 --2- 172.21.15.1:0/3662576646 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f94d40c2fa0 0x7f94d414c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:56.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.317+0000 7f94d3e00640 1 -- 172.21.15.1:0/3662576646 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f94d4150d00 msgr2=0x7f94d40c33e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:56.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.318+0000 7f94d3e00640 1 --2- 172.21.15.1:0/3662576646 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f94d4150d00 0x7f94d40c33e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:56.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.318+0000 7f94d3e00640 1 -- 172.21.15.1:0/3662576646 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94d40c4ec0 con 0x7f94d4157e60 2024-05-09T08:02:56.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.318+0000 7f94d3400640 1 --2- 172.21.15.1:0/3662576646 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f94d40c2fa0 0x7f94d414c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:02:56.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.318+0000 7f94cbe00640 1 --2- 172.21.15.1:0/3662576646 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f94d4150d00 0x7f94d40c33e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T08:02:56.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.318+0000 7f94d3e00640 1 --2- 172.21.15.1:0/3662576646 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f94d4157e60 0x7f94d40c3920 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f94cc009e80 tx=0x7f94cc004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:56.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.319+0000 7f94cb400640 1 -- 172.21.15.1:0/3662576646 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94cc040030 con 0x7f94d4157e60 2024-05-09T08:02:56.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.319+0000 7f94cb400640 1 -- 172.21.15.1:0/3662576646 <== mon.1 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94cc0318c0 con 0x7f94d4157e60 2024-05-09T08:02:56.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.319+0000 7f94da600640 1 -- 172.21.15.1:0/3662576646 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94d40c8de0 con 0x7f94d4157e60 2024-05-09T08:02:56.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.316+0000 7f92bfe00640 1 -- 172.21.15.1:0/2959884947 shutdown_connections 2024-05-09T08:02:56.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.317+0000 7f92bfe00640 1 -- 172.21.15.1:0/2959884947 wait complete. 2024-05-09T08:02:56.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.317+0000 7fc276a00640 1 --2- 172.21.15.1:0/4090439372 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc2700376a0 0x7fc27004fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T08:02:56.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.319+0000 7f94da600640 1 -- 172.21.15.1:0/3662576646 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94d40c5380 con 0x7f94d4157e60 2024-05-09T08:02:56.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.319+0000 7f94cb400640 1 -- 172.21.15.1:0/3662576646 <== mon.1 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94cc031c70 con 0x7f94d4157e60 2024-05-09T08:02:56.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.320+0000 7f92bfe00640 1 Processor -- start 2024-05-09T08:02:56.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.321+0000 7fc277400640 1 --2- 172.21.15.1:0/4090439372 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc2700deda0 0x7fc2700d1e30 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fc27806a1a0 tx=0x7fc278068e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:56.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.321+0000 7f92bfe00640 1 -- start start 2024-05-09T08:02:56.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.320+0000 7f90c3400640 1 -- 172.21.15.1:0/3286634966 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f90b400aa20 con 0x7f90cc0c6c50 2024-05-09T08:02:56.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.321+0000 7f90c3400640 1 --2- 172.21.15.1:0/3286634966 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f90a806be30 0x7f90a806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:56.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.319+0000 7fd14be00640 1 -- 172.21.15.1:0/1223524150 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd14c046460 con 0x7fd1380039b0 2024-05-09T08:02:56.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.321+0000 7fd14be00640 1 --2- 172.21.15.1:0/1223524150 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fd13006bf00 0x7fd13006e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:56.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.321+0000 7fd14be00640 1 -- 172.21.15.1:0/1223524150 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fd14c0ab6c0 con 0x7fd1380039b0 2024-05-09T08:02:56.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.321+0000 7f90c3400640 1 -- 172.21.15.1:0/3286634966 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f90b408a8a0 con 0x7f90cc0c6c50 2024-05-09T08:02:56.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.319+0000 7f329cc00640 1 -- 172.21.15.1:0/879103317 <== mon.1 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3294044020 con 0x7f3298037a10 2024-05-09T08:02:56.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.320+0000 7f329cc00640 1 --2- 172.21.15.1:0/879103317 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f328006be30 0x7f328006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:56.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.321+0000 7f329cc00640 1 -- 172.21.15.1:0/879103317 <== mon.1 v2:172.21.15.7:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f32940ab6f0 con 0x7f3298037a10 2024-05-09T08:02:56.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.322+0000 7fd153400640 1 --2- 172.21.15.1:0/1223524150 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fd13006bf00 0x7fd13006e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:56.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.322+0000 7f32a4a00640 1 --2- 172.21.15.1:0/879103317 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f328006be30 0x7f328006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:56.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.322+0000 7f90caa00640 1 --2- 172.21.15.1:0/3286634966 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f90a806be30 0x7f90a806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:56.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.323+0000 7f32a4a00640 1 --2- 172.21.15.1:0/879103317 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f328006be30 0x7f328006e2f0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f328c0099f0 tx=0x7f328c009110 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:56.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.323+0000 7fd153400640 1 --2- 172.21.15.1:0/1223524150 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fd13006bf00 0x7fd13006e3c0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fd1541e9c70 tx=0x7fd13c009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:56.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.323+0000 7f94da600640 1 -- 172.21.15.1:0/3662576646 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94ac005290 con 0x7f94d4157e60 2024-05-09T08:02:56.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.323+0000 7f92bfe00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f92c00cac60 0x7f92c0155e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:56.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.323+0000 7f92bfe00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f92c00cc5f0 0x7f92c01e5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:56.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.323+0000 7f92bea00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f92c00cac60 0x7f92c0155e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:56.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.323+0000 7f92bea00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f92c00cac60 0x7f92c0155e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3301/0 says I am v2:172.21.15.1:44736/0 (socket says 172.21.15.1:44736) 2024-05-09T08:02:56.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.324+0000 7f92bea00640 1 -- 172.21.15.1:0/3659492570 learned_addr learned my addr 172.21.15.1:0/3659492570 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:02:56.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.324+0000 7f92bfe00640 1 --2- 172.21.15.1:0/3659492570 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f92c00ce4e0 0x7f92c0150290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:56.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.324+0000 7f92bfe00640 1 -- 172.21.15.1:0/3659492570 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f92c00ce110 con 0x7f92c00cc5f0 2024-05-09T08:02:56.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.324+0000 7f92be000640 1 --2- 172.21.15.1:0/3659492570 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f92c00cc5f0 0x7f92c01e5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:56.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.324+0000 7f92bf400640 1 --2- 172.21.15.1:0/3659492570 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f92c00ce4e0 0x7f92c0150290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:56.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.324+0000 7f90caa00640 1 --2- 172.21.15.1:0/3286634966 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f90a806be30 0x7f90a806e2f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f90bc034000 tx=0x7f90bc02e000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:56.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.324+0000 7f92bfe00640 1 -- 172.21.15.1:0/3659492570 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f92c00cd550 con 0x7f92c00ce4e0 2024-05-09T08:02:56.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.325+0000 7f92bfe00640 1 -- 172.21.15.1:0/3659492570 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f92c00ce290 con 0x7f92c00cac60 2024-05-09T08:02:56.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.325+0000 7fc26ea00640 1 -- 172.21.15.1:0/4090439372 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc27806e650 con 0x7fc2700deda0 2024-05-09T08:02:56.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.325+0000 7fc26ea00640 1 -- 172.21.15.1:0/4090439372 <== mon.1 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc27806e830 con 0x7fc2700deda0 2024-05-09T08:02:56.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.325+0000 7fc26ea00640 1 -- 172.21.15.1:0/4090439372 <== mon.1 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc27806ec60 con 0x7fc2700deda0 2024-05-09T08:02:56.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.325+0000 7fc277e00640 1 -- 172.21.15.1:0/4090439372 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2701792b0 con 0x7fc2700deda0 2024-05-09T08:02:56.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.325+0000 7f329cc00640 1 -- 172.21.15.1:0/879103317 <== mon.1 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f329404b030 con 0x7f3298037a10 2024-05-09T08:02:56.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.325+0000 7fc277e00640 1 -- 172.21.15.1:0/4090439372 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc270179870 con 0x7fc2700deda0 2024-05-09T08:02:56.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.326+0000 7f92bf400640 1 -- 172.21.15.1:0/3659492570 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f92c00cac60 msgr2=0x7f92c0155e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:56.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.326+0000 7f92bf400640 1 --2- 172.21.15.1:0/3659492570 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f92c00cac60 0x7f92c0155e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:56.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.326+0000 7f92bf400640 1 -- 172.21.15.1:0/3659492570 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f92c00cc5f0 msgr2=0x7f92c01e5cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:56.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.326+0000 7f92bf400640 1 --2- 172.21.15.1:0/3659492570 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f92c00cc5f0 0x7f92c01e5cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:56.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.327+0000 7f92bf400640 1 -- 172.21.15.1:0/3659492570 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92c0156450 con 0x7f92c00ce4e0 2024-05-09T08:02:56.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.327+0000 7f92be000640 1 --2- 172.21.15.1:0/3659492570 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f92c00cc5f0 0x7f92c01e5cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:02:56.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.327+0000 7f90c3400640 1 -- 172.21.15.1:0/3286634966 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f90b4020020 con 0x7f90cc0c6c50 2024-05-09T08:02:56.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.327+0000 7f92bf400640 1 --2- 172.21.15.1:0/3659492570 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f92c00ce4e0 0x7f92c0150290 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f92b8030000 tx=0x7f92b8004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:56.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.327+0000 7fc277e00640 1 -- 172.21.15.1:0/4090439372 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2700052c0 con 0x7fc2700deda0 2024-05-09T08:02:56.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.328+0000 7f94cb400640 1 -- 172.21.15.1:0/3662576646 <== mon.1 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f94cc043030 con 0x7f94d4157e60 2024-05-09T08:02:56.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.330+0000 7f92b7400640 1 -- 172.21.15.1:0/3659492570 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92b8041020 con 0x7f92c00ce4e0 2024-05-09T08:02:56.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.330+0000 7f92b7400640 1 -- 172.21.15.1:0/3659492570 <== mon.1 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92b8007720 con 0x7f92c00ce4e0 2024-05-09T08:02:56.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.330+0000 7f92b7400640 1 -- 172.21.15.1:0/3659492570 <== mon.1 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92b8036040 con 0x7f92c00ce4e0 2024-05-09T08:02:56.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.331+0000 7f92bfe00640 1 -- 172.21.15.1:0/3659492570 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92c014f4d0 con 0x7f92c00ce4e0 2024-05-09T08:02:56.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.331+0000 7f92bfe00640 1 -- 172.21.15.1:0/3659492570 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92c014fa90 con 0x7f92c00ce4e0 2024-05-09T08:02:56.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.332+0000 7fc26ea00640 1 -- 172.21.15.1:0/4090439372 <== mon.1 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc27806d450 con 0x7fc2700deda0 2024-05-09T08:02:56.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.333+0000 7f92bfe00640 1 -- 172.21.15.1:0/3659492570 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9298005290 con 0x7f92c00ce4e0 2024-05-09T08:02:56.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.333+0000 7f94cb400640 1 --2- 172.21.15.1:0/3662576646 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f94b006bf00 0x7f94b006e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:56.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.333+0000 7f94cb400640 1 -- 172.21.15.1:0/3662576646 <== mon.1 v2:172.21.15.7:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f94cc0ac320 con 0x7f94d4157e60 2024-05-09T08:02:56.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.333+0000 7fc26ea00640 1 --2- 172.21.15.1:0/4090439372 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fc25806bf00 0x7fc25806e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:56.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.334+0000 7fc26ea00640 1 -- 172.21.15.1:0/4090439372 <== mon.1 v2:172.21.15.7:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fc278106940 con 0x7fc2700deda0 2024-05-09T08:02:56.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.334+0000 7f94cb400640 1 -- 172.21.15.1:0/3662576646 <== mon.1 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f94d40c4ec0 con 0x7f94d4157e60 2024-05-09T08:02:56.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.334+0000 7fc276a00640 1 --2- 172.21.15.1:0/4090439372 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fc25806bf00 0x7fc25806e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:56.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.334+0000 7f94d3400640 1 --2- 172.21.15.1:0/3662576646 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f94b006bf00 0x7f94b006e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:56.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.335+0000 7fd14be00640 1 -- 172.21.15.1:0/1223524150 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fd154068ab0 con 0x7fd1380039b0 2024-05-09T08:02:56.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.335+0000 7fc26ea00640 1 -- 172.21.15.1:0/4090439372 <== mon.1 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fc2780d49d0 con 0x7fc2700deda0 2024-05-09T08:02:56.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.336+0000 7f94d3400640 1 --2- 172.21.15.1:0/3662576646 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f94b006bf00 0x7f94b006e3c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f94c4000f60 tx=0x7f94c4007e40 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:56.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.337+0000 7f92b7400640 1 -- 172.21.15.1:0/3659492570 <== mon.1 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f92b8044020 con 0x7f92c00ce4e0 2024-05-09T08:02:56.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.336+0000 7fc276a00640 1 --2- 172.21.15.1:0/4090439372 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fc25806bf00 0x7fc25806e3c0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fc268001730 tx=0x7fc268009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:56.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.338+0000 7f92b7400640 1 --2- 172.21.15.1:0/3659492570 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f92a006beb0 0x7f92a006e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:56.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.338+0000 7f92b7400640 1 -- 172.21.15.1:0/3659492570 <== mon.1 v2:172.21.15.7:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f92b80abfb0 con 0x7f92c00ce4e0 2024-05-09T08:02:56.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.339+0000 7f92bea00640 1 --2- 172.21.15.1:0/3659492570 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f92a006beb0 0x7f92a006e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:56.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.340+0000 7f92b7400640 1 -- 172.21.15.1:0/3659492570 <== mon.1 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f92b80da2d0 con 0x7f92c00ce4e0 2024-05-09T08:02:56.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.340+0000 7f92bea00640 1 --2- 172.21.15.1:0/3659492570 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f92a006beb0 0x7f92a006e370 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f92ac009a30 tx=0x7f92ac009190 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:56.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.721+0000 7f546da00640 1 -- 172.21.15.1:0/2764213648 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f5440005580 con 0x7f54680724b0 2024-05-09T08:02:56.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.722+0000 7f545ea00640 1 -- 172.21.15.1:0/2764213648 <== mon.1 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f545007d770 con 0x7f54680724b0 2024-05-09T08:02:56.739 INFO:teuthology.orchestra.run.smithi001.stdout:55834574860 2024-05-09T08:02:56.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.725+0000 7f546da00640 1 -- 172.21.15.1:0/2764213648 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f544406c030 msgr2=0x7f544406e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:56.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.725+0000 7f546da00640 1 --2- 172.21.15.1:0/2764213648 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f544406c030 0x7f544406e4f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f546812d940 tx=0x7f5458009190 comp rx=0 tx=0).stop 2024-05-09T08:02:56.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.726+0000 7f546da00640 1 -- 172.21.15.1:0/2764213648 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54680724b0 msgr2=0x7f54681ea180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:56.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.726+0000 7f546da00640 1 --2- 172.21.15.1:0/2764213648 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54680724b0 0x7f54681ea180 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f5450009870 tx=0x7f54500046c0 comp rx=0 tx=0).stop 2024-05-09T08:02:56.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.726+0000 7f546da00640 1 -- 172.21.15.1:0/2764213648 shutdown_connections 2024-05-09T08:02:56.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.726+0000 7f546da00640 1 --2- 172.21.15.1:0/2764213648 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f544406c030 0x7f544406e4f0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:56.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.727+0000 7f546da00640 1 --2- 172.21.15.1:0/2764213648 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f5468158e20 0x7f54680cb670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:56.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.727+0000 7f546da00640 1 --2- 172.21.15.1:0/2764213648 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f54680c7e00 0x7f54680cb130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:56.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.727+0000 7f546da00640 1 --2- 172.21.15.1:0/2764213648 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54680724b0 0x7f54681ea180 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:56.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.727+0000 7f546da00640 1 -- 172.21.15.1:0/2764213648 >> 172.21.15.1:0/2764213648 conn(0x7f546806dad0 msgr2=0x7f546815d4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:02:56.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.727+0000 7f546da00640 1 -- 172.21.15.1:0/2764213648 shutdown_connections 2024-05-09T08:02:56.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.727+0000 7f546da00640 1 -- 172.21.15.1:0/2764213648 wait complete. 2024-05-09T08:02:56.771 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574860 got 55834574860 for osd.4 2024-05-09T08:02:56.771 DEBUG:teuthology.parallel:result is None 2024-05-09T08:02:56.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.765+0000 7f7991600640 1 -- 172.21.15.1:0/1143769927 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f7964005580 con 0x7f798c0cda80 2024-05-09T08:02:56.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.766+0000 7f7983e00640 1 -- 172.21.15.1:0/1143769927 <== mon.1 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f7984030e20 con 0x7f798c0cda80 2024-05-09T08:02:56.783 INFO:teuthology.orchestra.run.smithi001.stdout:55834574860 2024-05-09T08:02:56.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.767+0000 7f94da600640 1 -- 172.21.15.1:0/3662576646 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f94ac005850 con 0x7f94d4157e60 2024-05-09T08:02:56.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.768+0000 7f94cb400640 1 -- 172.21.15.1:0/3662576646 <== mon.1 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f94cc07a290 con 0x7f94d4157e60 2024-05-09T08:02:56.785 INFO:teuthology.orchestra.run.smithi001.stdout:55834574859 2024-05-09T08:02:56.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.769+0000 7f7991600640 1 -- 172.21.15.1:0/1143769927 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f796806be80 msgr2=0x7f796806e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:56.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.769+0000 7f7991600640 1 --2- 172.21.15.1:0/1143769927 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f796806be80 0x7f796806e340 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f798c157f80 tx=0x7f7974008040 comp rx=0 tx=0).stop 2024-05-09T08:02:56.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.769+0000 7f7991600640 1 -- 172.21.15.1:0/1143769927 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f798c0cda80 msgr2=0x7f798c154500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:56.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.769+0000 7f7991600640 1 --2- 172.21.15.1:0/1143769927 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f798c0cda80 0x7f798c154500 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f7984031000 tx=0x7f79840046c0 comp rx=0 tx=0).stop 2024-05-09T08:02:56.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.769+0000 7f7991600640 1 -- 172.21.15.1:0/1143769927 shutdown_connections 2024-05-09T08:02:56.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.769+0000 7f7991600640 1 --2- 172.21.15.1:0/1143769927 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f796806be80 0x7f796806e340 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:56.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.769+0000 7f7991600640 1 --2- 172.21.15.1:0/1143769927 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f798c15d730 0x7f798c1e5eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:56.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.769+0000 7f7991600640 1 --2- 172.21.15.1:0/1143769927 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f798c15ba80 0x7f798c1e5970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:56.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.769+0000 7f7991600640 1 --2- 172.21.15.1:0/1143769927 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f798c0cda80 0x7f798c154500 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:56.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.769+0000 7f7991600640 1 -- 172.21.15.1:0/1143769927 >> 172.21.15.1:0/1143769927 conn(0x7f798c066ed0 msgr2=0x7f798c158440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:02:56.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.770+0000 7f7991600640 1 -- 172.21.15.1:0/1143769927 shutdown_connections 2024-05-09T08:02:56.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.770+0000 7f7991600640 1 -- 172.21.15.1:0/1143769927 wait complete. 2024-05-09T08:02:56.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.771+0000 7f94da600640 1 -- 172.21.15.1:0/3662576646 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f94b006bf00 msgr2=0x7f94b006e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:56.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.771+0000 7f94da600640 1 --2- 172.21.15.1:0/3662576646 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f94b006bf00 0x7f94b006e3c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f94c4000f60 tx=0x7f94c4007e40 comp rx=0 tx=0).stop 2024-05-09T08:02:56.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.771+0000 7f94da600640 1 -- 172.21.15.1:0/3662576646 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f94d4157e60 msgr2=0x7f94d40c3920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:56.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.771+0000 7f94da600640 1 --2- 172.21.15.1:0/3662576646 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f94d4157e60 0x7f94d40c3920 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f94cc009e80 tx=0x7f94cc004a40 comp rx=0 tx=0).stop 2024-05-09T08:02:56.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.771+0000 7f94da600640 1 -- 172.21.15.1:0/3662576646 shutdown_connections 2024-05-09T08:02:56.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.771+0000 7f94da600640 1 --2- 172.21.15.1:0/3662576646 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f94b006bf00 0x7f94b006e3c0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:56.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.771+0000 7f94da600640 1 --2- 172.21.15.1:0/3662576646 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f94d4157e60 0x7f94d40c3920 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:56.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.771+0000 7f94da600640 1 --2- 172.21.15.1:0/3662576646 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f94d4150d00 0x7f94d40c33e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:56.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.771+0000 7f94da600640 1 --2- 172.21.15.1:0/3662576646 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f94d40c2fa0 0x7f94d414c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:56.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.771+0000 7f94da600640 1 -- 172.21.15.1:0/3662576646 >> 172.21.15.1:0/3662576646 conn(0x7f94d406dad0 msgr2=0x7f94d41582d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:02:56.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.771+0000 7f94da600640 1 -- 172.21.15.1:0/3662576646 shutdown_connections 2024-05-09T08:02:56.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.771+0000 7f94da600640 1 -- 172.21.15.1:0/3662576646 wait complete. 2024-05-09T08:02:56.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.776+0000 7f32a5e00640 1 -- 172.21.15.1:0/879103317 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f327c005580 con 0x7f3298037a10 2024-05-09T08:02:56.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.776+0000 7f329cc00640 1 -- 172.21.15.1:0/879103317 <== mon.1 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f3294079d50 con 0x7f3298037a10 2024-05-09T08:02:56.794 INFO:teuthology.orchestra.run.smithi001.stdout:55834574859 2024-05-09T08:02:56.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.780+0000 7f32a5e00640 1 -- 172.21.15.1:0/879103317 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f328006be30 msgr2=0x7f328006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:56.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.780+0000 7f32a5e00640 1 --2- 172.21.15.1:0/879103317 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f328006be30 0x7f328006e2f0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f328c0099f0 tx=0x7f328c009110 comp rx=0 tx=0).stop 2024-05-09T08:02:56.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.780+0000 7f32a5e00640 1 -- 172.21.15.1:0/879103317 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3298037a10 msgr2=0x7f32980d55c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:56.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.780+0000 7f32a5e00640 1 --2- 172.21.15.1:0/879103317 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3298037a10 0x7f32980d55c0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f32940090b0 tx=0x7f32940046c0 comp rx=0 tx=0).stop 2024-05-09T08:02:56.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.781+0000 7f32a5e00640 1 -- 172.21.15.1:0/879103317 shutdown_connections 2024-05-09T08:02:56.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.781+0000 7f32a5e00640 1 --2- 172.21.15.1:0/879103317 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f328006be30 0x7f328006e2f0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:56.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.781+0000 7f32a5e00640 1 --2- 172.21.15.1:0/879103317 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3298037a10 0x7f32980d55c0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:56.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.781+0000 7f32a5e00640 1 --2- 172.21.15.1:0/879103317 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3298037630 0x7f329804e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:56.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.781+0000 7f32a5e00640 1 --2- 172.21.15.1:0/879103317 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f3298007810 0x7f329804dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:56.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.781+0000 7f32a5e00640 1 -- 172.21.15.1:0/879103317 >> 172.21.15.1:0/879103317 conn(0x7f3298054610 msgr2=0x7f32980dde90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:02:56.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.781+0000 7f32a5e00640 1 -- 172.21.15.1:0/879103317 shutdown_connections 2024-05-09T08:02:56.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.781+0000 7f32a5e00640 1 -- 172.21.15.1:0/879103317 wait complete. 2024-05-09T08:02:56.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.784+0000 7f90d0e00640 1 -- 172.21.15.1:0/3286634966 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f90a4005850 con 0x7f90cc0c6c50 2024-05-09T08:02:56.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.785+0000 7f90c3400640 1 -- 172.21.15.1:0/3286634966 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f90b40010c0 con 0x7f90cc0c6c50 2024-05-09T08:02:56.802 INFO:teuthology.orchestra.run.smithi001.stdout:55834574860 2024-05-09T08:02:56.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.786+0000 7fc277e00640 1 -- 172.21.15.1:0/4090439372 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fc270050ba0 con 0x7fc2700deda0 2024-05-09T08:02:56.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.786+0000 7fc26ea00640 1 -- 172.21.15.1:0/4090439372 <== mon.1 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fc2780d8cd0 con 0x7fc2700deda0 2024-05-09T08:02:56.804 INFO:teuthology.orchestra.run.smithi001.stdout:55834574861 2024-05-09T08:02:56.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.788+0000 7f90d0e00640 1 -- 172.21.15.1:0/3286634966 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f90a806be30 msgr2=0x7f90a806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:56.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.788+0000 7f90d0e00640 1 --2- 172.21.15.1:0/3286634966 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f90a806be30 0x7f90a806e2f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f90bc034000 tx=0x7f90bc02e000 comp rx=0 tx=0).stop 2024-05-09T08:02:56.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.788+0000 7f90d0e00640 1 -- 172.21.15.1:0/3286634966 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f90cc0c6c50 msgr2=0x7f90cc15e9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:56.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.788+0000 7f90d0e00640 1 --2- 172.21.15.1:0/3286634966 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f90cc0c6c50 0x7f90cc15e9e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f90b400c780 tx=0x7f90b400cc40 comp rx=0 tx=0).stop 2024-05-09T08:02:56.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.789+0000 7f90d0e00640 1 -- 172.21.15.1:0/3286634966 shutdown_connections 2024-05-09T08:02:56.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.789+0000 7f90d0e00640 1 --2- 172.21.15.1:0/3286634966 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f90a806be30 0x7f90a806e2f0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:56.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.789+0000 7f90d0e00640 1 --2- 172.21.15.1:0/3286634966 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90cc0c7030 0x7f90cc0c9970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:56.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.789+0000 7f90d0e00640 1 --2- 172.21.15.1:0/3286634966 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f90cc15c5c0 0x7f90cc0c9eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:56.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.789+0000 7f90d0e00640 1 --2- 172.21.15.1:0/3286634966 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f90cc0c6c50 0x7f90cc15e9e0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:56.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.789+0000 7f90d0e00640 1 -- 172.21.15.1:0/3286634966 >> 172.21.15.1:0/3286634966 conn(0x7f90cc06b250 msgr2=0x7f90cc0c4600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:02:56.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.789+0000 7f90d0e00640 1 -- 172.21.15.1:0/3286634966 shutdown_connections 2024-05-09T08:02:56.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.789+0000 7f90d0e00640 1 -- 172.21.15.1:0/3286634966 wait complete. 2024-05-09T08:02:56.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.790+0000 7fc277e00640 1 -- 172.21.15.1:0/4090439372 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fc25806bf00 msgr2=0x7fc25806e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:56.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.790+0000 7fc277e00640 1 --2- 172.21.15.1:0/4090439372 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fc25806bf00 0x7fc25806e3c0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fc268001730 tx=0x7fc268009040 comp rx=0 tx=0).stop 2024-05-09T08:02:56.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.790+0000 7fc277e00640 1 -- 172.21.15.1:0/4090439372 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc2700deda0 msgr2=0x7fc2700d1e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:56.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.790+0000 7fc277e00640 1 --2- 172.21.15.1:0/4090439372 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc2700deda0 0x7fc2700d1e30 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fc27806a1a0 tx=0x7fc278068e80 comp rx=0 tx=0).stop 2024-05-09T08:02:56.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.791+0000 7fc277e00640 1 -- 172.21.15.1:0/4090439372 shutdown_connections 2024-05-09T08:02:56.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.791+0000 7fc277e00640 1 --2- 172.21.15.1:0/4090439372 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fc25806bf00 0x7fc25806e3c0 secure :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fc268001730 tx=0x7fc268009040 comp rx=0 tx=0).stop 2024-05-09T08:02:56.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.791+0000 7fc277e00640 1 --2- 172.21.15.1:0/4090439372 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc2700deda0 0x7fc2700d1e30 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:56.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.791+0000 7fc277e00640 1 --2- 172.21.15.1:0/4090439372 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fc2700d9090 0x7fc2700d18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:56.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.791+0000 7fc277e00640 1 --2- 172.21.15.1:0/4090439372 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc2700376a0 0x7fc27004fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:56.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.791+0000 7fc277e00640 1 -- 172.21.15.1:0/4090439372 >> 172.21.15.1:0/4090439372 conn(0x7fc270054610 msgr2=0x7fc270054b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:02:56.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.791+0000 7fc277e00640 1 -- 172.21.15.1:0/4090439372 shutdown_connections 2024-05-09T08:02:56.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.792+0000 7f92bfe00640 1 -- 172.21.15.1:0/3659492570 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f9298005850 con 0x7f92c00ce4e0 2024-05-09T08:02:56.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.792+0000 7fd159a00640 1 -- 172.21.15.1:0/1223524150 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fd15404f660 con 0x7fd1380039b0 2024-05-09T08:02:56.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.792+0000 7fc277e00640 1 -- 172.21.15.1:0/4090439372 wait complete. 2024-05-09T08:02:56.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.793+0000 7f92b7400640 1 -- 172.21.15.1:0/3659492570 <== mon.1 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f92b804b030 con 0x7f92c00ce4e0 2024-05-09T08:02:56.812 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574859 for osd.7 2024-05-09T08:02:56.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.793+0000 7fd14be00640 1 -- 172.21.15.1:0/1223524150 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fd14c04b020 con 0x7fd1380039b0 2024-05-09T08:02:56.812 INFO:teuthology.orchestra.run.smithi001.stdout:55834574860 2024-05-09T08:02:56.813 DEBUG:teuthology.parallel:result is None 2024-05-09T08:02:56.813 INFO:teuthology.orchestra.run.smithi001.stdout:55834574859 2024-05-09T08:02:56.814 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574860 got 55834574860 for osd.3 2024-05-09T08:02:56.815 DEBUG:teuthology.parallel:result is None 2024-05-09T08:02:56.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.797+0000 7fd159a00640 1 -- 172.21.15.1:0/1223524150 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fd13006bf00 msgr2=0x7fd13006e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:56.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.797+0000 7fd159a00640 1 --2- 172.21.15.1:0/1223524150 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fd13006bf00 0x7fd13006e3c0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fd1541e9c70 tx=0x7fd13c009040 comp rx=0 tx=0).stop 2024-05-09T08:02:56.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.797+0000 7fd159a00640 1 -- 172.21.15.1:0/1223524150 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd1380039b0 msgr2=0x7fd15414e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:56.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.797+0000 7fd159a00640 1 --2- 172.21.15.1:0/1223524150 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd1380039b0 0x7fd15414e160 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fd14c001e50 tx=0x7fd14c001440 comp rx=0 tx=0).stop 2024-05-09T08:02:56.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.797+0000 7fd159a00640 1 -- 172.21.15.1:0/1223524150 shutdown_connections 2024-05-09T08:02:56.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.797+0000 7fd159a00640 1 --2- 172.21.15.1:0/1223524150 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fd13006bf00 0x7fd13006e3c0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:56.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.797+0000 7fd159a00640 1 --2- 172.21.15.1:0/1223524150 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd154150790 0x7fd1541f0100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:56.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.797+0000 7fd159a00640 1 --2- 172.21.15.1:0/1223524150 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fd154150310 0x7fd1541efbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:56.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.797+0000 7fd159a00640 1 --2- 172.21.15.1:0/1223524150 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd1380039b0 0x7fd15414e160 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:56.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.797+0000 7fd159a00640 1 -- 172.21.15.1:0/1223524150 >> 172.21.15.1:0/1223524150 conn(0x7fd1540cfc20 msgr2=0x7fd154158300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:02:56.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.797+0000 7fd159a00640 1 -- 172.21.15.1:0/1223524150 shutdown_connections 2024-05-09T08:02:56.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.797+0000 7fd159a00640 1 -- 172.21.15.1:0/1223524150 wait complete. 2024-05-09T08:02:56.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.797+0000 7f92bfe00640 1 -- 172.21.15.1:0/3659492570 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f92a006beb0 msgr2=0x7f92a006e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:56.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.797+0000 7f92bfe00640 1 --2- 172.21.15.1:0/3659492570 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f92a006beb0 0x7f92a006e370 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f92ac009a30 tx=0x7f92ac009190 comp rx=0 tx=0).stop 2024-05-09T08:02:56.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.797+0000 7f92bfe00640 1 -- 172.21.15.1:0/3659492570 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f92c00ce4e0 msgr2=0x7f92c0150290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:56.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.797+0000 7f92bfe00640 1 --2- 172.21.15.1:0/3659492570 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f92c00ce4e0 0x7f92c0150290 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f92b8030000 tx=0x7f92b8004a40 comp rx=0 tx=0).stop 2024-05-09T08:02:56.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.797+0000 7f92bfe00640 1 -- 172.21.15.1:0/3659492570 shutdown_connections 2024-05-09T08:02:56.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.797+0000 7f92bfe00640 1 --2- 172.21.15.1:0/3659492570 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f92a006beb0 0x7f92a006e370 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:56.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.797+0000 7f92bfe00640 1 --2- 172.21.15.1:0/3659492570 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f92c00ce4e0 0x7f92c0150290 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:56.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.797+0000 7f92bfe00640 1 --2- 172.21.15.1:0/3659492570 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f92c00cc5f0 0x7f92c01e5cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:56.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.797+0000 7f92bfe00640 1 --2- 172.21.15.1:0/3659492570 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f92c00cac60 0x7f92c0155e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:56.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.797+0000 7f92bfe00640 1 -- 172.21.15.1:0/3659492570 >> 172.21.15.1:0/3659492570 conn(0x7f92c00cfc20 msgr2=0x7f92c00d0000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:02:56.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.798+0000 7f92bfe00640 1 -- 172.21.15.1:0/3659492570 shutdown_connections 2024-05-09T08:02:56.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:56.798+0000 7f92bfe00640 1 -- 172.21.15.1:0/3659492570 wait complete. 2024-05-09T08:02:56.821 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574859 for osd.6 2024-05-09T08:02:56.821 DEBUG:teuthology.parallel:result is None 2024-05-09T08:02:56.830 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574860 got 55834574860 for osd.0 2024-05-09T08:02:56.830 DEBUG:teuthology.parallel:result is None 2024-05-09T08:02:56.832 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574861 got 55834574861 for osd.1 2024-05-09T08:02:56.832 DEBUG:teuthology.parallel:result is None 2024-05-09T08:02:56.840 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574860 got 55834574860 for osd.2 2024-05-09T08:02:56.841 DEBUG:teuthology.parallel:result is None 2024-05-09T08:02:56.841 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574859 for osd.5 2024-05-09T08:02:56.841 DEBUG:teuthology.parallel:result is None 2024-05-09T08:02:56.841 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-05-09T08:02:56.842 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-09T08:02:57.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:57.069+0000 7f4706600640 1 Processor -- start 2024-05-09T08:02:57.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:57.070+0000 7f4706600640 1 -- start start 2024-05-09T08:02:57.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:57.071+0000 7f4706600640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f47000c46f0 0x7f47000ca210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:57.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:57.071+0000 7f4706600640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f470015a3c0 con 0x7f47001547d0 2024-05-09T08:02:57.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:57.071+0000 7f4706600640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f470015e100 con 0x7f47000c9880 2024-05-09T08:02:57.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:57.071+0000 7f4706600640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f4700154bb0 con 0x7f47000c46f0 2024-05-09T08:02:57.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:57.072+0000 7f46ffe00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f47000c46f0 0x7f47000ca210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:57.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:57.072+0000 7f4705c00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f47000c9880 0x7f47000c3bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.1:42146/0 (socket says 172.21.15.1:42146) 2024-05-09T08:02:57.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:57.072+0000 7f46ffe00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f47000c46f0 0x7f47000ca210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.1:59102/0 (socket says 172.21.15.1:59102) 2024-05-09T08:02:57.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:57.072+0000 7f4705c00640 1 -- 172.21.15.1:0/1665188889 learned_addr learned my addr 172.21.15.1:0/1665188889 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T08:02:57.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:57.072+0000 7f46ffe00640 1 -- 172.21.15.1:0/1665188889 >> v1:172.21.15.7:6789/0 conn(0x7f47000c9880 legacy=0x7f47000c3bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:57.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:57.072+0000 7f46ffe00640 1 -- 172.21.15.1:0/1665188889 >> v1:172.21.15.1:6789/0 conn(0x7f47001547d0 legacy=0x7f470015d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:02:57.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:57.072+0000 7f46ffe00640 1 -- 172.21.15.1:0/1665188889 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f470015e3e0 con 0x7f47000c46f0 2024-05-09T08:02:57.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:57.073+0000 7f46ffe00640 1 --2- 172.21.15.1:0/1665188889 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f47000c46f0 0x7f47000ca210 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f46ec0097a0 tx=0x7f46ec02a2c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9c41a7fb43f2ad06 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:57.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:57.074+0000 7f46ff400640 1 -- 172.21.15.1:0/1665188889 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46ec037070 con 0x7f47000c46f0 2024-05-09T08:02:57.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:57.074+0000 7f46ff400640 1 -- 172.21.15.1:0/1665188889 <== mon.1 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46ec032640 con 0x7f47000c46f0 2024-05-09T08:02:57.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:57.074+0000 7f46ff400640 1 -- 172.21.15.1:0/1665188889 <== mon.1 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46ec032a10 con 0x7f47000c46f0 2024-05-09T08:02:57.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:57.074+0000 7f4706600640 1 -- 172.21.15.1:0/1665188889 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f47000c46f0 msgr2=0x7f47000ca210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:57.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:57.074+0000 7f4706600640 1 --2- 172.21.15.1:0/1665188889 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f47000c46f0 0x7f47000ca210 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f46ec0097a0 tx=0x7f46ec02a2c0 comp rx=0 tx=0).stop 2024-05-09T08:02:57.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:57.074+0000 7f4706600640 1 -- 172.21.15.1:0/1665188889 shutdown_connections 2024-05-09T08:02:57.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:57.074+0000 7f4706600640 1 --2- 172.21.15.1:0/1665188889 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f47000c46f0 0x7f47000ca210 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:57.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:57.074+0000 7f4706600640 1 -- 172.21.15.1:0/1665188889 >> 172.21.15.1:0/1665188889 conn(0x7f4700071640 msgr2=0x7f4700071a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:02:57.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:57.074+0000 7f4706600640 1 -- 172.21.15.1:0/1665188889 shutdown_connections 2024-05-09T08:02:57.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:57.074+0000 7f4706600640 1 -- 172.21.15.1:0/1665188889 wait complete. 2024-05-09T08:02:57.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:57.076+0000 7f4706600640 1 Processor -- start 2024-05-09T08:02:57.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:57.076+0000 7f4706600640 1 -- start start 2024-05-09T08:02:57.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:57.077+0000 7f4706600640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f47000c46f0 0x7f47000cc6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:57.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:57.077+0000 7f4706600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f47000c9880 0x7f4700148be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:57.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:57.077+0000 7f4706600640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f47001547d0 0x7f4700155060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:57.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:57.077+0000 7f4706600640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f470015a960 con 0x7f47001547d0 2024-05-09T08:02:57.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:57.077+0000 7f4706600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f470015a680 con 0x7f47000c9880 2024-05-09T08:02:57.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:57.077+0000 7f4706600640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f47000c42c0 con 0x7f47000c46f0 2024-05-09T08:02:57.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:57.077+0000 7f4705c00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f47001547d0 0x7f4700155060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:57.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:57.077+0000 7f46ffe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f47000c9880 0x7f4700148be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:57.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:57.077+0000 7f4705c00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f47001547d0 0x7f4700155060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38316/0 (socket says 172.21.15.1:38316) 2024-05-09T08:02:57.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:57.077+0000 7f4705c00640 1 -- 172.21.15.1:0/1453365131 learned_addr learned my addr 172.21.15.1:0/1453365131 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:02:57.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:57.077+0000 7f4705200640 1 --2- 172.21.15.1:0/1453365131 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f47000c46f0 0x7f47000cc6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:57.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:57.078+0000 7f46ffe00640 1 -- 172.21.15.1:0/1453365131 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f47000c46f0 msgr2=0x7f47000cc6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:57.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:57.078+0000 7f46ffe00640 1 --2- 172.21.15.1:0/1453365131 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f47000c46f0 0x7f47000cc6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:57.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:57.078+0000 7f46ffe00640 1 -- 172.21.15.1:0/1453365131 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f47001547d0 msgr2=0x7f4700155060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:57.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:57.078+0000 7f46ffe00640 1 --2- 172.21.15.1:0/1453365131 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f47001547d0 0x7f4700155060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:57.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:57.078+0000 7f46ffe00640 1 -- 172.21.15.1:0/1453365131 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4700157950 con 0x7f47000c9880 2024-05-09T08:02:57.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:57.078+0000 7f4705c00640 1 --2- 172.21.15.1:0/1453365131 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f47001547d0 0x7f4700155060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:02:57.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:57.078+0000 7f46ffe00640 1 --2- 172.21.15.1:0/1453365131 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f47000c9880 0x7f4700148be0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f46ec031000 tx=0x7f46ec0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:57.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:57.078+0000 7f46fcc00640 1 -- 172.21.15.1:0/1453365131 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46ec041020 con 0x7f47000c9880 2024-05-09T08:02:57.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:57.078+0000 7f46fcc00640 1 -- 172.21.15.1:0/1453365131 <== mon.1 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46ec030910 con 0x7f47000c9880 2024-05-09T08:02:57.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:57.078+0000 7f46fcc00640 1 -- 172.21.15.1:0/1453365131 <== mon.1 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46ec037040 con 0x7f47000c9880 2024-05-09T08:02:57.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:57.078+0000 7f4706600640 1 -- 172.21.15.1:0/1453365131 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47001f4aa0 con 0x7f47000c9880 2024-05-09T08:02:57.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:57.078+0000 7f4706600640 1 -- 172.21.15.1:0/1453365131 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47001f3fc0 con 0x7f47000c9880 2024-05-09T08:02:57.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:57.079+0000 7f4706600640 1 -- 172.21.15.1:0/1453365131 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f470004f660 con 0x7f47000c9880 2024-05-09T08:02:57.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:57.080+0000 7f46fcc00640 1 -- 172.21.15.1:0/1453365131 <== mon.1 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f46ec044070 con 0x7f47000c9880 2024-05-09T08:02:57.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:57.081+0000 7f46fcc00640 1 --2- 172.21.15.1:0/1453365131 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f46e006be30 0x7f46e006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:57.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:57.081+0000 7f46fcc00640 1 -- 172.21.15.1:0/1453365131 <== mon.1 v2:172.21.15.7:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f46ec0aaba0 con 0x7f47000c9880 2024-05-09T08:02:57.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:57.082+0000 7f4705200640 1 --2- 172.21.15.1:0/1453365131 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f46e006be30 0x7f46e006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:57.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:57.082+0000 7f4705200640 1 --2- 172.21.15.1:0/1453365131 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f46e006be30 0x7f46e006e2f0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f46f0001e70 tx=0x7f46f0009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:57.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:57.086+0000 7f46fcc00640 1 -- 172.21.15.1:0/1453365131 <== mon.1 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f46ec078bb0 con 0x7f47000c9880 2024-05-09T08:02:57.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:57.278+0000 7f4706600640 1 -- 172.21.15.1:0/1453365131 --> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f47000c94b0 con 0x7f46e006be30 2024-05-09T08:02:57.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:57.289+0000 7f46fcc00640 1 -- 172.21.15.1:0/1453365131 <== mgr.4114 v2:172.21.15.7:6832/3562612464 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332535 (secure 0 0 0) 0x7f47000c94b0 con 0x7f46e006be30 2024-05-09T08:02:57.311 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T08:02:57.313 INFO:teuthology.orchestra.run.smithi001.stderr:dumped all 2024-05-09T08:02:57.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:57.294+0000 7f4706600640 1 -- 172.21.15.1:0/1453365131 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f46e006be30 msgr2=0x7f46e006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:57.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:57.294+0000 7f4706600640 1 --2- 172.21.15.1:0/1453365131 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f46e006be30 0x7f46e006e2f0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f46f0001e70 tx=0x7f46f0009040 comp rx=0 tx=0).stop 2024-05-09T08:02:57.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:57.294+0000 7f4706600640 1 -- 172.21.15.1:0/1453365131 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f47000c9880 msgr2=0x7f4700148be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:57.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:57.294+0000 7f4706600640 1 --2- 172.21.15.1:0/1453365131 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f47000c9880 0x7f4700148be0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f46ec031000 tx=0x7f46ec0046c0 comp rx=0 tx=0).stop 2024-05-09T08:02:57.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:57.295+0000 7f4706600640 1 -- 172.21.15.1:0/1453365131 shutdown_connections 2024-05-09T08:02:57.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:57.295+0000 7f4706600640 1 --2- 172.21.15.1:0/1453365131 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f46e006be30 0x7f46e006e2f0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:57.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:57.295+0000 7f4706600640 1 --2- 172.21.15.1:0/1453365131 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f47000c9880 0x7f4700148be0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:57.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:57.295+0000 7f4706600640 1 --2- 172.21.15.1:0/1453365131 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f47001547d0 0x7f4700155060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:57.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:57.295+0000 7f4706600640 1 --2- 172.21.15.1:0/1453365131 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f47000c46f0 0x7f47000cc6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:57.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:57.295+0000 7f4706600640 1 -- 172.21.15.1:0/1453365131 >> 172.21.15.1:0/1453365131 conn(0x7f4700071640 msgr2=0x7f4700071a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:02:57.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:57.295+0000 7f4706600640 1 -- 172.21.15.1:0/1453365131 shutdown_connections 2024-05-09T08:02:57.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:57.295+0000 7f4706600640 1 -- 172.21.15.1:0/1453365131 wait complete. 2024-05-09T08:02:57.329 INFO:teuthology.orchestra.run.smithi001.stdout:{"pg_ready":true,"pg_map":{"version":43,"stamp":"2024-05-09T08:02:57.106896+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":208,"num_write_kb":2435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160,"ondisk_log_size":160,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":240256,"kb_used_data":3304,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754734464,"statfs":{"total":773094113280,"available":772848091136,"internally_reserved":0,"allocated":3383296,"data_stored":2029432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":242535986},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003367"},"pg_stats":[{"pgid":"2.1f","version":"24'4","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:21.677957+0000","last_change":"2024-05-09T08:02:14.875108+0000","last_active":"2024-05-09T08:02:21.677957+0000","last_peered":"2024-05-09T08:02:21.677957+0000","last_clean":"2024-05-09T08:02:21.677957+0000","last_became_active":"2024-05-09T08:02:14.874841+0000","last_became_peered":"2024-05-09T08:02:14.874841+0000","last_unstale":"2024-05-09T08:02:21.677957+0000","last_undegraded":"2024-05-09T08:02:21.677957+0000","last_fullsized":"2024-05-09T08:02:21.677957+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:00:40.599475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:20.934410+0000","last_change":"2024-05-09T08:02:15.899818+0000","last_active":"2024-05-09T08:02:20.934410+0000","last_peered":"2024-05-09T08:02:20.934410+0000","last_clean":"2024-05-09T08:02:20.934410+0000","last_became_active":"2024-05-09T08:02:15.899479+0000","last_became_peered":"2024-05-09T08:02:15.899479+0000","last_unstale":"2024-05-09T08:02:20.934410+0000","last_undegraded":"2024-05-09T08:02:20.934410+0000","last_fullsized":"2024-05-09T08:02:20.934410+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:18:36.309627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:20.932807+0000","last_change":"2024-05-09T08:02:20.057944+0000","last_active":"2024-05-09T08:02:20.932807+0000","last_peered":"2024-05-09T08:02:20.932807+0000","last_clean":"2024-05-09T08:02:20.932807+0000","last_became_active":"2024-05-09T08:02:20.057016+0000","last_became_peered":"2024-05-09T08:02:20.057016+0000","last_unstale":"2024-05-09T08:02:20.932807+0000","last_undegraded":"2024-05-09T08:02:20.932807+0000","last_fullsized":"2024-05-09T08:02:20.932807+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:17:01.514168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"24'1","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:20.934743+0000","last_change":"2024-05-09T08:02:14.886137+0000","last_active":"2024-05-09T08:02:20.934743+0000","last_peered":"2024-05-09T08:02:20.934743+0000","last_clean":"2024-05-09T08:02:20.934743+0000","last_became_active":"2024-05-09T08:02:14.885830+0000","last_became_peered":"2024-05-09T08:02:14.885830+0000","last_unstale":"2024-05-09T08:02:20.934743+0000","last_undegraded":"2024-05-09T08:02:20.934743+0000","last_fullsized":"2024-05-09T08:02:20.934743+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:46:11.160815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:21.677645+0000","last_change":"2024-05-09T08:02:15.906280+0000","last_active":"2024-05-09T08:02:21.677645+0000","last_peered":"2024-05-09T08:02:21.677645+0000","last_clean":"2024-05-09T08:02:21.677645+0000","last_became_active":"2024-05-09T08:02:15.905590+0000","last_became_peered":"2024-05-09T08:02:15.905590+0000","last_unstale":"2024-05-09T08:02:21.677645+0000","last_undegraded":"2024-05-09T08:02:21.677645+0000","last_fullsized":"2024-05-09T08:02:21.677645+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:20:07.368327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.617387+0000","last_change":"2024-05-09T08:02:19.954548+0000","last_active":"2024-05-09T08:02:22.617387+0000","last_peered":"2024-05-09T08:02:22.617387+0000","last_clean":"2024-05-09T08:02:22.617387+0000","last_became_active":"2024-05-09T08:02:19.954161+0000","last_became_peered":"2024-05-09T08:02:19.954161+0000","last_unstale":"2024-05-09T08:02:22.617387+0000","last_undegraded":"2024-05-09T08:02:22.617387+0000","last_fullsized":"2024-05-09T08:02:22.617387+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:53:27.246334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"24'2","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:21.077047+0000","last_change":"2024-05-09T08:02:14.867128+0000","last_active":"2024-05-09T08:02:21.077047+0000","last_peered":"2024-05-09T08:02:21.077047+0000","last_clean":"2024-05-09T08:02:21.077047+0000","last_became_active":"2024-05-09T08:02:14.866809+0000","last_became_peered":"2024-05-09T08:02:14.866809+0000","last_unstale":"2024-05-09T08:02:21.077047+0000","last_undegraded":"2024-05-09T08:02:21.077047+0000","last_fullsized":"2024-05-09T08:02:21.077047+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:57:06.093718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.618173+0000","last_change":"2024-05-09T08:02:15.888431+0000","last_active":"2024-05-09T08:02:22.618173+0000","last_peered":"2024-05-09T08:02:22.618173+0000","last_clean":"2024-05-09T08:02:22.618173+0000","last_became_active":"2024-05-09T08:02:15.887509+0000","last_became_peered":"2024-05-09T08:02:15.887509+0000","last_unstale":"2024-05-09T08:02:22.618173+0000","last_undegraded":"2024-05-09T08:02:22.618173+0000","last_fullsized":"2024-05-09T08:02:22.618173+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:25:01.238816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.617803+0000","last_change":"2024-05-09T08:02:19.953600+0000","last_active":"2024-05-09T08:02:22.617803+0000","last_peered":"2024-05-09T08:02:22.617803+0000","last_clean":"2024-05-09T08:02:22.617803+0000","last_became_active":"2024-05-09T08:02:19.952275+0000","last_became_peered":"2024-05-09T08:02:19.952275+0000","last_unstale":"2024-05-09T08:02:22.617803+0000","last_undegraded":"2024-05-09T08:02:22.617803+0000","last_fullsized":"2024-05-09T08:02:22.617803+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:25:05.306541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"24'2","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:21.076799+0000","last_change":"2024-05-09T08:02:14.870819+0000","last_active":"2024-05-09T08:02:21.076799+0000","last_peered":"2024-05-09T08:02:21.076799+0000","last_clean":"2024-05-09T08:02:21.076799+0000","last_became_active":"2024-05-09T08:02:14.870476+0000","last_became_peered":"2024-05-09T08:02:14.870476+0000","last_unstale":"2024-05-09T08:02:21.076799+0000","last_undegraded":"2024-05-09T08:02:21.076799+0000","last_fullsized":"2024-05-09T08:02:21.076799+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:43:01.061801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.618291+0000","last_change":"2024-05-09T08:02:15.886471+0000","last_active":"2024-05-09T08:02:22.618291+0000","last_peered":"2024-05-09T08:02:22.618291+0000","last_clean":"2024-05-09T08:02:22.618291+0000","last_became_active":"2024-05-09T08:02:15.886168+0000","last_became_peered":"2024-05-09T08:02:15.886168+0000","last_unstale":"2024-05-09T08:02:22.618291+0000","last_undegraded":"2024-05-09T08:02:22.618291+0000","last_fullsized":"2024-05-09T08:02:22.618291+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:30:33.351158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.618044+0000","last_change":"2024-05-09T08:02:19.947347+0000","last_active":"2024-05-09T08:02:22.618044+0000","last_peered":"2024-05-09T08:02:22.618044+0000","last_clean":"2024-05-09T08:02:22.618044+0000","last_became_active":"2024-05-09T08:02:19.946906+0000","last_became_peered":"2024-05-09T08:02:19.946906+0000","last_unstale":"2024-05-09T08:02:22.618044+0000","last_undegraded":"2024-05-09T08:02:22.618044+0000","last_fullsized":"2024-05-09T08:02:22.618044+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:30:37.418883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.618712+0000","last_change":"2024-05-09T08:02:15.880765+0000","last_active":"2024-05-09T08:02:22.618712+0000","last_peered":"2024-05-09T08:02:22.618712+0000","last_clean":"2024-05-09T08:02:22.618712+0000","last_became_active":"2024-05-09T08:02:15.880439+0000","last_became_peered":"2024-05-09T08:02:15.880439+0000","last_unstale":"2024-05-09T08:02:22.618712+0000","last_undegraded":"2024-05-09T08:02:22.618712+0000","last_fullsized":"2024-05-09T08:02:22.618712+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:29:59.441314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"24'2","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:21.076978+0000","last_change":"2024-05-09T08:02:14.867587+0000","last_active":"2024-05-09T08:02:21.076978+0000","last_peered":"2024-05-09T08:02:21.076978+0000","last_clean":"2024-05-09T08:02:21.076978+0000","last_became_active":"2024-05-09T08:02:14.867291+0000","last_became_peered":"2024-05-09T08:02:14.867291+0000","last_unstale":"2024-05-09T08:02:21.076978+0000","last_undegraded":"2024-05-09T08:02:21.076978+0000","last_fullsized":"2024-05-09T08:02:21.076978+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:20:08.921752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.619129+0000","last_change":"2024-05-09T08:02:19.954902+0000","last_active":"2024-05-09T08:02:22.619129+0000","last_peered":"2024-05-09T08:02:22.619129+0000","last_clean":"2024-05-09T08:02:22.619129+0000","last_became_active":"2024-05-09T08:02:19.954007+0000","last_became_peered":"2024-05-09T08:02:19.954007+0000","last_unstale":"2024-05-09T08:02:22.619129+0000","last_undegraded":"2024-05-09T08:02:22.619129+0000","last_fullsized":"2024-05-09T08:02:22.619129+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:41:31.470546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:20.934123+0000","last_change":"2024-05-09T08:02:15.915155+0000","last_active":"2024-05-09T08:02:20.934123+0000","last_peered":"2024-05-09T08:02:20.934123+0000","last_clean":"2024-05-09T08:02:20.934123+0000","last_became_active":"2024-05-09T08:02:15.914852+0000","last_became_peered":"2024-05-09T08:02:15.914852+0000","last_unstale":"2024-05-09T08:02:20.934123+0000","last_undegraded":"2024-05-09T08:02:20.934123+0000","last_fullsized":"2024-05-09T08:02:20.934123+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:53:23.178610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:20.934852+0000","last_change":"2024-05-09T08:02:14.881770+0000","last_active":"2024-05-09T08:02:20.934852+0000","last_peered":"2024-05-09T08:02:20.934852+0000","last_clean":"2024-05-09T08:02:20.934852+0000","last_became_active":"2024-05-09T08:02:14.881437+0000","last_became_peered":"2024-05-09T08:02:14.881437+0000","last_unstale":"2024-05-09T08:02:20.934852+0000","last_undegraded":"2024-05-09T08:02:20.934852+0000","last_fullsized":"2024-05-09T08:02:20.934852+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:25:23.306827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.618007+0000","last_change":"2024-05-09T08:02:19.955928+0000","last_active":"2024-05-09T08:02:22.618007+0000","last_peered":"2024-05-09T08:02:22.618007+0000","last_clean":"2024-05-09T08:02:22.618007+0000","last_became_active":"2024-05-09T08:02:19.955432+0000","last_became_peered":"2024-05-09T08:02:19.955432+0000","last_unstale":"2024-05-09T08:02:22.618007+0000","last_undegraded":"2024-05-09T08:02:22.618007+0000","last_fullsized":"2024-05-09T08:02:22.618007+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:53:15.458120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:20.933351+0000","last_change":"2024-05-09T08:02:15.913311+0000","last_active":"2024-05-09T08:02:20.933351+0000","last_peered":"2024-05-09T08:02:20.933351+0000","last_clean":"2024-05-09T08:02:20.933351+0000","last_became_active":"2024-05-09T08:02:15.912484+0000","last_became_peered":"2024-05-09T08:02:15.912484+0000","last_unstale":"2024-05-09T08:02:20.933351+0000","last_undegraded":"2024-05-09T08:02:20.933351+0000","last_fullsized":"2024-05-09T08:02:20.933351+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:43:06.938904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:20.933542+0000","last_change":"2024-05-09T08:02:14.868884+0000","last_active":"2024-05-09T08:02:20.933542+0000","last_peered":"2024-05-09T08:02:20.933542+0000","last_clean":"2024-05-09T08:02:20.933542+0000","last_became_active":"2024-05-09T08:02:14.868553+0000","last_became_peered":"2024-05-09T08:02:14.868553+0000","last_unstale":"2024-05-09T08:02:20.933542+0000","last_undegraded":"2024-05-09T08:02:20.933542+0000","last_fullsized":"2024-05-09T08:02:20.933542+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:32:48.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:20.935948+0000","last_change":"2024-05-09T08:02:20.053648+0000","last_active":"2024-05-09T08:02:20.935948+0000","last_peered":"2024-05-09T08:02:20.935948+0000","last_clean":"2024-05-09T08:02:20.935948+0000","last_became_active":"2024-05-09T08:02:20.053152+0000","last_became_peered":"2024-05-09T08:02:20.053152+0000","last_unstale":"2024-05-09T08:02:20.935948+0000","last_undegraded":"2024-05-09T08:02:20.935948+0000","last_fullsized":"2024-05-09T08:02:20.935948+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:59:19.025530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.618049+0000","last_change":"2024-05-09T08:02:15.891115+0000","last_active":"2024-05-09T08:02:22.618049+0000","last_peered":"2024-05-09T08:02:22.618049+0000","last_clean":"2024-05-09T08:02:22.618049+0000","last_became_active":"2024-05-09T08:02:15.890806+0000","last_became_peered":"2024-05-09T08:02:15.890806+0000","last_unstale":"2024-05-09T08:02:22.618049+0000","last_undegraded":"2024-05-09T08:02:22.618049+0000","last_fullsized":"2024-05-09T08:02:22.618049+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:57:07.107359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:21.076482+0000","last_change":"2024-05-09T08:02:14.874987+0000","last_active":"2024-05-09T08:02:21.076482+0000","last_peered":"2024-05-09T08:02:21.076482+0000","last_clean":"2024-05-09T08:02:21.076482+0000","last_became_active":"2024-05-09T08:02:14.874549+0000","last_became_peered":"2024-05-09T08:02:14.874549+0000","last_unstale":"2024-05-09T08:02:21.076482+0000","last_undegraded":"2024-05-09T08:02:21.076482+0000","last_fullsized":"2024-05-09T08:02:21.076482+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:57:10.338510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.618392+0000","last_change":"2024-05-09T08:02:19.952790+0000","last_active":"2024-05-09T08:02:22.618392+0000","last_peered":"2024-05-09T08:02:22.618392+0000","last_clean":"2024-05-09T08:02:22.618392+0000","last_became_active":"2024-05-09T08:02:19.952378+0000","last_became_peered":"2024-05-09T08:02:19.952378+0000","last_unstale":"2024-05-09T08:02:22.618392+0000","last_undegraded":"2024-05-09T08:02:22.618392+0000","last_fullsized":"2024-05-09T08:02:22.618392+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:42:00.939587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:21.678285+0000","last_change":"2024-05-09T08:02:15.913011+0000","last_active":"2024-05-09T08:02:21.678285+0000","last_peered":"2024-05-09T08:02:21.678285+0000","last_clean":"2024-05-09T08:02:21.678285+0000","last_became_active":"2024-05-09T08:02:15.912590+0000","last_became_peered":"2024-05-09T08:02:15.912590+0000","last_unstale":"2024-05-09T08:02:21.678285+0000","last_undegraded":"2024-05-09T08:02:21.678285+0000","last_fullsized":"2024-05-09T08:02:21.678285+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:43:06.938904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"24'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.619767+0000","last_change":"2024-05-09T08:02:14.864763+0000","last_active":"2024-05-09T08:02:22.619767+0000","last_peered":"2024-05-09T08:02:22.619767+0000","last_clean":"2024-05-09T08:02:22.619767+0000","last_became_active":"2024-05-09T08:02:14.864470+0000","last_became_peered":"2024-05-09T08:02:14.864470+0000","last_unstale":"2024-05-09T08:02:22.619767+0000","last_undegraded":"2024-05-09T08:02:22.619767+0000","last_fullsized":"2024-05-09T08:02:22.619767+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:00:40.599475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.617800+0000","last_change":"2024-05-09T08:02:20.054896+0000","last_active":"2024-05-09T08:02:22.617800+0000","last_peered":"2024-05-09T08:02:22.617800+0000","last_clean":"2024-05-09T08:02:22.617800+0000","last_became_active":"2024-05-09T08:02:20.054424+0000","last_became_peered":"2024-05-09T08:02:20.054424+0000","last_unstale":"2024-05-09T08:02:22.617800+0000","last_undegraded":"2024-05-09T08:02:22.617800+0000","last_fullsized":"2024-05-09T08:02:22.617800+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:23:44.316545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:20.935333+0000","last_change":"2024-05-09T08:02:15.905448+0000","last_active":"2024-05-09T08:02:20.935333+0000","last_peered":"2024-05-09T08:02:20.935333+0000","last_clean":"2024-05-09T08:02:20.935333+0000","last_became_active":"2024-05-09T08:02:15.905131+0000","last_became_peered":"2024-05-09T08:02:15.905131+0000","last_unstale":"2024-05-09T08:02:20.935333+0000","last_undegraded":"2024-05-09T08:02:20.935333+0000","last_fullsized":"2024-05-09T08:02:20.935333+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:25:01.238816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"25'1","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:20.936562+0000","last_change":"2024-05-09T08:02:14.868410+0000","last_active":"2024-05-09T08:02:20.936562+0000","last_peered":"2024-05-09T08:02:20.936562+0000","last_clean":"2024-05-09T08:02:20.936562+0000","last_became_active":"2024-05-09T08:02:14.865621+0000","last_became_peered":"2024-05-09T08:02:14.865621+0000","last_unstale":"2024-05-09T08:02:20.936562+0000","last_undegraded":"2024-05-09T08:02:20.936562+0000","last_fullsized":"2024-05-09T08:02:20.936562+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:43:01.061801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:20.932915+0000","last_change":"2024-05-09T08:02:20.057603+0000","last_active":"2024-05-09T08:02:20.932915+0000","last_peered":"2024-05-09T08:02:20.932915+0000","last_clean":"2024-05-09T08:02:20.932915+0000","last_became_active":"2024-05-09T08:02:20.055365+0000","last_became_peered":"2024-05-09T08:02:20.055365+0000","last_unstale":"2024-05-09T08:02:20.932915+0000","last_undegraded":"2024-05-09T08:02:20.932915+0000","last_fullsized":"2024-05-09T08:02:20.932915+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:48:38.283562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:20.934935+0000","last_change":"2024-05-09T08:02:15.890980+0000","last_active":"2024-05-09T08:02:20.934935+0000","last_peered":"2024-05-09T08:02:20.934935+0000","last_clean":"2024-05-09T08:02:20.934935+0000","last_became_active":"2024-05-09T08:02:15.890346+0000","last_became_peered":"2024-05-09T08:02:15.890346+0000","last_unstale":"2024-05-09T08:02:20.934935+0000","last_undegraded":"2024-05-09T08:02:20.934935+0000","last_fullsized":"2024-05-09T08:02:20.934935+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:29:59.441314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:21.077241+0000","last_change":"2024-05-09T08:02:14.876206+0000","last_active":"2024-05-09T08:02:21.077241+0000","last_peered":"2024-05-09T08:02:21.077241+0000","last_clean":"2024-05-09T08:02:21.077241+0000","last_became_active":"2024-05-09T08:02:14.875894+0000","last_became_peered":"2024-05-09T08:02:14.875894+0000","last_unstale":"2024-05-09T08:02:21.077241+0000","last_undegraded":"2024-05-09T08:02:21.077241+0000","last_fullsized":"2024-05-09T08:02:21.077241+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:49:53.882513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:21.677241+0000","last_change":"2024-05-09T08:02:20.055222+0000","last_active":"2024-05-09T08:02:21.677241+0000","last_peered":"2024-05-09T08:02:21.677241+0000","last_clean":"2024-05-09T08:02:21.677241+0000","last_became_active":"2024-05-09T08:02:20.053802+0000","last_became_peered":"2024-05-09T08:02:20.053802+0000","last_unstale":"2024-05-09T08:02:21.677241+0000","last_undegraded":"2024-05-09T08:02:21.677241+0000","last_fullsized":"2024-05-09T08:02:21.677241+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:42:00.939587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.618451+0000","last_change":"2024-05-09T08:02:15.904528+0000","last_active":"2024-05-09T08:02:22.618451+0000","last_peered":"2024-05-09T08:02:22.618451+0000","last_clean":"2024-05-09T08:02:22.618451+0000","last_became_active":"2024-05-09T08:02:15.904049+0000","last_became_peered":"2024-05-09T08:02:15.904049+0000","last_unstale":"2024-05-09T08:02:22.618451+0000","last_undegraded":"2024-05-09T08:02:22.618451+0000","last_fullsized":"2024-05-09T08:02:22.618451+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:53:11.390395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:20.935040+0000","last_change":"2024-05-09T08:02:14.872897+0000","last_active":"2024-05-09T08:02:20.935040+0000","last_peered":"2024-05-09T08:02:20.935040+0000","last_clean":"2024-05-09T08:02:20.935040+0000","last_became_active":"2024-05-09T08:02:14.872096+0000","last_became_peered":"2024-05-09T08:02:14.872096+0000","last_unstale":"2024-05-09T08:02:20.935040+0000","last_undegraded":"2024-05-09T08:02:20.935040+0000","last_fullsized":"2024-05-09T08:02:20.935040+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:00:40.599475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:20.933696+0000","last_change":"2024-05-09T08:02:19.963716+0000","last_active":"2024-05-09T08:02:20.933696+0000","last_peered":"2024-05-09T08:02:20.933696+0000","last_clean":"2024-05-09T08:02:20.933696+0000","last_became_active":"2024-05-09T08:02:19.962922+0000","last_became_peered":"2024-05-09T08:02:19.962922+0000","last_unstale":"2024-05-09T08:02:20.933696+0000","last_undegraded":"2024-05-09T08:02:20.933696+0000","last_fullsized":"2024-05-09T08:02:20.933696+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:42:00.939587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.617918+0000","last_change":"2024-05-09T08:02:15.901450+0000","last_active":"2024-05-09T08:02:22.617918+0000","last_peered":"2024-05-09T08:02:22.617918+0000","last_clean":"2024-05-09T08:02:22.617918+0000","last_became_active":"2024-05-09T08:02:15.901180+0000","last_became_peered":"2024-05-09T08:02:15.901180+0000","last_unstale":"2024-05-09T08:02:22.617918+0000","last_undegraded":"2024-05-09T08:02:22.617918+0000","last_fullsized":"2024-05-09T08:02:22.617918+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:43:02.075442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.618076+0000","last_change":"2024-05-09T08:02:14.876518+0000","last_active":"2024-05-09T08:02:22.618076+0000","last_peered":"2024-05-09T08:02:22.618076+0000","last_clean":"2024-05-09T08:02:22.618076+0000","last_became_active":"2024-05-09T08:02:14.876175+0000","last_became_peered":"2024-05-09T08:02:14.876175+0000","last_unstale":"2024-05-09T08:02:22.618076+0000","last_undegraded":"2024-05-09T08:02:22.618076+0000","last_fullsized":"2024-05-09T08:02:22.618076+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:00:40.599475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:20.934897+0000","last_change":"2024-05-09T08:02:19.960940+0000","last_active":"2024-05-09T08:02:20.934897+0000","last_peered":"2024-05-09T08:02:20.934897+0000","last_clean":"2024-05-09T08:02:20.934897+0000","last_became_active":"2024-05-09T08:02:19.960496+0000","last_became_peered":"2024-05-09T08:02:19.960496+0000","last_unstale":"2024-05-09T08:02:20.934897+0000","last_undegraded":"2024-05-09T08:02:20.934897+0000","last_fullsized":"2024-05-09T08:02:20.934897+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:52:40.529638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:20.932615+0000","last_change":"2024-05-09T08:02:15.891774+0000","last_active":"2024-05-09T08:02:20.932615+0000","last_peered":"2024-05-09T08:02:20.932615+0000","last_clean":"2024-05-09T08:02:20.932615+0000","last_became_active":"2024-05-09T08:02:15.891407+0000","last_became_peered":"2024-05-09T08:02:15.891407+0000","last_unstale":"2024-05-09T08:02:20.932615+0000","last_undegraded":"2024-05-09T08:02:20.932615+0000","last_fullsized":"2024-05-09T08:02:20.932615+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:05:02.181367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.618768+0000","last_change":"2024-05-09T08:02:14.883655+0000","last_active":"2024-05-09T08:02:22.618768+0000","last_peered":"2024-05-09T08:02:22.618768+0000","last_clean":"2024-05-09T08:02:22.618768+0000","last_became_active":"2024-05-09T08:02:14.882359+0000","last_became_peered":"2024-05-09T08:02:14.882359+0000","last_unstale":"2024-05-09T08:02:22.618768+0000","last_undegraded":"2024-05-09T08:02:22.618768+0000","last_fullsized":"2024-05-09T08:02:22.618768+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:46:11.160815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:20.936081+0000","last_change":"2024-05-09T08:02:19.953319+0000","last_active":"2024-05-09T08:02:20.936081+0000","last_peered":"2024-05-09T08:02:20.936081+0000","last_clean":"2024-05-09T08:02:20.936081+0000","last_became_active":"2024-05-09T08:02:19.952740+0000","last_became_peered":"2024-05-09T08:02:19.952740+0000","last_unstale":"2024-05-09T08:02:20.936081+0000","last_undegraded":"2024-05-09T08:02:20.936081+0000","last_fullsized":"2024-05-09T08:02:20.936081+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:25:28.804048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:21.676635+0000","last_change":"2024-05-09T08:02:15.913828+0000","last_active":"2024-05-09T08:02:21.676635+0000","last_peered":"2024-05-09T08:02:21.676635+0000","last_clean":"2024-05-09T08:02:21.676635+0000","last_became_active":"2024-05-09T08:02:15.913313+0000","last_became_peered":"2024-05-09T08:02:15.913313+0000","last_unstale":"2024-05-09T08:02:21.676635+0000","last_undegraded":"2024-05-09T08:02:21.676635+0000","last_fullsized":"2024-05-09T08:02:21.676635+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:20:09.935394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:21.076590+0000","last_change":"2024-05-09T08:02:14.876438+0000","last_active":"2024-05-09T08:02:21.076590+0000","last_peered":"2024-05-09T08:02:21.076590+0000","last_clean":"2024-05-09T08:02:21.076590+0000","last_became_active":"2024-05-09T08:02:14.876146+0000","last_became_peered":"2024-05-09T08:02:14.876146+0000","last_unstale":"2024-05-09T08:02:21.076590+0000","last_undegraded":"2024-05-09T08:02:21.076590+0000","last_fullsized":"2024-05-09T08:02:21.076590+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:36:49.629991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:21.678130+0000","last_change":"2024-05-09T08:02:20.051067+0000","last_active":"2024-05-09T08:02:21.678130+0000","last_peered":"2024-05-09T08:02:21.678130+0000","last_clean":"2024-05-09T08:02:21.678130+0000","last_became_active":"2024-05-09T08:02:20.050593+0000","last_became_peered":"2024-05-09T08:02:20.050593+0000","last_unstale":"2024-05-09T08:02:21.678130+0000","last_undegraded":"2024-05-09T08:02:21.678130+0000","last_fullsized":"2024-05-09T08:02:21.678130+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:25:28.804048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:20.933046+0000","last_change":"2024-05-09T08:02:15.899137+0000","last_active":"2024-05-09T08:02:20.933046+0000","last_peered":"2024-05-09T08:02:20.933046+0000","last_clean":"2024-05-09T08:02:20.933046+0000","last_became_active":"2024-05-09T08:02:15.898766+0000","last_became_peered":"2024-05-09T08:02:15.898766+0000","last_unstale":"2024-05-09T08:02:20.933046+0000","last_undegraded":"2024-05-09T08:02:20.933046+0000","last_fullsized":"2024-05-09T08:02:20.933046+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:47:32.605288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.617934+0000","last_change":"2024-05-09T08:02:14.868850+0000","last_active":"2024-05-09T08:02:22.617934+0000","last_peered":"2024-05-09T08:02:22.617934+0000","last_clean":"2024-05-09T08:02:22.617934+0000","last_became_active":"2024-05-09T08:02:14.868384+0000","last_became_peered":"2024-05-09T08:02:14.868384+0000","last_unstale":"2024-05-09T08:02:22.617934+0000","last_undegraded":"2024-05-09T08:02:22.617934+0000","last_fullsized":"2024-05-09T08:02:22.617934+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:14:52.920972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":177,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:21.076748+0000","last_change":"2024-05-09T08:02:11.559804+0000","last_active":"2024-05-09T08:02:21.076748+0000","last_peered":"2024-05-09T08:02:21.076748+0000","last_clean":"2024-05-09T08:02:21.076748+0000","last_became_active":"2024-05-09T08:02:11.558761+0000","last_became_peered":"2024-05-09T08:02:11.558761+0000","last_unstale":"2024-05-09T08:02:21.076748+0000","last_undegraded":"2024-05-09T08:02:21.076748+0000","last_fullsized":"2024-05-09T08:02:21.076748+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:09.800143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:09.800143+0000","last_clean_scrub_stamp":"2024-05-09T08:02:09.800143+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:18:35.964149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.618755+0000","last_change":"2024-05-09T08:02:20.054746+0000","last_active":"2024-05-09T08:02:22.618755+0000","last_peered":"2024-05-09T08:02:22.618755+0000","last_clean":"2024-05-09T08:02:22.618755+0000","last_became_active":"2024-05-09T08:02:20.054186+0000","last_became_peered":"2024-05-09T08:02:20.054186+0000","last_unstale":"2024-05-09T08:02:22.618755+0000","last_undegraded":"2024-05-09T08:02:22.618755+0000","last_fullsized":"2024-05-09T08:02:22.618755+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:53:15.458120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:21.076861+0000","last_change":"2024-05-09T08:02:15.889657+0000","last_active":"2024-05-09T08:02:21.076861+0000","last_peered":"2024-05-09T08:02:21.076861+0000","last_clean":"2024-05-09T08:02:21.076861+0000","last_became_active":"2024-05-09T08:02:15.889381+0000","last_became_peered":"2024-05-09T08:02:15.889381+0000","last_unstale":"2024-05-09T08:02:21.076861+0000","last_undegraded":"2024-05-09T08:02:21.076861+0000","last_fullsized":"2024-05-09T08:02:21.076861+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:59:14.957806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.619305+0000","last_change":"2024-05-09T08:02:14.868892+0000","last_active":"2024-05-09T08:02:22.619305+0000","last_peered":"2024-05-09T08:02:22.619305+0000","last_clean":"2024-05-09T08:02:22.619305+0000","last_became_active":"2024-05-09T08:02:14.868513+0000","last_became_peered":"2024-05-09T08:02:14.868513+0000","last_unstale":"2024-05-09T08:02:22.619305+0000","last_undegraded":"2024-05-09T08:02:22.619305+0000","last_fullsized":"2024-05-09T08:02:22.619305+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:19:31.913236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.618654+0000","last_change":"2024-05-09T08:02:20.053097+0000","last_active":"2024-05-09T08:02:22.618654+0000","last_peered":"2024-05-09T08:02:22.618654+0000","last_clean":"2024-05-09T08:02:22.618654+0000","last_became_active":"2024-05-09T08:02:20.052653+0000","last_became_peered":"2024-05-09T08:02:20.052653+0000","last_unstale":"2024-05-09T08:02:22.618654+0000","last_undegraded":"2024-05-09T08:02:22.618654+0000","last_fullsized":"2024-05-09T08:02:22.618654+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:25:05.306541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.618299+0000","last_change":"2024-05-09T08:02:15.899059+0000","last_active":"2024-05-09T08:02:22.618299+0000","last_peered":"2024-05-09T08:02:22.618299+0000","last_clean":"2024-05-09T08:02:22.618299+0000","last_became_active":"2024-05-09T08:02:15.898692+0000","last_became_peered":"2024-05-09T08:02:15.898692+0000","last_unstale":"2024-05-09T08:02:22.618299+0000","last_undegraded":"2024-05-09T08:02:22.618299+0000","last_fullsized":"2024-05-09T08:02:22.618299+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:52:36.461913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"24'1","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:20.935331+0000","last_change":"2024-05-09T08:02:14.873071+0000","last_active":"2024-05-09T08:02:20.935331+0000","last_peered":"2024-05-09T08:02:20.935331+0000","last_clean":"2024-05-09T08:02:20.935331+0000","last_became_active":"2024-05-09T08:02:14.872556+0000","last_became_peered":"2024-05-09T08:02:14.872556+0000","last_unstale":"2024-05-09T08:02:20.935331+0000","last_undegraded":"2024-05-09T08:02:20.935331+0000","last_fullsized":"2024-05-09T08:02:20.935331+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:57:06.093718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:21.677388+0000","last_change":"2024-05-09T08:02:19.960414+0000","last_active":"2024-05-09T08:02:21.677388+0000","last_peered":"2024-05-09T08:02:21.677388+0000","last_clean":"2024-05-09T08:02:21.677388+0000","last_became_active":"2024-05-09T08:02:19.957941+0000","last_became_peered":"2024-05-09T08:02:19.957941+0000","last_unstale":"2024-05-09T08:02:21.677388+0000","last_undegraded":"2024-05-09T08:02:21.677388+0000","last_fullsized":"2024-05-09T08:02:21.677388+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:41:31.470546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:21.077406+0000","last_change":"2024-05-09T08:02:15.884395+0000","last_active":"2024-05-09T08:02:21.077406+0000","last_peered":"2024-05-09T08:02:21.077406+0000","last_clean":"2024-05-09T08:02:21.077406+0000","last_became_active":"2024-05-09T08:02:15.883788+0000","last_became_peered":"2024-05-09T08:02:15.883788+0000","last_unstale":"2024-05-09T08:02:21.077406+0000","last_undegraded":"2024-05-09T08:02:21.077406+0000","last_fullsized":"2024-05-09T08:02:21.077406+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:48:34.215837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"24'2","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.619345+0000","last_change":"2024-05-09T08:02:14.885044+0000","last_active":"2024-05-09T08:02:22.619345+0000","last_peered":"2024-05-09T08:02:22.619345+0000","last_clean":"2024-05-09T08:02:22.619345+0000","last_became_active":"2024-05-09T08:02:14.884706+0000","last_became_peered":"2024-05-09T08:02:14.884706+0000","last_unstale":"2024-05-09T08:02:22.619345+0000","last_undegraded":"2024-05-09T08:02:22.619345+0000","last_fullsized":"2024-05-09T08:02:22.619345+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:14:52.920972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.617383+0000","last_change":"2024-05-09T08:02:20.050865+0000","last_active":"2024-05-09T08:02:22.617383+0000","last_peered":"2024-05-09T08:02:22.617383+0000","last_clean":"2024-05-09T08:02:22.617383+0000","last_became_active":"2024-05-09T08:02:20.050270+0000","last_became_peered":"2024-05-09T08:02:20.050270+0000","last_unstale":"2024-05-09T08:02:22.617383+0000","last_undegraded":"2024-05-09T08:02:22.617383+0000","last_fullsized":"2024-05-09T08:02:22.617383+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:48:38.283562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:21.077190+0000","last_change":"2024-05-09T08:02:15.884737+0000","last_active":"2024-05-09T08:02:21.077190+0000","last_peered":"2024-05-09T08:02:21.077190+0000","last_clean":"2024-05-09T08:02:21.077190+0000","last_became_active":"2024-05-09T08:02:15.884465+0000","last_became_peered":"2024-05-09T08:02:15.884465+0000","last_unstale":"2024-05-09T08:02:21.077190+0000","last_undegraded":"2024-05-09T08:02:21.077190+0000","last_fullsized":"2024-05-09T08:02:21.077190+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:52:36.461913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"24'1","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.618658+0000","last_change":"2024-05-09T08:02:14.875003+0000","last_active":"2024-05-09T08:02:22.618658+0000","last_peered":"2024-05-09T08:02:22.618658+0000","last_clean":"2024-05-09T08:02:22.618658+0000","last_became_active":"2024-05-09T08:02:14.874665+0000","last_became_peered":"2024-05-09T08:02:14.874665+0000","last_unstale":"2024-05-09T08:02:22.618658+0000","last_undegraded":"2024-05-09T08:02:22.618658+0000","last_fullsized":"2024-05-09T08:02:22.618658+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:18:39.991319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.617254+0000","last_change":"2024-05-09T08:02:19.952692+0000","last_active":"2024-05-09T08:02:22.617254+0000","last_peered":"2024-05-09T08:02:22.617254+0000","last_clean":"2024-05-09T08:02:22.617254+0000","last_became_active":"2024-05-09T08:02:19.952224+0000","last_became_peered":"2024-05-09T08:02:19.952224+0000","last_unstale":"2024-05-09T08:02:22.617254+0000","last_undegraded":"2024-05-09T08:02:22.617254+0000","last_fullsized":"2024-05-09T08:02:22.617254+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:47:36.673013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:21.076666+0000","last_change":"2024-05-09T08:02:15.883408+0000","last_active":"2024-05-09T08:02:21.076666+0000","last_peered":"2024-05-09T08:02:21.076666+0000","last_clean":"2024-05-09T08:02:21.076666+0000","last_became_active":"2024-05-09T08:02:15.883103+0000","last_became_peered":"2024-05-09T08:02:15.883103+0000","last_unstale":"2024-05-09T08:02:21.076666+0000","last_undegraded":"2024-05-09T08:02:21.076666+0000","last_fullsized":"2024-05-09T08:02:21.076666+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:53:11.390395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"24'8","reported_seq":29,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.618529+0000","last_change":"2024-05-09T08:02:14.873184+0000","last_active":"2024-05-09T08:02:22.618529+0000","last_peered":"2024-05-09T08:02:22.618529+0000","last_clean":"2024-05-09T08:02:22.618529+0000","last_became_active":"2024-05-09T08:02:14.871055+0000","last_became_peered":"2024-05-09T08:02:14.871055+0000","last_unstale":"2024-05-09T08:02:22.618529+0000","last_undegraded":"2024-05-09T08:02:22.618529+0000","last_fullsized":"2024-05-09T08:02:22.618529+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:19:56.315972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:21.678092+0000","last_change":"2024-05-09T08:02:19.950823+0000","last_active":"2024-05-09T08:02:21.678092+0000","last_peered":"2024-05-09T08:02:21.678092+0000","last_clean":"2024-05-09T08:02:21.678092+0000","last_became_active":"2024-05-09T08:02:19.950352+0000","last_became_peered":"2024-05-09T08:02:19.950352+0000","last_unstale":"2024-05-09T08:02:21.678092+0000","last_undegraded":"2024-05-09T08:02:21.678092+0000","last_fullsized":"2024-05-09T08:02:21.678092+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:59:19.025530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.619335+0000","last_change":"2024-05-09T08:02:15.899115+0000","last_active":"2024-05-09T08:02:22.619335+0000","last_peered":"2024-05-09T08:02:22.619335+0000","last_clean":"2024-05-09T08:02:22.619335+0000","last_became_active":"2024-05-09T08:02:15.898811+0000","last_became_peered":"2024-05-09T08:02:15.898811+0000","last_unstale":"2024-05-09T08:02:22.619335+0000","last_undegraded":"2024-05-09T08:02:22.619335+0000","last_fullsized":"2024-05-09T08:02:22.619335+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:43:06.938904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.619023+0000","last_change":"2024-05-09T08:02:14.860818+0000","last_active":"2024-05-09T08:02:22.619023+0000","last_peered":"2024-05-09T08:02:22.619023+0000","last_clean":"2024-05-09T08:02:22.619023+0000","last_became_active":"2024-05-09T08:02:14.860518+0000","last_became_peered":"2024-05-09T08:02:14.860518+0000","last_unstale":"2024-05-09T08:02:22.619023+0000","last_undegraded":"2024-05-09T08:02:22.619023+0000","last_fullsized":"2024-05-09T08:02:22.619023+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:25:23.306827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:20.935214+0000","last_change":"2024-05-09T08:02:20.053251+0000","last_active":"2024-05-09T08:02:20.935214+0000","last_peered":"2024-05-09T08:02:20.935214+0000","last_clean":"2024-05-09T08:02:20.935214+0000","last_became_active":"2024-05-09T08:02:20.052783+0000","last_became_peered":"2024-05-09T08:02:20.052783+0000","last_unstale":"2024-05-09T08:02:20.935214+0000","last_undegraded":"2024-05-09T08:02:20.935214+0000","last_fullsized":"2024-05-09T08:02:20.935214+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:16:41.863341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:21.076854+0000","last_change":"2024-05-09T08:02:15.883972+0000","last_active":"2024-05-09T08:02:21.076854+0000","last_peered":"2024-05-09T08:02:21.076854+0000","last_clean":"2024-05-09T08:02:21.076854+0000","last_became_active":"2024-05-09T08:02:15.883613+0000","last_became_peered":"2024-05-09T08:02:15.883613+0000","last_unstale":"2024-05-09T08:02:21.076854+0000","last_undegraded":"2024-05-09T08:02:21.076854+0000","last_fullsized":"2024-05-09T08:02:21.076854+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:41:27.402821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.618038+0000","last_change":"2024-05-09T08:02:14.883366+0000","last_active":"2024-05-09T08:02:22.618038+0000","last_peered":"2024-05-09T08:02:22.618038+0000","last_clean":"2024-05-09T08:02:22.618038+0000","last_became_active":"2024-05-09T08:02:14.883135+0000","last_became_peered":"2024-05-09T08:02:14.883135+0000","last_unstale":"2024-05-09T08:02:22.618038+0000","last_undegraded":"2024-05-09T08:02:22.618038+0000","last_fullsized":"2024-05-09T08:02:22.618038+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:32:48.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:20.935375+0000","last_change":"2024-05-09T08:02:20.050888+0000","last_active":"2024-05-09T08:02:20.935375+0000","last_peered":"2024-05-09T08:02:20.935375+0000","last_clean":"2024-05-09T08:02:20.935375+0000","last_became_active":"2024-05-09T08:02:20.050390+0000","last_became_peered":"2024-05-09T08:02:20.050390+0000","last_unstale":"2024-05-09T08:02:20.935375+0000","last_undegraded":"2024-05-09T08:02:20.935375+0000","last_fullsized":"2024-05-09T08:02:20.935375+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:10:24.797526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:21.677358+0000","last_change":"2024-05-09T08:02:15.899096+0000","last_active":"2024-05-09T08:02:21.677358+0000","last_peered":"2024-05-09T08:02:21.677358+0000","last_clean":"2024-05-09T08:02:21.677358+0000","last_became_active":"2024-05-09T08:02:15.898811+0000","last_became_peered":"2024-05-09T08:02:15.898811+0000","last_unstale":"2024-05-09T08:02:21.677358+0000","last_undegraded":"2024-05-09T08:02:21.677358+0000","last_fullsized":"2024-05-09T08:02:21.677358+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:28:51.229262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"24'2","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:21.677657+0000","last_change":"2024-05-09T08:02:14.868657+0000","last_active":"2024-05-09T08:02:21.677657+0000","last_peered":"2024-05-09T08:02:21.677657+0000","last_clean":"2024-05-09T08:02:21.677657+0000","last_became_active":"2024-05-09T08:02:14.868385+0000","last_became_peered":"2024-05-09T08:02:14.868385+0000","last_unstale":"2024-05-09T08:02:21.677657+0000","last_undegraded":"2024-05-09T08:02:21.677657+0000","last_fullsized":"2024-05-09T08:02:21.677657+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:46:11.160815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.618792+0000","last_change":"2024-05-09T08:02:20.053155+0000","last_active":"2024-05-09T08:02:22.618792+0000","last_peered":"2024-05-09T08:02:22.618792+0000","last_clean":"2024-05-09T08:02:22.618792+0000","last_became_active":"2024-05-09T08:02:20.052629+0000","last_became_peered":"2024-05-09T08:02:20.052629+0000","last_unstale":"2024-05-09T08:02:22.618792+0000","last_undegraded":"2024-05-09T08:02:22.618792+0000","last_fullsized":"2024-05-09T08:02:22.618792+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:35:54.628874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:21.076864+0000","last_change":"2024-05-09T08:02:15.905406+0000","last_active":"2024-05-09T08:02:21.076864+0000","last_peered":"2024-05-09T08:02:21.076864+0000","last_clean":"2024-05-09T08:02:21.076864+0000","last_became_active":"2024-05-09T08:02:15.905076+0000","last_became_peered":"2024-05-09T08:02:15.905076+0000","last_unstale":"2024-05-09T08:02:21.076864+0000","last_undegraded":"2024-05-09T08:02:21.076864+0000","last_fullsized":"2024-05-09T08:02:21.076864+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:25:01.238816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"25'10","reported_seq":32,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:39.803593+0000","last_change":"2024-05-09T08:02:14.880410+0000","last_active":"2024-05-09T08:02:39.803593+0000","last_peered":"2024-05-09T08:02:39.803593+0000","last_clean":"2024-05-09T08:02:39.803593+0000","last_became_active":"2024-05-09T08:02:14.880045+0000","last_became_peered":"2024-05-09T08:02:14.880045+0000","last_unstale":"2024-05-09T08:02:39.803593+0000","last_undegraded":"2024-05-09T08:02:39.803593+0000","last_fullsized":"2024-05-09T08:02:39.803593+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:19:56.315972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:20.934006+0000","last_change":"2024-05-09T08:02:19.963378+0000","last_active":"2024-05-09T08:02:20.934006+0000","last_peered":"2024-05-09T08:02:20.934006+0000","last_clean":"2024-05-09T08:02:20.934006+0000","last_became_active":"2024-05-09T08:02:19.958964+0000","last_became_peered":"2024-05-09T08:02:19.958964+0000","last_unstale":"2024-05-09T08:02:20.934006+0000","last_undegraded":"2024-05-09T08:02:20.934006+0000","last_fullsized":"2024-05-09T08:02:20.934006+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:41:31.470546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.618268+0000","last_change":"2024-05-09T08:02:15.888428+0000","last_active":"2024-05-09T08:02:22.618268+0000","last_peered":"2024-05-09T08:02:22.618268+0000","last_clean":"2024-05-09T08:02:22.618268+0000","last_became_active":"2024-05-09T08:02:15.888071+0000","last_became_peered":"2024-05-09T08:02:15.888071+0000","last_unstale":"2024-05-09T08:02:22.618268+0000","last_undegraded":"2024-05-09T08:02:22.618268+0000","last_fullsized":"2024-05-09T08:02:22.618268+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:36:50.643632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"24'3","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:20.935171+0000","last_change":"2024-05-09T08:02:14.872697+0000","last_active":"2024-05-09T08:02:20.935171+0000","last_peered":"2024-05-09T08:02:20.935171+0000","last_clean":"2024-05-09T08:02:20.935171+0000","last_became_active":"2024-05-09T08:02:14.871945+0000","last_became_peered":"2024-05-09T08:02:14.871945+0000","last_unstale":"2024-05-09T08:02:20.935171+0000","last_undegraded":"2024-05-09T08:02:20.935171+0000","last_fullsized":"2024-05-09T08:02:20.935171+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:19:56.315972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.617683+0000","last_change":"2024-05-09T08:02:19.953635+0000","last_active":"2024-05-09T08:02:22.617683+0000","last_peered":"2024-05-09T08:02:22.617683+0000","last_clean":"2024-05-09T08:02:22.617683+0000","last_became_active":"2024-05-09T08:02:19.953166+0000","last_became_peered":"2024-05-09T08:02:19.953166+0000","last_unstale":"2024-05-09T08:02:22.617683+0000","last_undegraded":"2024-05-09T08:02:22.617683+0000","last_fullsized":"2024-05-09T08:02:22.617683+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:18:40.377352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:21.677885+0000","last_change":"2024-05-09T08:02:15.905285+0000","last_active":"2024-05-09T08:02:21.677885+0000","last_peered":"2024-05-09T08:02:21.677885+0000","last_clean":"2024-05-09T08:02:21.677885+0000","last_became_active":"2024-05-09T08:02:15.904898+0000","last_became_peered":"2024-05-09T08:02:15.904898+0000","last_unstale":"2024-05-09T08:02:21.677885+0000","last_undegraded":"2024-05-09T08:02:21.677885+0000","last_fullsized":"2024-05-09T08:02:21.677885+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:47:32.605288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.619004+0000","last_change":"2024-05-09T08:02:14.868682+0000","last_active":"2024-05-09T08:02:22.619004+0000","last_peered":"2024-05-09T08:02:22.619004+0000","last_clean":"2024-05-09T08:02:22.619004+0000","last_became_active":"2024-05-09T08:02:14.868294+0000","last_became_peered":"2024-05-09T08:02:14.868294+0000","last_unstale":"2024-05-09T08:02:22.619004+0000","last_undegraded":"2024-05-09T08:02:22.619004+0000","last_fullsized":"2024-05-09T08:02:22.619004+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:57:06.093718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:20.933284+0000","last_change":"2024-05-09T08:02:19.956115+0000","last_active":"2024-05-09T08:02:20.933284+0000","last_peered":"2024-05-09T08:02:20.933284+0000","last_clean":"2024-05-09T08:02:20.933284+0000","last_became_active":"2024-05-09T08:02:19.955662+0000","last_became_peered":"2024-05-09T08:02:19.955662+0000","last_unstale":"2024-05-09T08:02:20.933284+0000","last_undegraded":"2024-05-09T08:02:20.933284+0000","last_fullsized":"2024-05-09T08:02:20.933284+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:15:28.286330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.618890+0000","last_change":"2024-05-09T08:02:15.888406+0000","last_active":"2024-05-09T08:02:22.618890+0000","last_peered":"2024-05-09T08:02:22.618890+0000","last_clean":"2024-05-09T08:02:22.618890+0000","last_became_active":"2024-05-09T08:02:15.888090+0000","last_became_peered":"2024-05-09T08:02:15.888090+0000","last_unstale":"2024-05-09T08:02:22.618890+0000","last_undegraded":"2024-05-09T08:02:22.618890+0000","last_fullsized":"2024-05-09T08:02:22.618890+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:53:23.178610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.619656+0000","last_change":"2024-05-09T08:02:14.862463+0000","last_active":"2024-05-09T08:02:22.619656+0000","last_peered":"2024-05-09T08:02:22.619656+0000","last_clean":"2024-05-09T08:02:22.619656+0000","last_became_active":"2024-05-09T08:02:14.862104+0000","last_became_peered":"2024-05-09T08:02:14.862104+0000","last_unstale":"2024-05-09T08:02:22.619656+0000","last_undegraded":"2024-05-09T08:02:22.619656+0000","last_fullsized":"2024-05-09T08:02:22.619656+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:32:48.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:20.932952+0000","last_change":"2024-05-09T08:02:20.053600+0000","last_active":"2024-05-09T08:02:20.932952+0000","last_peered":"2024-05-09T08:02:20.932952+0000","last_clean":"2024-05-09T08:02:20.932952+0000","last_became_active":"2024-05-09T08:02:20.052485+0000","last_became_peered":"2024-05-09T08:02:20.052485+0000","last_unstale":"2024-05-09T08:02:20.932952+0000","last_undegraded":"2024-05-09T08:02:20.932952+0000","last_fullsized":"2024-05-09T08:02:20.932952+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:35:54.628874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:21.677902+0000","last_change":"2024-05-09T08:02:20.054016+0000","last_active":"2024-05-09T08:02:21.677902+0000","last_peered":"2024-05-09T08:02:21.677902+0000","last_clean":"2024-05-09T08:02:21.677902+0000","last_became_active":"2024-05-09T08:02:20.053250+0000","last_became_peered":"2024-05-09T08:02:20.053250+0000","last_unstale":"2024-05-09T08:02:21.677902+0000","last_undegraded":"2024-05-09T08:02:21.677902+0000","last_fullsized":"2024-05-09T08:02:21.677902+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:52:40.529638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:20.932409+0000","last_change":"2024-05-09T08:02:15.907080+0000","last_active":"2024-05-09T08:02:20.932409+0000","last_peered":"2024-05-09T08:02:20.932409+0000","last_clean":"2024-05-09T08:02:20.932409+0000","last_became_active":"2024-05-09T08:02:15.906600+0000","last_became_peered":"2024-05-09T08:02:15.906600+0000","last_unstale":"2024-05-09T08:02:20.932409+0000","last_undegraded":"2024-05-09T08:02:20.932409+0000","last_fullsized":"2024-05-09T08:02:20.932409+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:43:06.938904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.617668+0000","last_change":"2024-05-09T08:02:14.870990+0000","last_active":"2024-05-09T08:02:22.617668+0000","last_peered":"2024-05-09T08:02:22.617668+0000","last_clean":"2024-05-09T08:02:22.617668+0000","last_became_active":"2024-05-09T08:02:14.869818+0000","last_became_peered":"2024-05-09T08:02:14.869818+0000","last_unstale":"2024-05-09T08:02:22.617668+0000","last_undegraded":"2024-05-09T08:02:22.617668+0000","last_fullsized":"2024-05-09T08:02:22.617668+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:25:23.306827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.619498+0000","last_change":"2024-05-09T08:02:19.952582+0000","last_active":"2024-05-09T08:02:22.619498+0000","last_peered":"2024-05-09T08:02:22.619498+0000","last_clean":"2024-05-09T08:02:22.619498+0000","last_became_active":"2024-05-09T08:02:19.952117+0000","last_became_peered":"2024-05-09T08:02:19.952117+0000","last_unstale":"2024-05-09T08:02:22.619498+0000","last_undegraded":"2024-05-09T08:02:22.619498+0000","last_fullsized":"2024-05-09T08:02:22.619498+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:42:00.939587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:20.933300+0000","last_change":"2024-05-09T08:02:15.913109+0000","last_active":"2024-05-09T08:02:20.933300+0000","last_peered":"2024-05-09T08:02:20.933300+0000","last_clean":"2024-05-09T08:02:20.933300+0000","last_became_active":"2024-05-09T08:02:15.912883+0000","last_became_peered":"2024-05-09T08:02:15.912883+0000","last_unstale":"2024-05-09T08:02:20.933300+0000","last_undegraded":"2024-05-09T08:02:20.933300+0000","last_fullsized":"2024-05-09T08:02:20.933300+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:29:59.441314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:20.933233+0000","last_change":"2024-05-09T08:02:14.885256+0000","last_active":"2024-05-09T08:02:20.933233+0000","last_peered":"2024-05-09T08:02:20.933233+0000","last_clean":"2024-05-09T08:02:20.933233+0000","last_became_active":"2024-05-09T08:02:14.884965+0000","last_became_peered":"2024-05-09T08:02:14.884965+0000","last_unstale":"2024-05-09T08:02:20.933233+0000","last_undegraded":"2024-05-09T08:02:20.933233+0000","last_fullsized":"2024-05-09T08:02:20.933233+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:14:52.920972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"24'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.618840+0000","last_change":"2024-05-09T08:02:14.860339+0000","last_active":"2024-05-09T08:02:22.618840+0000","last_peered":"2024-05-09T08:02:22.618840+0000","last_clean":"2024-05-09T08:02:22.618840+0000","last_became_active":"2024-05-09T08:02:14.859917+0000","last_became_peered":"2024-05-09T08:02:14.859917+0000","last_unstale":"2024-05-09T08:02:22.618840+0000","last_undegraded":"2024-05-09T08:02:22.618840+0000","last_fullsized":"2024-05-09T08:02:22.618840+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:14:52.920972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:21.677460+0000","last_change":"2024-05-09T08:02:15.897001+0000","last_active":"2024-05-09T08:02:21.677460+0000","last_peered":"2024-05-09T08:02:21.677460+0000","last_clean":"2024-05-09T08:02:21.677460+0000","last_became_active":"2024-05-09T08:02:15.896708+0000","last_became_peered":"2024-05-09T08:02:15.896708+0000","last_unstale":"2024-05-09T08:02:21.677460+0000","last_undegraded":"2024-05-09T08:02:21.677460+0000","last_fullsized":"2024-05-09T08:02:21.677460+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:29:59.441314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.619069+0000","last_change":"2024-05-09T08:02:19.955369+0000","last_active":"2024-05-09T08:02:22.619069+0000","last_peered":"2024-05-09T08:02:22.619069+0000","last_clean":"2024-05-09T08:02:22.619069+0000","last_became_active":"2024-05-09T08:02:19.954829+0000","last_became_peered":"2024-05-09T08:02:19.954829+0000","last_unstale":"2024-05-09T08:02:22.619069+0000","last_undegraded":"2024-05-09T08:02:22.619069+0000","last_fullsized":"2024-05-09T08:02:22.619069+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:49:58.963879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"24'1","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:20.934324+0000","last_change":"2024-05-09T08:02:14.883856+0000","last_active":"2024-05-09T08:02:20.934324+0000","last_peered":"2024-05-09T08:02:20.934324+0000","last_clean":"2024-05-09T08:02:20.934324+0000","last_became_active":"2024-05-09T08:02:14.882610+0000","last_became_peered":"2024-05-09T08:02:14.882610+0000","last_unstale":"2024-05-09T08:02:20.934324+0000","last_undegraded":"2024-05-09T08:02:20.934324+0000","last_fullsized":"2024-05-09T08:02:20.934324+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:19:31.913236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.617629+0000","last_change":"2024-05-09T08:02:15.889974+0000","last_active":"2024-05-09T08:02:22.617629+0000","last_peered":"2024-05-09T08:02:22.617629+0000","last_clean":"2024-05-09T08:02:22.617629+0000","last_became_active":"2024-05-09T08:02:15.889650+0000","last_became_peered":"2024-05-09T08:02:15.889650+0000","last_unstale":"2024-05-09T08:02:22.617629+0000","last_undegraded":"2024-05-09T08:02:22.617629+0000","last_fullsized":"2024-05-09T08:02:22.617629+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:00:41.613116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:20.934556+0000","last_change":"2024-05-09T08:02:19.955362+0000","last_active":"2024-05-09T08:02:20.934556+0000","last_peered":"2024-05-09T08:02:20.934556+0000","last_clean":"2024-05-09T08:02:20.934556+0000","last_became_active":"2024-05-09T08:02:19.954947+0000","last_became_peered":"2024-05-09T08:02:19.954947+0000","last_unstale":"2024-05-09T08:02:20.934556+0000","last_undegraded":"2024-05-09T08:02:20.934556+0000","last_fullsized":"2024-05-09T08:02:20.934556+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:43:30.549966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.618982+0000","last_change":"2024-05-09T08:02:15.909379+0000","last_active":"2024-05-09T08:02:22.618982+0000","last_peered":"2024-05-09T08:02:22.618982+0000","last_clean":"2024-05-09T08:02:22.618982+0000","last_became_active":"2024-05-09T08:02:15.909086+0000","last_became_peered":"2024-05-09T08:02:15.909086+0000","last_unstale":"2024-05-09T08:02:22.618982+0000","last_undegraded":"2024-05-09T08:02:22.618982+0000","last_fullsized":"2024-05-09T08:02:22.618982+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:05:02.181367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.618963+0000","last_change":"2024-05-09T08:02:15.891242+0000","last_active":"2024-05-09T08:02:22.618963+0000","last_peered":"2024-05-09T08:02:22.618963+0000","last_clean":"2024-05-09T08:02:22.618963+0000","last_became_active":"2024-05-09T08:02:15.890640+0000","last_became_peered":"2024-05-09T08:02:15.890640+0000","last_unstale":"2024-05-09T08:02:22.618963+0000","last_undegraded":"2024-05-09T08:02:22.618963+0000","last_fullsized":"2024-05-09T08:02:22.618963+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:57:07.107359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.617950+0000","last_change":"2024-05-09T08:02:15.899001+0000","last_active":"2024-05-09T08:02:22.617950+0000","last_peered":"2024-05-09T08:02:22.617950+0000","last_clean":"2024-05-09T08:02:22.617950+0000","last_became_active":"2024-05-09T08:02:15.898695+0000","last_became_peered":"2024-05-09T08:02:15.898695+0000","last_unstale":"2024-05-09T08:02:22.617950+0000","last_undegraded":"2024-05-09T08:02:22.617950+0000","last_fullsized":"2024-05-09T08:02:22.617950+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:41:27.402821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:21.677395+0000","last_change":"2024-05-09T08:02:15.907374+0000","last_active":"2024-05-09T08:02:21.677395+0000","last_peered":"2024-05-09T08:02:21.677395+0000","last_clean":"2024-05-09T08:02:21.677395+0000","last_became_active":"2024-05-09T08:02:15.906767+0000","last_became_peered":"2024-05-09T08:02:15.906767+0000","last_unstale":"2024-05-09T08:02:21.677395+0000","last_undegraded":"2024-05-09T08:02:21.677395+0000","last_fullsized":"2024-05-09T08:02:21.677395+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:05:02.181367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:20.935527+0000","last_change":"2024-05-09T08:02:15.905326+0000","last_active":"2024-05-09T08:02:20.935527+0000","last_peered":"2024-05-09T08:02:20.935527+0000","last_clean":"2024-05-09T08:02:20.935527+0000","last_became_active":"2024-05-09T08:02:15.905083+0000","last_became_peered":"2024-05-09T08:02:15.905083+0000","last_unstale":"2024-05-09T08:02:20.935527+0000","last_undegraded":"2024-05-09T08:02:20.935527+0000","last_fullsized":"2024-05-09T08:02:20.935527+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:23:40.248821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:20.934866+0000","last_change":"2024-05-09T08:02:15.907753+0000","last_active":"2024-05-09T08:02:20.934866+0000","last_peered":"2024-05-09T08:02:20.934866+0000","last_clean":"2024-05-09T08:02:20.934866+0000","last_became_active":"2024-05-09T08:02:15.907480+0000","last_became_peered":"2024-05-09T08:02:15.907480+0000","last_unstale":"2024-05-09T08:02:20.934866+0000","last_undegraded":"2024-05-09T08:02:20.934866+0000","last_fullsized":"2024-05-09T08:02:20.934866+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:20:07.368327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.619474+0000","last_change":"2024-05-09T08:02:15.888605+0000","last_active":"2024-05-09T08:02:22.619474+0000","last_peered":"2024-05-09T08:02:22.619474+0000","last_clean":"2024-05-09T08:02:22.619474+0000","last_became_active":"2024-05-09T08:02:15.887981+0000","last_became_peered":"2024-05-09T08:02:15.887981+0000","last_unstale":"2024-05-09T08:02:22.619474+0000","last_undegraded":"2024-05-09T08:02:22.619474+0000","last_fullsized":"2024-05-09T08:02:22.619474+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:47:32.605288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:20.935282+0000","last_change":"2024-05-09T08:02:15.904744+0000","last_active":"2024-05-09T08:02:20.935282+0000","last_peered":"2024-05-09T08:02:20.935282+0000","last_clean":"2024-05-09T08:02:20.935282+0000","last_became_active":"2024-05-09T08:02:15.904334+0000","last_became_peered":"2024-05-09T08:02:15.904334+0000","last_unstale":"2024-05-09T08:02:20.935282+0000","last_undegraded":"2024-05-09T08:02:20.935282+0000","last_fullsized":"2024-05-09T08:02:20.935282+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:30:33.351158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.617634+0000","last_change":"2024-05-09T08:02:15.904835+0000","last_active":"2024-05-09T08:02:22.617634+0000","last_peered":"2024-05-09T08:02:22.617634+0000","last_clean":"2024-05-09T08:02:22.617634+0000","last_became_active":"2024-05-09T08:02:15.904416+0000","last_became_peered":"2024-05-09T08:02:15.904416+0000","last_unstale":"2024-05-09T08:02:22.617634+0000","last_undegraded":"2024-05-09T08:02:22.617634+0000","last_fullsized":"2024-05-09T08:02:22.617634+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:47:32.605288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:20.933091+0000","last_change":"2024-05-09T08:02:15.898324+0000","last_active":"2024-05-09T08:02:20.933091+0000","last_peered":"2024-05-09T08:02:20.933091+0000","last_clean":"2024-05-09T08:02:20.933091+0000","last_became_active":"2024-05-09T08:02:15.897981+0000","last_became_peered":"2024-05-09T08:02:15.897981+0000","last_unstale":"2024-05-09T08:02:20.933091+0000","last_undegraded":"2024-05-09T08:02:20.933091+0000","last_fullsized":"2024-05-09T08:02:20.933091+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:01:50.341133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.617909+0000","last_change":"2024-05-09T08:02:15.890945+0000","last_active":"2024-05-09T08:02:22.617909+0000","last_peered":"2024-05-09T08:02:22.617909+0000","last_clean":"2024-05-09T08:02:22.617909+0000","last_became_active":"2024-05-09T08:02:15.890647+0000","last_became_peered":"2024-05-09T08:02:15.890647+0000","last_unstale":"2024-05-09T08:02:22.617909+0000","last_undegraded":"2024-05-09T08:02:22.617909+0000","last_fullsized":"2024-05-09T08:02:22.617909+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:41:56.871863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:21.677105+0000","last_change":"2024-05-09T08:02:15.913728+0000","last_active":"2024-05-09T08:02:21.677105+0000","last_peered":"2024-05-09T08:02:21.677105+0000","last_clean":"2024-05-09T08:02:21.677105+0000","last_became_active":"2024-05-09T08:02:15.909818+0000","last_became_peered":"2024-05-09T08:02:15.909818+0000","last_unstale":"2024-05-09T08:02:21.677105+0000","last_undegraded":"2024-05-09T08:02:21.677105+0000","last_fullsized":"2024-05-09T08:02:21.677105+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:41:55.788033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.619452+0000","last_change":"2024-05-09T08:02:15.885025+0000","last_active":"2024-05-09T08:02:22.619452+0000","last_peered":"2024-05-09T08:02:22.619452+0000","last_clean":"2024-05-09T08:02:22.619452+0000","last_became_active":"2024-05-09T08:02:15.884517+0000","last_became_peered":"2024-05-09T08:02:15.884517+0000","last_unstale":"2024-05-09T08:02:22.619452+0000","last_undegraded":"2024-05-09T08:02:22.619452+0000","last_fullsized":"2024-05-09T08:02:22.619452+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:01:50.341133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:21.077234+0000","last_change":"2024-05-09T08:02:15.890306+0000","last_active":"2024-05-09T08:02:21.077234+0000","last_peered":"2024-05-09T08:02:21.077234+0000","last_clean":"2024-05-09T08:02:21.077234+0000","last_became_active":"2024-05-09T08:02:15.889998+0000","last_became_peered":"2024-05-09T08:02:15.889998+0000","last_unstale":"2024-05-09T08:02:21.077234+0000","last_undegraded":"2024-05-09T08:02:21.077234+0000","last_fullsized":"2024-05-09T08:02:21.077234+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:16:57.446444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.619318+0000","last_change":"2024-05-09T08:02:15.886417+0000","last_active":"2024-05-09T08:02:22.619318+0000","last_peered":"2024-05-09T08:02:22.619318+0000","last_clean":"2024-05-09T08:02:22.619318+0000","last_became_active":"2024-05-09T08:02:15.885525+0000","last_became_peered":"2024-05-09T08:02:15.885525+0000","last_unstale":"2024-05-09T08:02:22.619318+0000","last_undegraded":"2024-05-09T08:02:22.619318+0000","last_fullsized":"2024-05-09T08:02:22.619318+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:43:02.075442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.618616+0000","last_change":"2024-05-09T08:02:15.893172+0000","last_active":"2024-05-09T08:02:22.618616+0000","last_peered":"2024-05-09T08:02:22.618616+0000","last_clean":"2024-05-09T08:02:22.618616+0000","last_became_active":"2024-05-09T08:02:15.892886+0000","last_became_peered":"2024-05-09T08:02:15.892886+0000","last_unstale":"2024-05-09T08:02:22.618616+0000","last_undegraded":"2024-05-09T08:02:22.618616+0000","last_fullsized":"2024-05-09T08:02:22.618616+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:59:14.957806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.617513+0000","last_change":"2024-05-09T08:02:15.908659+0000","last_active":"2024-05-09T08:02:22.617513+0000","last_peered":"2024-05-09T08:02:22.617513+0000","last_clean":"2024-05-09T08:02:22.617513+0000","last_became_active":"2024-05-09T08:02:15.908307+0000","last_became_peered":"2024-05-09T08:02:15.908307+0000","last_unstale":"2024-05-09T08:02:22.617513+0000","last_undegraded":"2024-05-09T08:02:22.617513+0000","last_fullsized":"2024-05-09T08:02:22.617513+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:20:07.368327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:20.933418+0000","last_change":"2024-05-09T08:02:15.885800+0000","last_active":"2024-05-09T08:02:20.933418+0000","last_peered":"2024-05-09T08:02:20.933418+0000","last_clean":"2024-05-09T08:02:20.933418+0000","last_became_active":"2024-05-09T08:02:15.885185+0000","last_became_peered":"2024-05-09T08:02:15.885185+0000","last_unstale":"2024-05-09T08:02:20.933418+0000","last_undegraded":"2024-05-09T08:02:20.933418+0000","last_fullsized":"2024-05-09T08:02:20.933418+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:01:50.341133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:21.677190+0000","last_change":"2024-05-09T08:02:15.911496+0000","last_active":"2024-05-09T08:02:21.677190+0000","last_peered":"2024-05-09T08:02:21.677190+0000","last_clean":"2024-05-09T08:02:21.677190+0000","last_became_active":"2024-05-09T08:02:15.911142+0000","last_became_peered":"2024-05-09T08:02:15.911142+0000","last_unstale":"2024-05-09T08:02:21.677190+0000","last_undegraded":"2024-05-09T08:02:21.677190+0000","last_fullsized":"2024-05-09T08:02:21.677190+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:27:59.315750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:20.933605+0000","last_change":"2024-05-09T08:02:15.898847+0000","last_active":"2024-05-09T08:02:20.933605+0000","last_peered":"2024-05-09T08:02:20.933605+0000","last_clean":"2024-05-09T08:02:20.933605+0000","last_became_active":"2024-05-09T08:02:15.898480+0000","last_became_peered":"2024-05-09T08:02:15.898480+0000","last_unstale":"2024-05-09T08:02:20.933605+0000","last_undegraded":"2024-05-09T08:02:20.933605+0000","last_fullsized":"2024-05-09T08:02:20.933605+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:20:07.368327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.618261+0000","last_change":"2024-05-09T08:02:15.883117+0000","last_active":"2024-05-09T08:02:22.618261+0000","last_peered":"2024-05-09T08:02:22.618261+0000","last_clean":"2024-05-09T08:02:22.618261+0000","last_became_active":"2024-05-09T08:02:15.882141+0000","last_became_peered":"2024-05-09T08:02:15.882141+0000","last_unstale":"2024-05-09T08:02:22.618261+0000","last_undegraded":"2024-05-09T08:02:22.618261+0000","last_fullsized":"2024-05-09T08:02:22.618261+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:57:11.352151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.618965+0000","last_change":"2024-05-09T08:02:15.893318+0000","last_active":"2024-05-09T08:02:22.618965+0000","last_peered":"2024-05-09T08:02:22.618965+0000","last_clean":"2024-05-09T08:02:22.618965+0000","last_became_active":"2024-05-09T08:02:15.892964+0000","last_became_peered":"2024-05-09T08:02:15.892964+0000","last_unstale":"2024-05-09T08:02:22.618965+0000","last_undegraded":"2024-05-09T08:02:22.618965+0000","last_fullsized":"2024-05-09T08:02:22.618965+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:20:07.368327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.619428+0000","last_change":"2024-05-09T08:02:15.886482+0000","last_active":"2024-05-09T08:02:22.619428+0000","last_peered":"2024-05-09T08:02:22.619428+0000","last_clean":"2024-05-09T08:02:22.619428+0000","last_became_active":"2024-05-09T08:02:15.886183+0000","last_became_peered":"2024-05-09T08:02:15.886183+0000","last_unstale":"2024-05-09T08:02:22.619428+0000","last_undegraded":"2024-05-09T08:02:22.619428+0000","last_fullsized":"2024-05-09T08:02:22.619428+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:23:40.248821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:21.077365+0000","last_change":"2024-05-09T08:02:15.887506+0000","last_active":"2024-05-09T08:02:21.077365+0000","last_peered":"2024-05-09T08:02:21.077365+0000","last_clean":"2024-05-09T08:02:21.077365+0000","last_became_active":"2024-05-09T08:02:15.887225+0000","last_became_peered":"2024-05-09T08:02:15.887225+0000","last_unstale":"2024-05-09T08:02:21.077365+0000","last_undegraded":"2024-05-09T08:02:21.077365+0000","last_fullsized":"2024-05-09T08:02:21.077365+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:25:24.736323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:20.933062+0000","last_change":"2024-05-09T08:02:15.902065+0000","last_active":"2024-05-09T08:02:20.933062+0000","last_peered":"2024-05-09T08:02:20.933062+0000","last_clean":"2024-05-09T08:02:20.933062+0000","last_became_active":"2024-05-09T08:02:15.901766+0000","last_became_peered":"2024-05-09T08:02:15.901766+0000","last_unstale":"2024-05-09T08:02:20.933062+0000","last_undegraded":"2024-05-09T08:02:20.933062+0000","last_fullsized":"2024-05-09T08:02:20.933062+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:05:02.181367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:21.076616+0000","last_change":"2024-05-09T08:02:15.911058+0000","last_active":"2024-05-09T08:02:21.076616+0000","last_peered":"2024-05-09T08:02:21.076616+0000","last_clean":"2024-05-09T08:02:21.076616+0000","last_became_active":"2024-05-09T08:02:15.910716+0000","last_became_peered":"2024-05-09T08:02:15.910716+0000","last_unstale":"2024-05-09T08:02:21.076616+0000","last_undegraded":"2024-05-09T08:02:21.076616+0000","last_fullsized":"2024-05-09T08:02:21.076616+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:41:56.871863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.617832+0000","last_change":"2024-05-09T08:02:15.908653+0000","last_active":"2024-05-09T08:02:22.617832+0000","last_peered":"2024-05-09T08:02:22.617832+0000","last_clean":"2024-05-09T08:02:22.617832+0000","last_became_active":"2024-05-09T08:02:15.908290+0000","last_became_peered":"2024-05-09T08:02:15.908290+0000","last_unstale":"2024-05-09T08:02:22.617832+0000","last_undegraded":"2024-05-09T08:02:22.617832+0000","last_fullsized":"2024-05-09T08:02:22.617832+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:18:36.309627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:21.076688+0000","last_change":"2024-05-09T08:02:15.896652+0000","last_active":"2024-05-09T08:02:21.076688+0000","last_peered":"2024-05-09T08:02:21.076688+0000","last_clean":"2024-05-09T08:02:21.076688+0000","last_became_active":"2024-05-09T08:02:15.896384+0000","last_became_peered":"2024-05-09T08:02:15.896384+0000","last_unstale":"2024-05-09T08:02:21.076688+0000","last_undegraded":"2024-05-09T08:02:21.076688+0000","last_fullsized":"2024-05-09T08:02:21.076688+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:18:36.309627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.617988+0000","last_change":"2024-05-09T08:02:15.898998+0000","last_active":"2024-05-09T08:02:22.617988+0000","last_peered":"2024-05-09T08:02:22.617988+0000","last_clean":"2024-05-09T08:02:22.617988+0000","last_became_active":"2024-05-09T08:02:15.898670+0000","last_became_peered":"2024-05-09T08:02:15.898670+0000","last_unstale":"2024-05-09T08:02:22.617988+0000","last_undegraded":"2024-05-09T08:02:22.617988+0000","last_fullsized":"2024-05-09T08:02:22.617988+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:23:40.248821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:21.677605+0000","last_change":"2024-05-09T08:02:15.905078+0000","last_active":"2024-05-09T08:02:21.677605+0000","last_peered":"2024-05-09T08:02:21.677605+0000","last_clean":"2024-05-09T08:02:21.677605+0000","last_became_active":"2024-05-09T08:02:15.904736+0000","last_became_peered":"2024-05-09T08:02:15.904736+0000","last_unstale":"2024-05-09T08:02:21.677605+0000","last_undegraded":"2024-05-09T08:02:21.677605+0000","last_fullsized":"2024-05-09T08:02:21.677605+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:01:50.341133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:20.936449+0000","last_change":"2024-05-09T08:02:15.897743+0000","last_active":"2024-05-09T08:02:20.936449+0000","last_peered":"2024-05-09T08:02:20.936449+0000","last_clean":"2024-05-09T08:02:20.936449+0000","last_became_active":"2024-05-09T08:02:15.897367+0000","last_became_peered":"2024-05-09T08:02:15.897367+0000","last_unstale":"2024-05-09T08:02:20.936449+0000","last_undegraded":"2024-05-09T08:02:20.936449+0000","last_fullsized":"2024-05-09T08:02:20.936449+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:47:32.605288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:21.077273+0000","last_change":"2024-05-09T08:02:15.884163+0000","last_active":"2024-05-09T08:02:21.077273+0000","last_peered":"2024-05-09T08:02:21.077273+0000","last_clean":"2024-05-09T08:02:21.077273+0000","last_became_active":"2024-05-09T08:02:15.883430+0000","last_became_peered":"2024-05-09T08:02:15.883430+0000","last_unstale":"2024-05-09T08:02:21.077273+0000","last_undegraded":"2024-05-09T08:02:21.077273+0000","last_fullsized":"2024-05-09T08:02:21.077273+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:05:15.555592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574859,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32024,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339816,"statfs":{"total":96636764160,"available":96603971584,"internally_reserved":0,"allocated":864256,"data_stored":695877,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574859,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31448,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340392,"statfs":{"total":96636764160,"available":96604561408,"internally_reserved":0,"allocated":278528,"data_stored":108769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574859,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31444,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340396,"statfs":{"total":96636764160,"available":96604565504,"internally_reserved":0,"allocated":270336,"data_stored":105017,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574860,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31452,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340388,"statfs":{"total":96636764160,"available":96604557312,"internally_reserved":0,"allocated":278528,"data_stored":105053,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574860,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":108769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574860,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":266240,"data_stored":104927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574861,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":274432,"data_stored":105087,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574860,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32032,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339808,"statfs":{"total":96636764160,"available":96603963392,"internally_reserved":0,"allocated":872448,"data_stored":695933,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T08:02:57.334 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-09T08:02:57.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:57.559+0000 7efe68a00640 1 Processor -- start 2024-05-09T08:02:57.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:57.560+0000 7efe68a00640 1 -- start start 2024-05-09T08:02:57.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:57.561+0000 7efe68a00640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7efe64152500 0x7efe640c8180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:57.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:57.561+0000 7efe68a00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7efe640cc730 0x7efe6415c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:57.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:57.561+0000 7efe68a00640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efe640c6a30 con 0x7efe640c7660 2024-05-09T08:02:57.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:57.561+0000 7efe68a00640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7efe640c5e50 con 0x7efe64152500 2024-05-09T08:02:57.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:57.561+0000 7efe68a00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7efe640c22d0 con 0x7efe640cc730 2024-05-09T08:02:57.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:57.561+0000 7efe63e00640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7efe640c7660 0x7efe6406b250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:56086/0 (socket says 172.21.15.1:56086) 2024-05-09T08:02:57.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:57.561+0000 7efe63e00640 1 -- 172.21.15.1:0/3700724627 learned_addr learned my addr 172.21.15.1:0/3700724627 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T08:02:57.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:57.561+0000 7efe63400640 1 --2- 172.21.15.1:0/3700724627 >> v2:172.21.15.1:3300/0 conn(0x7efe64152500 0x7efe640c8180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:57.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:57.561+0000 7efe62a00640 1 --2- 172.21.15.1:0/3700724627 >> v2:172.21.15.7:3300/0 conn(0x7efe640cc730 0x7efe6415c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:57.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:57.562+0000 7efe62000640 1 -- 172.21.15.1:0/3700724627 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 633681981 0 0) 0x7efe640c6a30 con 0x7efe640c7660 2024-05-09T08:02:57.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:57.562+0000 7efe62000640 1 -- 172.21.15.1:0/3700724627 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efe48003520 con 0x7efe640c7660 2024-05-09T08:02:57.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:57.562+0000 7efe63400640 1 -- 172.21.15.1:0/3700724627 >> v2:172.21.15.7:3300/0 conn(0x7efe640cc730 msgr2=0x7efe6415c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:02:57.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:57.562+0000 7efe63400640 1 --2- 172.21.15.1:0/3700724627 >> v2:172.21.15.7:3300/0 conn(0x7efe640cc730 0x7efe6415c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:57.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:57.562+0000 7efe63400640 1 -- 172.21.15.1:0/3700724627 >> v1:172.21.15.1:6789/0 conn(0x7efe640c7660 legacy=0x7efe6406b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:57.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:57.562+0000 7efe63400640 1 -- 172.21.15.1:0/3700724627 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe640c6cf0 con 0x7efe64152500 2024-05-09T08:02:57.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:57.563+0000 7efe63400640 1 --2- 172.21.15.1:0/3700724627 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efe64152500 0x7efe640c8180 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7efe64155f40 tx=0x7efe5402a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da14965092e214f4 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:57.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:57.563+0000 7efe62000640 1 -- 172.21.15.1:0/3700724627 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe54037070 con 0x7efe64152500 2024-05-09T08:02:57.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:57.563+0000 7efe62000640 1 -- 172.21.15.1:0/3700724627 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe54032440 con 0x7efe64152500 2024-05-09T08:02:57.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:57.563+0000 7efe62000640 1 -- 172.21.15.1:0/3700724627 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe54032810 con 0x7efe64152500 2024-05-09T08:02:57.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:57.564+0000 7efe68a00640 1 -- 172.21.15.1:0/3700724627 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efe64152500 msgr2=0x7efe640c8180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:57.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:57.564+0000 7efe68a00640 1 --2- 172.21.15.1:0/3700724627 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efe64152500 0x7efe640c8180 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7efe64155f40 tx=0x7efe5402a250 comp rx=0 tx=0).stop 2024-05-09T08:02:57.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:57.564+0000 7efe68a00640 1 -- 172.21.15.1:0/3700724627 shutdown_connections 2024-05-09T08:02:57.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:57.564+0000 7efe68a00640 1 --2- 172.21.15.1:0/3700724627 >> v2:172.21.15.7:3300/0 conn(0x7efe640cc730 0x7efe6415c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:57.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:57.564+0000 7efe68a00640 1 --2- 172.21.15.1:0/3700724627 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efe64152500 0x7efe640c8180 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:57.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:57.564+0000 7efe68a00640 1 -- 172.21.15.1:0/3700724627 >> 172.21.15.1:0/3700724627 conn(0x7efe640cbc10 msgr2=0x7efe640cc020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:02:57.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:57.564+0000 7efe68a00640 1 -- 172.21.15.1:0/3700724627 shutdown_connections 2024-05-09T08:02:57.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:57.564+0000 7efe68a00640 1 -- 172.21.15.1:0/3700724627 wait complete. 2024-05-09T08:02:57.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:57.566+0000 7efe68a00640 1 Processor -- start 2024-05-09T08:02:57.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:57.566+0000 7efe68a00640 1 -- start start 2024-05-09T08:02:57.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:57.567+0000 7efe68a00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efe640c7660 0x7efe641f3360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:57.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:57.567+0000 7efe68a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efe640cc730 0x7efe640c4930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:57.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:57.567+0000 7efe68a00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7efe64152500 0x7efe6414f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:57.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:57.567+0000 7efe68a00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7efe640c72b0 con 0x7efe640c7660 2024-05-09T08:02:57.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:57.567+0000 7efe68a00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7efe640ccb50 con 0x7efe640cc730 2024-05-09T08:02:57.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:57.567+0000 7efe68a00640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7efe6415b880 con 0x7efe64152500 2024-05-09T08:02:57.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:57.567+0000 7efe63400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efe640c7660 0x7efe641f3360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:57.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:57.567+0000 7efe63400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efe640c7660 0x7efe641f3360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38340/0 (socket says 172.21.15.1:38340) 2024-05-09T08:02:57.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:57.567+0000 7efe63400640 1 -- 172.21.15.1:0/1825773215 learned_addr learned my addr 172.21.15.1:0/1825773215 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:02:57.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:57.567+0000 7efe62a00640 1 --2- 172.21.15.1:0/1825773215 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efe640cc730 0x7efe640c4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:57.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:57.567+0000 7efe63e00640 1 --2- 172.21.15.1:0/1825773215 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7efe64152500 0x7efe6414f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:57.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:57.568+0000 7efe63400640 1 -- 172.21.15.1:0/1825773215 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7efe64152500 msgr2=0x7efe6414f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:57.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:57.568+0000 7efe63400640 1 --2- 172.21.15.1:0/1825773215 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7efe64152500 0x7efe6414f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:57.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:57.568+0000 7efe63400640 1 -- 172.21.15.1:0/1825773215 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efe640cc730 msgr2=0x7efe640c4930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:57.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:57.568+0000 7efe63400640 1 --2- 172.21.15.1:0/1825773215 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efe640cc730 0x7efe640c4930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:57.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:57.568+0000 7efe63400640 1 -- 172.21.15.1:0/1825773215 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe640b9230 con 0x7efe640c7660 2024-05-09T08:02:57.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:57.568+0000 7efe63400640 1 --2- 172.21.15.1:0/1825773215 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efe640c7660 0x7efe641f3360 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7efe54000c00 tx=0x7efe540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:57.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:57.568+0000 7efe5b400640 1 -- 172.21.15.1:0/1825773215 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe54041020 con 0x7efe640c7660 2024-05-09T08:02:57.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:57.568+0000 7efe5b400640 1 -- 172.21.15.1:0/1825773215 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe5403d700 con 0x7efe640c7660 2024-05-09T08:02:57.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:57.568+0000 7efe5b400640 1 -- 172.21.15.1:0/1825773215 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe54037040 con 0x7efe640c7660 2024-05-09T08:02:57.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:57.568+0000 7efe68a00640 1 -- 172.21.15.1:0/1825773215 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe640b94f0 con 0x7efe640c7660 2024-05-09T08:02:57.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:57.569+0000 7efe68a00640 1 -- 172.21.15.1:0/1825773215 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe640c3110 con 0x7efe640c7660 2024-05-09T08:02:57.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:57.570+0000 7efe68a00640 1 -- 172.21.15.1:0/1825773215 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe6404f660 con 0x7efe640c7660 2024-05-09T08:02:57.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:57.571+0000 7efe5b400640 1 -- 172.21.15.1:0/1825773215 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7efe54044020 con 0x7efe640c7660 2024-05-09T08:02:57.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:57.572+0000 7efe5b400640 1 --2- 172.21.15.1:0/1825773215 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7efe4406be30 0x7efe4406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:57.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:57.572+0000 7efe5b400640 1 -- 172.21.15.1:0/1825773215 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7efe540ab820 con 0x7efe640c7660 2024-05-09T08:02:57.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:57.572+0000 7efe62a00640 1 --2- 172.21.15.1:0/1825773215 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7efe4406be30 0x7efe4406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:57.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:57.572+0000 7efe62a00640 1 --2- 172.21.15.1:0/1825773215 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7efe4406be30 0x7efe4406e2f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7efe640c3fc0 tx=0x7efe50009190 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:57.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:57.576+0000 7efe5b400640 1 -- 172.21.15.1:0/1825773215 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7efe5404b030 con 0x7efe640c7660 2024-05-09T08:02:57.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:57.765+0000 7efe68a00640 1 -- 172.21.15.1:0/1825773215 --> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7efe640c2a40 con 0x7efe4406be30 2024-05-09T08:02:57.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:57.775+0000 7efe5b400640 1 -- 172.21.15.1:0/1825773215 <== mgr.4114 v2:172.21.15.7:6832/3562612464 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332535 (secure 0 0 0) 0x7efe640c2a40 con 0x7efe4406be30 2024-05-09T08:02:57.797 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T08:02:57.799 INFO:teuthology.orchestra.run.smithi001.stderr:dumped all 2024-05-09T08:02:57.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:57.782+0000 7efe68a00640 1 -- 172.21.15.1:0/1825773215 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7efe4406be30 msgr2=0x7efe4406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:57.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:57.782+0000 7efe68a00640 1 --2- 172.21.15.1:0/1825773215 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7efe4406be30 0x7efe4406e2f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7efe640c3fc0 tx=0x7efe50009190 comp rx=0 tx=0).stop 2024-05-09T08:02:57.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:57.782+0000 7efe68a00640 1 -- 172.21.15.1:0/1825773215 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efe640c7660 msgr2=0x7efe641f3360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:57.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:57.782+0000 7efe68a00640 1 --2- 172.21.15.1:0/1825773215 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efe640c7660 0x7efe641f3360 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7efe54000c00 tx=0x7efe540046c0 comp rx=0 tx=0).stop 2024-05-09T08:02:57.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:57.782+0000 7efe68a00640 1 -- 172.21.15.1:0/1825773215 shutdown_connections 2024-05-09T08:02:57.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:57.782+0000 7efe68a00640 1 --2- 172.21.15.1:0/1825773215 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7efe4406be30 0x7efe4406e2f0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:57.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:57.782+0000 7efe68a00640 1 --2- 172.21.15.1:0/1825773215 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efe640cc730 0x7efe640c4930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:57.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:57.782+0000 7efe68a00640 1 --2- 172.21.15.1:0/1825773215 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7efe64152500 0x7efe6414f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:57.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:57.782+0000 7efe68a00640 1 --2- 172.21.15.1:0/1825773215 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efe640c7660 0x7efe641f3360 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:57.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:57.782+0000 7efe68a00640 1 -- 172.21.15.1:0/1825773215 >> 172.21.15.1:0/1825773215 conn(0x7efe640cbc10 msgr2=0x7efe640c7ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:02:57.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:57.782+0000 7efe68a00640 1 -- 172.21.15.1:0/1825773215 shutdown_connections 2024-05-09T08:02:57.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:57.782+0000 7efe68a00640 1 -- 172.21.15.1:0/1825773215 wait complete. 2024-05-09T08:02:57.817 INFO:teuthology.orchestra.run.smithi001.stdout:{"pg_ready":true,"pg_map":{"version":43,"stamp":"2024-05-09T08:02:57.106896+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":208,"num_write_kb":2435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160,"ondisk_log_size":160,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":240256,"kb_used_data":3304,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754734464,"statfs":{"total":773094113280,"available":772848091136,"internally_reserved":0,"allocated":3383296,"data_stored":2029432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":242535986},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003367"},"pg_stats":[{"pgid":"2.1f","version":"24'4","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:21.677957+0000","last_change":"2024-05-09T08:02:14.875108+0000","last_active":"2024-05-09T08:02:21.677957+0000","last_peered":"2024-05-09T08:02:21.677957+0000","last_clean":"2024-05-09T08:02:21.677957+0000","last_became_active":"2024-05-09T08:02:14.874841+0000","last_became_peered":"2024-05-09T08:02:14.874841+0000","last_unstale":"2024-05-09T08:02:21.677957+0000","last_undegraded":"2024-05-09T08:02:21.677957+0000","last_fullsized":"2024-05-09T08:02:21.677957+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:00:40.599475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:20.934410+0000","last_change":"2024-05-09T08:02:15.899818+0000","last_active":"2024-05-09T08:02:20.934410+0000","last_peered":"2024-05-09T08:02:20.934410+0000","last_clean":"2024-05-09T08:02:20.934410+0000","last_became_active":"2024-05-09T08:02:15.899479+0000","last_became_peered":"2024-05-09T08:02:15.899479+0000","last_unstale":"2024-05-09T08:02:20.934410+0000","last_undegraded":"2024-05-09T08:02:20.934410+0000","last_fullsized":"2024-05-09T08:02:20.934410+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:18:36.309627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:20.932807+0000","last_change":"2024-05-09T08:02:20.057944+0000","last_active":"2024-05-09T08:02:20.932807+0000","last_peered":"2024-05-09T08:02:20.932807+0000","last_clean":"2024-05-09T08:02:20.932807+0000","last_became_active":"2024-05-09T08:02:20.057016+0000","last_became_peered":"2024-05-09T08:02:20.057016+0000","last_unstale":"2024-05-09T08:02:20.932807+0000","last_undegraded":"2024-05-09T08:02:20.932807+0000","last_fullsized":"2024-05-09T08:02:20.932807+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:17:01.514168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"24'1","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:20.934743+0000","last_change":"2024-05-09T08:02:14.886137+0000","last_active":"2024-05-09T08:02:20.934743+0000","last_peered":"2024-05-09T08:02:20.934743+0000","last_clean":"2024-05-09T08:02:20.934743+0000","last_became_active":"2024-05-09T08:02:14.885830+0000","last_became_peered":"2024-05-09T08:02:14.885830+0000","last_unstale":"2024-05-09T08:02:20.934743+0000","last_undegraded":"2024-05-09T08:02:20.934743+0000","last_fullsized":"2024-05-09T08:02:20.934743+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:46:11.160815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:21.677645+0000","last_change":"2024-05-09T08:02:15.906280+0000","last_active":"2024-05-09T08:02:21.677645+0000","last_peered":"2024-05-09T08:02:21.677645+0000","last_clean":"2024-05-09T08:02:21.677645+0000","last_became_active":"2024-05-09T08:02:15.905590+0000","last_became_peered":"2024-05-09T08:02:15.905590+0000","last_unstale":"2024-05-09T08:02:21.677645+0000","last_undegraded":"2024-05-09T08:02:21.677645+0000","last_fullsized":"2024-05-09T08:02:21.677645+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:20:07.368327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.617387+0000","last_change":"2024-05-09T08:02:19.954548+0000","last_active":"2024-05-09T08:02:22.617387+0000","last_peered":"2024-05-09T08:02:22.617387+0000","last_clean":"2024-05-09T08:02:22.617387+0000","last_became_active":"2024-05-09T08:02:19.954161+0000","last_became_peered":"2024-05-09T08:02:19.954161+0000","last_unstale":"2024-05-09T08:02:22.617387+0000","last_undegraded":"2024-05-09T08:02:22.617387+0000","last_fullsized":"2024-05-09T08:02:22.617387+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:53:27.246334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"24'2","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:21.077047+0000","last_change":"2024-05-09T08:02:14.867128+0000","last_active":"2024-05-09T08:02:21.077047+0000","last_peered":"2024-05-09T08:02:21.077047+0000","last_clean":"2024-05-09T08:02:21.077047+0000","last_became_active":"2024-05-09T08:02:14.866809+0000","last_became_peered":"2024-05-09T08:02:14.866809+0000","last_unstale":"2024-05-09T08:02:21.077047+0000","last_undegraded":"2024-05-09T08:02:21.077047+0000","last_fullsized":"2024-05-09T08:02:21.077047+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:57:06.093718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.618173+0000","last_change":"2024-05-09T08:02:15.888431+0000","last_active":"2024-05-09T08:02:22.618173+0000","last_peered":"2024-05-09T08:02:22.618173+0000","last_clean":"2024-05-09T08:02:22.618173+0000","last_became_active":"2024-05-09T08:02:15.887509+0000","last_became_peered":"2024-05-09T08:02:15.887509+0000","last_unstale":"2024-05-09T08:02:22.618173+0000","last_undegraded":"2024-05-09T08:02:22.618173+0000","last_fullsized":"2024-05-09T08:02:22.618173+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:25:01.238816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.617803+0000","last_change":"2024-05-09T08:02:19.953600+0000","last_active":"2024-05-09T08:02:22.617803+0000","last_peered":"2024-05-09T08:02:22.617803+0000","last_clean":"2024-05-09T08:02:22.617803+0000","last_became_active":"2024-05-09T08:02:19.952275+0000","last_became_peered":"2024-05-09T08:02:19.952275+0000","last_unstale":"2024-05-09T08:02:22.617803+0000","last_undegraded":"2024-05-09T08:02:22.617803+0000","last_fullsized":"2024-05-09T08:02:22.617803+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:25:05.306541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"24'2","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:21.076799+0000","last_change":"2024-05-09T08:02:14.870819+0000","last_active":"2024-05-09T08:02:21.076799+0000","last_peered":"2024-05-09T08:02:21.076799+0000","last_clean":"2024-05-09T08:02:21.076799+0000","last_became_active":"2024-05-09T08:02:14.870476+0000","last_became_peered":"2024-05-09T08:02:14.870476+0000","last_unstale":"2024-05-09T08:02:21.076799+0000","last_undegraded":"2024-05-09T08:02:21.076799+0000","last_fullsized":"2024-05-09T08:02:21.076799+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:43:01.061801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.618291+0000","last_change":"2024-05-09T08:02:15.886471+0000","last_active":"2024-05-09T08:02:22.618291+0000","last_peered":"2024-05-09T08:02:22.618291+0000","last_clean":"2024-05-09T08:02:22.618291+0000","last_became_active":"2024-05-09T08:02:15.886168+0000","last_became_peered":"2024-05-09T08:02:15.886168+0000","last_unstale":"2024-05-09T08:02:22.618291+0000","last_undegraded":"2024-05-09T08:02:22.618291+0000","last_fullsized":"2024-05-09T08:02:22.618291+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:30:33.351158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.618044+0000","last_change":"2024-05-09T08:02:19.947347+0000","last_active":"2024-05-09T08:02:22.618044+0000","last_peered":"2024-05-09T08:02:22.618044+0000","last_clean":"2024-05-09T08:02:22.618044+0000","last_became_active":"2024-05-09T08:02:19.946906+0000","last_became_peered":"2024-05-09T08:02:19.946906+0000","last_unstale":"2024-05-09T08:02:22.618044+0000","last_undegraded":"2024-05-09T08:02:22.618044+0000","last_fullsized":"2024-05-09T08:02:22.618044+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:30:37.418883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.618712+0000","last_change":"2024-05-09T08:02:15.880765+0000","last_active":"2024-05-09T08:02:22.618712+0000","last_peered":"2024-05-09T08:02:22.618712+0000","last_clean":"2024-05-09T08:02:22.618712+0000","last_became_active":"2024-05-09T08:02:15.880439+0000","last_became_peered":"2024-05-09T08:02:15.880439+0000","last_unstale":"2024-05-09T08:02:22.618712+0000","last_undegraded":"2024-05-09T08:02:22.618712+0000","last_fullsized":"2024-05-09T08:02:22.618712+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:29:59.441314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"24'2","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:21.076978+0000","last_change":"2024-05-09T08:02:14.867587+0000","last_active":"2024-05-09T08:02:21.076978+0000","last_peered":"2024-05-09T08:02:21.076978+0000","last_clean":"2024-05-09T08:02:21.076978+0000","last_became_active":"2024-05-09T08:02:14.867291+0000","last_became_peered":"2024-05-09T08:02:14.867291+0000","last_unstale":"2024-05-09T08:02:21.076978+0000","last_undegraded":"2024-05-09T08:02:21.076978+0000","last_fullsized":"2024-05-09T08:02:21.076978+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:20:08.921752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.619129+0000","last_change":"2024-05-09T08:02:19.954902+0000","last_active":"2024-05-09T08:02:22.619129+0000","last_peered":"2024-05-09T08:02:22.619129+0000","last_clean":"2024-05-09T08:02:22.619129+0000","last_became_active":"2024-05-09T08:02:19.954007+0000","last_became_peered":"2024-05-09T08:02:19.954007+0000","last_unstale":"2024-05-09T08:02:22.619129+0000","last_undegraded":"2024-05-09T08:02:22.619129+0000","last_fullsized":"2024-05-09T08:02:22.619129+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:41:31.470546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:20.934123+0000","last_change":"2024-05-09T08:02:15.915155+0000","last_active":"2024-05-09T08:02:20.934123+0000","last_peered":"2024-05-09T08:02:20.934123+0000","last_clean":"2024-05-09T08:02:20.934123+0000","last_became_active":"2024-05-09T08:02:15.914852+0000","last_became_peered":"2024-05-09T08:02:15.914852+0000","last_unstale":"2024-05-09T08:02:20.934123+0000","last_undegraded":"2024-05-09T08:02:20.934123+0000","last_fullsized":"2024-05-09T08:02:20.934123+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:53:23.178610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:20.934852+0000","last_change":"2024-05-09T08:02:14.881770+0000","last_active":"2024-05-09T08:02:20.934852+0000","last_peered":"2024-05-09T08:02:20.934852+0000","last_clean":"2024-05-09T08:02:20.934852+0000","last_became_active":"2024-05-09T08:02:14.881437+0000","last_became_peered":"2024-05-09T08:02:14.881437+0000","last_unstale":"2024-05-09T08:02:20.934852+0000","last_undegraded":"2024-05-09T08:02:20.934852+0000","last_fullsized":"2024-05-09T08:02:20.934852+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:25:23.306827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.618007+0000","last_change":"2024-05-09T08:02:19.955928+0000","last_active":"2024-05-09T08:02:22.618007+0000","last_peered":"2024-05-09T08:02:22.618007+0000","last_clean":"2024-05-09T08:02:22.618007+0000","last_became_active":"2024-05-09T08:02:19.955432+0000","last_became_peered":"2024-05-09T08:02:19.955432+0000","last_unstale":"2024-05-09T08:02:22.618007+0000","last_undegraded":"2024-05-09T08:02:22.618007+0000","last_fullsized":"2024-05-09T08:02:22.618007+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:53:15.458120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:20.933351+0000","last_change":"2024-05-09T08:02:15.913311+0000","last_active":"2024-05-09T08:02:20.933351+0000","last_peered":"2024-05-09T08:02:20.933351+0000","last_clean":"2024-05-09T08:02:20.933351+0000","last_became_active":"2024-05-09T08:02:15.912484+0000","last_became_peered":"2024-05-09T08:02:15.912484+0000","last_unstale":"2024-05-09T08:02:20.933351+0000","last_undegraded":"2024-05-09T08:02:20.933351+0000","last_fullsized":"2024-05-09T08:02:20.933351+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:43:06.938904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:20.933542+0000","last_change":"2024-05-09T08:02:14.868884+0000","last_active":"2024-05-09T08:02:20.933542+0000","last_peered":"2024-05-09T08:02:20.933542+0000","last_clean":"2024-05-09T08:02:20.933542+0000","last_became_active":"2024-05-09T08:02:14.868553+0000","last_became_peered":"2024-05-09T08:02:14.868553+0000","last_unstale":"2024-05-09T08:02:20.933542+0000","last_undegraded":"2024-05-09T08:02:20.933542+0000","last_fullsized":"2024-05-09T08:02:20.933542+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:32:48.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:20.935948+0000","last_change":"2024-05-09T08:02:20.053648+0000","last_active":"2024-05-09T08:02:20.935948+0000","last_peered":"2024-05-09T08:02:20.935948+0000","last_clean":"2024-05-09T08:02:20.935948+0000","last_became_active":"2024-05-09T08:02:20.053152+0000","last_became_peered":"2024-05-09T08:02:20.053152+0000","last_unstale":"2024-05-09T08:02:20.935948+0000","last_undegraded":"2024-05-09T08:02:20.935948+0000","last_fullsized":"2024-05-09T08:02:20.935948+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:59:19.025530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.618049+0000","last_change":"2024-05-09T08:02:15.891115+0000","last_active":"2024-05-09T08:02:22.618049+0000","last_peered":"2024-05-09T08:02:22.618049+0000","last_clean":"2024-05-09T08:02:22.618049+0000","last_became_active":"2024-05-09T08:02:15.890806+0000","last_became_peered":"2024-05-09T08:02:15.890806+0000","last_unstale":"2024-05-09T08:02:22.618049+0000","last_undegraded":"2024-05-09T08:02:22.618049+0000","last_fullsized":"2024-05-09T08:02:22.618049+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:57:07.107359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:21.076482+0000","last_change":"2024-05-09T08:02:14.874987+0000","last_active":"2024-05-09T08:02:21.076482+0000","last_peered":"2024-05-09T08:02:21.076482+0000","last_clean":"2024-05-09T08:02:21.076482+0000","last_became_active":"2024-05-09T08:02:14.874549+0000","last_became_peered":"2024-05-09T08:02:14.874549+0000","last_unstale":"2024-05-09T08:02:21.076482+0000","last_undegraded":"2024-05-09T08:02:21.076482+0000","last_fullsized":"2024-05-09T08:02:21.076482+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:57:10.338510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.618392+0000","last_change":"2024-05-09T08:02:19.952790+0000","last_active":"2024-05-09T08:02:22.618392+0000","last_peered":"2024-05-09T08:02:22.618392+0000","last_clean":"2024-05-09T08:02:22.618392+0000","last_became_active":"2024-05-09T08:02:19.952378+0000","last_became_peered":"2024-05-09T08:02:19.952378+0000","last_unstale":"2024-05-09T08:02:22.618392+0000","last_undegraded":"2024-05-09T08:02:22.618392+0000","last_fullsized":"2024-05-09T08:02:22.618392+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:42:00.939587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:21.678285+0000","last_change":"2024-05-09T08:02:15.913011+0000","last_active":"2024-05-09T08:02:21.678285+0000","last_peered":"2024-05-09T08:02:21.678285+0000","last_clean":"2024-05-09T08:02:21.678285+0000","last_became_active":"2024-05-09T08:02:15.912590+0000","last_became_peered":"2024-05-09T08:02:15.912590+0000","last_unstale":"2024-05-09T08:02:21.678285+0000","last_undegraded":"2024-05-09T08:02:21.678285+0000","last_fullsized":"2024-05-09T08:02:21.678285+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:43:06.938904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"24'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.619767+0000","last_change":"2024-05-09T08:02:14.864763+0000","last_active":"2024-05-09T08:02:22.619767+0000","last_peered":"2024-05-09T08:02:22.619767+0000","last_clean":"2024-05-09T08:02:22.619767+0000","last_became_active":"2024-05-09T08:02:14.864470+0000","last_became_peered":"2024-05-09T08:02:14.864470+0000","last_unstale":"2024-05-09T08:02:22.619767+0000","last_undegraded":"2024-05-09T08:02:22.619767+0000","last_fullsized":"2024-05-09T08:02:22.619767+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:00:40.599475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.617800+0000","last_change":"2024-05-09T08:02:20.054896+0000","last_active":"2024-05-09T08:02:22.617800+0000","last_peered":"2024-05-09T08:02:22.617800+0000","last_clean":"2024-05-09T08:02:22.617800+0000","last_became_active":"2024-05-09T08:02:20.054424+0000","last_became_peered":"2024-05-09T08:02:20.054424+0000","last_unstale":"2024-05-09T08:02:22.617800+0000","last_undegraded":"2024-05-09T08:02:22.617800+0000","last_fullsized":"2024-05-09T08:02:22.617800+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:23:44.316545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:20.935333+0000","last_change":"2024-05-09T08:02:15.905448+0000","last_active":"2024-05-09T08:02:20.935333+0000","last_peered":"2024-05-09T08:02:20.935333+0000","last_clean":"2024-05-09T08:02:20.935333+0000","last_became_active":"2024-05-09T08:02:15.905131+0000","last_became_peered":"2024-05-09T08:02:15.905131+0000","last_unstale":"2024-05-09T08:02:20.935333+0000","last_undegraded":"2024-05-09T08:02:20.935333+0000","last_fullsized":"2024-05-09T08:02:20.935333+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:25:01.238816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"25'1","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:20.936562+0000","last_change":"2024-05-09T08:02:14.868410+0000","last_active":"2024-05-09T08:02:20.936562+0000","last_peered":"2024-05-09T08:02:20.936562+0000","last_clean":"2024-05-09T08:02:20.936562+0000","last_became_active":"2024-05-09T08:02:14.865621+0000","last_became_peered":"2024-05-09T08:02:14.865621+0000","last_unstale":"2024-05-09T08:02:20.936562+0000","last_undegraded":"2024-05-09T08:02:20.936562+0000","last_fullsized":"2024-05-09T08:02:20.936562+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:43:01.061801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:20.932915+0000","last_change":"2024-05-09T08:02:20.057603+0000","last_active":"2024-05-09T08:02:20.932915+0000","last_peered":"2024-05-09T08:02:20.932915+0000","last_clean":"2024-05-09T08:02:20.932915+0000","last_became_active":"2024-05-09T08:02:20.055365+0000","last_became_peered":"2024-05-09T08:02:20.055365+0000","last_unstale":"2024-05-09T08:02:20.932915+0000","last_undegraded":"2024-05-09T08:02:20.932915+0000","last_fullsized":"2024-05-09T08:02:20.932915+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:48:38.283562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:20.934935+0000","last_change":"2024-05-09T08:02:15.890980+0000","last_active":"2024-05-09T08:02:20.934935+0000","last_peered":"2024-05-09T08:02:20.934935+0000","last_clean":"2024-05-09T08:02:20.934935+0000","last_became_active":"2024-05-09T08:02:15.890346+0000","last_became_peered":"2024-05-09T08:02:15.890346+0000","last_unstale":"2024-05-09T08:02:20.934935+0000","last_undegraded":"2024-05-09T08:02:20.934935+0000","last_fullsized":"2024-05-09T08:02:20.934935+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:29:59.441314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:21.077241+0000","last_change":"2024-05-09T08:02:14.876206+0000","last_active":"2024-05-09T08:02:21.077241+0000","last_peered":"2024-05-09T08:02:21.077241+0000","last_clean":"2024-05-09T08:02:21.077241+0000","last_became_active":"2024-05-09T08:02:14.875894+0000","last_became_peered":"2024-05-09T08:02:14.875894+0000","last_unstale":"2024-05-09T08:02:21.077241+0000","last_undegraded":"2024-05-09T08:02:21.077241+0000","last_fullsized":"2024-05-09T08:02:21.077241+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:49:53.882513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:21.677241+0000","last_change":"2024-05-09T08:02:20.055222+0000","last_active":"2024-05-09T08:02:21.677241+0000","last_peered":"2024-05-09T08:02:21.677241+0000","last_clean":"2024-05-09T08:02:21.677241+0000","last_became_active":"2024-05-09T08:02:20.053802+0000","last_became_peered":"2024-05-09T08:02:20.053802+0000","last_unstale":"2024-05-09T08:02:21.677241+0000","last_undegraded":"2024-05-09T08:02:21.677241+0000","last_fullsized":"2024-05-09T08:02:21.677241+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:42:00.939587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.618451+0000","last_change":"2024-05-09T08:02:15.904528+0000","last_active":"2024-05-09T08:02:22.618451+0000","last_peered":"2024-05-09T08:02:22.618451+0000","last_clean":"2024-05-09T08:02:22.618451+0000","last_became_active":"2024-05-09T08:02:15.904049+0000","last_became_peered":"2024-05-09T08:02:15.904049+0000","last_unstale":"2024-05-09T08:02:22.618451+0000","last_undegraded":"2024-05-09T08:02:22.618451+0000","last_fullsized":"2024-05-09T08:02:22.618451+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:53:11.390395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:20.935040+0000","last_change":"2024-05-09T08:02:14.872897+0000","last_active":"2024-05-09T08:02:20.935040+0000","last_peered":"2024-05-09T08:02:20.935040+0000","last_clean":"2024-05-09T08:02:20.935040+0000","last_became_active":"2024-05-09T08:02:14.872096+0000","last_became_peered":"2024-05-09T08:02:14.872096+0000","last_unstale":"2024-05-09T08:02:20.935040+0000","last_undegraded":"2024-05-09T08:02:20.935040+0000","last_fullsized":"2024-05-09T08:02:20.935040+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:00:40.599475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:20.933696+0000","last_change":"2024-05-09T08:02:19.963716+0000","last_active":"2024-05-09T08:02:20.933696+0000","last_peered":"2024-05-09T08:02:20.933696+0000","last_clean":"2024-05-09T08:02:20.933696+0000","last_became_active":"2024-05-09T08:02:19.962922+0000","last_became_peered":"2024-05-09T08:02:19.962922+0000","last_unstale":"2024-05-09T08:02:20.933696+0000","last_undegraded":"2024-05-09T08:02:20.933696+0000","last_fullsized":"2024-05-09T08:02:20.933696+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:42:00.939587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.617918+0000","last_change":"2024-05-09T08:02:15.901450+0000","last_active":"2024-05-09T08:02:22.617918+0000","last_peered":"2024-05-09T08:02:22.617918+0000","last_clean":"2024-05-09T08:02:22.617918+0000","last_became_active":"2024-05-09T08:02:15.901180+0000","last_became_peered":"2024-05-09T08:02:15.901180+0000","last_unstale":"2024-05-09T08:02:22.617918+0000","last_undegraded":"2024-05-09T08:02:22.617918+0000","last_fullsized":"2024-05-09T08:02:22.617918+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:43:02.075442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.618076+0000","last_change":"2024-05-09T08:02:14.876518+0000","last_active":"2024-05-09T08:02:22.618076+0000","last_peered":"2024-05-09T08:02:22.618076+0000","last_clean":"2024-05-09T08:02:22.618076+0000","last_became_active":"2024-05-09T08:02:14.876175+0000","last_became_peered":"2024-05-09T08:02:14.876175+0000","last_unstale":"2024-05-09T08:02:22.618076+0000","last_undegraded":"2024-05-09T08:02:22.618076+0000","last_fullsized":"2024-05-09T08:02:22.618076+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:00:40.599475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:20.934897+0000","last_change":"2024-05-09T08:02:19.960940+0000","last_active":"2024-05-09T08:02:20.934897+0000","last_peered":"2024-05-09T08:02:20.934897+0000","last_clean":"2024-05-09T08:02:20.934897+0000","last_became_active":"2024-05-09T08:02:19.960496+0000","last_became_peered":"2024-05-09T08:02:19.960496+0000","last_unstale":"2024-05-09T08:02:20.934897+0000","last_undegraded":"2024-05-09T08:02:20.934897+0000","last_fullsized":"2024-05-09T08:02:20.934897+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:52:40.529638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:20.932615+0000","last_change":"2024-05-09T08:02:15.891774+0000","last_active":"2024-05-09T08:02:20.932615+0000","last_peered":"2024-05-09T08:02:20.932615+0000","last_clean":"2024-05-09T08:02:20.932615+0000","last_became_active":"2024-05-09T08:02:15.891407+0000","last_became_peered":"2024-05-09T08:02:15.891407+0000","last_unstale":"2024-05-09T08:02:20.932615+0000","last_undegraded":"2024-05-09T08:02:20.932615+0000","last_fullsized":"2024-05-09T08:02:20.932615+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:05:02.181367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.618768+0000","last_change":"2024-05-09T08:02:14.883655+0000","last_active":"2024-05-09T08:02:22.618768+0000","last_peered":"2024-05-09T08:02:22.618768+0000","last_clean":"2024-05-09T08:02:22.618768+0000","last_became_active":"2024-05-09T08:02:14.882359+0000","last_became_peered":"2024-05-09T08:02:14.882359+0000","last_unstale":"2024-05-09T08:02:22.618768+0000","last_undegraded":"2024-05-09T08:02:22.618768+0000","last_fullsized":"2024-05-09T08:02:22.618768+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:46:11.160815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:20.936081+0000","last_change":"2024-05-09T08:02:19.953319+0000","last_active":"2024-05-09T08:02:20.936081+0000","last_peered":"2024-05-09T08:02:20.936081+0000","last_clean":"2024-05-09T08:02:20.936081+0000","last_became_active":"2024-05-09T08:02:19.952740+0000","last_became_peered":"2024-05-09T08:02:19.952740+0000","last_unstale":"2024-05-09T08:02:20.936081+0000","last_undegraded":"2024-05-09T08:02:20.936081+0000","last_fullsized":"2024-05-09T08:02:20.936081+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:25:28.804048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:21.676635+0000","last_change":"2024-05-09T08:02:15.913828+0000","last_active":"2024-05-09T08:02:21.676635+0000","last_peered":"2024-05-09T08:02:21.676635+0000","last_clean":"2024-05-09T08:02:21.676635+0000","last_became_active":"2024-05-09T08:02:15.913313+0000","last_became_peered":"2024-05-09T08:02:15.913313+0000","last_unstale":"2024-05-09T08:02:21.676635+0000","last_undegraded":"2024-05-09T08:02:21.676635+0000","last_fullsized":"2024-05-09T08:02:21.676635+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:20:09.935394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:21.076590+0000","last_change":"2024-05-09T08:02:14.876438+0000","last_active":"2024-05-09T08:02:21.076590+0000","last_peered":"2024-05-09T08:02:21.076590+0000","last_clean":"2024-05-09T08:02:21.076590+0000","last_became_active":"2024-05-09T08:02:14.876146+0000","last_became_peered":"2024-05-09T08:02:14.876146+0000","last_unstale":"2024-05-09T08:02:21.076590+0000","last_undegraded":"2024-05-09T08:02:21.076590+0000","last_fullsized":"2024-05-09T08:02:21.076590+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:36:49.629991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:21.678130+0000","last_change":"2024-05-09T08:02:20.051067+0000","last_active":"2024-05-09T08:02:21.678130+0000","last_peered":"2024-05-09T08:02:21.678130+0000","last_clean":"2024-05-09T08:02:21.678130+0000","last_became_active":"2024-05-09T08:02:20.050593+0000","last_became_peered":"2024-05-09T08:02:20.050593+0000","last_unstale":"2024-05-09T08:02:21.678130+0000","last_undegraded":"2024-05-09T08:02:21.678130+0000","last_fullsized":"2024-05-09T08:02:21.678130+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:25:28.804048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:20.933046+0000","last_change":"2024-05-09T08:02:15.899137+0000","last_active":"2024-05-09T08:02:20.933046+0000","last_peered":"2024-05-09T08:02:20.933046+0000","last_clean":"2024-05-09T08:02:20.933046+0000","last_became_active":"2024-05-09T08:02:15.898766+0000","last_became_peered":"2024-05-09T08:02:15.898766+0000","last_unstale":"2024-05-09T08:02:20.933046+0000","last_undegraded":"2024-05-09T08:02:20.933046+0000","last_fullsized":"2024-05-09T08:02:20.933046+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:47:32.605288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.617934+0000","last_change":"2024-05-09T08:02:14.868850+0000","last_active":"2024-05-09T08:02:22.617934+0000","last_peered":"2024-05-09T08:02:22.617934+0000","last_clean":"2024-05-09T08:02:22.617934+0000","last_became_active":"2024-05-09T08:02:14.868384+0000","last_became_peered":"2024-05-09T08:02:14.868384+0000","last_unstale":"2024-05-09T08:02:22.617934+0000","last_undegraded":"2024-05-09T08:02:22.617934+0000","last_fullsized":"2024-05-09T08:02:22.617934+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:14:52.920972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":177,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:21.076748+0000","last_change":"2024-05-09T08:02:11.559804+0000","last_active":"2024-05-09T08:02:21.076748+0000","last_peered":"2024-05-09T08:02:21.076748+0000","last_clean":"2024-05-09T08:02:21.076748+0000","last_became_active":"2024-05-09T08:02:11.558761+0000","last_became_peered":"2024-05-09T08:02:11.558761+0000","last_unstale":"2024-05-09T08:02:21.076748+0000","last_undegraded":"2024-05-09T08:02:21.076748+0000","last_fullsized":"2024-05-09T08:02:21.076748+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:09.800143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:09.800143+0000","last_clean_scrub_stamp":"2024-05-09T08:02:09.800143+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:18:35.964149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.618755+0000","last_change":"2024-05-09T08:02:20.054746+0000","last_active":"2024-05-09T08:02:22.618755+0000","last_peered":"2024-05-09T08:02:22.618755+0000","last_clean":"2024-05-09T08:02:22.618755+0000","last_became_active":"2024-05-09T08:02:20.054186+0000","last_became_peered":"2024-05-09T08:02:20.054186+0000","last_unstale":"2024-05-09T08:02:22.618755+0000","last_undegraded":"2024-05-09T08:02:22.618755+0000","last_fullsized":"2024-05-09T08:02:22.618755+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:53:15.458120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:21.076861+0000","last_change":"2024-05-09T08:02:15.889657+0000","last_active":"2024-05-09T08:02:21.076861+0000","last_peered":"2024-05-09T08:02:21.076861+0000","last_clean":"2024-05-09T08:02:21.076861+0000","last_became_active":"2024-05-09T08:02:15.889381+0000","last_became_peered":"2024-05-09T08:02:15.889381+0000","last_unstale":"2024-05-09T08:02:21.076861+0000","last_undegraded":"2024-05-09T08:02:21.076861+0000","last_fullsized":"2024-05-09T08:02:21.076861+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:59:14.957806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.619305+0000","last_change":"2024-05-09T08:02:14.868892+0000","last_active":"2024-05-09T08:02:22.619305+0000","last_peered":"2024-05-09T08:02:22.619305+0000","last_clean":"2024-05-09T08:02:22.619305+0000","last_became_active":"2024-05-09T08:02:14.868513+0000","last_became_peered":"2024-05-09T08:02:14.868513+0000","last_unstale":"2024-05-09T08:02:22.619305+0000","last_undegraded":"2024-05-09T08:02:22.619305+0000","last_fullsized":"2024-05-09T08:02:22.619305+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:19:31.913236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.618654+0000","last_change":"2024-05-09T08:02:20.053097+0000","last_active":"2024-05-09T08:02:22.618654+0000","last_peered":"2024-05-09T08:02:22.618654+0000","last_clean":"2024-05-09T08:02:22.618654+0000","last_became_active":"2024-05-09T08:02:20.052653+0000","last_became_peered":"2024-05-09T08:02:20.052653+0000","last_unstale":"2024-05-09T08:02:22.618654+0000","last_undegraded":"2024-05-09T08:02:22.618654+0000","last_fullsized":"2024-05-09T08:02:22.618654+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:25:05.306541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.618299+0000","last_change":"2024-05-09T08:02:15.899059+0000","last_active":"2024-05-09T08:02:22.618299+0000","last_peered":"2024-05-09T08:02:22.618299+0000","last_clean":"2024-05-09T08:02:22.618299+0000","last_became_active":"2024-05-09T08:02:15.898692+0000","last_became_peered":"2024-05-09T08:02:15.898692+0000","last_unstale":"2024-05-09T08:02:22.618299+0000","last_undegraded":"2024-05-09T08:02:22.618299+0000","last_fullsized":"2024-05-09T08:02:22.618299+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:52:36.461913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"24'1","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:20.935331+0000","last_change":"2024-05-09T08:02:14.873071+0000","last_active":"2024-05-09T08:02:20.935331+0000","last_peered":"2024-05-09T08:02:20.935331+0000","last_clean":"2024-05-09T08:02:20.935331+0000","last_became_active":"2024-05-09T08:02:14.872556+0000","last_became_peered":"2024-05-09T08:02:14.872556+0000","last_unstale":"2024-05-09T08:02:20.935331+0000","last_undegraded":"2024-05-09T08:02:20.935331+0000","last_fullsized":"2024-05-09T08:02:20.935331+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:57:06.093718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:21.677388+0000","last_change":"2024-05-09T08:02:19.960414+0000","last_active":"2024-05-09T08:02:21.677388+0000","last_peered":"2024-05-09T08:02:21.677388+0000","last_clean":"2024-05-09T08:02:21.677388+0000","last_became_active":"2024-05-09T08:02:19.957941+0000","last_became_peered":"2024-05-09T08:02:19.957941+0000","last_unstale":"2024-05-09T08:02:21.677388+0000","last_undegraded":"2024-05-09T08:02:21.677388+0000","last_fullsized":"2024-05-09T08:02:21.677388+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:41:31.470546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:21.077406+0000","last_change":"2024-05-09T08:02:15.884395+0000","last_active":"2024-05-09T08:02:21.077406+0000","last_peered":"2024-05-09T08:02:21.077406+0000","last_clean":"2024-05-09T08:02:21.077406+0000","last_became_active":"2024-05-09T08:02:15.883788+0000","last_became_peered":"2024-05-09T08:02:15.883788+0000","last_unstale":"2024-05-09T08:02:21.077406+0000","last_undegraded":"2024-05-09T08:02:21.077406+0000","last_fullsized":"2024-05-09T08:02:21.077406+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:48:34.215837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"24'2","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.619345+0000","last_change":"2024-05-09T08:02:14.885044+0000","last_active":"2024-05-09T08:02:22.619345+0000","last_peered":"2024-05-09T08:02:22.619345+0000","last_clean":"2024-05-09T08:02:22.619345+0000","last_became_active":"2024-05-09T08:02:14.884706+0000","last_became_peered":"2024-05-09T08:02:14.884706+0000","last_unstale":"2024-05-09T08:02:22.619345+0000","last_undegraded":"2024-05-09T08:02:22.619345+0000","last_fullsized":"2024-05-09T08:02:22.619345+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:14:52.920972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.617383+0000","last_change":"2024-05-09T08:02:20.050865+0000","last_active":"2024-05-09T08:02:22.617383+0000","last_peered":"2024-05-09T08:02:22.617383+0000","last_clean":"2024-05-09T08:02:22.617383+0000","last_became_active":"2024-05-09T08:02:20.050270+0000","last_became_peered":"2024-05-09T08:02:20.050270+0000","last_unstale":"2024-05-09T08:02:22.617383+0000","last_undegraded":"2024-05-09T08:02:22.617383+0000","last_fullsized":"2024-05-09T08:02:22.617383+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:48:38.283562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:21.077190+0000","last_change":"2024-05-09T08:02:15.884737+0000","last_active":"2024-05-09T08:02:21.077190+0000","last_peered":"2024-05-09T08:02:21.077190+0000","last_clean":"2024-05-09T08:02:21.077190+0000","last_became_active":"2024-05-09T08:02:15.884465+0000","last_became_peered":"2024-05-09T08:02:15.884465+0000","last_unstale":"2024-05-09T08:02:21.077190+0000","last_undegraded":"2024-05-09T08:02:21.077190+0000","last_fullsized":"2024-05-09T08:02:21.077190+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:52:36.461913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"24'1","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.618658+0000","last_change":"2024-05-09T08:02:14.875003+0000","last_active":"2024-05-09T08:02:22.618658+0000","last_peered":"2024-05-09T08:02:22.618658+0000","last_clean":"2024-05-09T08:02:22.618658+0000","last_became_active":"2024-05-09T08:02:14.874665+0000","last_became_peered":"2024-05-09T08:02:14.874665+0000","last_unstale":"2024-05-09T08:02:22.618658+0000","last_undegraded":"2024-05-09T08:02:22.618658+0000","last_fullsized":"2024-05-09T08:02:22.618658+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:18:39.991319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.617254+0000","last_change":"2024-05-09T08:02:19.952692+0000","last_active":"2024-05-09T08:02:22.617254+0000","last_peered":"2024-05-09T08:02:22.617254+0000","last_clean":"2024-05-09T08:02:22.617254+0000","last_became_active":"2024-05-09T08:02:19.952224+0000","last_became_peered":"2024-05-09T08:02:19.952224+0000","last_unstale":"2024-05-09T08:02:22.617254+0000","last_undegraded":"2024-05-09T08:02:22.617254+0000","last_fullsized":"2024-05-09T08:02:22.617254+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:47:36.673013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:21.076666+0000","last_change":"2024-05-09T08:02:15.883408+0000","last_active":"2024-05-09T08:02:21.076666+0000","last_peered":"2024-05-09T08:02:21.076666+0000","last_clean":"2024-05-09T08:02:21.076666+0000","last_became_active":"2024-05-09T08:02:15.883103+0000","last_became_peered":"2024-05-09T08:02:15.883103+0000","last_unstale":"2024-05-09T08:02:21.076666+0000","last_undegraded":"2024-05-09T08:02:21.076666+0000","last_fullsized":"2024-05-09T08:02:21.076666+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:53:11.390395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"24'8","reported_seq":29,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.618529+0000","last_change":"2024-05-09T08:02:14.873184+0000","last_active":"2024-05-09T08:02:22.618529+0000","last_peered":"2024-05-09T08:02:22.618529+0000","last_clean":"2024-05-09T08:02:22.618529+0000","last_became_active":"2024-05-09T08:02:14.871055+0000","last_became_peered":"2024-05-09T08:02:14.871055+0000","last_unstale":"2024-05-09T08:02:22.618529+0000","last_undegraded":"2024-05-09T08:02:22.618529+0000","last_fullsized":"2024-05-09T08:02:22.618529+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:19:56.315972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:21.678092+0000","last_change":"2024-05-09T08:02:19.950823+0000","last_active":"2024-05-09T08:02:21.678092+0000","last_peered":"2024-05-09T08:02:21.678092+0000","last_clean":"2024-05-09T08:02:21.678092+0000","last_became_active":"2024-05-09T08:02:19.950352+0000","last_became_peered":"2024-05-09T08:02:19.950352+0000","last_unstale":"2024-05-09T08:02:21.678092+0000","last_undegraded":"2024-05-09T08:02:21.678092+0000","last_fullsized":"2024-05-09T08:02:21.678092+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:59:19.025530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.619335+0000","last_change":"2024-05-09T08:02:15.899115+0000","last_active":"2024-05-09T08:02:22.619335+0000","last_peered":"2024-05-09T08:02:22.619335+0000","last_clean":"2024-05-09T08:02:22.619335+0000","last_became_active":"2024-05-09T08:02:15.898811+0000","last_became_peered":"2024-05-09T08:02:15.898811+0000","last_unstale":"2024-05-09T08:02:22.619335+0000","last_undegraded":"2024-05-09T08:02:22.619335+0000","last_fullsized":"2024-05-09T08:02:22.619335+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:43:06.938904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.619023+0000","last_change":"2024-05-09T08:02:14.860818+0000","last_active":"2024-05-09T08:02:22.619023+0000","last_peered":"2024-05-09T08:02:22.619023+0000","last_clean":"2024-05-09T08:02:22.619023+0000","last_became_active":"2024-05-09T08:02:14.860518+0000","last_became_peered":"2024-05-09T08:02:14.860518+0000","last_unstale":"2024-05-09T08:02:22.619023+0000","last_undegraded":"2024-05-09T08:02:22.619023+0000","last_fullsized":"2024-05-09T08:02:22.619023+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:25:23.306827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:20.935214+0000","last_change":"2024-05-09T08:02:20.053251+0000","last_active":"2024-05-09T08:02:20.935214+0000","last_peered":"2024-05-09T08:02:20.935214+0000","last_clean":"2024-05-09T08:02:20.935214+0000","last_became_active":"2024-05-09T08:02:20.052783+0000","last_became_peered":"2024-05-09T08:02:20.052783+0000","last_unstale":"2024-05-09T08:02:20.935214+0000","last_undegraded":"2024-05-09T08:02:20.935214+0000","last_fullsized":"2024-05-09T08:02:20.935214+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:16:41.863341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:21.076854+0000","last_change":"2024-05-09T08:02:15.883972+0000","last_active":"2024-05-09T08:02:21.076854+0000","last_peered":"2024-05-09T08:02:21.076854+0000","last_clean":"2024-05-09T08:02:21.076854+0000","last_became_active":"2024-05-09T08:02:15.883613+0000","last_became_peered":"2024-05-09T08:02:15.883613+0000","last_unstale":"2024-05-09T08:02:21.076854+0000","last_undegraded":"2024-05-09T08:02:21.076854+0000","last_fullsized":"2024-05-09T08:02:21.076854+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:41:27.402821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.618038+0000","last_change":"2024-05-09T08:02:14.883366+0000","last_active":"2024-05-09T08:02:22.618038+0000","last_peered":"2024-05-09T08:02:22.618038+0000","last_clean":"2024-05-09T08:02:22.618038+0000","last_became_active":"2024-05-09T08:02:14.883135+0000","last_became_peered":"2024-05-09T08:02:14.883135+0000","last_unstale":"2024-05-09T08:02:22.618038+0000","last_undegraded":"2024-05-09T08:02:22.618038+0000","last_fullsized":"2024-05-09T08:02:22.618038+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:32:48.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:20.935375+0000","last_change":"2024-05-09T08:02:20.050888+0000","last_active":"2024-05-09T08:02:20.935375+0000","last_peered":"2024-05-09T08:02:20.935375+0000","last_clean":"2024-05-09T08:02:20.935375+0000","last_became_active":"2024-05-09T08:02:20.050390+0000","last_became_peered":"2024-05-09T08:02:20.050390+0000","last_unstale":"2024-05-09T08:02:20.935375+0000","last_undegraded":"2024-05-09T08:02:20.935375+0000","last_fullsized":"2024-05-09T08:02:20.935375+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:10:24.797526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:21.677358+0000","last_change":"2024-05-09T08:02:15.899096+0000","last_active":"2024-05-09T08:02:21.677358+0000","last_peered":"2024-05-09T08:02:21.677358+0000","last_clean":"2024-05-09T08:02:21.677358+0000","last_became_active":"2024-05-09T08:02:15.898811+0000","last_became_peered":"2024-05-09T08:02:15.898811+0000","last_unstale":"2024-05-09T08:02:21.677358+0000","last_undegraded":"2024-05-09T08:02:21.677358+0000","last_fullsized":"2024-05-09T08:02:21.677358+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:28:51.229262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"24'2","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:21.677657+0000","last_change":"2024-05-09T08:02:14.868657+0000","last_active":"2024-05-09T08:02:21.677657+0000","last_peered":"2024-05-09T08:02:21.677657+0000","last_clean":"2024-05-09T08:02:21.677657+0000","last_became_active":"2024-05-09T08:02:14.868385+0000","last_became_peered":"2024-05-09T08:02:14.868385+0000","last_unstale":"2024-05-09T08:02:21.677657+0000","last_undegraded":"2024-05-09T08:02:21.677657+0000","last_fullsized":"2024-05-09T08:02:21.677657+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:46:11.160815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.618792+0000","last_change":"2024-05-09T08:02:20.053155+0000","last_active":"2024-05-09T08:02:22.618792+0000","last_peered":"2024-05-09T08:02:22.618792+0000","last_clean":"2024-05-09T08:02:22.618792+0000","last_became_active":"2024-05-09T08:02:20.052629+0000","last_became_peered":"2024-05-09T08:02:20.052629+0000","last_unstale":"2024-05-09T08:02:22.618792+0000","last_undegraded":"2024-05-09T08:02:22.618792+0000","last_fullsized":"2024-05-09T08:02:22.618792+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:35:54.628874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:21.076864+0000","last_change":"2024-05-09T08:02:15.905406+0000","last_active":"2024-05-09T08:02:21.076864+0000","last_peered":"2024-05-09T08:02:21.076864+0000","last_clean":"2024-05-09T08:02:21.076864+0000","last_became_active":"2024-05-09T08:02:15.905076+0000","last_became_peered":"2024-05-09T08:02:15.905076+0000","last_unstale":"2024-05-09T08:02:21.076864+0000","last_undegraded":"2024-05-09T08:02:21.076864+0000","last_fullsized":"2024-05-09T08:02:21.076864+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:25:01.238816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"25'10","reported_seq":32,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:39.803593+0000","last_change":"2024-05-09T08:02:14.880410+0000","last_active":"2024-05-09T08:02:39.803593+0000","last_peered":"2024-05-09T08:02:39.803593+0000","last_clean":"2024-05-09T08:02:39.803593+0000","last_became_active":"2024-05-09T08:02:14.880045+0000","last_became_peered":"2024-05-09T08:02:14.880045+0000","last_unstale":"2024-05-09T08:02:39.803593+0000","last_undegraded":"2024-05-09T08:02:39.803593+0000","last_fullsized":"2024-05-09T08:02:39.803593+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:19:56.315972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:20.934006+0000","last_change":"2024-05-09T08:02:19.963378+0000","last_active":"2024-05-09T08:02:20.934006+0000","last_peered":"2024-05-09T08:02:20.934006+0000","last_clean":"2024-05-09T08:02:20.934006+0000","last_became_active":"2024-05-09T08:02:19.958964+0000","last_became_peered":"2024-05-09T08:02:19.958964+0000","last_unstale":"2024-05-09T08:02:20.934006+0000","last_undegraded":"2024-05-09T08:02:20.934006+0000","last_fullsized":"2024-05-09T08:02:20.934006+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:41:31.470546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.618268+0000","last_change":"2024-05-09T08:02:15.888428+0000","last_active":"2024-05-09T08:02:22.618268+0000","last_peered":"2024-05-09T08:02:22.618268+0000","last_clean":"2024-05-09T08:02:22.618268+0000","last_became_active":"2024-05-09T08:02:15.888071+0000","last_became_peered":"2024-05-09T08:02:15.888071+0000","last_unstale":"2024-05-09T08:02:22.618268+0000","last_undegraded":"2024-05-09T08:02:22.618268+0000","last_fullsized":"2024-05-09T08:02:22.618268+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:36:50.643632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"24'3","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:20.935171+0000","last_change":"2024-05-09T08:02:14.872697+0000","last_active":"2024-05-09T08:02:20.935171+0000","last_peered":"2024-05-09T08:02:20.935171+0000","last_clean":"2024-05-09T08:02:20.935171+0000","last_became_active":"2024-05-09T08:02:14.871945+0000","last_became_peered":"2024-05-09T08:02:14.871945+0000","last_unstale":"2024-05-09T08:02:20.935171+0000","last_undegraded":"2024-05-09T08:02:20.935171+0000","last_fullsized":"2024-05-09T08:02:20.935171+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:19:56.315972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.617683+0000","last_change":"2024-05-09T08:02:19.953635+0000","last_active":"2024-05-09T08:02:22.617683+0000","last_peered":"2024-05-09T08:02:22.617683+0000","last_clean":"2024-05-09T08:02:22.617683+0000","last_became_active":"2024-05-09T08:02:19.953166+0000","last_became_peered":"2024-05-09T08:02:19.953166+0000","last_unstale":"2024-05-09T08:02:22.617683+0000","last_undegraded":"2024-05-09T08:02:22.617683+0000","last_fullsized":"2024-05-09T08:02:22.617683+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:18:40.377352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:21.677885+0000","last_change":"2024-05-09T08:02:15.905285+0000","last_active":"2024-05-09T08:02:21.677885+0000","last_peered":"2024-05-09T08:02:21.677885+0000","last_clean":"2024-05-09T08:02:21.677885+0000","last_became_active":"2024-05-09T08:02:15.904898+0000","last_became_peered":"2024-05-09T08:02:15.904898+0000","last_unstale":"2024-05-09T08:02:21.677885+0000","last_undegraded":"2024-05-09T08:02:21.677885+0000","last_fullsized":"2024-05-09T08:02:21.677885+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:47:32.605288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.619004+0000","last_change":"2024-05-09T08:02:14.868682+0000","last_active":"2024-05-09T08:02:22.619004+0000","last_peered":"2024-05-09T08:02:22.619004+0000","last_clean":"2024-05-09T08:02:22.619004+0000","last_became_active":"2024-05-09T08:02:14.868294+0000","last_became_peered":"2024-05-09T08:02:14.868294+0000","last_unstale":"2024-05-09T08:02:22.619004+0000","last_undegraded":"2024-05-09T08:02:22.619004+0000","last_fullsized":"2024-05-09T08:02:22.619004+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:57:06.093718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:20.933284+0000","last_change":"2024-05-09T08:02:19.956115+0000","last_active":"2024-05-09T08:02:20.933284+0000","last_peered":"2024-05-09T08:02:20.933284+0000","last_clean":"2024-05-09T08:02:20.933284+0000","last_became_active":"2024-05-09T08:02:19.955662+0000","last_became_peered":"2024-05-09T08:02:19.955662+0000","last_unstale":"2024-05-09T08:02:20.933284+0000","last_undegraded":"2024-05-09T08:02:20.933284+0000","last_fullsized":"2024-05-09T08:02:20.933284+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:15:28.286330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.618890+0000","last_change":"2024-05-09T08:02:15.888406+0000","last_active":"2024-05-09T08:02:22.618890+0000","last_peered":"2024-05-09T08:02:22.618890+0000","last_clean":"2024-05-09T08:02:22.618890+0000","last_became_active":"2024-05-09T08:02:15.888090+0000","last_became_peered":"2024-05-09T08:02:15.888090+0000","last_unstale":"2024-05-09T08:02:22.618890+0000","last_undegraded":"2024-05-09T08:02:22.618890+0000","last_fullsized":"2024-05-09T08:02:22.618890+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:53:23.178610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.619656+0000","last_change":"2024-05-09T08:02:14.862463+0000","last_active":"2024-05-09T08:02:22.619656+0000","last_peered":"2024-05-09T08:02:22.619656+0000","last_clean":"2024-05-09T08:02:22.619656+0000","last_became_active":"2024-05-09T08:02:14.862104+0000","last_became_peered":"2024-05-09T08:02:14.862104+0000","last_unstale":"2024-05-09T08:02:22.619656+0000","last_undegraded":"2024-05-09T08:02:22.619656+0000","last_fullsized":"2024-05-09T08:02:22.619656+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:32:48.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:20.932952+0000","last_change":"2024-05-09T08:02:20.053600+0000","last_active":"2024-05-09T08:02:20.932952+0000","last_peered":"2024-05-09T08:02:20.932952+0000","last_clean":"2024-05-09T08:02:20.932952+0000","last_became_active":"2024-05-09T08:02:20.052485+0000","last_became_peered":"2024-05-09T08:02:20.052485+0000","last_unstale":"2024-05-09T08:02:20.932952+0000","last_undegraded":"2024-05-09T08:02:20.932952+0000","last_fullsized":"2024-05-09T08:02:20.932952+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:35:54.628874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:21.677902+0000","last_change":"2024-05-09T08:02:20.054016+0000","last_active":"2024-05-09T08:02:21.677902+0000","last_peered":"2024-05-09T08:02:21.677902+0000","last_clean":"2024-05-09T08:02:21.677902+0000","last_became_active":"2024-05-09T08:02:20.053250+0000","last_became_peered":"2024-05-09T08:02:20.053250+0000","last_unstale":"2024-05-09T08:02:21.677902+0000","last_undegraded":"2024-05-09T08:02:21.677902+0000","last_fullsized":"2024-05-09T08:02:21.677902+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:52:40.529638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:20.932409+0000","last_change":"2024-05-09T08:02:15.907080+0000","last_active":"2024-05-09T08:02:20.932409+0000","last_peered":"2024-05-09T08:02:20.932409+0000","last_clean":"2024-05-09T08:02:20.932409+0000","last_became_active":"2024-05-09T08:02:15.906600+0000","last_became_peered":"2024-05-09T08:02:15.906600+0000","last_unstale":"2024-05-09T08:02:20.932409+0000","last_undegraded":"2024-05-09T08:02:20.932409+0000","last_fullsized":"2024-05-09T08:02:20.932409+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:43:06.938904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.617668+0000","last_change":"2024-05-09T08:02:14.870990+0000","last_active":"2024-05-09T08:02:22.617668+0000","last_peered":"2024-05-09T08:02:22.617668+0000","last_clean":"2024-05-09T08:02:22.617668+0000","last_became_active":"2024-05-09T08:02:14.869818+0000","last_became_peered":"2024-05-09T08:02:14.869818+0000","last_unstale":"2024-05-09T08:02:22.617668+0000","last_undegraded":"2024-05-09T08:02:22.617668+0000","last_fullsized":"2024-05-09T08:02:22.617668+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:25:23.306827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.619498+0000","last_change":"2024-05-09T08:02:19.952582+0000","last_active":"2024-05-09T08:02:22.619498+0000","last_peered":"2024-05-09T08:02:22.619498+0000","last_clean":"2024-05-09T08:02:22.619498+0000","last_became_active":"2024-05-09T08:02:19.952117+0000","last_became_peered":"2024-05-09T08:02:19.952117+0000","last_unstale":"2024-05-09T08:02:22.619498+0000","last_undegraded":"2024-05-09T08:02:22.619498+0000","last_fullsized":"2024-05-09T08:02:22.619498+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:42:00.939587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:20.933300+0000","last_change":"2024-05-09T08:02:15.913109+0000","last_active":"2024-05-09T08:02:20.933300+0000","last_peered":"2024-05-09T08:02:20.933300+0000","last_clean":"2024-05-09T08:02:20.933300+0000","last_became_active":"2024-05-09T08:02:15.912883+0000","last_became_peered":"2024-05-09T08:02:15.912883+0000","last_unstale":"2024-05-09T08:02:20.933300+0000","last_undegraded":"2024-05-09T08:02:20.933300+0000","last_fullsized":"2024-05-09T08:02:20.933300+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:29:59.441314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:20.933233+0000","last_change":"2024-05-09T08:02:14.885256+0000","last_active":"2024-05-09T08:02:20.933233+0000","last_peered":"2024-05-09T08:02:20.933233+0000","last_clean":"2024-05-09T08:02:20.933233+0000","last_became_active":"2024-05-09T08:02:14.884965+0000","last_became_peered":"2024-05-09T08:02:14.884965+0000","last_unstale":"2024-05-09T08:02:20.933233+0000","last_undegraded":"2024-05-09T08:02:20.933233+0000","last_fullsized":"2024-05-09T08:02:20.933233+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:14:52.920972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"24'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.618840+0000","last_change":"2024-05-09T08:02:14.860339+0000","last_active":"2024-05-09T08:02:22.618840+0000","last_peered":"2024-05-09T08:02:22.618840+0000","last_clean":"2024-05-09T08:02:22.618840+0000","last_became_active":"2024-05-09T08:02:14.859917+0000","last_became_peered":"2024-05-09T08:02:14.859917+0000","last_unstale":"2024-05-09T08:02:22.618840+0000","last_undegraded":"2024-05-09T08:02:22.618840+0000","last_fullsized":"2024-05-09T08:02:22.618840+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:14:52.920972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:21.677460+0000","last_change":"2024-05-09T08:02:15.897001+0000","last_active":"2024-05-09T08:02:21.677460+0000","last_peered":"2024-05-09T08:02:21.677460+0000","last_clean":"2024-05-09T08:02:21.677460+0000","last_became_active":"2024-05-09T08:02:15.896708+0000","last_became_peered":"2024-05-09T08:02:15.896708+0000","last_unstale":"2024-05-09T08:02:21.677460+0000","last_undegraded":"2024-05-09T08:02:21.677460+0000","last_fullsized":"2024-05-09T08:02:21.677460+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:29:59.441314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.619069+0000","last_change":"2024-05-09T08:02:19.955369+0000","last_active":"2024-05-09T08:02:22.619069+0000","last_peered":"2024-05-09T08:02:22.619069+0000","last_clean":"2024-05-09T08:02:22.619069+0000","last_became_active":"2024-05-09T08:02:19.954829+0000","last_became_peered":"2024-05-09T08:02:19.954829+0000","last_unstale":"2024-05-09T08:02:22.619069+0000","last_undegraded":"2024-05-09T08:02:22.619069+0000","last_fullsized":"2024-05-09T08:02:22.619069+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:49:58.963879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"24'1","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:20.934324+0000","last_change":"2024-05-09T08:02:14.883856+0000","last_active":"2024-05-09T08:02:20.934324+0000","last_peered":"2024-05-09T08:02:20.934324+0000","last_clean":"2024-05-09T08:02:20.934324+0000","last_became_active":"2024-05-09T08:02:14.882610+0000","last_became_peered":"2024-05-09T08:02:14.882610+0000","last_unstale":"2024-05-09T08:02:20.934324+0000","last_undegraded":"2024-05-09T08:02:20.934324+0000","last_fullsized":"2024-05-09T08:02:20.934324+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:19:31.913236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.617629+0000","last_change":"2024-05-09T08:02:15.889974+0000","last_active":"2024-05-09T08:02:22.617629+0000","last_peered":"2024-05-09T08:02:22.617629+0000","last_clean":"2024-05-09T08:02:22.617629+0000","last_became_active":"2024-05-09T08:02:15.889650+0000","last_became_peered":"2024-05-09T08:02:15.889650+0000","last_unstale":"2024-05-09T08:02:22.617629+0000","last_undegraded":"2024-05-09T08:02:22.617629+0000","last_fullsized":"2024-05-09T08:02:22.617629+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:00:41.613116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:20.934556+0000","last_change":"2024-05-09T08:02:19.955362+0000","last_active":"2024-05-09T08:02:20.934556+0000","last_peered":"2024-05-09T08:02:20.934556+0000","last_clean":"2024-05-09T08:02:20.934556+0000","last_became_active":"2024-05-09T08:02:19.954947+0000","last_became_peered":"2024-05-09T08:02:19.954947+0000","last_unstale":"2024-05-09T08:02:20.934556+0000","last_undegraded":"2024-05-09T08:02:20.934556+0000","last_fullsized":"2024-05-09T08:02:20.934556+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:43:30.549966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.618982+0000","last_change":"2024-05-09T08:02:15.909379+0000","last_active":"2024-05-09T08:02:22.618982+0000","last_peered":"2024-05-09T08:02:22.618982+0000","last_clean":"2024-05-09T08:02:22.618982+0000","last_became_active":"2024-05-09T08:02:15.909086+0000","last_became_peered":"2024-05-09T08:02:15.909086+0000","last_unstale":"2024-05-09T08:02:22.618982+0000","last_undegraded":"2024-05-09T08:02:22.618982+0000","last_fullsized":"2024-05-09T08:02:22.618982+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:05:02.181367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.618963+0000","last_change":"2024-05-09T08:02:15.891242+0000","last_active":"2024-05-09T08:02:22.618963+0000","last_peered":"2024-05-09T08:02:22.618963+0000","last_clean":"2024-05-09T08:02:22.618963+0000","last_became_active":"2024-05-09T08:02:15.890640+0000","last_became_peered":"2024-05-09T08:02:15.890640+0000","last_unstale":"2024-05-09T08:02:22.618963+0000","last_undegraded":"2024-05-09T08:02:22.618963+0000","last_fullsized":"2024-05-09T08:02:22.618963+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:57:07.107359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.617950+0000","last_change":"2024-05-09T08:02:15.899001+0000","last_active":"2024-05-09T08:02:22.617950+0000","last_peered":"2024-05-09T08:02:22.617950+0000","last_clean":"2024-05-09T08:02:22.617950+0000","last_became_active":"2024-05-09T08:02:15.898695+0000","last_became_peered":"2024-05-09T08:02:15.898695+0000","last_unstale":"2024-05-09T08:02:22.617950+0000","last_undegraded":"2024-05-09T08:02:22.617950+0000","last_fullsized":"2024-05-09T08:02:22.617950+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:41:27.402821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:21.677395+0000","last_change":"2024-05-09T08:02:15.907374+0000","last_active":"2024-05-09T08:02:21.677395+0000","last_peered":"2024-05-09T08:02:21.677395+0000","last_clean":"2024-05-09T08:02:21.677395+0000","last_became_active":"2024-05-09T08:02:15.906767+0000","last_became_peered":"2024-05-09T08:02:15.906767+0000","last_unstale":"2024-05-09T08:02:21.677395+0000","last_undegraded":"2024-05-09T08:02:21.677395+0000","last_fullsized":"2024-05-09T08:02:21.677395+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:05:02.181367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:20.935527+0000","last_change":"2024-05-09T08:02:15.905326+0000","last_active":"2024-05-09T08:02:20.935527+0000","last_peered":"2024-05-09T08:02:20.935527+0000","last_clean":"2024-05-09T08:02:20.935527+0000","last_became_active":"2024-05-09T08:02:15.905083+0000","last_became_peered":"2024-05-09T08:02:15.905083+0000","last_unstale":"2024-05-09T08:02:20.935527+0000","last_undegraded":"2024-05-09T08:02:20.935527+0000","last_fullsized":"2024-05-09T08:02:20.935527+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:23:40.248821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:20.934866+0000","last_change":"2024-05-09T08:02:15.907753+0000","last_active":"2024-05-09T08:02:20.934866+0000","last_peered":"2024-05-09T08:02:20.934866+0000","last_clean":"2024-05-09T08:02:20.934866+0000","last_became_active":"2024-05-09T08:02:15.907480+0000","last_became_peered":"2024-05-09T08:02:15.907480+0000","last_unstale":"2024-05-09T08:02:20.934866+0000","last_undegraded":"2024-05-09T08:02:20.934866+0000","last_fullsized":"2024-05-09T08:02:20.934866+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:20:07.368327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.619474+0000","last_change":"2024-05-09T08:02:15.888605+0000","last_active":"2024-05-09T08:02:22.619474+0000","last_peered":"2024-05-09T08:02:22.619474+0000","last_clean":"2024-05-09T08:02:22.619474+0000","last_became_active":"2024-05-09T08:02:15.887981+0000","last_became_peered":"2024-05-09T08:02:15.887981+0000","last_unstale":"2024-05-09T08:02:22.619474+0000","last_undegraded":"2024-05-09T08:02:22.619474+0000","last_fullsized":"2024-05-09T08:02:22.619474+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:47:32.605288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:20.935282+0000","last_change":"2024-05-09T08:02:15.904744+0000","last_active":"2024-05-09T08:02:20.935282+0000","last_peered":"2024-05-09T08:02:20.935282+0000","last_clean":"2024-05-09T08:02:20.935282+0000","last_became_active":"2024-05-09T08:02:15.904334+0000","last_became_peered":"2024-05-09T08:02:15.904334+0000","last_unstale":"2024-05-09T08:02:20.935282+0000","last_undegraded":"2024-05-09T08:02:20.935282+0000","last_fullsized":"2024-05-09T08:02:20.935282+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:30:33.351158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.617634+0000","last_change":"2024-05-09T08:02:15.904835+0000","last_active":"2024-05-09T08:02:22.617634+0000","last_peered":"2024-05-09T08:02:22.617634+0000","last_clean":"2024-05-09T08:02:22.617634+0000","last_became_active":"2024-05-09T08:02:15.904416+0000","last_became_peered":"2024-05-09T08:02:15.904416+0000","last_unstale":"2024-05-09T08:02:22.617634+0000","last_undegraded":"2024-05-09T08:02:22.617634+0000","last_fullsized":"2024-05-09T08:02:22.617634+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:47:32.605288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:20.933091+0000","last_change":"2024-05-09T08:02:15.898324+0000","last_active":"2024-05-09T08:02:20.933091+0000","last_peered":"2024-05-09T08:02:20.933091+0000","last_clean":"2024-05-09T08:02:20.933091+0000","last_became_active":"2024-05-09T08:02:15.897981+0000","last_became_peered":"2024-05-09T08:02:15.897981+0000","last_unstale":"2024-05-09T08:02:20.933091+0000","last_undegraded":"2024-05-09T08:02:20.933091+0000","last_fullsized":"2024-05-09T08:02:20.933091+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:01:50.341133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.617909+0000","last_change":"2024-05-09T08:02:15.890945+0000","last_active":"2024-05-09T08:02:22.617909+0000","last_peered":"2024-05-09T08:02:22.617909+0000","last_clean":"2024-05-09T08:02:22.617909+0000","last_became_active":"2024-05-09T08:02:15.890647+0000","last_became_peered":"2024-05-09T08:02:15.890647+0000","last_unstale":"2024-05-09T08:02:22.617909+0000","last_undegraded":"2024-05-09T08:02:22.617909+0000","last_fullsized":"2024-05-09T08:02:22.617909+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:41:56.871863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:21.677105+0000","last_change":"2024-05-09T08:02:15.913728+0000","last_active":"2024-05-09T08:02:21.677105+0000","last_peered":"2024-05-09T08:02:21.677105+0000","last_clean":"2024-05-09T08:02:21.677105+0000","last_became_active":"2024-05-09T08:02:15.909818+0000","last_became_peered":"2024-05-09T08:02:15.909818+0000","last_unstale":"2024-05-09T08:02:21.677105+0000","last_undegraded":"2024-05-09T08:02:21.677105+0000","last_fullsized":"2024-05-09T08:02:21.677105+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:41:55.788033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.619452+0000","last_change":"2024-05-09T08:02:15.885025+0000","last_active":"2024-05-09T08:02:22.619452+0000","last_peered":"2024-05-09T08:02:22.619452+0000","last_clean":"2024-05-09T08:02:22.619452+0000","last_became_active":"2024-05-09T08:02:15.884517+0000","last_became_peered":"2024-05-09T08:02:15.884517+0000","last_unstale":"2024-05-09T08:02:22.619452+0000","last_undegraded":"2024-05-09T08:02:22.619452+0000","last_fullsized":"2024-05-09T08:02:22.619452+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:01:50.341133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:21.077234+0000","last_change":"2024-05-09T08:02:15.890306+0000","last_active":"2024-05-09T08:02:21.077234+0000","last_peered":"2024-05-09T08:02:21.077234+0000","last_clean":"2024-05-09T08:02:21.077234+0000","last_became_active":"2024-05-09T08:02:15.889998+0000","last_became_peered":"2024-05-09T08:02:15.889998+0000","last_unstale":"2024-05-09T08:02:21.077234+0000","last_undegraded":"2024-05-09T08:02:21.077234+0000","last_fullsized":"2024-05-09T08:02:21.077234+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:16:57.446444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.619318+0000","last_change":"2024-05-09T08:02:15.886417+0000","last_active":"2024-05-09T08:02:22.619318+0000","last_peered":"2024-05-09T08:02:22.619318+0000","last_clean":"2024-05-09T08:02:22.619318+0000","last_became_active":"2024-05-09T08:02:15.885525+0000","last_became_peered":"2024-05-09T08:02:15.885525+0000","last_unstale":"2024-05-09T08:02:22.619318+0000","last_undegraded":"2024-05-09T08:02:22.619318+0000","last_fullsized":"2024-05-09T08:02:22.619318+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:43:02.075442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.618616+0000","last_change":"2024-05-09T08:02:15.893172+0000","last_active":"2024-05-09T08:02:22.618616+0000","last_peered":"2024-05-09T08:02:22.618616+0000","last_clean":"2024-05-09T08:02:22.618616+0000","last_became_active":"2024-05-09T08:02:15.892886+0000","last_became_peered":"2024-05-09T08:02:15.892886+0000","last_unstale":"2024-05-09T08:02:22.618616+0000","last_undegraded":"2024-05-09T08:02:22.618616+0000","last_fullsized":"2024-05-09T08:02:22.618616+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:59:14.957806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.617513+0000","last_change":"2024-05-09T08:02:15.908659+0000","last_active":"2024-05-09T08:02:22.617513+0000","last_peered":"2024-05-09T08:02:22.617513+0000","last_clean":"2024-05-09T08:02:22.617513+0000","last_became_active":"2024-05-09T08:02:15.908307+0000","last_became_peered":"2024-05-09T08:02:15.908307+0000","last_unstale":"2024-05-09T08:02:22.617513+0000","last_undegraded":"2024-05-09T08:02:22.617513+0000","last_fullsized":"2024-05-09T08:02:22.617513+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:20:07.368327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:20.933418+0000","last_change":"2024-05-09T08:02:15.885800+0000","last_active":"2024-05-09T08:02:20.933418+0000","last_peered":"2024-05-09T08:02:20.933418+0000","last_clean":"2024-05-09T08:02:20.933418+0000","last_became_active":"2024-05-09T08:02:15.885185+0000","last_became_peered":"2024-05-09T08:02:15.885185+0000","last_unstale":"2024-05-09T08:02:20.933418+0000","last_undegraded":"2024-05-09T08:02:20.933418+0000","last_fullsized":"2024-05-09T08:02:20.933418+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:01:50.341133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:21.677190+0000","last_change":"2024-05-09T08:02:15.911496+0000","last_active":"2024-05-09T08:02:21.677190+0000","last_peered":"2024-05-09T08:02:21.677190+0000","last_clean":"2024-05-09T08:02:21.677190+0000","last_became_active":"2024-05-09T08:02:15.911142+0000","last_became_peered":"2024-05-09T08:02:15.911142+0000","last_unstale":"2024-05-09T08:02:21.677190+0000","last_undegraded":"2024-05-09T08:02:21.677190+0000","last_fullsized":"2024-05-09T08:02:21.677190+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:27:59.315750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:20.933605+0000","last_change":"2024-05-09T08:02:15.898847+0000","last_active":"2024-05-09T08:02:20.933605+0000","last_peered":"2024-05-09T08:02:20.933605+0000","last_clean":"2024-05-09T08:02:20.933605+0000","last_became_active":"2024-05-09T08:02:15.898480+0000","last_became_peered":"2024-05-09T08:02:15.898480+0000","last_unstale":"2024-05-09T08:02:20.933605+0000","last_undegraded":"2024-05-09T08:02:20.933605+0000","last_fullsized":"2024-05-09T08:02:20.933605+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:20:07.368327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.618261+0000","last_change":"2024-05-09T08:02:15.883117+0000","last_active":"2024-05-09T08:02:22.618261+0000","last_peered":"2024-05-09T08:02:22.618261+0000","last_clean":"2024-05-09T08:02:22.618261+0000","last_became_active":"2024-05-09T08:02:15.882141+0000","last_became_peered":"2024-05-09T08:02:15.882141+0000","last_unstale":"2024-05-09T08:02:22.618261+0000","last_undegraded":"2024-05-09T08:02:22.618261+0000","last_fullsized":"2024-05-09T08:02:22.618261+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:57:11.352151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.618965+0000","last_change":"2024-05-09T08:02:15.893318+0000","last_active":"2024-05-09T08:02:22.618965+0000","last_peered":"2024-05-09T08:02:22.618965+0000","last_clean":"2024-05-09T08:02:22.618965+0000","last_became_active":"2024-05-09T08:02:15.892964+0000","last_became_peered":"2024-05-09T08:02:15.892964+0000","last_unstale":"2024-05-09T08:02:22.618965+0000","last_undegraded":"2024-05-09T08:02:22.618965+0000","last_fullsized":"2024-05-09T08:02:22.618965+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:20:07.368327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.619428+0000","last_change":"2024-05-09T08:02:15.886482+0000","last_active":"2024-05-09T08:02:22.619428+0000","last_peered":"2024-05-09T08:02:22.619428+0000","last_clean":"2024-05-09T08:02:22.619428+0000","last_became_active":"2024-05-09T08:02:15.886183+0000","last_became_peered":"2024-05-09T08:02:15.886183+0000","last_unstale":"2024-05-09T08:02:22.619428+0000","last_undegraded":"2024-05-09T08:02:22.619428+0000","last_fullsized":"2024-05-09T08:02:22.619428+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:23:40.248821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:21.077365+0000","last_change":"2024-05-09T08:02:15.887506+0000","last_active":"2024-05-09T08:02:21.077365+0000","last_peered":"2024-05-09T08:02:21.077365+0000","last_clean":"2024-05-09T08:02:21.077365+0000","last_became_active":"2024-05-09T08:02:15.887225+0000","last_became_peered":"2024-05-09T08:02:15.887225+0000","last_unstale":"2024-05-09T08:02:21.077365+0000","last_undegraded":"2024-05-09T08:02:21.077365+0000","last_fullsized":"2024-05-09T08:02:21.077365+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:25:24.736323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:20.933062+0000","last_change":"2024-05-09T08:02:15.902065+0000","last_active":"2024-05-09T08:02:20.933062+0000","last_peered":"2024-05-09T08:02:20.933062+0000","last_clean":"2024-05-09T08:02:20.933062+0000","last_became_active":"2024-05-09T08:02:15.901766+0000","last_became_peered":"2024-05-09T08:02:15.901766+0000","last_unstale":"2024-05-09T08:02:20.933062+0000","last_undegraded":"2024-05-09T08:02:20.933062+0000","last_fullsized":"2024-05-09T08:02:20.933062+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:05:02.181367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:21.076616+0000","last_change":"2024-05-09T08:02:15.911058+0000","last_active":"2024-05-09T08:02:21.076616+0000","last_peered":"2024-05-09T08:02:21.076616+0000","last_clean":"2024-05-09T08:02:21.076616+0000","last_became_active":"2024-05-09T08:02:15.910716+0000","last_became_peered":"2024-05-09T08:02:15.910716+0000","last_unstale":"2024-05-09T08:02:21.076616+0000","last_undegraded":"2024-05-09T08:02:21.076616+0000","last_fullsized":"2024-05-09T08:02:21.076616+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:41:56.871863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.617832+0000","last_change":"2024-05-09T08:02:15.908653+0000","last_active":"2024-05-09T08:02:22.617832+0000","last_peered":"2024-05-09T08:02:22.617832+0000","last_clean":"2024-05-09T08:02:22.617832+0000","last_became_active":"2024-05-09T08:02:15.908290+0000","last_became_peered":"2024-05-09T08:02:15.908290+0000","last_unstale":"2024-05-09T08:02:22.617832+0000","last_undegraded":"2024-05-09T08:02:22.617832+0000","last_fullsized":"2024-05-09T08:02:22.617832+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:18:36.309627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:21.076688+0000","last_change":"2024-05-09T08:02:15.896652+0000","last_active":"2024-05-09T08:02:21.076688+0000","last_peered":"2024-05-09T08:02:21.076688+0000","last_clean":"2024-05-09T08:02:21.076688+0000","last_became_active":"2024-05-09T08:02:15.896384+0000","last_became_peered":"2024-05-09T08:02:15.896384+0000","last_unstale":"2024-05-09T08:02:21.076688+0000","last_undegraded":"2024-05-09T08:02:21.076688+0000","last_fullsized":"2024-05-09T08:02:21.076688+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:18:36.309627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:22.617988+0000","last_change":"2024-05-09T08:02:15.898998+0000","last_active":"2024-05-09T08:02:22.617988+0000","last_peered":"2024-05-09T08:02:22.617988+0000","last_clean":"2024-05-09T08:02:22.617988+0000","last_became_active":"2024-05-09T08:02:15.898670+0000","last_became_peered":"2024-05-09T08:02:15.898670+0000","last_unstale":"2024-05-09T08:02:22.617988+0000","last_undegraded":"2024-05-09T08:02:22.617988+0000","last_fullsized":"2024-05-09T08:02:22.617988+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:23:40.248821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:21.677605+0000","last_change":"2024-05-09T08:02:15.905078+0000","last_active":"2024-05-09T08:02:21.677605+0000","last_peered":"2024-05-09T08:02:21.677605+0000","last_clean":"2024-05-09T08:02:21.677605+0000","last_became_active":"2024-05-09T08:02:15.904736+0000","last_became_peered":"2024-05-09T08:02:15.904736+0000","last_unstale":"2024-05-09T08:02:21.677605+0000","last_undegraded":"2024-05-09T08:02:21.677605+0000","last_fullsized":"2024-05-09T08:02:21.677605+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:01:50.341133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:20.936449+0000","last_change":"2024-05-09T08:02:15.897743+0000","last_active":"2024-05-09T08:02:20.936449+0000","last_peered":"2024-05-09T08:02:20.936449+0000","last_clean":"2024-05-09T08:02:20.936449+0000","last_became_active":"2024-05-09T08:02:15.897367+0000","last_became_peered":"2024-05-09T08:02:15.897367+0000","last_unstale":"2024-05-09T08:02:20.936449+0000","last_undegraded":"2024-05-09T08:02:20.936449+0000","last_fullsized":"2024-05-09T08:02:20.936449+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:47:32.605288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T08:02:21.077273+0000","last_change":"2024-05-09T08:02:15.884163+0000","last_active":"2024-05-09T08:02:21.077273+0000","last_peered":"2024-05-09T08:02:21.077273+0000","last_clean":"2024-05-09T08:02:21.077273+0000","last_became_active":"2024-05-09T08:02:15.883430+0000","last_became_peered":"2024-05-09T08:02:15.883430+0000","last_unstale":"2024-05-09T08:02:21.077273+0000","last_undegraded":"2024-05-09T08:02:21.077273+0000","last_fullsized":"2024-05-09T08:02:21.077273+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:05:15.555592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574859,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32024,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339816,"statfs":{"total":96636764160,"available":96603971584,"internally_reserved":0,"allocated":864256,"data_stored":695877,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574859,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31448,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340392,"statfs":{"total":96636764160,"available":96604561408,"internally_reserved":0,"allocated":278528,"data_stored":108769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574859,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31444,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340396,"statfs":{"total":96636764160,"available":96604565504,"internally_reserved":0,"allocated":270336,"data_stored":105017,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574860,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31452,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340388,"statfs":{"total":96636764160,"available":96604557312,"internally_reserved":0,"allocated":278528,"data_stored":105053,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574860,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":108769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574860,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":266240,"data_stored":104927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574861,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":274432,"data_stored":105087,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574860,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32032,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339808,"statfs":{"total":96636764160,"available":96603963392,"internally_reserved":0,"allocated":872448,"data_stored":695933,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T08:02:57.821 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-05-09T08:02:57.822 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-05-09T08:02:57.822 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-05-09T08:02:57.822 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-05-09T08:02:58.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.049+0000 7f31ba600640 1 Processor -- start 2024-05-09T08:02:58.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.049+0000 7f31ba600640 1 -- start start 2024-05-09T08:02:58.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.051+0000 7f31ba600640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f31b415a440 0x7f31b40c7dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:58.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.051+0000 7f31ba600640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f31b40c48c0 con 0x7f31b40c44b0 2024-05-09T08:02:58.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.051+0000 7f31ba600640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f31b40c6a50 con 0x7f31b4154790 2024-05-09T08:02:58.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.051+0000 7f31ba600640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f31b4154b70 con 0x7f31b415a440 2024-05-09T08:02:58.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.051+0000 7f31b9200640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f31b4154790 0x7f31b406b250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.1:42160/0 (socket says 172.21.15.1:42160) 2024-05-09T08:02:58.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.051+0000 7f31b9200640 1 -- 172.21.15.1:0/2908968756 learned_addr learned my addr 172.21.15.1:0/2908968756 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T08:02:58.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.051+0000 7f31b3e00640 1 --2- 172.21.15.1:0/2908968756 >> v2:172.21.15.7:3300/0 conn(0x7f31b415a440 0x7f31b40c7dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:58.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.052+0000 7f31b3400640 1 -- 172.21.15.1:0/2908968756 <== mon.1 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2213746251 0 0) 0x7f31b40c6a50 con 0x7f31b4154790 2024-05-09T08:02:58.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.052+0000 7f31b3400640 1 -- 172.21.15.1:0/2908968756 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3198003520 con 0x7f31b4154790 2024-05-09T08:02:58.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.052+0000 7f31b3e00640 1 -- 172.21.15.1:0/2908968756 >> v1:172.21.15.7:6789/0 conn(0x7f31b4154790 legacy=0x7f31b406b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:58.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.052+0000 7f31b3e00640 1 -- 172.21.15.1:0/2908968756 >> v1:172.21.15.1:6789/0 conn(0x7f31b40c44b0 legacy=0x7f31b406b960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:58.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.052+0000 7f31b3400640 1 -- 172.21.15.1:0/2908968756 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1538245485 0 0) 0x7f31b40c48c0 con 0x7f31b40c44b0 2024-05-09T08:02:58.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.052+0000 7f31b3e00640 1 -- 172.21.15.1:0/2908968756 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31b40c6d30 con 0x7f31b415a440 2024-05-09T08:02:58.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.053+0000 7f31b3e00640 1 --2- 172.21.15.1:0/2908968756 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31b415a440 0x7f31b40c7dd0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f31a00097a0 tx=0x7f31a002a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c730eadf1c575e45 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:58.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.053+0000 7f31b3400640 1 -- 172.21.15.1:0/2908968756 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31a0037070 con 0x7f31b415a440 2024-05-09T08:02:58.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.053+0000 7f31b3400640 1 -- 172.21.15.1:0/2908968756 <== mon.1 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31a0032440 con 0x7f31b415a440 2024-05-09T08:02:58.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.053+0000 7f31b3400640 1 -- 172.21.15.1:0/2908968756 <== mon.1 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31a0032810 con 0x7f31b415a440 2024-05-09T08:02:58.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.053+0000 7f31ba600640 1 -- 172.21.15.1:0/2908968756 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31b415a440 msgr2=0x7f31b40c7dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:58.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.053+0000 7f31ba600640 1 --2- 172.21.15.1:0/2908968756 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31b415a440 0x7f31b40c7dd0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f31a00097a0 tx=0x7f31a002a230 comp rx=0 tx=0).stop 2024-05-09T08:02:58.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.054+0000 7f31ba600640 1 -- 172.21.15.1:0/2908968756 shutdown_connections 2024-05-09T08:02:58.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.054+0000 7f31ba600640 1 --2- 172.21.15.1:0/2908968756 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31b415a440 0x7f31b40c7dd0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:58.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.054+0000 7f31ba600640 1 -- 172.21.15.1:0/2908968756 >> 172.21.15.1:0/2908968756 conn(0x7f31b40cbc10 msgr2=0x7f31b40cc020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:02:58.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.054+0000 7f31ba600640 1 -- 172.21.15.1:0/2908968756 shutdown_connections 2024-05-09T08:02:58.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.054+0000 7f31ba600640 1 -- 172.21.15.1:0/2908968756 wait complete. 2024-05-09T08:02:58.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.055+0000 7f31ba600640 1 Processor -- start 2024-05-09T08:02:58.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.055+0000 7f31ba600640 1 -- start start 2024-05-09T08:02:58.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.056+0000 7f31ba600640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f31b40c44b0 0x7f31b4142c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:58.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.056+0000 7f31ba600640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f31b4154790 0x7f31b4151d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:58.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.056+0000 7f31ba600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31b415a440 0x7f31b4156f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:58.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.056+0000 7f31ba600640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f31b40c75d0 con 0x7f31b40c44b0 2024-05-09T08:02:58.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.056+0000 7f31ba600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f31b40c72f0 con 0x7f31b415a440 2024-05-09T08:02:58.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.056+0000 7f31ba600640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f31b40c78b0 con 0x7f31b4154790 2024-05-09T08:02:58.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.057+0000 7f31b9c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31b415a440 0x7f31b4156f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:58.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.057+0000 7f31b9200640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f31b40c44b0 0x7f31b4142c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:58.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.057+0000 7f31b3e00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f31b4154790 0x7f31b4151d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:58.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.057+0000 7f31b9c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31b415a440 0x7f31b4156f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.1:59148/0 (socket says 172.21.15.1:59148) 2024-05-09T08:02:58.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.057+0000 7f31b9c00640 1 -- 172.21.15.1:0/4277776574 learned_addr learned my addr 172.21.15.1:0/4277776574 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:02:58.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.057+0000 7f31b9c00640 1 -- 172.21.15.1:0/4277776574 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f31b4154790 msgr2=0x7f31b4151d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:58.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.057+0000 7f31b9c00640 1 --2- 172.21.15.1:0/4277776574 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f31b4154790 0x7f31b4151d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:58.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.057+0000 7f31b9c00640 1 -- 172.21.15.1:0/4277776574 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f31b40c44b0 msgr2=0x7f31b4142c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:58.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.057+0000 7f31b9c00640 1 --2- 172.21.15.1:0/4277776574 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f31b40c44b0 0x7f31b4142c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:58.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.057+0000 7f31b9c00640 1 -- 172.21.15.1:0/4277776574 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31b41f03d0 con 0x7f31b415a440 2024-05-09T08:02:58.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.057+0000 7f31b3e00640 1 --2- 172.21.15.1:0/4277776574 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f31b4154790 0x7f31b4151d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:02:58.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.057+0000 7f31b9c00640 1 --2- 172.21.15.1:0/4277776574 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31b415a440 0x7f31b4156f90 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f31ac00c480 tx=0x7f31ac00c940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:58.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.058+0000 7f31b0c00640 1 -- 172.21.15.1:0/4277776574 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31ac005480 con 0x7f31b415a440 2024-05-09T08:02:58.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.058+0000 7f31b0c00640 1 -- 172.21.15.1:0/4277776574 <== mon.1 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31ac0035f0 con 0x7f31b415a440 2024-05-09T08:02:58.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.058+0000 7f31b0c00640 1 -- 172.21.15.1:0/4277776574 <== mon.1 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31ac0039c0 con 0x7f31b415a440 2024-05-09T08:02:58.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.058+0000 7f31ba600640 1 -- 172.21.15.1:0/4277776574 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31b41f0670 con 0x7f31b415a440 2024-05-09T08:02:58.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.058+0000 7f31ba600640 1 -- 172.21.15.1:0/4277776574 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31b4071b80 con 0x7f31b415a440 2024-05-09T08:02:58.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.059+0000 7f31ba600640 1 -- 172.21.15.1:0/4277776574 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31b404f660 con 0x7f31b415a440 2024-05-09T08:02:58.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.060+0000 7f31b0c00640 1 -- 172.21.15.1:0/4277776574 <== mon.1 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f31ac003b60 con 0x7f31b415a440 2024-05-09T08:02:58.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.061+0000 7f31b0c00640 1 --2- 172.21.15.1:0/4277776574 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f319406bfd0 0x7f319406e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:58.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.061+0000 7f31b0c00640 1 -- 172.21.15.1:0/4277776574 <== mon.1 v2:172.21.15.7:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f31ac08ad30 con 0x7f31b415a440 2024-05-09T08:02:58.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.061+0000 7f31b9200640 1 --2- 172.21.15.1:0/4277776574 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f319406bfd0 0x7f319406e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:58.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.062+0000 7f31b9200640 1 --2- 172.21.15.1:0/4277776574 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f319406bfd0 0x7f319406e490 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f31a4001560 tx=0x7f31a4001a90 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:58.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.066+0000 7f31b0c00640 1 -- 172.21.15.1:0/4277776574 <== mon.1 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f31ac058cb0 con 0x7f31b415a440 2024-05-09T08:02:58.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.297+0000 7f31ba600640 1 -- 172.21.15.1:0/4277776574 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f31b40c2990 con 0x7f31b415a440 2024-05-09T08:02:58.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.298+0000 7f31b0c00640 1 -- 172.21.15.1:0/4277776574 <== mon.1 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f31ac05cfb0 con 0x7f31b415a440 2024-05-09T08:02:58.315 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T08:02:58.315 INFO:teuthology.orchestra.run.smithi001.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-05-09T08:02:58.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.301+0000 7f31ba600640 1 -- 172.21.15.1:0/4277776574 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f319406bfd0 msgr2=0x7f319406e490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:58.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.301+0000 7f31ba600640 1 --2- 172.21.15.1:0/4277776574 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f319406bfd0 0x7f319406e490 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f31a4001560 tx=0x7f31a4001a90 comp rx=0 tx=0).stop 2024-05-09T08:02:58.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.301+0000 7f31ba600640 1 -- 172.21.15.1:0/4277776574 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31b415a440 msgr2=0x7f31b4156f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:58.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.301+0000 7f31ba600640 1 --2- 172.21.15.1:0/4277776574 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31b415a440 0x7f31b4156f90 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f31ac00c480 tx=0x7f31ac00c940 comp rx=0 tx=0).stop 2024-05-09T08:02:58.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.301+0000 7f31ba600640 1 -- 172.21.15.1:0/4277776574 shutdown_connections 2024-05-09T08:02:58.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.301+0000 7f31ba600640 1 --2- 172.21.15.1:0/4277776574 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f319406bfd0 0x7f319406e490 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:58.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.301+0000 7f31ba600640 1 --2- 172.21.15.1:0/4277776574 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31b415a440 0x7f31b4156f90 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:58.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.301+0000 7f31ba600640 1 --2- 172.21.15.1:0/4277776574 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f31b4154790 0x7f31b4151d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:58.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.301+0000 7f31ba600640 1 --2- 172.21.15.1:0/4277776574 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f31b40c44b0 0x7f31b4142c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:58.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.301+0000 7f31ba600640 1 -- 172.21.15.1:0/4277776574 >> 172.21.15.1:0/4277776574 conn(0x7f31b40cbc10 msgr2=0x7f31b40c61e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:02:58.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.301+0000 7f31ba600640 1 -- 172.21.15.1:0/4277776574 shutdown_connections 2024-05-09T08:02:58.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.301+0000 7f31ba600640 1 -- 172.21.15.1:0/4277776574 wait complete. 2024-05-09T08:02:58.337 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-05-09T08:02:58.337 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T08:02:58.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.516+0000 7f68a3200640 1 Processor -- start 2024-05-09T08:02:58.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.516+0000 7f68a3200640 1 -- start start 2024-05-09T08:02:58.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.518+0000 7f68a3200640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f689c152500 0x7f689c0c8180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:58.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.518+0000 7f68a3200640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f689c0cc730 0x7f689c15c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:58.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.518+0000 7f68a3200640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f689c0c6a30 con 0x7f689c0c7660 2024-05-09T08:02:58.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.518+0000 7f68a3200640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f689c0c5e50 con 0x7f689c0cc730 2024-05-09T08:02:58.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.518+0000 7f68a3200640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f689c0c22d0 con 0x7f689c152500 2024-05-09T08:02:58.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.518+0000 7f68a1400640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f689c0cc730 0x7f689c15c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:58.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.518+0000 7f68a1400640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f689c0cc730 0x7f689c15c080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38360/0 (socket says 172.21.15.1:38360) 2024-05-09T08:02:58.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.519+0000 7f68a1400640 1 -- 172.21.15.1:0/1128227307 learned_addr learned my addr 172.21.15.1:0/1128227307 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:02:58.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.519+0000 7f68a1e00640 1 --2- 172.21.15.1:0/1128227307 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f689c152500 0x7f689c0c8180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:58.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.519+0000 7f68a1400640 1 -- 172.21.15.1:0/1128227307 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f689c152500 msgr2=0x7f689c0c8180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:02:58.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.519+0000 7f68a1400640 1 --2- 172.21.15.1:0/1128227307 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f689c152500 0x7f689c0c8180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:58.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.519+0000 7f68a1400640 1 -- 172.21.15.1:0/1128227307 >> v1:172.21.15.7:6789/0 conn(0x7f689c0c7660 legacy=0x7f689c06b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:58.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.519+0000 7f68a1400640 1 -- 172.21.15.1:0/1128227307 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f689c0c6cf0 con 0x7f689c0cc730 2024-05-09T08:02:58.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.520+0000 7f68a1400640 1 --2- 172.21.15.1:0/1128227307 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f689c0cc730 0x7f689c15c080 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f688c009870 tx=0x7f688c02a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=196d937a6c273d3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:58.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.520+0000 7f68a0a00640 1 -- 172.21.15.1:0/1128227307 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f688c037070 con 0x7f689c0cc730 2024-05-09T08:02:58.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.520+0000 7f68a0a00640 1 -- 172.21.15.1:0/1128227307 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f688c02ae50 con 0x7f689c0cc730 2024-05-09T08:02:58.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.520+0000 7f68a0a00640 1 -- 172.21.15.1:0/1128227307 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f688c032860 con 0x7f689c0cc730 2024-05-09T08:02:58.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.521+0000 7f68a3200640 1 -- 172.21.15.1:0/1128227307 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f689c0cc730 msgr2=0x7f689c15c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:58.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.521+0000 7f68a3200640 1 --2- 172.21.15.1:0/1128227307 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f689c0cc730 0x7f689c15c080 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f688c009870 tx=0x7f688c02a300 comp rx=0 tx=0).stop 2024-05-09T08:02:58.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.521+0000 7f68a3200640 1 -- 172.21.15.1:0/1128227307 shutdown_connections 2024-05-09T08:02:58.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.521+0000 7f68a3200640 1 --2- 172.21.15.1:0/1128227307 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f689c0cc730 0x7f689c15c080 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:58.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.521+0000 7f68a3200640 1 --2- 172.21.15.1:0/1128227307 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f689c152500 0x7f689c0c8180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:58.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.521+0000 7f68a3200640 1 -- 172.21.15.1:0/1128227307 >> 172.21.15.1:0/1128227307 conn(0x7f689c0cbc10 msgr2=0x7f689c0cc020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:02:58.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.521+0000 7f68a3200640 1 -- 172.21.15.1:0/1128227307 shutdown_connections 2024-05-09T08:02:58.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.521+0000 7f68a3200640 1 -- 172.21.15.1:0/1128227307 wait complete. 2024-05-09T08:02:58.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.523+0000 7f68a3200640 1 Processor -- start 2024-05-09T08:02:58.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.523+0000 7f68a3200640 1 -- start start 2024-05-09T08:02:58.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.524+0000 7f68a3200640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f689c0c7660 0x7f689c15a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:58.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.524+0000 7f68a3200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f689c0cc730 0x7f689c159470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:58.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.524+0000 7f68a3200640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f689c152500 0x7f689c1599b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:58.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.524+0000 7f68a3200640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f689c0c72b0 con 0x7f689c0c7660 2024-05-09T08:02:58.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.524+0000 7f68a3200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f689c0ccb50 con 0x7f689c0cc730 2024-05-09T08:02:58.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.524+0000 7f68a3200640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f689c15b880 con 0x7f689c152500 2024-05-09T08:02:58.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.524+0000 7f68a1e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f689c0c7660 0x7f689c15a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:58.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.524+0000 7f68a1e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f689c0c7660 0x7f689c15a9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38362/0 (socket says 172.21.15.1:38362) 2024-05-09T08:02:58.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.524+0000 7f68a1e00640 1 -- 172.21.15.1:0/3276485406 learned_addr learned my addr 172.21.15.1:0/3276485406 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:02:58.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.524+0000 7f68a1400640 1 --2- 172.21.15.1:0/3276485406 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f689c0cc730 0x7f689c159470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:58.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.524+0000 7f68a2800640 1 --2- 172.21.15.1:0/3276485406 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f689c152500 0x7f689c1599b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:58.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.525+0000 7f68a1e00640 1 -- 172.21.15.1:0/3276485406 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f689c152500 msgr2=0x7f689c1599b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:58.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.525+0000 7f68a1e00640 1 --2- 172.21.15.1:0/3276485406 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f689c152500 0x7f689c1599b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:58.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.525+0000 7f68a1e00640 1 -- 172.21.15.1:0/3276485406 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f689c0cc730 msgr2=0x7f689c159470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:58.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.525+0000 7f68a1e00640 1 --2- 172.21.15.1:0/3276485406 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f689c0cc730 0x7f689c159470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:58.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.525+0000 7f68a1e00640 1 -- 172.21.15.1:0/3276485406 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f689c0c9bb0 con 0x7f689c0c7660 2024-05-09T08:02:58.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.525+0000 7f68a1e00640 1 --2- 172.21.15.1:0/3276485406 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f689c0c7660 0x7f689c15a9d0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f689000e8a0 tx=0x7f689000ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:58.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.525+0000 7f6896000640 1 -- 172.21.15.1:0/3276485406 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6890017070 con 0x7f689c0c7660 2024-05-09T08:02:58.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.525+0000 7f6896000640 1 -- 172.21.15.1:0/3276485406 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f689000c940 con 0x7f689c0c7660 2024-05-09T08:02:58.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.525+0000 7f6896000640 1 -- 172.21.15.1:0/3276485406 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f689000cd30 con 0x7f689c0c7660 2024-05-09T08:02:58.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.526+0000 7f68a3200640 1 -- 172.21.15.1:0/3276485406 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f689c1f1f80 con 0x7f689c0c7660 2024-05-09T08:02:58.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.526+0000 7f68a3200640 1 -- 172.21.15.1:0/3276485406 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f689c1476f0 con 0x7f689c0c7660 2024-05-09T08:02:58.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.527+0000 7f68a3200640 1 -- 172.21.15.1:0/3276485406 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f689c04f660 con 0x7f689c0c7660 2024-05-09T08:02:58.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.528+0000 7f6896000640 1 -- 172.21.15.1:0/3276485406 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f689001a070 con 0x7f689c0c7660 2024-05-09T08:02:58.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.529+0000 7f6896000640 1 --2- 172.21.15.1:0/3276485406 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f688006be30 0x7f688006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:58.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.529+0000 7f6896000640 1 -- 172.21.15.1:0/3276485406 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f689008af70 con 0x7f689c0c7660 2024-05-09T08:02:58.547 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.529+0000 7f68a1400640 1 --2- 172.21.15.1:0/3276485406 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f688006be30 0x7f688006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:58.547 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.530+0000 7f68a1400640 1 --2- 172.21.15.1:0/3276485406 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f688006be30 0x7f688006e2f0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f688c034000 tx=0x7f688c02e000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:58.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.533+0000 7f6896000640 1 -- 172.21.15.1:0/3276485406 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f6890058f80 con 0x7f689c0c7660 2024-05-09T08:02:58.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.728+0000 7f68a3200640 1 -- 172.21.15.1:0/3276485406 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f689c0c2a40 con 0x7f689c0c7660 2024-05-09T08:02:58.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.730+0000 7f6896000640 1 -- 172.21.15.1:0/3276485406 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18382 (secure 0 0 0) 0x7f689005d280 con 0x7f689c0c7660 2024-05-09T08:02:58.748 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T08:02:58.748 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":25,"fsid":"54329ec8-3270-4043-9a82-c6cd7420aa7f","created":"2024-05-09T08:01:56.666970+0000","modified":"2024-05-09T08:02:20.403456+0000","last_up_change":"2024-05-09T08:02:07.146082+0000","last_in_change":"2024-05-09T08:01:59.008225+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T08:02:09.165511+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T08:02:12.925485+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T08:02:14.367455+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-09T08:02:17.967064+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c064454e-a9b5-4a24-9a77-17f0a4085952","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6800","nonce":1513349547},{"type":"v1","addr":"172.21.15.1:6801","nonce":1513349547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":1513349547},{"type":"v1","addr":"172.21.15.1:6803","nonce":1513349547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":1513349547},{"type":"v1","addr":"172.21.15.1:6807","nonce":1513349547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":1513349547},{"type":"v1","addr":"172.21.15.1:6805","nonce":1513349547}]},"public_addr":"172.21.15.1:6801/1513349547","cluster_addr":"172.21.15.1:6803/1513349547","heartbeat_back_addr":"172.21.15.1:6807/1513349547","heartbeat_front_addr":"172.21.15.1:6805/1513349547","state":["exists","up"]},{"osd":1,"uuid":"1a9ea687-2ff6-407a-949b-0e7e1083af21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":2942255259},{"type":"v1","addr":"172.21.15.1:6817","nonce":2942255259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":2942255259},{"type":"v1","addr":"172.21.15.1:6819","nonce":2942255259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":2942255259},{"type":"v1","addr":"172.21.15.1:6823","nonce":2942255259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6820","nonce":2942255259},{"type":"v1","addr":"172.21.15.1:6821","nonce":2942255259}]},"public_addr":"172.21.15.1:6817/2942255259","cluster_addr":"172.21.15.1:6819/2942255259","heartbeat_back_addr":"172.21.15.1:6823/2942255259","heartbeat_front_addr":"172.21.15.1:6821/2942255259","state":["exists","up"]},{"osd":2,"uuid":"c88af08b-7c9a-4dd8-9640-44138239addb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":2925031811},{"type":"v1","addr":"172.21.15.1:6809","nonce":2925031811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":2925031811},{"type":"v1","addr":"172.21.15.1:6811","nonce":2925031811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":2925031811},{"type":"v1","addr":"172.21.15.1:6815","nonce":2925031811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":2925031811},{"type":"v1","addr":"172.21.15.1:6813","nonce":2925031811}]},"public_addr":"172.21.15.1:6809/2925031811","cluster_addr":"172.21.15.1:6811/2925031811","heartbeat_back_addr":"172.21.15.1:6815/2925031811","heartbeat_front_addr":"172.21.15.1:6813/2925031811","state":["exists","up"]},{"osd":3,"uuid":"c087b7e3-a41d-4333-9365-32e3bd8c8e6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6824","nonce":2497776654},{"type":"v1","addr":"172.21.15.1:6825","nonce":2497776654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":2497776654},{"type":"v1","addr":"172.21.15.1:6827","nonce":2497776654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":2497776654},{"type":"v1","addr":"172.21.15.1:6831","nonce":2497776654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6828","nonce":2497776654},{"type":"v1","addr":"172.21.15.1:6829","nonce":2497776654}]},"public_addr":"172.21.15.1:6825/2497776654","cluster_addr":"172.21.15.1:6827/2497776654","heartbeat_back_addr":"172.21.15.1:6831/2497776654","heartbeat_front_addr":"172.21.15.1:6829/2497776654","state":["exists","up"]},{"osd":4,"uuid":"b4576b9c-871d-47a8-8d36-65af525d0200","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":2957026670},{"type":"v1","addr":"172.21.15.7:6801","nonce":2957026670}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2957026670},{"type":"v1","addr":"172.21.15.7:6803","nonce":2957026670}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":2957026670},{"type":"v1","addr":"172.21.15.7:6807","nonce":2957026670}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2957026670},{"type":"v1","addr":"172.21.15.7:6805","nonce":2957026670}]},"public_addr":"172.21.15.7:6801/2957026670","cluster_addr":"172.21.15.7:6803/2957026670","heartbeat_back_addr":"172.21.15.7:6807/2957026670","heartbeat_front_addr":"172.21.15.7:6805/2957026670","state":["exists","up"]},{"osd":5,"uuid":"902b3d94-72f3-4499-b21a-e32453ef5144","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1670295242},{"type":"v1","addr":"172.21.15.7:6809","nonce":1670295242}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1670295242},{"type":"v1","addr":"172.21.15.7:6811","nonce":1670295242}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1670295242},{"type":"v1","addr":"172.21.15.7:6815","nonce":1670295242}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1670295242},{"type":"v1","addr":"172.21.15.7:6813","nonce":1670295242}]},"public_addr":"172.21.15.7:6809/1670295242","cluster_addr":"172.21.15.7:6811/1670295242","heartbeat_back_addr":"172.21.15.7:6815/1670295242","heartbeat_front_addr":"172.21.15.7:6813/1670295242","state":["exists","up"]},{"osd":6,"uuid":"ead1d771-1728-4f85-adb4-26be21cedff3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":2852086880},{"type":"v1","addr":"172.21.15.7:6817","nonce":2852086880}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":2852086880},{"type":"v1","addr":"172.21.15.7:6819","nonce":2852086880}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":2852086880},{"type":"v1","addr":"172.21.15.7:6823","nonce":2852086880}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":2852086880},{"type":"v1","addr":"172.21.15.7:6821","nonce":2852086880}]},"public_addr":"172.21.15.7:6817/2852086880","cluster_addr":"172.21.15.7:6819/2852086880","heartbeat_back_addr":"172.21.15.7:6823/2852086880","heartbeat_front_addr":"172.21.15.7:6821/2852086880","state":["exists","up"]},{"osd":7,"uuid":"3ab5a201-9394-48a8-8918-e95197893669","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":1442884138},{"type":"v1","addr":"172.21.15.7:6825","nonce":1442884138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1442884138},{"type":"v1","addr":"172.21.15.7:6827","nonce":1442884138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1442884138},{"type":"v1","addr":"172.21.15.7:6831","nonce":1442884138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1442884138},{"type":"v1","addr":"172.21.15.7:6829","nonce":1442884138}]},"public_addr":"172.21.15.7:6825/1442884138","cluster_addr":"172.21.15.7:6827/1442884138","heartbeat_back_addr":"172.21.15.7:6831/1442884138","heartbeat_front_addr":"172.21.15.7:6829/1442884138","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:02:04.615675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:02:04.385974+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:02:04.619217+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:02:04.401681+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:02:04.097081+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:02:04.129981+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:02:04.225327+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:02:04.193351+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T08:02:58.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.733+0000 7f68a3200640 1 -- 172.21.15.1:0/3276485406 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f688006be30 msgr2=0x7f688006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:58.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.733+0000 7f68a3200640 1 --2- 172.21.15.1:0/3276485406 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f688006be30 0x7f688006e2f0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f688c034000 tx=0x7f688c02e000 comp rx=0 tx=0).stop 2024-05-09T08:02:58.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.733+0000 7f68a3200640 1 -- 172.21.15.1:0/3276485406 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f689c0c7660 msgr2=0x7f689c15a9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:58.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.733+0000 7f68a3200640 1 --2- 172.21.15.1:0/3276485406 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f689c0c7660 0x7f689c15a9d0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f689000e8a0 tx=0x7f689000ed60 comp rx=0 tx=0).stop 2024-05-09T08:02:58.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.733+0000 7f68a3200640 1 -- 172.21.15.1:0/3276485406 shutdown_connections 2024-05-09T08:02:58.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.733+0000 7f68a3200640 1 --2- 172.21.15.1:0/3276485406 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f688006be30 0x7f688006e2f0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:58.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.733+0000 7f68a3200640 1 --2- 172.21.15.1:0/3276485406 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f689c0cc730 0x7f689c159470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:58.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.733+0000 7f68a3200640 1 --2- 172.21.15.1:0/3276485406 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f689c152500 0x7f689c1599b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:58.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.733+0000 7f68a3200640 1 --2- 172.21.15.1:0/3276485406 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f689c0c7660 0x7f689c15a9d0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:58.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.733+0000 7f68a3200640 1 -- 172.21.15.1:0/3276485406 >> 172.21.15.1:0/3276485406 conn(0x7f689c0cbc10 msgr2=0x7f689c156000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:02:58.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.733+0000 7f68a3200640 1 -- 172.21.15.1:0/3276485406 shutdown_connections 2024-05-09T08:02:58.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.733+0000 7f68a3200640 1 -- 172.21.15.1:0/3276485406 wait complete. 2024-05-09T08:02:58.768 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T08:02:09.165511+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-09T08:02:12.925485+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-09T08:02:14.367455+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-09T08:02:17.967064+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-09T08:02:58.769 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-09T08:02:58.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.963+0000 7fdccd400640 1 Processor -- start 2024-05-09T08:02:58.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.963+0000 7fdccd400640 1 -- start start 2024-05-09T08:02:58.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.965+0000 7fdccd400640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fdcc8157d60 0x7fdcc806cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:58.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.965+0000 7fdccd400640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fdcc80b3440 0x7fdcc806d260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:58.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.965+0000 7fdccd400640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdcc815b300 con 0x7fdcc80c9860 2024-05-09T08:02:58.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.965+0000 7fdccd400640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7fdcc80b3910 con 0x7fdcc80b3440 2024-05-09T08:02:58.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.965+0000 7fdccd400640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7fdcc80c8870 con 0x7fdcc8157d60 2024-05-09T08:02:58.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.965+0000 7fdcbde00640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fdcc80b3440 0x7fdcc806d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:58.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.966+0000 7fdcbde00640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fdcc80b3440 0x7fdcc806d260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38370/0 (socket says 172.21.15.1:38370) 2024-05-09T08:02:58.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.966+0000 7fdcbde00640 1 -- 172.21.15.1:0/2685032385 learned_addr learned my addr 172.21.15.1:0/2685032385 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:02:58.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.966+0000 7fdcc6800640 1 --2- 172.21.15.1:0/2685032385 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fdcc8157d60 0x7fdcc806cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:58.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.966+0000 7fdcbde00640 1 -- 172.21.15.1:0/2685032385 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fdcc8157d60 msgr2=0x7fdcc806cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:02:58.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.966+0000 7fdcbde00640 1 --2- 172.21.15.1:0/2685032385 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fdcc8157d60 0x7fdcc806cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:58.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.966+0000 7fdcbde00640 1 -- 172.21.15.1:0/2685032385 >> v1:172.21.15.7:6789/0 conn(0x7fdcc80c9860 legacy=0x7fdcc806f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:58.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.966+0000 7fdcbde00640 1 -- 172.21.15.1:0/2685032385 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcc80c8b50 con 0x7fdcc80b3440 2024-05-09T08:02:58.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.966+0000 7fdcc6800640 1 --2- 172.21.15.1:0/2685032385 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fdcc8157d60 0x7fdcc806cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:02:58.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.967+0000 7fdcbde00640 1 --2- 172.21.15.1:0/2685032385 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdcc80b3440 0x7fdcc806d260 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fdcb0009870 tx=0x7fdcb002a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcb7a96308c8336 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:58.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.967+0000 7fdcc5e00640 1 -- 172.21.15.1:0/2685032385 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcb0037070 con 0x7fdcc80b3440 2024-05-09T08:02:58.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.967+0000 7fdcc5e00640 1 -- 172.21.15.1:0/2685032385 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdcb0032440 con 0x7fdcc80b3440 2024-05-09T08:02:58.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.967+0000 7fdcc5e00640 1 -- 172.21.15.1:0/2685032385 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcb0032810 con 0x7fdcc80b3440 2024-05-09T08:02:58.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.968+0000 7fdccd400640 1 -- 172.21.15.1:0/2685032385 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdcc80b3440 msgr2=0x7fdcc806d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:58.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.968+0000 7fdccd400640 1 --2- 172.21.15.1:0/2685032385 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdcc80b3440 0x7fdcc806d260 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fdcb0009870 tx=0x7fdcb002a250 comp rx=0 tx=0).stop 2024-05-09T08:02:58.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.968+0000 7fdccd400640 1 -- 172.21.15.1:0/2685032385 shutdown_connections 2024-05-09T08:02:58.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.968+0000 7fdccd400640 1 --2- 172.21.15.1:0/2685032385 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdcc80b3440 0x7fdcc806d260 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:58.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.968+0000 7fdccd400640 1 --2- 172.21.15.1:0/2685032385 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fdcc8157d60 0x7fdcc806cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:58.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.968+0000 7fdccd400640 1 -- 172.21.15.1:0/2685032385 >> 172.21.15.1:0/2685032385 conn(0x7fdcc8071870 msgr2=0x7fdcc8071c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:02:58.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.968+0000 7fdccd400640 1 -- 172.21.15.1:0/2685032385 shutdown_connections 2024-05-09T08:02:58.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.968+0000 7fdccd400640 1 -- 172.21.15.1:0/2685032385 wait complete. 2024-05-09T08:02:58.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.970+0000 7fdccd400640 1 Processor -- start 2024-05-09T08:02:58.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.970+0000 7fdccd400640 1 -- start start 2024-05-09T08:02:58.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.971+0000 7fdccd400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdcc80b3440 0x7fdcc80cbbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:58.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.971+0000 7fdccd400640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fdcc80c9860 0x7fdcc81f4000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:58.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.971+0000 7fdccd400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdcc8157d60 0x7fdcc80c4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:58.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.971+0000 7fdccd400640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc815b5e0 con 0x7fdcc8157d60 2024-05-09T08:02:58.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.971+0000 7fdccd400640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc806c4f0 con 0x7fdcc80b3440 2024-05-09T08:02:58.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.971+0000 7fdccd400640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7fdcc806fda0 con 0x7fdcc80c9860 2024-05-09T08:02:58.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.971+0000 7fdcc6800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdcc80b3440 0x7fdcc80cbbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:58.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.972+0000 7fdcbde00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fdcc80c9860 0x7fdcc81f4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:58.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.972+0000 7fdcc6800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdcc80b3440 0x7fdcc80cbbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.1:59170/0 (socket says 172.21.15.1:59170) 2024-05-09T08:02:58.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.972+0000 7fdcc6800640 1 -- 172.21.15.1:0/177782115 learned_addr learned my addr 172.21.15.1:0/177782115 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:02:58.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.972+0000 7fdcc7200640 1 --2- 172.21.15.1:0/177782115 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdcc8157d60 0x7fdcc80c4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:58.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.972+0000 7fdcbde00640 1 -- 172.21.15.1:0/177782115 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdcc80b3440 msgr2=0x7fdcc80cbbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:58.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.972+0000 7fdcbde00640 1 --2- 172.21.15.1:0/177782115 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdcc80b3440 0x7fdcc80cbbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:58.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.972+0000 7fdcbde00640 1 -- 172.21.15.1:0/177782115 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdcc8157d60 msgr2=0x7fdcc80c4010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:58.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.972+0000 7fdcbde00640 1 --2- 172.21.15.1:0/177782115 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdcc8157d60 0x7fdcc80c4010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:58.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.972+0000 7fdcbde00640 1 -- 172.21.15.1:0/177782115 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcc80c7370 con 0x7fdcc80c9860 2024-05-09T08:02:58.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.972+0000 7fdcbde00640 1 --2- 172.21.15.1:0/177782115 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fdcc80c9860 0x7fdcc81f4000 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fdcb0031000 tx=0x7fdcb00046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:58.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.973+0000 7fdcbf400640 1 -- 172.21.15.1:0/177782115 <== mon.2 v2:172.21.15.7:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcb0041020 con 0x7fdcc80c9860 2024-05-09T08:02:58.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.973+0000 7fdcbf400640 1 -- 172.21.15.1:0/177782115 <== mon.2 v2:172.21.15.7:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdcb0030660 con 0x7fdcc80c9860 2024-05-09T08:02:58.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.973+0000 7fdccd400640 1 -- 172.21.15.1:0/177782115 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcc81508b0 con 0x7fdcc80c9860 2024-05-09T08:02:58.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.973+0000 7fdcbf400640 1 -- 172.21.15.1:0/177782115 <== mon.2 v2:172.21.15.7:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcb0037040 con 0x7fdcc80c9860 2024-05-09T08:02:58.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.973+0000 7fdccd400640 1 -- 172.21.15.1:0/177782115 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcc8150e30 con 0x7fdcc80c9860 2024-05-09T08:02:58.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.974+0000 7fdccd400640 1 -- 172.21.15.1:0/177782115 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdcc8074d70 con 0x7fdcc80c9860 2024-05-09T08:02:58.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.975+0000 7fdcbf400640 1 -- 172.21.15.1:0/177782115 <== mon.2 v2:172.21.15.7:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fdcb0044020 con 0x7fdcc80c9860 2024-05-09T08:02:58.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.976+0000 7fdcbf400640 1 --2- 172.21.15.1:0/177782115 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fdca406be30 0x7fdca406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:58.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.976+0000 7fdcbf400640 1 -- 172.21.15.1:0/177782115 <== mon.2 v2:172.21.15.7:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fdcb00ab7c0 con 0x7fdcc80c9860 2024-05-09T08:02:58.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.976+0000 7fdcc6800640 1 --2- 172.21.15.1:0/177782115 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fdca406be30 0x7fdca406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:58.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.977+0000 7fdcc6800640 1 --2- 172.21.15.1:0/177782115 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fdca406be30 0x7fdca406e2f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fdcb8009c50 tx=0x7fdcb800b040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:58.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:58.981+0000 7fdcbf400640 1 -- 172.21.15.1:0/177782115 <== mon.2 v2:172.21.15.7:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fdcb004b030 con 0x7fdcc80c9860 2024-05-09T08:02:59.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:59.180+0000 7fdccd400640 1 -- 172.21.15.1:0/177782115 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fdcc80c9040 con 0x7fdcc80c9860 2024-05-09T08:02:59.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:59.181+0000 7fdcbf400640 1 -- 172.21.15.1:0/177782115 <== mon.2 v2:172.21.15.7:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) ==== 93+0+10 (secure 0 0 0) 0x7fdcb00796b0 con 0x7fdcc80c9860 2024-05-09T08:02:59.198 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 1 2024-05-09T08:02:59.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:59.183+0000 7fdccd400640 1 -- 172.21.15.1:0/177782115 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fdca406be30 msgr2=0x7fdca406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:59.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:59.183+0000 7fdccd400640 1 --2- 172.21.15.1:0/177782115 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fdca406be30 0x7fdca406e2f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fdcb8009c50 tx=0x7fdcb800b040 comp rx=0 tx=0).stop 2024-05-09T08:02:59.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:59.183+0000 7fdccd400640 1 -- 172.21.15.1:0/177782115 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fdcc80c9860 msgr2=0x7fdcc81f4000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:59.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:59.183+0000 7fdccd400640 1 --2- 172.21.15.1:0/177782115 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fdcc80c9860 0x7fdcc81f4000 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fdcb0031000 tx=0x7fdcb00046c0 comp rx=0 tx=0).stop 2024-05-09T08:02:59.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:59.184+0000 7fdccd400640 1 -- 172.21.15.1:0/177782115 shutdown_connections 2024-05-09T08:02:59.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:59.184+0000 7fdccd400640 1 --2- 172.21.15.1:0/177782115 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fdca406be30 0x7fdca406e2f0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:59.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:59.184+0000 7fdccd400640 1 --2- 172.21.15.1:0/177782115 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdcc8157d60 0x7fdcc80c4010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:59.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:59.184+0000 7fdccd400640 1 --2- 172.21.15.1:0/177782115 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fdcc80c9860 0x7fdcc81f4000 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:59.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:59.184+0000 7fdccd400640 1 --2- 172.21.15.1:0/177782115 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdcc80b3440 0x7fdcc80cbbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:59.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:59.184+0000 7fdccd400640 1 -- 172.21.15.1:0/177782115 >> 172.21.15.1:0/177782115 conn(0x7fdcc8071870 msgr2=0x7fdcc80cac30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:02:59.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:59.184+0000 7fdccd400640 1 -- 172.21.15.1:0/177782115 shutdown_connections 2024-05-09T08:02:59.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:59.184+0000 7fdccd400640 1 -- 172.21.15.1:0/177782115 wait complete. 2024-05-09T08:02:59.221 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-09T08:02:59.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:59.414+0000 7f7531a00640 1 Processor -- start 2024-05-09T08:02:59.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:59.415+0000 7f7531a00640 1 -- start start 2024-05-09T08:02:59.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:59.416+0000 7f7531a00640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f752c0c5df0 0x7f752c0c3070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:59.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:59.416+0000 7f7531a00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f752c0c7b10 0x7f752c0c7f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:59.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:59.416+0000 7f7531a00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f752c0c5350 con 0x7f752c1513a0 2024-05-09T08:02:59.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:59.417+0000 7f7531a00640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f752c0c3670 con 0x7f752c0c5df0 2024-05-09T08:02:59.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:59.417+0000 7f7531a00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f752c0c3950 con 0x7f752c0c7b10 2024-05-09T08:02:59.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:59.417+0000 7f752b400640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f752c0c5df0 0x7f752c0c3070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:59.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:59.417+0000 7f7531000640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f752c0c7b10 0x7f752c0c7f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:59.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:59.417+0000 7f752be00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f752c1513a0 0x7f752c157240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.1:42184/0 (socket says 172.21.15.1:42184) 2024-05-09T08:02:59.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:59.417+0000 7f752be00640 1 -- 172.21.15.1:0/41533507 learned_addr learned my addr 172.21.15.1:0/41533507 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T08:02:59.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:59.417+0000 7f7531000640 1 -- 172.21.15.1:0/41533507 >> v2:172.21.15.1:3300/0 conn(0x7f752c0c5df0 msgr2=0x7f752c0c3070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:02:59.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:59.417+0000 7f7531000640 1 --2- 172.21.15.1:0/41533507 >> v2:172.21.15.1:3300/0 conn(0x7f752c0c5df0 0x7f752c0c3070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:59.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:59.417+0000 7f7531000640 1 -- 172.21.15.1:0/41533507 >> v1:172.21.15.7:6789/0 conn(0x7f752c1513a0 legacy=0x7f752c157240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:59.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:59.417+0000 7f7531000640 1 -- 172.21.15.1:0/41533507 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f752c0c55f0 con 0x7f752c0c7b10 2024-05-09T08:02:59.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:59.419+0000 7f7531000640 1 --2- 172.21.15.1:0/41533507 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f752c0c7b10 0x7f752c0c7f80 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f752c0c51b0 tx=0x7f752402a1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ab90f9dcc7c68ad server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:59.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:59.419+0000 7f752aa00640 1 -- 172.21.15.1:0/41533507 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7524037070 con 0x7f752c0c7b10 2024-05-09T08:02:59.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:59.419+0000 7f752aa00640 1 -- 172.21.15.1:0/41533507 <== mon.1 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7524032440 con 0x7f752c0c7b10 2024-05-09T08:02:59.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:59.419+0000 7f752aa00640 1 -- 172.21.15.1:0/41533507 <== mon.1 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7524032810 con 0x7f752c0c7b10 2024-05-09T08:02:59.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:59.420+0000 7f7531a00640 1 -- 172.21.15.1:0/41533507 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f752c0c7b10 msgr2=0x7f752c0c7f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:59.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:59.420+0000 7f7531a00640 1 --2- 172.21.15.1:0/41533507 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f752c0c7b10 0x7f752c0c7f80 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f752c0c51b0 tx=0x7f752402a1a0 comp rx=0 tx=0).stop 2024-05-09T08:02:59.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:59.420+0000 7f7531a00640 1 -- 172.21.15.1:0/41533507 shutdown_connections 2024-05-09T08:02:59.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:59.420+0000 7f7531a00640 1 --2- 172.21.15.1:0/41533507 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f752c0c7b10 0x7f752c0c7f80 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:59.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:59.420+0000 7f7531a00640 1 --2- 172.21.15.1:0/41533507 >> v2:172.21.15.1:3300/0 conn(0x7f752c0c5df0 0x7f752c0c3070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:59.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:59.420+0000 7f7531a00640 1 -- 172.21.15.1:0/41533507 >> 172.21.15.1:0/41533507 conn(0x7f752c06b250 msgr2=0x7f752c06b630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:02:59.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:59.420+0000 7f7531a00640 1 -- 172.21.15.1:0/41533507 shutdown_connections 2024-05-09T08:02:59.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:59.420+0000 7f7531a00640 1 -- 172.21.15.1:0/41533507 wait complete. 2024-05-09T08:02:59.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:59.422+0000 7f7531a00640 1 Processor -- start 2024-05-09T08:02:59.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:59.422+0000 7f7531a00640 1 -- start start 2024-05-09T08:02:59.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:59.423+0000 7f7531a00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f752c0c5df0 0x7f752c1f0be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:59.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:59.423+0000 7f7531a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f752c0c7b10 0x7f752c15ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:59.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:59.423+0000 7f7531a00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f752c1513a0 0x7f752c15b150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:59.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:59.423+0000 7f7531a00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f752c151900 con 0x7f752c0c5df0 2024-05-09T08:02:59.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:59.423+0000 7f7531a00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f752c151780 con 0x7f752c0c7b10 2024-05-09T08:02:59.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:59.423+0000 7f7531a00640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f752c0c8600 con 0x7f752c1513a0 2024-05-09T08:02:59.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:59.424+0000 7f752be00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f752c0c5df0 0x7f752c1f0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:59.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:59.424+0000 7f752b400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f752c0c7b10 0x7f752c15ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:59.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:59.424+0000 7f752b400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f752c0c7b10 0x7f752c15ac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.1:59180/0 (socket says 172.21.15.1:59180) 2024-05-09T08:02:59.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:59.424+0000 7f752be00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f752c0c5df0 0x7f752c1f0be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38404/0 (socket says 172.21.15.1:38404) 2024-05-09T08:02:59.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:59.424+0000 7f7531000640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f752c1513a0 0x7f752c15b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:59.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:59.424+0000 7f752b400640 1 -- 172.21.15.1:0/2969049577 learned_addr learned my addr 172.21.15.1:0/2969049577 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:02:59.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:59.424+0000 7f752b400640 1 -- 172.21.15.1:0/2969049577 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f752c1513a0 msgr2=0x7f752c15b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:59.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:59.424+0000 7f752b400640 1 --2- 172.21.15.1:0/2969049577 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f752c1513a0 0x7f752c15b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:59.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:59.424+0000 7f752b400640 1 -- 172.21.15.1:0/2969049577 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f752c0c5df0 msgr2=0x7f752c1f0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:59.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:59.424+0000 7f752b400640 1 --2- 172.21.15.1:0/2969049577 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f752c0c5df0 0x7f752c1f0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:59.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:59.424+0000 7f752b400640 1 -- 172.21.15.1:0/2969049577 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f752c14f9d0 con 0x7f752c0c7b10 2024-05-09T08:02:59.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:59.424+0000 7f752be00640 1 --2- 172.21.15.1:0/2969049577 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f752c0c5df0 0x7f752c1f0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:02:59.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:59.425+0000 7f752b400640 1 --2- 172.21.15.1:0/2969049577 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f752c0c7b10 0x7f752c15ac10 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f751800efc0 tx=0x7f751800c490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:59.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:59.425+0000 7f7523e00640 1 -- 172.21.15.1:0/2969049577 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7518018020 con 0x7f752c0c7b10 2024-05-09T08:02:59.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:59.425+0000 7f7523e00640 1 -- 172.21.15.1:0/2969049577 <== mon.1 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7518014480 con 0x7f752c0c7b10 2024-05-09T08:02:59.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:59.425+0000 7f7523e00640 1 -- 172.21.15.1:0/2969049577 <== mon.1 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7518014850 con 0x7f752c0c7b10 2024-05-09T08:02:59.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:59.425+0000 7f7531a00640 1 -- 172.21.15.1:0/2969049577 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f752c1595c0 con 0x7f752c0c7b10 2024-05-09T08:02:59.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:59.425+0000 7f7531a00640 1 -- 172.21.15.1:0/2969049577 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f752c159b80 con 0x7f752c0c7b10 2024-05-09T08:02:59.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:59.426+0000 7f7531a00640 1 -- 172.21.15.1:0/2969049577 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f752c159d80 con 0x7f752c0c7b10 2024-05-09T08:02:59.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:59.427+0000 7f7523e00640 1 -- 172.21.15.1:0/2969049577 <== mon.1 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7518005d10 con 0x7f752c0c7b10 2024-05-09T08:02:59.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:59.428+0000 7f7523e00640 1 --2- 172.21.15.1:0/2969049577 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f750c06be30 0x7f750c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:59.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:59.428+0000 7f7523e00640 1 -- 172.21.15.1:0/2969049577 <== mon.1 v2:172.21.15.7:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f751808bf70 con 0x7f752c0c7b10 2024-05-09T08:02:59.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:59.428+0000 7f752be00640 1 --2- 172.21.15.1:0/2969049577 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f750c06be30 0x7f750c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:59.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:59.429+0000 7f752be00640 1 --2- 172.21.15.1:0/2969049577 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f750c06be30 0x7f750c06e2f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f751c007fd0 tx=0x7f751c007e80 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:59.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:59.433+0000 7f7523e00640 1 -- 172.21.15.1:0/2969049577 <== mon.1 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f752c159d80 con 0x7f752c0c7b10 2024-05-09T08:02:59.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:59.625+0000 7f7531a00640 1 -- 172.21.15.1:0/2969049577 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f752c04f660 con 0x7f752c0c7b10 2024-05-09T08:02:59.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:59.625+0000 7f7523e00640 1 -- 172.21.15.1:0/2969049577 <== mon.1 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) ==== 104+0+11 (secure 0 0 0) 0x7f7518022030 con 0x7f752c0c7b10 2024-05-09T08:02:59.642 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 32 2024-05-09T08:02:59.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:59.628+0000 7f7531a00640 1 -- 172.21.15.1:0/2969049577 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f750c06be30 msgr2=0x7f750c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:59.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:59.628+0000 7f7531a00640 1 --2- 172.21.15.1:0/2969049577 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f750c06be30 0x7f750c06e2f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f751c007fd0 tx=0x7f751c007e80 comp rx=0 tx=0).stop 2024-05-09T08:02:59.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:59.628+0000 7f7531a00640 1 -- 172.21.15.1:0/2969049577 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f752c0c7b10 msgr2=0x7f752c15ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:59.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:59.628+0000 7f7531a00640 1 --2- 172.21.15.1:0/2969049577 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f752c0c7b10 0x7f752c15ac10 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f751800efc0 tx=0x7f751800c490 comp rx=0 tx=0).stop 2024-05-09T08:02:59.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:59.628+0000 7f7531a00640 1 -- 172.21.15.1:0/2969049577 shutdown_connections 2024-05-09T08:02:59.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:59.628+0000 7f7531a00640 1 --2- 172.21.15.1:0/2969049577 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f750c06be30 0x7f750c06e2f0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:59.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:59.628+0000 7f7531a00640 1 --2- 172.21.15.1:0/2969049577 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f752c0c7b10 0x7f752c15ac10 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:59.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:59.628+0000 7f7531a00640 1 --2- 172.21.15.1:0/2969049577 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f752c1513a0 0x7f752c15b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:59.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:59.628+0000 7f7531a00640 1 --2- 172.21.15.1:0/2969049577 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f752c0c5df0 0x7f752c1f0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:59.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:59.628+0000 7f7531a00640 1 -- 172.21.15.1:0/2969049577 >> 172.21.15.1:0/2969049577 conn(0x7f752c06b250 msgr2=0x7f752c150290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:02:59.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:59.629+0000 7f7531a00640 1 -- 172.21.15.1:0/2969049577 shutdown_connections 2024-05-09T08:02:59.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:59.629+0000 7f7531a00640 1 -- 172.21.15.1:0/2969049577 wait complete. 2024-05-09T08:02:59.663 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-09T08:02:59.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:59.849+0000 7f036e600640 1 Processor -- start 2024-05-09T08:02:59.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:59.849+0000 7f036e600640 1 -- start start 2024-05-09T08:02:59.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:59.850+0000 7f036e600640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f03680cac60 0x7f03680cda00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:59.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:59.850+0000 7f036e600640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f03680cc5f0 0x7f03680cdf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:59.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:59.850+0000 7f036e600640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f03680ce4e0 0x7f036815a570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:59.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:59.850+0000 7f036e600640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f03680ccb20 con 0x7f03680cc5f0 2024-05-09T08:02:59.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:59.850+0000 7f036e600640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f03680cce00 con 0x7f03680ce4e0 2024-05-09T08:02:59.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:59.850+0000 7f036e600640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f03680cf120 con 0x7f03680cac60 2024-05-09T08:02:59.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:59.851+0000 7f0367e00640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f03680cc5f0 0x7f03680cdf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:59.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:59.851+0000 7f036d200640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f03680cac60 0x7f03680cda00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:59.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:59.851+0000 7f036dc00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f03680ce4e0 0x7f036815a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:59.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:59.851+0000 7f036d200640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f03680cac60 0x7f03680cda00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3301/0 says I am v2:172.21.15.1:44816/0 (socket says 172.21.15.1:44816) 2024-05-09T08:02:59.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:59.851+0000 7f036d200640 1 -- 172.21.15.1:0/2765614226 learned_addr learned my addr 172.21.15.1:0/2765614226 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:02:59.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:59.851+0000 7f036dc00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f03680ce4e0 0x7f036815a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.1:59192/0 (socket says 172.21.15.1:59192) 2024-05-09T08:02:59.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:59.851+0000 7f036dc00640 1 -- 172.21.15.1:0/2765614226 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f03680cac60 msgr2=0x7f03680cda00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:02:59.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:59.851+0000 7f036dc00640 1 --2- 172.21.15.1:0/2765614226 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f03680cac60 0x7f03680cda00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:59.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:59.851+0000 7f036dc00640 1 -- 172.21.15.1:0/2765614226 >> v2:172.21.15.1:3300/0 conn(0x7f03680cc5f0 msgr2=0x7f03680cdf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:02:59.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:59.851+0000 7f036dc00640 1 --2- 172.21.15.1:0/2765614226 >> v2:172.21.15.1:3300/0 conn(0x7f03680cc5f0 0x7f03680cdf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:59.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:59.851+0000 7f036dc00640 1 -- 172.21.15.1:0/2765614226 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03680cf400 con 0x7f03680ce4e0 2024-05-09T08:02:59.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:59.851+0000 7f036d200640 1 --2- 172.21.15.1:0/2765614226 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f03680cac60 0x7f03680cda00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:02:59.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:59.851+0000 7f0367e00640 1 --2- 172.21.15.1:0/2765614226 >> v2:172.21.15.1:3300/0 conn(0x7f03680cc5f0 0x7f03680cdf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:02:59.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:59.852+0000 7f036dc00640 1 --2- 172.21.15.1:0/2765614226 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03680ce4e0 0x7f036815a570 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f036806b590 tx=0x7f036002a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2d8e8adf9c453522 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:59.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:59.853+0000 7f0367400640 1 -- 172.21.15.1:0/2765614226 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0360007740 con 0x7f03680ce4e0 2024-05-09T08:02:59.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:59.853+0000 7f0367400640 1 -- 172.21.15.1:0/2765614226 <== mon.1 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03600078e0 con 0x7f03680ce4e0 2024-05-09T08:02:59.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:59.853+0000 7f0367400640 1 -- 172.21.15.1:0/2765614226 <== mon.1 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0360007cb0 con 0x7f03680ce4e0 2024-05-09T08:02:59.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:59.853+0000 7f036e600640 1 -- 172.21.15.1:0/2765614226 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03680ce4e0 msgr2=0x7f036815a570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:59.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:59.853+0000 7f036e600640 1 --2- 172.21.15.1:0/2765614226 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03680ce4e0 0x7f036815a570 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f036806b590 tx=0x7f036002a660 comp rx=0 tx=0).stop 2024-05-09T08:02:59.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:59.854+0000 7f036e600640 1 -- 172.21.15.1:0/2765614226 shutdown_connections 2024-05-09T08:02:59.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:59.854+0000 7f036e600640 1 --2- 172.21.15.1:0/2765614226 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03680ce4e0 0x7f036815a570 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:59.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:59.854+0000 7f036e600640 1 --2- 172.21.15.1:0/2765614226 >> v2:172.21.15.1:3300/0 conn(0x7f03680cc5f0 0x7f03680cdf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:59.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:59.854+0000 7f036e600640 1 --2- 172.21.15.1:0/2765614226 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f03680cac60 0x7f03680cda00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:59.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:59.854+0000 7f036e600640 1 -- 172.21.15.1:0/2765614226 >> 172.21.15.1:0/2765614226 conn(0x7f03680cfc20 msgr2=0x7f03680d0030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:02:59.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:59.854+0000 7f036e600640 1 -- 172.21.15.1:0/2765614226 shutdown_connections 2024-05-09T08:02:59.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:59.854+0000 7f036e600640 1 -- 172.21.15.1:0/2765614226 wait complete. 2024-05-09T08:02:59.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:59.856+0000 7f036e600640 1 Processor -- start 2024-05-09T08:02:59.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:59.856+0000 7f036e600640 1 -- start start 2024-05-09T08:02:59.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:59.857+0000 7f036e600640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f03680cac60 0x7f0368155470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:59.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:59.857+0000 7f036e600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03680cc5f0 0x7f036814f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:59.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:59.857+0000 7f036e600640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f03680ce4e0 0x7f036814fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:59.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:59.857+0000 7f036e600640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f03680cea30 con 0x7f03680ce4e0 2024-05-09T08:02:59.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:59.857+0000 7f036e600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f03680cd550 con 0x7f03680cc5f0 2024-05-09T08:02:59.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:59.857+0000 7f036e600640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f03680cebb0 con 0x7f03680cac60 2024-05-09T08:02:59.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:59.858+0000 7f036d200640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f03680cac60 0x7f0368155470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:59.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:59.858+0000 7f036dc00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f03680ce4e0 0x7f036814fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:59.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:59.858+0000 7f0367e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03680cc5f0 0x7f036814f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:59.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:59.858+0000 7f036d200640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f03680cac60 0x7f0368155470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3301/0 says I am v2:172.21.15.1:44832/0 (socket says 172.21.15.1:44832) 2024-05-09T08:02:59.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:59.858+0000 7f036d200640 1 -- 172.21.15.1:0/3801616232 learned_addr learned my addr 172.21.15.1:0/3801616232 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:02:59.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:59.858+0000 7f036d200640 1 -- 172.21.15.1:0/3801616232 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03680cc5f0 msgr2=0x7f036814f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:59.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:59.858+0000 7f036d200640 1 --2- 172.21.15.1:0/3801616232 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03680cc5f0 0x7f036814f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:59.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:59.858+0000 7f036d200640 1 -- 172.21.15.1:0/3801616232 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f03680ce4e0 msgr2=0x7f036814fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:02:59.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:59.858+0000 7f036d200640 1 --2- 172.21.15.1:0/3801616232 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f03680ce4e0 0x7f036814fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:02:59.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:59.858+0000 7f036d200640 1 -- 172.21.15.1:0/3801616232 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03681e3800 con 0x7f03680cac60 2024-05-09T08:02:59.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:59.858+0000 7f0367e00640 1 --2- 172.21.15.1:0/3801616232 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03680cc5f0 0x7f036814f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:02:59.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:59.859+0000 7f036d200640 1 --2- 172.21.15.1:0/3801616232 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f03680cac60 0x7f0368155470 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f035800e9d0 tx=0x7f035800ee90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:59.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:59.859+0000 7f0364c00640 1 -- 172.21.15.1:0/3801616232 <== mon.2 v2:172.21.15.7:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0358017070 con 0x7f03680cac60 2024-05-09T08:02:59.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:59.859+0000 7f0364c00640 1 -- 172.21.15.1:0/3801616232 <== mon.2 v2:172.21.15.7:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f035800ca60 con 0x7f03680cac60 2024-05-09T08:02:59.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:59.859+0000 7f036e600640 1 -- 172.21.15.1:0/3801616232 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0368155f20 con 0x7f03680cac60 2024-05-09T08:02:59.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:59.859+0000 7f0364c00640 1 -- 172.21.15.1:0/3801616232 <== mon.2 v2:172.21.15.7:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f035800ce10 con 0x7f03680cac60 2024-05-09T08:02:59.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:59.859+0000 7f036e600640 1 -- 172.21.15.1:0/3801616232 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f03681e3d60 con 0x7f03680cac60 2024-05-09T08:02:59.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:59.860+0000 7f036e600640 1 -- 172.21.15.1:0/3801616232 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f036804f660 con 0x7f03680cac60 2024-05-09T08:02:59.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:59.861+0000 7f0364c00640 1 -- 172.21.15.1:0/3801616232 <== mon.2 v2:172.21.15.7:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f035801a070 con 0x7f03680cac60 2024-05-09T08:02:59.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:59.862+0000 7f0364c00640 1 --2- 172.21.15.1:0/3801616232 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f034806be60 0x7f034806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:02:59.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:59.863+0000 7f0364c00640 1 -- 172.21.15.1:0/3801616232 <== mon.2 v2:172.21.15.7:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f035808b1b0 con 0x7f03680cac60 2024-05-09T08:02:59.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:59.863+0000 7f0367e00640 1 --2- 172.21.15.1:0/3801616232 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f034806be60 0x7f034806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:02:59.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:59.864+0000 7f0367e00640 1 --2- 172.21.15.1:0/3801616232 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f034806be60 0x7f034806e320 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f0354009560 tx=0x7f0354009190 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:02:59.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:02:59.867+0000 7f0364c00640 1 -- 172.21.15.1:0/3801616232 <== mon.2 v2:172.21.15.7:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f03580590a0 con 0x7f03680cac60 2024-05-09T08:03:00.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:00.080+0000 7f036e600640 1 -- 172.21.15.1:0/3801616232 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f036814f2e0 con 0x7f03680cac60 2024-05-09T08:03:00.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:00.081+0000 7f0364c00640 1 -- 172.21.15.1:0/3801616232 <== mon.2 v2:172.21.15.7:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) ==== 100+0+11 (secure 0 0 0) 0x7f035805d3a0 con 0x7f03680cac60 2024-05-09T08:03:00.098 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 64 2024-05-09T08:03:00.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:00.083+0000 7f036e600640 1 -- 172.21.15.1:0/3801616232 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f034806be60 msgr2=0x7f034806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:00.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:00.083+0000 7f036e600640 1 --2- 172.21.15.1:0/3801616232 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f034806be60 0x7f034806e320 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f0354009560 tx=0x7f0354009190 comp rx=0 tx=0).stop 2024-05-09T08:03:00.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:00.084+0000 7f036e600640 1 -- 172.21.15.1:0/3801616232 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f03680cac60 msgr2=0x7f0368155470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:00.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:00.084+0000 7f036e600640 1 --2- 172.21.15.1:0/3801616232 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f03680cac60 0x7f0368155470 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f035800e9d0 tx=0x7f035800ee90 comp rx=0 tx=0).stop 2024-05-09T08:03:00.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:00.084+0000 7f036e600640 1 -- 172.21.15.1:0/3801616232 shutdown_connections 2024-05-09T08:03:00.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:00.084+0000 7f036e600640 1 --2- 172.21.15.1:0/3801616232 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f034806be60 0x7f034806e320 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:00.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:00.084+0000 7f036e600640 1 --2- 172.21.15.1:0/3801616232 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03680cc5f0 0x7f036814f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:00.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:00.084+0000 7f036e600640 1 --2- 172.21.15.1:0/3801616232 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f03680ce4e0 0x7f036814fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:00.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:00.084+0000 7f036e600640 1 --2- 172.21.15.1:0/3801616232 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f03680cac60 0x7f0368155470 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:00.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:00.084+0000 7f036e600640 1 -- 172.21.15.1:0/3801616232 >> 172.21.15.1:0/3801616232 conn(0x7f03680cfc20 msgr2=0x7f036806d660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:03:00.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:00.084+0000 7f036e600640 1 -- 172.21.15.1:0/3801616232 shutdown_connections 2024-05-09T08:03:00.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:00.084+0000 7f036e600640 1 -- 172.21.15.1:0/3801616232 wait complete. 2024-05-09T08:03:00.120 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-09T08:03:00.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:00.314+0000 7fa345600640 1 Processor -- start 2024-05-09T08:03:00.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:00.315+0000 7fa345600640 1 -- start start 2024-05-09T08:03:00.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:00.316+0000 7fa345600640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fa340071490 0x7fa340071870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:00.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:00.316+0000 7fa345600640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa340152110 0x7fa340152580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:00.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:00.316+0000 7fa345600640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa3400c95f0 con 0x7fa3400ce470 2024-05-09T08:03:00.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:00.316+0000 7fa345600640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fa3400c9310 con 0x7fa340152110 2024-05-09T08:03:00.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:00.317+0000 7fa345600640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7fa34015bc30 con 0x7fa340071490 2024-05-09T08:03:00.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:00.317+0000 7fa33fe00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fa340071490 0x7fa340071870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:00.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:00.317+0000 7fa337e00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fa3400ce470 0x7fa3400c3430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.1:42196/0 (socket says 172.21.15.1:42196) 2024-05-09T08:03:00.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:00.317+0000 7fa337e00640 1 -- 172.21.15.1:0/4257632623 learned_addr learned my addr 172.21.15.1:0/4257632623 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T08:03:00.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:00.317+0000 7fa344c00640 1 --2- 172.21.15.1:0/4257632623 >> v2:172.21.15.7:3300/0 conn(0x7fa340152110 0x7fa340152580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:00.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:00.317+0000 7fa33f400640 1 -- 172.21.15.1:0/4257632623 <== mon.1 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3331228411 0 0) 0x7fa328001110 con 0x7fa3400ce470 2024-05-09T08:03:00.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:00.317+0000 7fa33fe00640 1 -- 172.21.15.1:0/4257632623 >> v2:172.21.15.7:3300/0 conn(0x7fa340152110 msgr2=0x7fa340152580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:03:00.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:00.317+0000 7fa33fe00640 1 --2- 172.21.15.1:0/4257632623 >> v2:172.21.15.7:3300/0 conn(0x7fa340152110 0x7fa340152580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:00.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:00.317+0000 7fa33fe00640 1 -- 172.21.15.1:0/4257632623 >> v1:172.21.15.7:6789/0 conn(0x7fa3400ce470 legacy=0x7fa3400c3430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:00.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:00.317+0000 7fa33fe00640 1 -- 172.21.15.1:0/4257632623 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa34015bef0 con 0x7fa340071490 2024-05-09T08:03:00.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:00.317+0000 7fa344c00640 1 --2- 172.21.15.1:0/4257632623 >> v2:172.21.15.7:3300/0 conn(0x7fa340152110 0x7fa340152580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:03:00.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:00.318+0000 7fa33fe00640 1 --2- 172.21.15.1:0/4257632623 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fa340071490 0x7fa340071870 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fa3300097a0 tx=0x7fa33002a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=85ac104eb01736d9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:03:00.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:00.319+0000 7fa33f400640 1 -- 172.21.15.1:0/4257632623 <== mon.2 v2:172.21.15.7:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa330037070 con 0x7fa340071490 2024-05-09T08:03:00.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:00.319+0000 7fa33f400640 1 -- 172.21.15.1:0/4257632623 <== mon.2 v2:172.21.15.7:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa330032440 con 0x7fa340071490 2024-05-09T08:03:00.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:00.319+0000 7fa33f400640 1 -- 172.21.15.1:0/4257632623 <== mon.2 v2:172.21.15.7:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa330032810 con 0x7fa340071490 2024-05-09T08:03:00.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:00.319+0000 7fa345600640 1 -- 172.21.15.1:0/4257632623 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fa340071490 msgr2=0x7fa340071870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:00.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:00.319+0000 7fa345600640 1 --2- 172.21.15.1:0/4257632623 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fa340071490 0x7fa340071870 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fa3300097a0 tx=0x7fa33002a180 comp rx=0 tx=0).stop 2024-05-09T08:03:00.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:00.319+0000 7fa345600640 1 -- 172.21.15.1:0/4257632623 shutdown_connections 2024-05-09T08:03:00.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:00.319+0000 7fa345600640 1 --2- 172.21.15.1:0/4257632623 >> v2:172.21.15.7:3300/0 conn(0x7fa340152110 0x7fa340152580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:00.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:00.319+0000 7fa345600640 1 --2- 172.21.15.1:0/4257632623 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fa340071490 0x7fa340071870 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:00.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:00.319+0000 7fa345600640 1 -- 172.21.15.1:0/4257632623 >> 172.21.15.1:0/4257632623 conn(0x7fa34006b8b0 msgr2=0x7fa34006f840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:03:00.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:00.319+0000 7fa345600640 1 -- 172.21.15.1:0/4257632623 shutdown_connections 2024-05-09T08:03:00.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:00.319+0000 7fa345600640 1 -- 172.21.15.1:0/4257632623 wait complete. 2024-05-09T08:03:00.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:00.321+0000 7fa345600640 1 Processor -- start 2024-05-09T08:03:00.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:00.321+0000 7fa345600640 1 -- start start 2024-05-09T08:03:00.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:00.322+0000 7fa345600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa340071490 0x7fa3401f3ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:00.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:00.322+0000 7fa345600640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fa3400ce470 0x7fa3400c6540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:00.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:00.322+0000 7fa345600640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa340152110 0x7fa3401f0830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:00.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:00.322+0000 7fa345600640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fa3400c2c30 con 0x7fa340152110 2024-05-09T08:03:00.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:00.322+0000 7fa345600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa34015c4b0 con 0x7fa340071490 2024-05-09T08:03:00.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:00.322+0000 7fa345600640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7fa3400c2f10 con 0x7fa3400ce470 2024-05-09T08:03:00.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:00.322+0000 7fa344c00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa340152110 0x7fa3401f0830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:00.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:00.323+0000 7fa344c00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa340152110 0x7fa3401f0830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38442/0 (socket says 172.21.15.1:38442) 2024-05-09T08:03:00.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:00.323+0000 7fa344c00640 1 -- 172.21.15.1:0/1718661843 learned_addr learned my addr 172.21.15.1:0/1718661843 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:03:00.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:00.323+0000 7fa33fe00640 1 --2- 172.21.15.1:0/1718661843 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa340071490 0x7fa3401f3ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:00.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:00.323+0000 7fa337e00640 1 --2- 172.21.15.1:0/1718661843 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fa3400ce470 0x7fa3400c6540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:00.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:00.323+0000 7fa344c00640 1 -- 172.21.15.1:0/1718661843 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fa3400ce470 msgr2=0x7fa3400c6540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:00.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:00.323+0000 7fa344c00640 1 --2- 172.21.15.1:0/1718661843 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fa3400ce470 0x7fa3400c6540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:00.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:00.323+0000 7fa344c00640 1 -- 172.21.15.1:0/1718661843 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa340071490 msgr2=0x7fa3401f3ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:00.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:00.323+0000 7fa344c00640 1 --2- 172.21.15.1:0/1718661843 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa340071490 0x7fa3401f3ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:00.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:00.323+0000 7fa344c00640 1 -- 172.21.15.1:0/1718661843 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa34006d9f0 con 0x7fa340152110 2024-05-09T08:03:00.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:00.323+0000 7fa344c00640 1 --2- 172.21.15.1:0/1718661843 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa340152110 0x7fa3401f0830 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fa338009f00 tx=0x7fa33800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:03:00.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:00.324+0000 7fa33cc00640 1 -- 172.21.15.1:0/1718661843 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa338018030 con 0x7fa340152110 2024-05-09T08:03:00.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:00.324+0000 7fa33cc00640 1 -- 172.21.15.1:0/1718661843 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa33800ce50 con 0x7fa340152110 2024-05-09T08:03:00.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:00.324+0000 7fa33cc00640 1 -- 172.21.15.1:0/1718661843 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa338003bb0 con 0x7fa340152110 2024-05-09T08:03:00.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:00.324+0000 7fa345600640 1 -- 172.21.15.1:0/1718661843 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa34006dcd0 con 0x7fa340152110 2024-05-09T08:03:00.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:00.324+0000 7fa345600640 1 -- 172.21.15.1:0/1718661843 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa34006e290 con 0x7fa340152110 2024-05-09T08:03:00.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:00.325+0000 7fa345600640 1 -- 172.21.15.1:0/1718661843 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3400caf30 con 0x7fa340152110 2024-05-09T08:03:00.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:00.326+0000 7fa33cc00640 1 -- 172.21.15.1:0/1718661843 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa33801b070 con 0x7fa340152110 2024-05-09T08:03:00.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:00.327+0000 7fa33cc00640 1 --2- 172.21.15.1:0/1718661843 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fa31c06be60 0x7fa31c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:00.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:00.327+0000 7fa33cc00640 1 -- 172.21.15.1:0/1718661843 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fa33808bb90 con 0x7fa340152110 2024-05-09T08:03:00.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:00.328+0000 7fa33fe00640 1 --2- 172.21.15.1:0/1718661843 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fa31c06be60 0x7fa31c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:00.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:00.328+0000 7fa33fe00640 1 --2- 172.21.15.1:0/1718661843 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fa31c06be60 0x7fa31c06e320 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fa330034000 tx=0x7fa33002e000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:03:00.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:00.331+0000 7fa33cc00640 1 -- 172.21.15.1:0/1718661843 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fa3400caf30 con 0x7fa340152110 2024-05-09T08:03:00.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:00.534+0000 7fa345600640 1 -- 172.21.15.1:0/1718661843 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fa34004f660 con 0x7fa340152110 2024-05-09T08:03:00.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:00.535+0000 7fa33cc00640 1 -- 172.21.15.1:0/1718661843 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) ==== 103+0+11 (secure 0 0 0) 0x7fa338010040 con 0x7fa340152110 2024-05-09T08:03:00.552 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 32 2024-05-09T08:03:00.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:00.537+0000 7fa345600640 1 -- 172.21.15.1:0/1718661843 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fa31c06be60 msgr2=0x7fa31c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:00.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:00.537+0000 7fa345600640 1 --2- 172.21.15.1:0/1718661843 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fa31c06be60 0x7fa31c06e320 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fa330034000 tx=0x7fa33002e000 comp rx=0 tx=0).stop 2024-05-09T08:03:00.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:00.537+0000 7fa345600640 1 -- 172.21.15.1:0/1718661843 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa340152110 msgr2=0x7fa3401f0830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:00.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:00.537+0000 7fa345600640 1 --2- 172.21.15.1:0/1718661843 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa340152110 0x7fa3401f0830 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fa338009f00 tx=0x7fa33800c490 comp rx=0 tx=0).stop 2024-05-09T08:03:00.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:00.538+0000 7fa345600640 1 -- 172.21.15.1:0/1718661843 shutdown_connections 2024-05-09T08:03:00.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:00.538+0000 7fa345600640 1 --2- 172.21.15.1:0/1718661843 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fa31c06be60 0x7fa31c06e320 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:00.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:00.538+0000 7fa345600640 1 --2- 172.21.15.1:0/1718661843 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa340152110 0x7fa3401f0830 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:00.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:00.538+0000 7fa345600640 1 --2- 172.21.15.1:0/1718661843 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fa3400ce470 0x7fa3400c6540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:00.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:00.538+0000 7fa345600640 1 --2- 172.21.15.1:0/1718661843 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa340071490 0x7fa3401f3ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:00.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:00.538+0000 7fa345600640 1 -- 172.21.15.1:0/1718661843 >> 172.21.15.1:0/1718661843 conn(0x7fa34006b8b0 msgr2=0x7fa3400c3330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:03:00.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:00.538+0000 7fa345600640 1 -- 172.21.15.1:0/1718661843 shutdown_connections 2024-05-09T08:03:00.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:00.538+0000 7fa345600640 1 -- 172.21.15.1:0/1718661843 wait complete. 2024-05-09T08:03:00.574 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T08:03:00.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:00.768+0000 7fa75d400640 1 Processor -- start 2024-05-09T08:03:00.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:00.769+0000 7fa75d400640 1 -- start start 2024-05-09T08:03:00.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:00.770+0000 7fa75d400640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fa758152500 0x7fa7580c8180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:00.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:00.770+0000 7fa75d400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa7580cc730 0x7fa75815c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:00.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:00.770+0000 7fa75d400640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7580c6a30 con 0x7fa7580c7660 2024-05-09T08:03:00.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:00.770+0000 7fa75d400640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7fa7580c5e50 con 0x7fa758152500 2024-05-09T08:03:00.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:00.770+0000 7fa75d400640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fa7580c22d0 con 0x7fa7580cc730 2024-05-09T08:03:00.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:00.771+0000 7fa757e00640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fa758152500 0x7fa7580c8180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:00.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:00.771+0000 7fa757e00640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fa758152500 0x7fa7580c8180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38456/0 (socket says 172.21.15.1:38456) 2024-05-09T08:03:00.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:00.771+0000 7fa757e00640 1 -- 172.21.15.1:0/1691852989 learned_addr learned my addr 172.21.15.1:0/1691852989 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:03:00.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:00.771+0000 7fa75ca00640 1 --1- 172.21.15.1:0/1691852989 >> v1:172.21.15.1:6789/0 conn(0x7fa7580c7660 0x7fa75806b250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:56108/0 (socket says 172.21.15.1:56108) 2024-05-09T08:03:00.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:00.771+0000 7fa757400640 1 --2- 172.21.15.1:0/1691852989 >> v2:172.21.15.7:3300/0 conn(0x7fa7580cc730 0x7fa75815c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:00.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:00.771+0000 7fa757e00640 1 -- 172.21.15.1:0/1691852989 >> v2:172.21.15.7:3300/0 conn(0x7fa7580cc730 msgr2=0x7fa75815c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:03:00.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:00.771+0000 7fa757e00640 1 --2- 172.21.15.1:0/1691852989 >> v2:172.21.15.7:3300/0 conn(0x7fa7580cc730 0x7fa75815c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:00.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:00.771+0000 7fa757e00640 1 -- 172.21.15.1:0/1691852989 >> v1:172.21.15.1:6789/0 conn(0x7fa7580c7660 legacy=0x7fa75806b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:00.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:00.771+0000 7fa757e00640 1 -- 172.21.15.1:0/1691852989 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7580c6cf0 con 0x7fa758152500 2024-05-09T08:03:00.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:00.771+0000 7fa756a00640 1 -- 172.21.15.1:0/1691852989 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3146926640 0 0) 0x7fa7580c6a30 con 0x7fa7580c7660 2024-05-09T08:03:00.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:00.773+0000 7fa757e00640 1 --2- 172.21.15.1:0/1691852989 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa758152500 0x7fa7580c8180 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fa758155f40 tx=0x7fa74802a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c66a7423d44c61a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:03:00.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:00.773+0000 7fa756a00640 1 -- 172.21.15.1:0/1691852989 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa748037070 con 0x7fa758152500 2024-05-09T08:03:00.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:00.773+0000 7fa756a00640 1 -- 172.21.15.1:0/1691852989 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa74802ae50 con 0x7fa758152500 2024-05-09T08:03:00.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:00.773+0000 7fa756a00640 1 -- 172.21.15.1:0/1691852989 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa748032860 con 0x7fa758152500 2024-05-09T08:03:00.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:00.773+0000 7fa75d400640 1 -- 172.21.15.1:0/1691852989 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa758152500 msgr2=0x7fa7580c8180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:00.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:00.773+0000 7fa75d400640 1 --2- 172.21.15.1:0/1691852989 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa758152500 0x7fa7580c8180 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fa758155f40 tx=0x7fa74802a300 comp rx=0 tx=0).stop 2024-05-09T08:03:00.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:00.774+0000 7fa75d400640 1 -- 172.21.15.1:0/1691852989 shutdown_connections 2024-05-09T08:03:00.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:00.774+0000 7fa75d400640 1 --2- 172.21.15.1:0/1691852989 >> v2:172.21.15.7:3300/0 conn(0x7fa7580cc730 0x7fa75815c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:00.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:00.774+0000 7fa75d400640 1 --2- 172.21.15.1:0/1691852989 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa758152500 0x7fa7580c8180 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:00.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:00.774+0000 7fa75d400640 1 -- 172.21.15.1:0/1691852989 >> 172.21.15.1:0/1691852989 conn(0x7fa7580cbc10 msgr2=0x7fa7580cc020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:03:00.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:00.774+0000 7fa75d400640 1 -- 172.21.15.1:0/1691852989 shutdown_connections 2024-05-09T08:03:00.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:00.774+0000 7fa75d400640 1 -- 172.21.15.1:0/1691852989 wait complete. 2024-05-09T08:03:00.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:00.775+0000 7fa75d400640 1 Processor -- start 2024-05-09T08:03:00.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:00.776+0000 7fa75d400640 1 -- start start 2024-05-09T08:03:00.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:00.777+0000 7fa75d400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa7580c7660 0x7fa7580c8d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:00.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:00.777+0000 7fa75d400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa7580cc730 0x7fa7580c3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:00.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:00.777+0000 7fa75d400640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fa758152500 0x7fa7580c4210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:00.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:00.777+0000 7fa75d400640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fa7580c72b0 con 0x7fa7580cc730 2024-05-09T08:03:00.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:00.777+0000 7fa75d400640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa7580ccb50 con 0x7fa7580c7660 2024-05-09T08:03:00.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:00.777+0000 7fa75d400640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7fa75815b880 con 0x7fa758152500 2024-05-09T08:03:00.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:00.777+0000 7fa757400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa7580cc730 0x7fa7580c3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:00.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:00.777+0000 7fa757400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa7580cc730 0x7fa7580c3cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38458/0 (socket says 172.21.15.1:38458) 2024-05-09T08:03:00.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:00.777+0000 7fa757400640 1 -- 172.21.15.1:0/604517725 learned_addr learned my addr 172.21.15.1:0/604517725 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:03:00.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:00.777+0000 7fa757e00640 1 --2- 172.21.15.1:0/604517725 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa7580c7660 0x7fa7580c8d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:00.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:00.777+0000 7fa75ca00640 1 --2- 172.21.15.1:0/604517725 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fa758152500 0x7fa7580c4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:00.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:00.777+0000 7fa757400640 1 -- 172.21.15.1:0/604517725 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fa758152500 msgr2=0x7fa7580c4210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:00.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:00.777+0000 7fa757400640 1 --2- 172.21.15.1:0/604517725 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fa758152500 0x7fa7580c4210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:00.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:00.777+0000 7fa757400640 1 -- 172.21.15.1:0/604517725 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa7580c7660 msgr2=0x7fa7580c8d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:00.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:00.777+0000 7fa757400640 1 --2- 172.21.15.1:0/604517725 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa7580c7660 0x7fa7580c8d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:00.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:00.777+0000 7fa757400640 1 -- 172.21.15.1:0/604517725 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7580c37f0 con 0x7fa7580cc730 2024-05-09T08:03:00.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:00.777+0000 7fa75ca00640 1 --2- 172.21.15.1:0/604517725 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fa758152500 0x7fa7580c4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T08:03:00.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:00.778+0000 7fa757400640 1 --2- 172.21.15.1:0/604517725 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa7580cc730 0x7fa7580c3cd0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fa74000efc0 tx=0x7fa74000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:03:00.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:00.778+0000 7fa74fe00640 1 -- 172.21.15.1:0/604517725 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa740018020 con 0x7fa7580cc730 2024-05-09T08:03:00.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:00.778+0000 7fa74fe00640 1 -- 172.21.15.1:0/604517725 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa740014480 con 0x7fa7580cc730 2024-05-09T08:03:00.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:00.778+0000 7fa74fe00640 1 -- 172.21.15.1:0/604517725 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa740014850 con 0x7fa7580cc730 2024-05-09T08:03:00.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:00.778+0000 7fa75d400640 1 -- 172.21.15.1:0/604517725 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7581e3420 con 0x7fa7580cc730 2024-05-09T08:03:00.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:00.778+0000 7fa75d400640 1 -- 172.21.15.1:0/604517725 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7581e39e0 con 0x7fa7580cc730 2024-05-09T08:03:00.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:00.779+0000 7fa75d400640 1 -- 172.21.15.1:0/604517725 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7580c5fe0 con 0x7fa7580cc730 2024-05-09T08:03:00.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:00.780+0000 7fa74fe00640 1 -- 172.21.15.1:0/604517725 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa740005d10 con 0x7fa7580cc730 2024-05-09T08:03:00.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:00.782+0000 7fa74fe00640 1 --2- 172.21.15.1:0/604517725 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fa73406be30 0x7fa73406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:00.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:00.782+0000 7fa74fe00640 1 -- 172.21.15.1:0/604517725 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fa74008bc10 con 0x7fa7580cc730 2024-05-09T08:03:00.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:00.782+0000 7fa757e00640 1 --2- 172.21.15.1:0/604517725 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fa73406be30 0x7fa73406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:00.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:00.783+0000 7fa757e00640 1 --2- 172.21.15.1:0/604517725 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fa73406be30 0x7fa73406e2f0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fa748034000 tx=0x7fa74802e000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:03:00.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:00.787+0000 7fa74fe00640 1 -- 172.21.15.1:0/604517725 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fa7580c5fe0 con 0x7fa7580cc730 2024-05-09T08:03:01.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:01.036+0000 7fa75d400640 1 -- 172.21.15.1:0/604517725 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa75804f660 con 0x7fa7580cc730 2024-05-09T08:03:01.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:01.038+0000 7fa74fe00640 1 -- 172.21.15.1:0/604517725 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+6807 (secure 0 0 0) 0x7fa740022070 con 0x7fa7580cc730 2024-05-09T08:03:01.055 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T08:03:01.055 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":10,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4220,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/1157980939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":1157980939},{"type":"v1","addr":"172.21.15.7:6837","nonce":1157980939}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4235,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6833/3269798795","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6832","nonce":3269798795},{"type":"v1","addr":"172.21.15.1:6833","nonce":3269798795}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4241,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/725115499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6835","nonce":725115499},{"type":"v1","addr":"172.21.15.1:6837","nonce":725115499}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4276,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6838/1827115274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1827115274},{"type":"v1","addr":"172.21.15.7:6838","nonce":1827115274}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/1032767352","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":1032767352},{"type":"v1","addr":"172.21.15.7:6839","nonce":1032767352}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":10,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-09T08:02:15.332696+0000","modified":"2024-05-09T08:02:49.734561+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4294},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4294":{"gid":4294,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":6,"addr":"172.21.15.1:6836/1509134635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":1509134635},{"type":"v1","addr":"172.21.15.1:6836","nonce":1509134635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4294,"qdb_cluster":[4294]},"id":1}]} 2024-05-09T08:03:01.055 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 10 2024-05-09T08:03:01.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:01.041+0000 7fa75d400640 1 -- 172.21.15.1:0/604517725 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fa73406be30 msgr2=0x7fa73406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:01.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:01.041+0000 7fa75d400640 1 --2- 172.21.15.1:0/604517725 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fa73406be30 0x7fa73406e2f0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fa748034000 tx=0x7fa74802e000 comp rx=0 tx=0).stop 2024-05-09T08:03:01.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:01.041+0000 7fa75d400640 1 -- 172.21.15.1:0/604517725 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa7580cc730 msgr2=0x7fa7580c3cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:01.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:01.041+0000 7fa75d400640 1 --2- 172.21.15.1:0/604517725 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa7580cc730 0x7fa7580c3cd0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fa74000efc0 tx=0x7fa74000c490 comp rx=0 tx=0).stop 2024-05-09T08:03:01.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:01.041+0000 7fa75d400640 1 -- 172.21.15.1:0/604517725 shutdown_connections 2024-05-09T08:03:01.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:01.041+0000 7fa75d400640 1 --2- 172.21.15.1:0/604517725 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fa73406be30 0x7fa73406e2f0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:01.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:01.041+0000 7fa75d400640 1 --2- 172.21.15.1:0/604517725 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fa758152500 0x7fa7580c4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:01.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:01.041+0000 7fa75d400640 1 --2- 172.21.15.1:0/604517725 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa7580cc730 0x7fa7580c3cd0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:01.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:01.041+0000 7fa75d400640 1 --2- 172.21.15.1:0/604517725 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa7580c7660 0x7fa7580c8d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:01.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:01.041+0000 7fa75d400640 1 -- 172.21.15.1:0/604517725 >> 172.21.15.1:0/604517725 conn(0x7fa7580cbc10 msgr2=0x7fa7580c9a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:03:01.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:01.041+0000 7fa75d400640 1 -- 172.21.15.1:0/604517725 shutdown_connections 2024-05-09T08:03:01.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:01.041+0000 7fa75d400640 1 -- 172.21.15.1:0/604517725 wait complete. 2024-05-09T08:03:01.078 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T08:03:01.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:01.276+0000 7f76de400640 1 Processor -- start 2024-05-09T08:03:01.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:01.277+0000 7f76de400640 1 -- start start 2024-05-09T08:03:01.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:01.278+0000 7f76de400640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f76d80c1ba0 0x7f76d80c6c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:01.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:01.278+0000 7f76de400640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f76d80c72f0 con 0x7f76d80b1d30 2024-05-09T08:03:01.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:01.278+0000 7f76de400640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f76d8157620 con 0x7f76d8151660 2024-05-09T08:03:01.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:01.278+0000 7f76de400640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f76d8151a40 con 0x7f76d80c1ba0 2024-05-09T08:03:01.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:01.278+0000 7f76d7400640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f76d8151660 0x7f76d8072d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.1:42198/0 (socket says 172.21.15.1:42198) 2024-05-09T08:03:01.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:01.279+0000 7f76d7400640 1 -- 172.21.15.1:0/1064898091 learned_addr learned my addr 172.21.15.1:0/1064898091 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T08:03:01.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:01.279+0000 7f76cfe00640 1 --2- 172.21.15.1:0/1064898091 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f76d80c1ba0 0x7f76d80c6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:01.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:01.279+0000 7f76d6a00640 1 -- 172.21.15.1:0/1064898091 <== mon.1 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3347505495 0 0) 0x7f76d8157620 con 0x7f76d8151660 2024-05-09T08:03:01.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:01.279+0000 7f76d6a00640 1 -- 172.21.15.1:0/1064898091 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f76bc005730 con 0x7f76d8151660 2024-05-09T08:03:01.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:01.279+0000 7f76d6a00640 1 -- 172.21.15.1:0/1064898091 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 142831368 0 0) 0x7f76d80c72f0 con 0x7f76d80b1d30 2024-05-09T08:03:01.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:01.279+0000 7f76d6a00640 1 -- 172.21.15.1:0/1064898091 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f76d8157620 con 0x7f76d80b1d30 2024-05-09T08:03:01.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:01.279+0000 7f76cfe00640 1 -- 172.21.15.1:0/1064898091 >> v1:172.21.15.7:6789/0 conn(0x7f76d8151660 legacy=0x7f76d8072d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:01.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:01.279+0000 7f76cfe00640 1 -- 172.21.15.1:0/1064898091 >> v1:172.21.15.1:6789/0 conn(0x7f76d80b1d30 legacy=0x7f76d80c7a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:01.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:01.279+0000 7f76cfe00640 1 -- 172.21.15.1:0/1064898091 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76d8157900 con 0x7f76d80c1ba0 2024-05-09T08:03:01.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:01.280+0000 7f76cfe00640 1 --2- 172.21.15.1:0/1064898091 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f76d80c1ba0 0x7f76d80c6c90 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f76c00097d0 tx=0x7f76c002a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9733a55b228e88f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:03:01.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:01.281+0000 7f76d6a00640 1 -- 172.21.15.1:0/1064898091 <== mon.2 v2:172.21.15.7:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76c0037070 con 0x7f76d80c1ba0 2024-05-09T08:03:01.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:01.281+0000 7f76d6a00640 1 -- 172.21.15.1:0/1064898091 <== mon.2 v2:172.21.15.7:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76c0032440 con 0x7f76d80c1ba0 2024-05-09T08:03:01.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:01.281+0000 7f76d6a00640 1 -- 172.21.15.1:0/1064898091 <== mon.2 v2:172.21.15.7:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76c0032810 con 0x7f76d80c1ba0 2024-05-09T08:03:01.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:01.281+0000 7f76de400640 1 -- 172.21.15.1:0/1064898091 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f76d80c1ba0 msgr2=0x7f76d80c6c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:01.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:01.281+0000 7f76de400640 1 --2- 172.21.15.1:0/1064898091 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f76d80c1ba0 0x7f76d80c6c90 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f76c00097d0 tx=0x7f76c002a210 comp rx=0 tx=0).stop 2024-05-09T08:03:01.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:01.281+0000 7f76de400640 1 -- 172.21.15.1:0/1064898091 shutdown_connections 2024-05-09T08:03:01.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:01.281+0000 7f76de400640 1 --2- 172.21.15.1:0/1064898091 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f76d80c1ba0 0x7f76d80c6c90 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:01.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:01.281+0000 7f76de400640 1 -- 172.21.15.1:0/1064898091 >> 172.21.15.1:0/1064898091 conn(0x7f76d80cc070 msgr2=0x7f76d80cc480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:03:01.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:01.281+0000 7f76de400640 1 -- 172.21.15.1:0/1064898091 shutdown_connections 2024-05-09T08:03:01.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:01.281+0000 7f76de400640 1 -- 172.21.15.1:0/1064898091 wait complete. 2024-05-09T08:03:01.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:01.283+0000 7f76de400640 1 Processor -- start 2024-05-09T08:03:01.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:01.283+0000 7f76de400640 1 -- start start 2024-05-09T08:03:01.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:01.284+0000 7f76de400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f76d80b1d30 0x7f76d81e3300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:01.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:01.284+0000 7f76de400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f76d80c1ba0 0x7f76d81ec4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:01.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:01.284+0000 7f76de400640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f76d8151660 0x7f76d80cb020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:01.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:01.284+0000 7f76de400640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f76d81581a0 con 0x7f76d80c1ba0 2024-05-09T08:03:01.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:01.284+0000 7f76de400640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f76d8157ec0 con 0x7f76d80b1d30 2024-05-09T08:03:01.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:01.284+0000 7f76de400640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f76d81572d0 con 0x7f76d8151660 2024-05-09T08:03:01.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:01.284+0000 7f76cfe00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f76d80c1ba0 0x7f76d81ec4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:01.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:01.285+0000 7f76cfe00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f76d80c1ba0 0x7f76d81ec4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38472/0 (socket says 172.21.15.1:38472) 2024-05-09T08:03:01.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:01.285+0000 7f76cfe00640 1 -- 172.21.15.1:0/3529862194 learned_addr learned my addr 172.21.15.1:0/3529862194 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:03:01.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:01.285+0000 7f76d7400640 1 --2- 172.21.15.1:0/3529862194 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f76d80b1d30 0x7f76d81e3300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:01.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:01.285+0000 7f76d7e00640 1 --2- 172.21.15.1:0/3529862194 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f76d8151660 0x7f76d80cb020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:01.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:01.285+0000 7f76cfe00640 1 -- 172.21.15.1:0/3529862194 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f76d8151660 msgr2=0x7f76d80cb020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:01.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:01.285+0000 7f76cfe00640 1 --2- 172.21.15.1:0/3529862194 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f76d8151660 0x7f76d80cb020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:01.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:01.285+0000 7f76cfe00640 1 -- 172.21.15.1:0/3529862194 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f76d80b1d30 msgr2=0x7f76d81e3300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:01.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:01.285+0000 7f76cfe00640 1 --2- 172.21.15.1:0/3529862194 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f76d80b1d30 0x7f76d81e3300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:01.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:01.285+0000 7f76cfe00640 1 -- 172.21.15.1:0/3529862194 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76d80cb9b0 con 0x7f76d80c1ba0 2024-05-09T08:03:01.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:01.285+0000 7f76cfe00640 1 --2- 172.21.15.1:0/3529862194 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f76d80c1ba0 0x7f76d81ec4d0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f76c0035040 tx=0x7f76c00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:03:01.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:01.286+0000 7f76cf400640 1 -- 172.21.15.1:0/3529862194 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76c0041020 con 0x7f76d80c1ba0 2024-05-09T08:03:01.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:01.286+0000 7f76cf400640 1 -- 172.21.15.1:0/3529862194 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76c003d720 con 0x7f76d80c1ba0 2024-05-09T08:03:01.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:01.286+0000 7f76cf400640 1 -- 172.21.15.1:0/3529862194 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76c0037040 con 0x7f76d80c1ba0 2024-05-09T08:03:01.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:01.286+0000 7f76de400640 1 -- 172.21.15.1:0/3529862194 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76d80cbc90 con 0x7f76d80c1ba0 2024-05-09T08:03:01.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:01.286+0000 7f76de400640 1 -- 172.21.15.1:0/3529862194 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76d80ca820 con 0x7f76d80c1ba0 2024-05-09T08:03:01.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:01.287+0000 7f76de400640 1 -- 172.21.15.1:0/3529862194 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76d814b670 con 0x7f76d80c1ba0 2024-05-09T08:03:01.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:01.288+0000 7f76cf400640 1 -- 172.21.15.1:0/3529862194 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f76c0044020 con 0x7f76d80c1ba0 2024-05-09T08:03:01.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:01.289+0000 7f76cf400640 1 --2- 172.21.15.1:0/3529862194 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f76b406be60 0x7f76b406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:01.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:01.289+0000 7f76cf400640 1 -- 172.21.15.1:0/3529862194 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f76c00ab740 con 0x7f76d80c1ba0 2024-05-09T08:03:01.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:01.289+0000 7f76d7400640 1 --2- 172.21.15.1:0/3529862194 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f76b406be60 0x7f76b406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:01.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:01.290+0000 7f76d7400640 1 --2- 172.21.15.1:0/3529862194 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f76b406be60 0x7f76b406e320 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f76c8000f60 tx=0x7f76c8007d70 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:03:01.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:01.293+0000 7f76cf400640 1 -- 172.21.15.1:0/3529862194 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f76d814b670 con 0x7f76d80c1ba0 2024-05-09T08:03:01.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:01.506+0000 7f76de400640 1 -- 172.21.15.1:0/3529862194 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f76d8074d70 con 0x7f76d80c1ba0 2024-05-09T08:03:01.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:01.508+0000 7f76cf400640 1 -- 172.21.15.1:0/3529862194 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18741 (secure 0 0 0) 0x7f76c004b030 con 0x7f76d80c1ba0 2024-05-09T08:03:01.526 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T08:03:01.526 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":26,"fsid":"54329ec8-3270-4043-9a82-c6cd7420aa7f","created":"2024-05-09T08:01:56.666970+0000","modified":"2024-05-09T08:03:01.298709+0000","last_up_change":"2024-05-09T08:02:07.146082+0000","last_in_change":"2024-05-09T08:01:59.008225+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T08:02:09.165511+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T08:02:12.925485+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T08:02:14.367455+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-09T08:02:17.967064+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c064454e-a9b5-4a24-9a77-17f0a4085952","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6800","nonce":1513349547},{"type":"v1","addr":"172.21.15.1:6801","nonce":1513349547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":1513349547},{"type":"v1","addr":"172.21.15.1:6803","nonce":1513349547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":1513349547},{"type":"v1","addr":"172.21.15.1:6807","nonce":1513349547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":1513349547},{"type":"v1","addr":"172.21.15.1:6805","nonce":1513349547}]},"public_addr":"172.21.15.1:6801/1513349547","cluster_addr":"172.21.15.1:6803/1513349547","heartbeat_back_addr":"172.21.15.1:6807/1513349547","heartbeat_front_addr":"172.21.15.1:6805/1513349547","state":["exists","up"]},{"osd":1,"uuid":"1a9ea687-2ff6-407a-949b-0e7e1083af21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":2942255259},{"type":"v1","addr":"172.21.15.1:6817","nonce":2942255259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":2942255259},{"type":"v1","addr":"172.21.15.1:6819","nonce":2942255259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":2942255259},{"type":"v1","addr":"172.21.15.1:6823","nonce":2942255259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6820","nonce":2942255259},{"type":"v1","addr":"172.21.15.1:6821","nonce":2942255259}]},"public_addr":"172.21.15.1:6817/2942255259","cluster_addr":"172.21.15.1:6819/2942255259","heartbeat_back_addr":"172.21.15.1:6823/2942255259","heartbeat_front_addr":"172.21.15.1:6821/2942255259","state":["exists","up"]},{"osd":2,"uuid":"c88af08b-7c9a-4dd8-9640-44138239addb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":2925031811},{"type":"v1","addr":"172.21.15.1:6809","nonce":2925031811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":2925031811},{"type":"v1","addr":"172.21.15.1:6811","nonce":2925031811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":2925031811},{"type":"v1","addr":"172.21.15.1:6815","nonce":2925031811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":2925031811},{"type":"v1","addr":"172.21.15.1:6813","nonce":2925031811}]},"public_addr":"172.21.15.1:6809/2925031811","cluster_addr":"172.21.15.1:6811/2925031811","heartbeat_back_addr":"172.21.15.1:6815/2925031811","heartbeat_front_addr":"172.21.15.1:6813/2925031811","state":["exists","up"]},{"osd":3,"uuid":"c087b7e3-a41d-4333-9365-32e3bd8c8e6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6824","nonce":2497776654},{"type":"v1","addr":"172.21.15.1:6825","nonce":2497776654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":2497776654},{"type":"v1","addr":"172.21.15.1:6827","nonce":2497776654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":2497776654},{"type":"v1","addr":"172.21.15.1:6831","nonce":2497776654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6828","nonce":2497776654},{"type":"v1","addr":"172.21.15.1:6829","nonce":2497776654}]},"public_addr":"172.21.15.1:6825/2497776654","cluster_addr":"172.21.15.1:6827/2497776654","heartbeat_back_addr":"172.21.15.1:6831/2497776654","heartbeat_front_addr":"172.21.15.1:6829/2497776654","state":["exists","up"]},{"osd":4,"uuid":"b4576b9c-871d-47a8-8d36-65af525d0200","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":2957026670},{"type":"v1","addr":"172.21.15.7:6801","nonce":2957026670}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2957026670},{"type":"v1","addr":"172.21.15.7:6803","nonce":2957026670}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":2957026670},{"type":"v1","addr":"172.21.15.7:6807","nonce":2957026670}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2957026670},{"type":"v1","addr":"172.21.15.7:6805","nonce":2957026670}]},"public_addr":"172.21.15.7:6801/2957026670","cluster_addr":"172.21.15.7:6803/2957026670","heartbeat_back_addr":"172.21.15.7:6807/2957026670","heartbeat_front_addr":"172.21.15.7:6805/2957026670","state":["exists","up"]},{"osd":5,"uuid":"902b3d94-72f3-4499-b21a-e32453ef5144","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1670295242},{"type":"v1","addr":"172.21.15.7:6809","nonce":1670295242}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1670295242},{"type":"v1","addr":"172.21.15.7:6811","nonce":1670295242}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1670295242},{"type":"v1","addr":"172.21.15.7:6815","nonce":1670295242}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1670295242},{"type":"v1","addr":"172.21.15.7:6813","nonce":1670295242}]},"public_addr":"172.21.15.7:6809/1670295242","cluster_addr":"172.21.15.7:6811/1670295242","heartbeat_back_addr":"172.21.15.7:6815/1670295242","heartbeat_front_addr":"172.21.15.7:6813/1670295242","state":["exists","up"]},{"osd":6,"uuid":"ead1d771-1728-4f85-adb4-26be21cedff3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":2852086880},{"type":"v1","addr":"172.21.15.7:6817","nonce":2852086880}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":2852086880},{"type":"v1","addr":"172.21.15.7:6819","nonce":2852086880}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":2852086880},{"type":"v1","addr":"172.21.15.7:6823","nonce":2852086880}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":2852086880},{"type":"v1","addr":"172.21.15.7:6821","nonce":2852086880}]},"public_addr":"172.21.15.7:6817/2852086880","cluster_addr":"172.21.15.7:6819/2852086880","heartbeat_back_addr":"172.21.15.7:6823/2852086880","heartbeat_front_addr":"172.21.15.7:6821/2852086880","state":["exists","up"]},{"osd":7,"uuid":"3ab5a201-9394-48a8-8918-e95197893669","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":1442884138},{"type":"v1","addr":"172.21.15.7:6825","nonce":1442884138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1442884138},{"type":"v1","addr":"172.21.15.7:6827","nonce":1442884138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1442884138},{"type":"v1","addr":"172.21.15.7:6831","nonce":1442884138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1442884138},{"type":"v1","addr":"172.21.15.7:6829","nonce":1442884138}]},"public_addr":"172.21.15.7:6825/1442884138","cluster_addr":"172.21.15.7:6827/1442884138","heartbeat_back_addr":"172.21.15.7:6831/1442884138","heartbeat_front_addr":"172.21.15.7:6829/1442884138","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:02:04.615675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:02:04.385974+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:02:04.619217+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:02:04.401681+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:02:04.097081+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:02:04.129981+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:02:04.225327+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:02:04.193351+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.35","mappings":[{"from":5,"to":2}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2},{"from":3,"to":1}]},{"pgid":"3.3f","mappings":[{"from":7,"to":2}]},{"pgid":"4.4","mappings":[{"from":1,"to":5}]},{"pgid":"4.e","mappings":[{"from":1,"to":2}]},{"pgid":"4.f","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T08:03:01.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:01.511+0000 7f76de400640 1 -- 172.21.15.1:0/3529862194 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f76b406be60 msgr2=0x7f76b406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:01.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:01.511+0000 7f76de400640 1 --2- 172.21.15.1:0/3529862194 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f76b406be60 0x7f76b406e320 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f76c8000f60 tx=0x7f76c8007d70 comp rx=0 tx=0).stop 2024-05-09T08:03:01.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:01.511+0000 7f76de400640 1 -- 172.21.15.1:0/3529862194 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f76d80c1ba0 msgr2=0x7f76d81ec4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:01.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:01.511+0000 7f76de400640 1 --2- 172.21.15.1:0/3529862194 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f76d80c1ba0 0x7f76d81ec4d0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f76c0035040 tx=0x7f76c00046c0 comp rx=0 tx=0).stop 2024-05-09T08:03:01.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:01.511+0000 7f76de400640 1 -- 172.21.15.1:0/3529862194 shutdown_connections 2024-05-09T08:03:01.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:01.511+0000 7f76de400640 1 --2- 172.21.15.1:0/3529862194 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f76b406be60 0x7f76b406e320 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:01.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:01.511+0000 7f76de400640 1 --2- 172.21.15.1:0/3529862194 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f76d8151660 0x7f76d80cb020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:01.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:01.511+0000 7f76de400640 1 --2- 172.21.15.1:0/3529862194 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f76d80c1ba0 0x7f76d81ec4d0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:01.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:01.511+0000 7f76de400640 1 --2- 172.21.15.1:0/3529862194 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f76d80b1d30 0x7f76d81e3300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:01.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:01.511+0000 7f76de400640 1 -- 172.21.15.1:0/3529862194 >> 172.21.15.1:0/3529862194 conn(0x7f76d80cc070 msgr2=0x7f76d8154f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:03:01.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:01.512+0000 7f76de400640 1 -- 172.21.15.1:0/3529862194 shutdown_connections 2024-05-09T08:03:01.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:01.512+0000 7f76de400640 1 -- 172.21.15.1:0/3529862194 wait complete. 2024-05-09T08:03:01.547 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T08:02:09.165511+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-09T08:02:12.925485+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-09T08:02:14.367455+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-09T08:02:17.967064+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-09T08:03:01.548 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-09T08:03:01.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:01.744+0000 7f60afe00640 1 Processor -- start 2024-05-09T08:03:01.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:01.744+0000 7f60afe00640 1 -- start start 2024-05-09T08:03:01.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:01.746+0000 7f60afe00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f60b00c54f0 0x7f60b006f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:01.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:01.746+0000 7f60afe00640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f60b00c58d0 0x7f60b01589e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:01.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:01.746+0000 7f60afe00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f60b00718e0 con 0x7f60b00c6470 2024-05-09T08:03:01.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:01.746+0000 7f60afe00640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f60b00726e0 con 0x7f60b00c58d0 2024-05-09T08:03:01.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:01.746+0000 7f60afe00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f60b0071b60 con 0x7f60b00c54f0 2024-05-09T08:03:01.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:01.746+0000 7f60ae000640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f60b00c58d0 0x7f60b01589e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:01.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:01.746+0000 7f60ae000640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f60b00c58d0 0x7f60b01589e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38474/0 (socket says 172.21.15.1:38474) 2024-05-09T08:03:01.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:01.746+0000 7f60ae000640 1 -- 172.21.15.1:0/806753117 learned_addr learned my addr 172.21.15.1:0/806753117 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:03:01.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:01.746+0000 7f60aea00640 1 --2- 172.21.15.1:0/806753117 >> v2:172.21.15.7:3300/0 conn(0x7f60b00c54f0 0x7f60b006f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:01.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:01.747+0000 7f60ae000640 1 -- 172.21.15.1:0/806753117 >> v2:172.21.15.7:3300/0 conn(0x7f60b00c54f0 msgr2=0x7f60b006f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:03:01.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:01.747+0000 7f60ae000640 1 --2- 172.21.15.1:0/806753117 >> v2:172.21.15.7:3300/0 conn(0x7f60b00c54f0 0x7f60b006f930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:01.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:01.747+0000 7f60ae000640 1 -- 172.21.15.1:0/806753117 >> v1:172.21.15.7:6789/0 conn(0x7f60b00c6470 legacy=0x7f60b0151d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:01.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:01.747+0000 7f60ae000640 1 -- 172.21.15.1:0/806753117 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60b0071e40 con 0x7f60b00c58d0 2024-05-09T08:03:01.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:01.748+0000 7f60ae000640 1 --2- 172.21.15.1:0/806753117 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f60b00c58d0 0x7f60b01589e0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f60b00c96b0 tx=0x7f609c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96d407ad193a2543 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:03:01.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:01.748+0000 7f60ad600640 1 -- 172.21.15.1:0/806753117 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f609c037070 con 0x7f60b00c58d0 2024-05-09T08:03:01.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:01.748+0000 7f60ad600640 1 -- 172.21.15.1:0/806753117 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f609c032440 con 0x7f60b00c58d0 2024-05-09T08:03:01.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:01.748+0000 7f60ad600640 1 -- 172.21.15.1:0/806753117 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f609c032810 con 0x7f60b00c58d0 2024-05-09T08:03:01.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:01.749+0000 7f60afe00640 1 -- 172.21.15.1:0/806753117 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f60b00c58d0 msgr2=0x7f60b01589e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:01.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:01.749+0000 7f60afe00640 1 --2- 172.21.15.1:0/806753117 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f60b00c58d0 0x7f60b01589e0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f60b00c96b0 tx=0x7f609c02a270 comp rx=0 tx=0).stop 2024-05-09T08:03:01.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:01.749+0000 7f60afe00640 1 -- 172.21.15.1:0/806753117 shutdown_connections 2024-05-09T08:03:01.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:01.749+0000 7f60afe00640 1 --2- 172.21.15.1:0/806753117 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f60b00c58d0 0x7f60b01589e0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:01.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:01.749+0000 7f60afe00640 1 --2- 172.21.15.1:0/806753117 >> v2:172.21.15.7:3300/0 conn(0x7f60b00c54f0 0x7f60b006f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:01.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:01.749+0000 7f60afe00640 1 -- 172.21.15.1:0/806753117 >> 172.21.15.1:0/806753117 conn(0x7f60b006b250 msgr2=0x7f60b006b630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:03:01.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:01.749+0000 7f60afe00640 1 -- 172.21.15.1:0/806753117 shutdown_connections 2024-05-09T08:03:01.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:01.749+0000 7f60afe00640 1 -- 172.21.15.1:0/806753117 wait complete. 2024-05-09T08:03:01.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:01.751+0000 7f60afe00640 1 Processor -- start 2024-05-09T08:03:01.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:01.751+0000 7f60afe00640 1 -- start start 2024-05-09T08:03:01.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:01.752+0000 7f60afe00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f60b00c54f0 0x7f60b00c0400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:01.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:01.752+0000 7f60afe00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f60b00c58d0 0x7f60b00c28a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:01.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:01.752+0000 7f60afe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f60b00c6470 0x7f60b00c8870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:01.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:01.752+0000 7f60afe00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f60b01584c0 con 0x7f60b00c54f0 2024-05-09T08:03:01.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:01.752+0000 7f60afe00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f60b0072400 con 0x7f60b00c6470 2024-05-09T08:03:01.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:01.752+0000 7f60afe00640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f60b006fe70 con 0x7f60b00c58d0 2024-05-09T08:03:01.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:01.753+0000 7f60af400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f60b00c6470 0x7f60b00c8870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:01.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:01.753+0000 7f60ae000640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f60b00c58d0 0x7f60b00c28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:01.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:01.753+0000 7f60af400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f60b00c6470 0x7f60b00c8870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.1:59254/0 (socket says 172.21.15.1:59254) 2024-05-09T08:03:01.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:01.753+0000 7f60af400640 1 -- 172.21.15.1:0/2318773720 learned_addr learned my addr 172.21.15.1:0/2318773720 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:03:01.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:01.753+0000 7f60aea00640 1 --2- 172.21.15.1:0/2318773720 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f60b00c54f0 0x7f60b00c0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:01.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:01.753+0000 7f60af400640 1 -- 172.21.15.1:0/2318773720 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f60b00c58d0 msgr2=0x7f60b00c28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:01.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:01.753+0000 7f60af400640 1 --2- 172.21.15.1:0/2318773720 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f60b00c58d0 0x7f60b00c28a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:01.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:01.753+0000 7f60af400640 1 -- 172.21.15.1:0/2318773720 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f60b00c54f0 msgr2=0x7f60b00c0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:01.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:01.753+0000 7f60af400640 1 --2- 172.21.15.1:0/2318773720 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f60b00c54f0 0x7f60b00c0400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:01.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:01.753+0000 7f60af400640 1 -- 172.21.15.1:0/2318773720 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60b00c0060 con 0x7f60b00c6470 2024-05-09T08:03:01.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:01.753+0000 7f60ae000640 1 --2- 172.21.15.1:0/2318773720 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f60b00c58d0 0x7f60b00c28a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:03:01.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:01.753+0000 7f60af400640 1 --2- 172.21.15.1:0/2318773720 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f60b00c6470 0x7f60b00c8870 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f60a800c6b0 tx=0x7f60a800cb70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:03:01.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:01.754+0000 7f60a7400640 1 -- 172.21.15.1:0/2318773720 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60a8003b70 con 0x7f60b00c6470 2024-05-09T08:03:01.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:01.754+0000 7f60a7400640 1 -- 172.21.15.1:0/2318773720 <== mon.1 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60a8003d30 con 0x7f60b00c6470 2024-05-09T08:03:01.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:01.754+0000 7f60a7400640 1 -- 172.21.15.1:0/2318773720 <== mon.1 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60a8005310 con 0x7f60b00c6470 2024-05-09T08:03:01.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:01.754+0000 7f60afe00640 1 -- 172.21.15.1:0/2318773720 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60b00bfdc0 con 0x7f60b00c6470 2024-05-09T08:03:01.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:01.754+0000 7f60afe00640 1 -- 172.21.15.1:0/2318773720 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60b00c20a0 con 0x7f60b00c6470 2024-05-09T08:03:01.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:01.755+0000 7f60afe00640 1 -- 172.21.15.1:0/2318773720 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60b0147340 con 0x7f60b00c6470 2024-05-09T08:03:01.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:01.756+0000 7f60a7400640 1 -- 172.21.15.1:0/2318773720 <== mon.1 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f60a80054b0 con 0x7f60b00c6470 2024-05-09T08:03:01.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:01.757+0000 7f60a7400640 1 --2- 172.21.15.1:0/2318773720 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f609006be60 0x7f609006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:01.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:01.757+0000 7f60a7400640 1 -- 172.21.15.1:0/2318773720 <== mon.1 v2:172.21.15.7:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6714+0+0 (secure 0 0 0) 0x7f60a808afc0 con 0x7f60b00c6470 2024-05-09T08:03:01.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:01.758+0000 7f60aea00640 1 --2- 172.21.15.1:0/2318773720 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f609006be60 0x7f609006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:01.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:01.758+0000 7f60aea00640 1 --2- 172.21.15.1:0/2318773720 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f609006be60 0x7f609006e320 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f60a0009b40 tx=0x7f60a0009190 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:03:01.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:01.762+0000 7f60a7400640 1 -- 172.21.15.1:0/2318773720 <== mon.1 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f60b00718e0 con 0x7f60b00c6470 2024-05-09T08:03:01.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:01.964+0000 7f60afe00640 1 -- 172.21.15.1:0/2318773720 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f60b004f660 con 0x7f60b00c6470 2024-05-09T08:03:01.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:01.964+0000 7f60a7400640 1 -- 172.21.15.1:0/2318773720 <== mon.1 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7f60a8058e90 con 0x7f60b00c6470 2024-05-09T08:03:01.982 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 1 2024-05-09T08:03:01.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:01.967+0000 7f60afe00640 1 -- 172.21.15.1:0/2318773720 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f609006be60 msgr2=0x7f609006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:01.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:01.967+0000 7f60afe00640 1 --2- 172.21.15.1:0/2318773720 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f609006be60 0x7f609006e320 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f60a0009b40 tx=0x7f60a0009190 comp rx=0 tx=0).stop 2024-05-09T08:03:01.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:01.967+0000 7f60afe00640 1 -- 172.21.15.1:0/2318773720 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f60b00c6470 msgr2=0x7f60b00c8870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:01.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:01.967+0000 7f60afe00640 1 --2- 172.21.15.1:0/2318773720 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f60b00c6470 0x7f60b00c8870 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f60a800c6b0 tx=0x7f60a800cb70 comp rx=0 tx=0).stop 2024-05-09T08:03:01.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:01.967+0000 7f60afe00640 1 -- 172.21.15.1:0/2318773720 shutdown_connections 2024-05-09T08:03:01.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:01.967+0000 7f60afe00640 1 --2- 172.21.15.1:0/2318773720 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f609006be60 0x7f609006e320 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:01.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:01.967+0000 7f60afe00640 1 --2- 172.21.15.1:0/2318773720 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f60b00c6470 0x7f60b00c8870 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:01.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:01.967+0000 7f60afe00640 1 --2- 172.21.15.1:0/2318773720 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f60b00c58d0 0x7f60b00c28a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:01.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:01.967+0000 7f60afe00640 1 --2- 172.21.15.1:0/2318773720 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f60b00c54f0 0x7f60b00c0400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:01.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:01.967+0000 7f60afe00640 1 -- 172.21.15.1:0/2318773720 >> 172.21.15.1:0/2318773720 conn(0x7f60b006b250 msgr2=0x7f60b014d690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:03:01.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:01.968+0000 7f60afe00640 1 -- 172.21.15.1:0/2318773720 shutdown_connections 2024-05-09T08:03:01.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:01.968+0000 7f60afe00640 1 -- 172.21.15.1:0/2318773720 wait complete. 2024-05-09T08:03:02.004 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-09T08:03:02.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:02.204+0000 7f181d000640 1 Processor -- start 2024-05-09T08:03:02.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:02.204+0000 7f181d000640 1 -- start start 2024-05-09T08:03:02.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:02.206+0000 7f181d000640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f18181547c0 0x7f18180c26f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:02.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:02.206+0000 7f181d000640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f18180c9460 0x7f181815dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:02.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:02.206+0000 7f181d000640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f18180c2da0 con 0x7f18180c1ee0 2024-05-09T08:03:02.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:02.206+0000 7f181d000640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f181815e000 con 0x7f18180c9460 2024-05-09T08:03:02.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:02.206+0000 7f181d000640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f18180c3360 con 0x7f18181547c0 2024-05-09T08:03:02.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:02.207+0000 7f1816800640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f18180c1ee0 0x7f1818070300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:56136/0 (socket says 172.21.15.1:56136) 2024-05-09T08:03:02.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:02.207+0000 7f1816800640 1 -- 172.21.15.1:0/959032396 learned_addr learned my addr 172.21.15.1:0/959032396 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T08:03:02.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:02.207+0000 7f1815e00640 1 --2- 172.21.15.1:0/959032396 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f18181547c0 0x7f18180c26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:02.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:02.207+0000 7f1815400640 1 --2- 172.21.15.1:0/959032396 >> v2:172.21.15.7:3300/0 conn(0x7f18180c9460 0x7f181815dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:02.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:02.207+0000 7f1815e00640 1 -- 172.21.15.1:0/959032396 >> v2:172.21.15.7:3300/0 conn(0x7f18180c9460 msgr2=0x7f181815dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:03:02.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:02.207+0000 7f1815e00640 1 --2- 172.21.15.1:0/959032396 >> v2:172.21.15.7:3300/0 conn(0x7f18180c9460 0x7f181815dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:02.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:02.207+0000 7f1815e00640 1 -- 172.21.15.1:0/959032396 >> v1:172.21.15.1:6789/0 conn(0x7f18180c1ee0 legacy=0x7f1818070300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:02.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:02.207+0000 7f1815e00640 1 -- 172.21.15.1:0/959032396 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18180c3080 con 0x7f18181547c0 2024-05-09T08:03:02.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:02.207+0000 7f1814a00640 1 -- 172.21.15.1:0/959032396 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3825356241 0 0) 0x7f18180c2da0 con 0x7f18180c1ee0 2024-05-09T08:03:02.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:02.209+0000 7f1815e00640 1 --2- 172.21.15.1:0/959032396 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f18181547c0 0x7f18180c26f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f1808009d90 tx=0x7f180802a6f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4f706d35fe638bb1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:03:02.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:02.209+0000 7f1814a00640 1 -- 172.21.15.1:0/959032396 <== mon.2 v2:172.21.15.7:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1808007740 con 0x7f18181547c0 2024-05-09T08:03:02.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:02.209+0000 7f1814a00640 1 -- 172.21.15.1:0/959032396 <== mon.2 v2:172.21.15.7:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18080078e0 con 0x7f18181547c0 2024-05-09T08:03:02.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:02.209+0000 7f1814a00640 1 -- 172.21.15.1:0/959032396 <== mon.2 v2:172.21.15.7:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1808007cb0 con 0x7f18181547c0 2024-05-09T08:03:02.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:02.210+0000 7f181d000640 1 -- 172.21.15.1:0/959032396 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f18181547c0 msgr2=0x7f18180c26f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:02.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:02.210+0000 7f181d000640 1 --2- 172.21.15.1:0/959032396 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f18181547c0 0x7f18180c26f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f1808009d90 tx=0x7f180802a6f0 comp rx=0 tx=0).stop 2024-05-09T08:03:02.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:02.210+0000 7f181d000640 1 -- 172.21.15.1:0/959032396 shutdown_connections 2024-05-09T08:03:02.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:02.210+0000 7f181d000640 1 --2- 172.21.15.1:0/959032396 >> v2:172.21.15.7:3300/0 conn(0x7f18180c9460 0x7f181815dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:02.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:02.210+0000 7f181d000640 1 --2- 172.21.15.1:0/959032396 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f18181547c0 0x7f18180c26f0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:02.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:02.210+0000 7f181d000640 1 -- 172.21.15.1:0/959032396 >> 172.21.15.1:0/959032396 conn(0x7f18180731b0 msgr2=0x7f18180cce30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:03:02.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:02.210+0000 7f181d000640 1 -- 172.21.15.1:0/959032396 shutdown_connections 2024-05-09T08:03:02.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:02.210+0000 7f181d000640 1 -- 172.21.15.1:0/959032396 wait complete. 2024-05-09T08:03:02.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:02.212+0000 7f181d000640 1 Processor -- start 2024-05-09T08:03:02.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:02.213+0000 7f181d000640 1 -- start start 2024-05-09T08:03:02.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:02.214+0000 7f181d000640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f18180c1ee0 0x7f1818156df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:02.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:02.214+0000 7f181d000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18180c9460 0x7f18181f2510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:02.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:02.214+0000 7f181d000640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f18181547c0 0x7f18181f2a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:02.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:02.214+0000 7f181d000640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f18180c8f40 con 0x7f18181547c0 2024-05-09T08:03:02.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:02.214+0000 7f181d000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f18180c8c60 con 0x7f18180c9460 2024-05-09T08:03:02.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:02.214+0000 7f181d000640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f18180c2370 con 0x7f18180c1ee0 2024-05-09T08:03:02.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:02.214+0000 7f1815e00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f18180c1ee0 0x7f1818156df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:02.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:02.214+0000 7f1816800640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f18181547c0 0x7f18181f2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:02.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:02.214+0000 7f1815400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18180c9460 0x7f18181f2510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:02.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:02.214+0000 7f1815e00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f18180c1ee0 0x7f1818156df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3301/0 says I am v2:172.21.15.1:44924/0 (socket says 172.21.15.1:44924) 2024-05-09T08:03:02.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:02.214+0000 7f1815e00640 1 -- 172.21.15.1:0/137104065 learned_addr learned my addr 172.21.15.1:0/137104065 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:03:02.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:02.215+0000 7f1815400640 1 -- 172.21.15.1:0/137104065 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f18180c1ee0 msgr2=0x7f1818156df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:02.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:02.215+0000 7f1815400640 1 --2- 172.21.15.1:0/137104065 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f18180c1ee0 0x7f1818156df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:02.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:02.215+0000 7f1815400640 1 -- 172.21.15.1:0/137104065 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f18181547c0 msgr2=0x7f18181f2a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:02.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:02.215+0000 7f1815400640 1 --2- 172.21.15.1:0/137104065 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f18181547c0 0x7f18181f2a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:02.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:02.215+0000 7f1815400640 1 -- 172.21.15.1:0/137104065 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1818152e70 con 0x7f18180c9460 2024-05-09T08:03:02.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:02.215+0000 7f1815e00640 1 --2- 172.21.15.1:0/137104065 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f18180c1ee0 0x7f1818156df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:03:02.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:02.215+0000 7f1815400640 1 --2- 172.21.15.1:0/137104065 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18180c9460 0x7f18181f2510 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f180400e850 tx=0x7f180400ed10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:03:02.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:02.216+0000 7f180e000640 1 -- 172.21.15.1:0/137104065 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1804017070 con 0x7f18180c9460 2024-05-09T08:03:02.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:02.216+0000 7f180e000640 1 -- 172.21.15.1:0/137104065 <== mon.1 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f180400c940 con 0x7f18180c9460 2024-05-09T08:03:02.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:02.216+0000 7f180e000640 1 -- 172.21.15.1:0/137104065 <== mon.1 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f180400cd10 con 0x7f18180c9460 2024-05-09T08:03:02.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:02.216+0000 7f181d000640 1 -- 172.21.15.1:0/137104065 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18181f1a80 con 0x7f18180c9460 2024-05-09T08:03:02.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:02.216+0000 7f181d000640 1 -- 172.21.15.1:0/137104065 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18181533c0 con 0x7f18180c9460 2024-05-09T08:03:02.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:02.217+0000 7f181d000640 1 -- 172.21.15.1:0/137104065 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1818074d70 con 0x7f18180c9460 2024-05-09T08:03:02.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:02.218+0000 7f180e000640 1 -- 172.21.15.1:0/137104065 <== mon.1 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f180401a070 con 0x7f18180c9460 2024-05-09T08:03:02.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:02.219+0000 7f180e000640 1 --2- 172.21.15.1:0/137104065 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f17f806beb0 0x7f17f806e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:02.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:02.219+0000 7f180e000640 1 -- 172.21.15.1:0/137104065 <== mon.1 v2:172.21.15.7:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6714+0+0 (secure 0 0 0) 0x7f180408b090 con 0x7f18180c9460 2024-05-09T08:03:02.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:02.219+0000 7f1815e00640 1 --2- 172.21.15.1:0/137104065 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f17f806beb0 0x7f17f806e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:02.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:02.220+0000 7f1815e00640 1 --2- 172.21.15.1:0/137104065 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f17f806beb0 0x7f17f806e370 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f1808009bd0 tx=0x7f180802f000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:03:02.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:02.224+0000 7f180e000640 1 -- 172.21.15.1:0/137104065 <== mon.1 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f1804059070 con 0x7f18180c9460 2024-05-09T08:03:02.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:02.431+0000 7f181d000640 1 -- 172.21.15.1:0/137104065 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f18180c7ed0 con 0x7f18180c9460 2024-05-09T08:03:02.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:02.432+0000 7f180e000640 1 -- 172.21.15.1:0/137104065 <== mon.1 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) ==== 104+0+11 (secure 0 0 0) 0x7f180405d370 con 0x7f18180c9460 2024-05-09T08:03:02.449 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 32 2024-05-09T08:03:02.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:02.434+0000 7f181d000640 1 -- 172.21.15.1:0/137104065 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f17f806beb0 msgr2=0x7f17f806e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:02.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:02.435+0000 7f181d000640 1 --2- 172.21.15.1:0/137104065 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f17f806beb0 0x7f17f806e370 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f1808009bd0 tx=0x7f180802f000 comp rx=0 tx=0).stop 2024-05-09T08:03:02.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:02.435+0000 7f181d000640 1 -- 172.21.15.1:0/137104065 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18180c9460 msgr2=0x7f18181f2510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:02.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:02.435+0000 7f181d000640 1 --2- 172.21.15.1:0/137104065 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18180c9460 0x7f18181f2510 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f180400e850 tx=0x7f180400ed10 comp rx=0 tx=0).stop 2024-05-09T08:03:02.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:02.435+0000 7f181d000640 1 -- 172.21.15.1:0/137104065 shutdown_connections 2024-05-09T08:03:02.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:02.435+0000 7f181d000640 1 --2- 172.21.15.1:0/137104065 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f17f806beb0 0x7f17f806e370 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:02.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:02.435+0000 7f181d000640 1 --2- 172.21.15.1:0/137104065 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18180c9460 0x7f18181f2510 secure :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f180400e850 tx=0x7f180400ed10 comp rx=0 tx=0).stop 2024-05-09T08:03:02.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:02.435+0000 7f181d000640 1 --2- 172.21.15.1:0/137104065 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f18181547c0 0x7f18181f2a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:02.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:02.435+0000 7f181d000640 1 --2- 172.21.15.1:0/137104065 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f18180c1ee0 0x7f1818156df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:02.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:02.435+0000 7f181d000640 1 -- 172.21.15.1:0/137104065 >> 172.21.15.1:0/137104065 conn(0x7f18180731b0 msgr2=0x7f181815b880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:03:02.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:02.435+0000 7f181d000640 1 -- 172.21.15.1:0/137104065 shutdown_connections 2024-05-09T08:03:02.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:02.435+0000 7f181d000640 1 -- 172.21.15.1:0/137104065 wait complete. 2024-05-09T08:03:02.471 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-09T08:03:02.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:02.673+0000 7fdbfc600640 1 Processor -- start 2024-05-09T08:03:02.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:02.673+0000 7fdbfc600640 1 -- start start 2024-05-09T08:03:02.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:02.675+0000 7fdbfc600640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fdbf406c900 0x7fdbf40c7280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:02.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:02.675+0000 7fdbfc600640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fdbf40c77c0 0x7fdbf40c81f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:02.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:02.675+0000 7fdbfc600640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fdbf40c8730 0x7fdbf41547c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:02.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:02.675+0000 7fdbfc600640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7fdbf40c7f30 con 0x7fdbf406c900 2024-05-09T08:03:02.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:02.675+0000 7fdbfc600640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fdbf415b860 con 0x7fdbf40c8730 2024-05-09T08:03:02.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:02.675+0000 7fdbfc600640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7fdbf40b5de0 con 0x7fdbf40c77c0 2024-05-09T08:03:02.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:02.675+0000 7fdbf9c00640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fdbf406c900 0x7fdbf40c7280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:02.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:02.675+0000 7fdbf9200640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fdbf40c77c0 0x7fdbf40c81f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:02.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:02.675+0000 7fdbfa600640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fdbf40c8730 0x7fdbf41547c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:02.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:02.675+0000 7fdbf9200640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fdbf40c77c0 0x7fdbf40c81f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3301/0 says I am v2:172.21.15.1:44936/0 (socket says 172.21.15.1:44936) 2024-05-09T08:03:02.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:02.675+0000 7fdbf9200640 1 -- 172.21.15.1:0/4290260317 learned_addr learned my addr 172.21.15.1:0/4290260317 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:03:02.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:02.675+0000 7fdbf9c00640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fdbf406c900 0x7fdbf40c7280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38514/0 (socket says 172.21.15.1:38514) 2024-05-09T08:03:02.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:02.676+0000 7fdbf9200640 1 -- 172.21.15.1:0/4290260317 >> v2:172.21.15.7:3300/0 conn(0x7fdbf40c8730 msgr2=0x7fdbf41547c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:03:02.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:02.676+0000 7fdbf9200640 1 --2- 172.21.15.1:0/4290260317 >> v2:172.21.15.7:3300/0 conn(0x7fdbf40c8730 0x7fdbf41547c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:02.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:02.676+0000 7fdbf9200640 1 -- 172.21.15.1:0/4290260317 >> v2:172.21.15.1:3300/0 conn(0x7fdbf406c900 msgr2=0x7fdbf40c7280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:03:02.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:02.676+0000 7fdbf9200640 1 --2- 172.21.15.1:0/4290260317 >> v2:172.21.15.1:3300/0 conn(0x7fdbf406c900 0x7fdbf40c7280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:02.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:02.676+0000 7fdbf9200640 1 -- 172.21.15.1:0/4290260317 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbf40c7c50 con 0x7fdbf40c77c0 2024-05-09T08:03:02.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:02.676+0000 7fdbf9c00640 1 --2- 172.21.15.1:0/4290260317 >> v2:172.21.15.1:3300/0 conn(0x7fdbf406c900 0x7fdbf40c7280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:03:02.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:02.677+0000 7fdbf9200640 1 --2- 172.21.15.1:0/4290260317 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fdbf40c77c0 0x7fdbf40c81f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fdbf40cd930 tx=0x7fdbe002a1d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e0829d2fbc3476a8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:03:02.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:02.677+0000 7fdbf3e00640 1 -- 172.21.15.1:0/4290260317 <== mon.2 v2:172.21.15.7:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbe0037070 con 0x7fdbf40c77c0 2024-05-09T08:03:02.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:02.677+0000 7fdbf3e00640 1 -- 172.21.15.1:0/4290260317 <== mon.2 v2:172.21.15.7:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdbe0032440 con 0x7fdbf40c77c0 2024-05-09T08:03:02.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:02.677+0000 7fdbf3e00640 1 -- 172.21.15.1:0/4290260317 <== mon.2 v2:172.21.15.7:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbe0032810 con 0x7fdbf40c77c0 2024-05-09T08:03:02.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:02.678+0000 7fdbfc600640 1 -- 172.21.15.1:0/4290260317 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fdbf40c77c0 msgr2=0x7fdbf40c81f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:02.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:02.678+0000 7fdbfc600640 1 --2- 172.21.15.1:0/4290260317 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fdbf40c77c0 0x7fdbf40c81f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fdbf40cd930 tx=0x7fdbe002a1d0 comp rx=0 tx=0).stop 2024-05-09T08:03:02.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:02.678+0000 7fdbfc600640 1 -- 172.21.15.1:0/4290260317 shutdown_connections 2024-05-09T08:03:02.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:02.678+0000 7fdbfc600640 1 --2- 172.21.15.1:0/4290260317 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fdbf40c77c0 0x7fdbf40c81f0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:02.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:02.678+0000 7fdbfc600640 1 --2- 172.21.15.1:0/4290260317 >> v2:172.21.15.7:3300/0 conn(0x7fdbf40c8730 0x7fdbf41547c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:02.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:02.678+0000 7fdbfc600640 1 --2- 172.21.15.1:0/4290260317 >> v2:172.21.15.1:3300/0 conn(0x7fdbf406c900 0x7fdbf40c7280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:02.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:02.678+0000 7fdbfc600640 1 -- 172.21.15.1:0/4290260317 >> 172.21.15.1:0/4290260317 conn(0x7fdbf40cfe80 msgr2=0x7fdbf40d0290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:03:02.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:02.678+0000 7fdbfc600640 1 -- 172.21.15.1:0/4290260317 shutdown_connections 2024-05-09T08:03:02.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:02.678+0000 7fdbfc600640 1 -- 172.21.15.1:0/4290260317 wait complete. 2024-05-09T08:03:02.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:02.680+0000 7fdbfc600640 1 Processor -- start 2024-05-09T08:03:02.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:02.680+0000 7fdbfc600640 1 -- start start 2024-05-09T08:03:02.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:02.681+0000 7fdbfc600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdbf406c900 0x7fdbf40c6950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:02.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:02.681+0000 7fdbfc600640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fdbf40c8730 0x7fdbf40c6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:02.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:02.681+0000 7fdbfc600640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdbf40cf5e0 0x7fdbf41f3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:02.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:02.681+0000 7fdbfc600640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fdbf415b060 con 0x7fdbf40cf5e0 2024-05-09T08:03:02.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:02.681+0000 7fdbfc600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fdbf415ad80 con 0x7fdbf406c900 2024-05-09T08:03:02.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:02.681+0000 7fdbfc600640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7fdbf406cce0 con 0x7fdbf40c8730 2024-05-09T08:03:02.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:02.681+0000 7fdbfa600640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdbf40cf5e0 0x7fdbf41f3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:02.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:02.681+0000 7fdbf9c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdbf406c900 0x7fdbf40c6950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:02.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:02.681+0000 7fdbfa600640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdbf40cf5e0 0x7fdbf41f3ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38516/0 (socket says 172.21.15.1:38516) 2024-05-09T08:03:02.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:02.681+0000 7fdbfa600640 1 -- 172.21.15.1:0/129034838 learned_addr learned my addr 172.21.15.1:0/129034838 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:03:02.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:02.681+0000 7fdbf9200640 1 --2- 172.21.15.1:0/129034838 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fdbf40c8730 0x7fdbf40c6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:02.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:02.682+0000 7fdbf9c00640 1 -- 172.21.15.1:0/129034838 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fdbf40c8730 msgr2=0x7fdbf40c6300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:02.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:02.682+0000 7fdbf9c00640 1 --2- 172.21.15.1:0/129034838 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fdbf40c8730 0x7fdbf40c6300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:02.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:02.682+0000 7fdbf9c00640 1 -- 172.21.15.1:0/129034838 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdbf40cf5e0 msgr2=0x7fdbf41f3ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:02.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:02.682+0000 7fdbf9c00640 1 --2- 172.21.15.1:0/129034838 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdbf40cf5e0 0x7fdbf41f3ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:02.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:02.682+0000 7fdbf9c00640 1 -- 172.21.15.1:0/129034838 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbf40c5390 con 0x7fdbf406c900 2024-05-09T08:03:02.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:02.682+0000 7fdbf9200640 1 --2- 172.21.15.1:0/129034838 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fdbf40c8730 0x7fdbf40c6300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:03:02.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:02.682+0000 7fdbf9c00640 1 --2- 172.21.15.1:0/129034838 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdbf406c900 0x7fdbf40c6950 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fdbec00ef00 tx=0x7fdbec00b480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:03:02.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:02.682+0000 7fdbf1600640 1 -- 172.21.15.1:0/129034838 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbec018020 con 0x7fdbf406c900 2024-05-09T08:03:02.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:02.682+0000 7fdbf1600640 1 -- 172.21.15.1:0/129034838 <== mon.1 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdbec014400 con 0x7fdbf406c900 2024-05-09T08:03:02.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:02.682+0000 7fdbfc600640 1 -- 172.21.15.1:0/129034838 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbf40c5670 con 0x7fdbf406c900 2024-05-09T08:03:02.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:02.682+0000 7fdbf1600640 1 -- 172.21.15.1:0/129034838 <== mon.1 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbec0147d0 con 0x7fdbf406c900 2024-05-09T08:03:02.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:02.683+0000 7fdbfc600640 1 -- 172.21.15.1:0/129034838 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbf4150780 con 0x7fdbf406c900 2024-05-09T08:03:02.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:02.684+0000 7fdbfc600640 1 -- 172.21.15.1:0/129034838 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdbf4068cc0 con 0x7fdbf406c900 2024-05-09T08:03:02.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:02.685+0000 7fdbf1600640 1 -- 172.21.15.1:0/129034838 <== mon.1 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fdbec005d10 con 0x7fdbf406c900 2024-05-09T08:03:02.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:02.686+0000 7fdbf1600640 1 --2- 172.21.15.1:0/129034838 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fdbd406be30 0x7fdbd406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:02.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:02.686+0000 7fdbf1600640 1 -- 172.21.15.1:0/129034838 <== mon.1 v2:172.21.15.7:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7fdbec08ba00 con 0x7fdbf406c900 2024-05-09T08:03:02.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:02.686+0000 7fdbf9200640 1 --2- 172.21.15.1:0/129034838 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fdbd406be30 0x7fdbd406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:02.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:02.687+0000 7fdbf9200640 1 --2- 172.21.15.1:0/129034838 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fdbd406be30 0x7fdbd406e2f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fdbe0034000 tx=0x7fdbe002e000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:03:02.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:02.691+0000 7fdbf1600640 1 -- 172.21.15.1:0/129034838 <== mon.1 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fdbf4068cc0 con 0x7fdbf406c900 2024-05-09T08:03:02.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:02.896+0000 7fdbfc600640 1 -- 172.21.15.1:0/129034838 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fdbf40d05e0 con 0x7fdbf406c900 2024-05-09T08:03:02.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:02.897+0000 7fdbf1600640 1 -- 172.21.15.1:0/129034838 <== mon.1 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) ==== 100+0+11 (secure 0 0 0) 0x7fdbec022030 con 0x7fdbf406c900 2024-05-09T08:03:02.914 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 64 2024-05-09T08:03:02.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:02.900+0000 7fdbfc600640 1 -- 172.21.15.1:0/129034838 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fdbd406be30 msgr2=0x7fdbd406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:02.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:02.900+0000 7fdbfc600640 1 --2- 172.21.15.1:0/129034838 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fdbd406be30 0x7fdbd406e2f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fdbe0034000 tx=0x7fdbe002e000 comp rx=0 tx=0).stop 2024-05-09T08:03:02.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:02.900+0000 7fdbfc600640 1 -- 172.21.15.1:0/129034838 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdbf406c900 msgr2=0x7fdbf40c6950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:02.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:02.900+0000 7fdbfc600640 1 --2- 172.21.15.1:0/129034838 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdbf406c900 0x7fdbf40c6950 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fdbec00ef00 tx=0x7fdbec00b480 comp rx=0 tx=0).stop 2024-05-09T08:03:02.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:02.900+0000 7fdbfc600640 1 -- 172.21.15.1:0/129034838 shutdown_connections 2024-05-09T08:03:02.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:02.900+0000 7fdbfc600640 1 --2- 172.21.15.1:0/129034838 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fdbd406be30 0x7fdbd406e2f0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:02.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:02.900+0000 7fdbfc600640 1 --2- 172.21.15.1:0/129034838 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdbf40cf5e0 0x7fdbf41f3ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:02.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:02.900+0000 7fdbfc600640 1 --2- 172.21.15.1:0/129034838 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fdbf40c8730 0x7fdbf40c6300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:02.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:02.900+0000 7fdbfc600640 1 --2- 172.21.15.1:0/129034838 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdbf406c900 0x7fdbf40c6950 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:02.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:02.900+0000 7fdbfc600640 1 -- 172.21.15.1:0/129034838 >> 172.21.15.1:0/129034838 conn(0x7fdbf40cfe80 msgr2=0x7fdbf4154590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:03:02.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:02.900+0000 7fdbfc600640 1 -- 172.21.15.1:0/129034838 shutdown_connections 2024-05-09T08:03:02.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:02.900+0000 7fdbfc600640 1 -- 172.21.15.1:0/129034838 wait complete. 2024-05-09T08:03:02.936 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-09T08:03:03.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:03.132+0000 7f2bf3200640 1 Processor -- start 2024-05-09T08:03:03.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:03.132+0000 7f2bf3200640 1 -- start start 2024-05-09T08:03:03.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:03.134+0000 7f2bf3200640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f2bf40c5890 0x7f2bf415b050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:03.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:03.134+0000 7f2bf3200640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f2bf40718e0 0x7f2bf40c69e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:03.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:03.134+0000 7f2bf3200640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2bf406f060 con 0x7f2bf40c6f20 2024-05-09T08:03:03.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:03.134+0000 7f2bf3200640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f2bf415b590 con 0x7f2bf40c5890 2024-05-09T08:03:03.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:03.134+0000 7f2bf3200640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f2bf4072150 con 0x7f2bf40718e0 2024-05-09T08:03:03.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:03.134+0000 7f2bf1e00640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f2bf40c5890 0x7f2bf415b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:03.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:03.134+0000 7f2bf1e00640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f2bf40c5890 0x7f2bf415b050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38526/0 (socket says 172.21.15.1:38526) 2024-05-09T08:03:03.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:03.134+0000 7f2bf1e00640 1 -- 172.21.15.1:0/1388856180 learned_addr learned my addr 172.21.15.1:0/1388856180 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:03:03.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:03.134+0000 7f2be9400640 1 --2- 172.21.15.1:0/1388856180 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f2bf40718e0 0x7f2bf40c69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:03.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:03.135+0000 7f2bf1400640 1 -- 172.21.15.1:0/1388856180 <== mon.1 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2091870001 0 0) 0x7f2bf406f060 con 0x7f2bf40c6f20 2024-05-09T08:03:03.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:03.135+0000 7f2bf1400640 1 -- 172.21.15.1:0/1388856180 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2bd8003520 con 0x7f2bf40c6f20 2024-05-09T08:03:03.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:03.135+0000 7f2bf1e00640 1 -- 172.21.15.1:0/1388856180 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f2bf40718e0 msgr2=0x7f2bf40c69e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:03:03.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:03.135+0000 7f2bf1e00640 1 --2- 172.21.15.1:0/1388856180 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f2bf40718e0 0x7f2bf40c69e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:03.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:03.135+0000 7f2bf1e00640 1 -- 172.21.15.1:0/1388856180 >> v1:172.21.15.7:6789/0 conn(0x7f2bf40c6f20 legacy=0x7f2bf4072940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:03.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:03.135+0000 7f2bf1e00640 1 -- 172.21.15.1:0/1388856180 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bf406fe80 con 0x7f2bf40c5890 2024-05-09T08:03:03.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:03.136+0000 7f2bf1e00640 1 --2- 172.21.15.1:0/1388856180 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2bf40c5890 0x7f2bf415b050 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f2be4009e50 tx=0x7f2be402a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79a8c82809381899 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:03:03.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:03.136+0000 7f2bf1400640 1 -- 172.21.15.1:0/1388856180 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2be4036070 con 0x7f2bf40c5890 2024-05-09T08:03:03.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:03.136+0000 7f2bf1400640 1 -- 172.21.15.1:0/1388856180 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2be4031600 con 0x7f2bf40c5890 2024-05-09T08:03:03.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:03.136+0000 7f2bf1400640 1 -- 172.21.15.1:0/1388856180 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2be40319b0 con 0x7f2bf40c5890 2024-05-09T08:03:03.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:03.137+0000 7f2bf3200640 1 -- 172.21.15.1:0/1388856180 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2bf40c5890 msgr2=0x7f2bf415b050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:03.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:03.137+0000 7f2bf3200640 1 --2- 172.21.15.1:0/1388856180 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2bf40c5890 0x7f2bf415b050 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f2be4009e50 tx=0x7f2be402a7a0 comp rx=0 tx=0).stop 2024-05-09T08:03:03.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:03.137+0000 7f2bf3200640 1 -- 172.21.15.1:0/1388856180 shutdown_connections 2024-05-09T08:03:03.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:03.137+0000 7f2bf3200640 1 --2- 172.21.15.1:0/1388856180 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f2bf40718e0 0x7f2bf40c69e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:03.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:03.137+0000 7f2bf3200640 1 --2- 172.21.15.1:0/1388856180 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2bf40c5890 0x7f2bf415b050 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:03.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:03.137+0000 7f2bf3200640 1 -- 172.21.15.1:0/1388856180 >> 172.21.15.1:0/1388856180 conn(0x7f2bf406b250 msgr2=0x7f2bf406b630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:03:03.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:03.137+0000 7f2bf3200640 1 -- 172.21.15.1:0/1388856180 shutdown_connections 2024-05-09T08:03:03.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:03.137+0000 7f2bf3200640 1 -- 172.21.15.1:0/1388856180 wait complete. 2024-05-09T08:03:03.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:03.139+0000 7f2bf3200640 1 Processor -- start 2024-05-09T08:03:03.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:03.139+0000 7f2bf3200640 1 -- start start 2024-05-09T08:03:03.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:03.140+0000 7f2bf3200640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f2bf40718e0 0x7f2bf4155170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:03.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:03.140+0000 7f2bf3200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2bf40c5890 0x7f2bf414fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:03.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:03.140+0000 7f2bf3200640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2bf40c6f20 0x7f2bf41f0500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:03.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:03.140+0000 7f2bf3200640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f2bf406f8c0 con 0x7f2bf40c6f20 2024-05-09T08:03:03.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:03.140+0000 7f2bf3200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2bf406f5e0 con 0x7f2bf40c5890 2024-05-09T08:03:03.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:03.140+0000 7f2bf3200640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f2bf406fba0 con 0x7f2bf40718e0 2024-05-09T08:03:03.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:03.140+0000 7f2bf2800640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2bf40c6f20 0x7f2bf41f0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:03.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:03.140+0000 7f2be9400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2bf40c5890 0x7f2bf414fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:03.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:03.140+0000 7f2bf2800640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2bf40c6f20 0x7f2bf41f0500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38542/0 (socket says 172.21.15.1:38542) 2024-05-09T08:03:03.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:03.140+0000 7f2bf2800640 1 -- 172.21.15.1:0/3850413606 learned_addr learned my addr 172.21.15.1:0/3850413606 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:03:03.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:03.140+0000 7f2bf1e00640 1 --2- 172.21.15.1:0/3850413606 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f2bf40718e0 0x7f2bf4155170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:03.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:03.141+0000 7f2bf2800640 1 -- 172.21.15.1:0/3850413606 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f2bf40718e0 msgr2=0x7f2bf4155170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:03.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:03.141+0000 7f2bf2800640 1 --2- 172.21.15.1:0/3850413606 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f2bf40718e0 0x7f2bf4155170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:03.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:03.141+0000 7f2bf2800640 1 -- 172.21.15.1:0/3850413606 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2bf40c5890 msgr2=0x7f2bf414fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:03.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:03.141+0000 7f2bf2800640 1 --2- 172.21.15.1:0/3850413606 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2bf40c5890 0x7f2bf414fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:03.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:03.141+0000 7f2bf2800640 1 -- 172.21.15.1:0/3850413606 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bf4156320 con 0x7f2bf40c6f20 2024-05-09T08:03:03.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:03.141+0000 7f2bf2800640 1 --2- 172.21.15.1:0/3850413606 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2bf40c6f20 0x7f2bf41f0500 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f2bec00c400 tx=0x7f2bec00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:03:03.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:03.141+0000 7f2beaa00640 1 -- 172.21.15.1:0/3850413606 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bec017070 con 0x7f2bf40c6f20 2024-05-09T08:03:03.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:03.141+0000 7f2beaa00640 1 -- 172.21.15.1:0/3850413606 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2bec0052d0 con 0x7f2bf40c6f20 2024-05-09T08:03:03.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:03.141+0000 7f2beaa00640 1 -- 172.21.15.1:0/3850413606 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bec0037f0 con 0x7f2bf40c6f20 2024-05-09T08:03:03.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:03.142+0000 7f2bf3200640 1 -- 172.21.15.1:0/3850413606 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bf4156600 con 0x7f2bf40c6f20 2024-05-09T08:03:03.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:03.142+0000 7f2bf3200640 1 -- 172.21.15.1:0/3850413606 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bf4156bc0 con 0x7f2bf40c6f20 2024-05-09T08:03:03.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:03.143+0000 7f2bf3200640 1 -- 172.21.15.1:0/3850413606 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2bf40b1ce0 con 0x7f2bf40c6f20 2024-05-09T08:03:03.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:03.144+0000 7f2beaa00640 1 -- 172.21.15.1:0/3850413606 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2bec01a070 con 0x7f2bf40c6f20 2024-05-09T08:03:03.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:03.145+0000 7f2beaa00640 1 --2- 172.21.15.1:0/3850413606 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f2bd006be30 0x7f2bd006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:03.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:03.145+0000 7f2beaa00640 1 -- 172.21.15.1:0/3850413606 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7f2bec08a530 con 0x7f2bf40c6f20 2024-05-09T08:03:03.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:03.145+0000 7f2bf1e00640 1 --2- 172.21.15.1:0/3850413606 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f2bd006be30 0x7f2bd006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:03.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:03.146+0000 7f2bf1e00640 1 --2- 172.21.15.1:0/3850413606 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f2bd006be30 0x7f2bd006e2f0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f2be4033000 tx=0x7f2be4009000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:03:03.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:03.150+0000 7f2beaa00640 1 -- 172.21.15.1:0/3850413606 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f2bec058490 con 0x7f2bf40c6f20 2024-05-09T08:03:03.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:03.355+0000 7f2bf3200640 1 -- 172.21.15.1:0/3850413606 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f2bf404f660 con 0x7f2bf40c6f20 2024-05-09T08:03:03.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:03.356+0000 7f2beaa00640 1 -- 172.21.15.1:0/3850413606 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) ==== 103+0+11 (secure 0 0 0) 0x7f2bec05c790 con 0x7f2bf40c6f20 2024-05-09T08:03:03.373 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 32 2024-05-09T08:03:03.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:03.359+0000 7f2bf3200640 1 -- 172.21.15.1:0/3850413606 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f2bd006be30 msgr2=0x7f2bd006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:03.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:03.359+0000 7f2bf3200640 1 --2- 172.21.15.1:0/3850413606 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f2bd006be30 0x7f2bd006e2f0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f2be4033000 tx=0x7f2be4009000 comp rx=0 tx=0).stop 2024-05-09T08:03:03.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:03.359+0000 7f2bf3200640 1 -- 172.21.15.1:0/3850413606 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2bf40c6f20 msgr2=0x7f2bf41f0500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:03.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:03.359+0000 7f2bf3200640 1 --2- 172.21.15.1:0/3850413606 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2bf40c6f20 0x7f2bf41f0500 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f2bec00c400 tx=0x7f2bec00c8c0 comp rx=0 tx=0).stop 2024-05-09T08:03:03.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:03.359+0000 7f2bf3200640 1 -- 172.21.15.1:0/3850413606 shutdown_connections 2024-05-09T08:03:03.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:03.359+0000 7f2bf3200640 1 --2- 172.21.15.1:0/3850413606 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f2bd006be30 0x7f2bd006e2f0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:03.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:03.359+0000 7f2bf3200640 1 --2- 172.21.15.1:0/3850413606 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2bf40c5890 0x7f2bf414fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:03.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:03.359+0000 7f2bf3200640 1 --2- 172.21.15.1:0/3850413606 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2bf40c6f20 0x7f2bf41f0500 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:03.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:03.359+0000 7f2bf3200640 1 --2- 172.21.15.1:0/3850413606 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f2bf40718e0 0x7f2bf4155170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:03.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:03.359+0000 7f2bf3200640 1 -- 172.21.15.1:0/3850413606 >> 172.21.15.1:0/3850413606 conn(0x7f2bf406b250 msgr2=0x7f2bf4073070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:03:03.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:03.359+0000 7f2bf3200640 1 -- 172.21.15.1:0/3850413606 shutdown_connections 2024-05-09T08:03:03.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:03.359+0000 7f2bf3200640 1 -- 172.21.15.1:0/3850413606 wait complete. 2024-05-09T08:03:03.396 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T08:03:03.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:03.585+0000 7fe46c600640 1 Processor -- start 2024-05-09T08:03:03.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:03.585+0000 7fe46c600640 1 -- start start 2024-05-09T08:03:03.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:03.587+0000 7fe46c600640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fe464152b20 0x7fe464072900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:03.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:03.587+0000 7fe46c600640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe464073200 con 0x7fe4640c6ea0 2024-05-09T08:03:03.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:03.587+0000 7fe46c600640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe4640734e0 con 0x7fe46406dca0 2024-05-09T08:03:03.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:03.587+0000 7fe46c600640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7fe464072e40 con 0x7fe464152b20 2024-05-09T08:03:03.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:03.587+0000 7fe46a600640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fe464152b20 0x7fe464072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:03.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:03.588+0000 7fe469200640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7fe4640c6ea0 0x7fe464152410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:56148/0 (socket says 172.21.15.1:56148) 2024-05-09T08:03:03.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:03.588+0000 7fe469200640 1 -- 172.21.15.1:0/1024341774 learned_addr learned my addr 172.21.15.1:0/1024341774 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T08:03:03.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:03.588+0000 7fe46a600640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fe464152b20 0x7fe464072900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38550/0 (socket says 172.21.15.1:38550) 2024-05-09T08:03:03.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:03.588+0000 7fe469c00640 1 --1- 172.21.15.1:0/1024341774 >> v1:172.21.15.7:6789/0 conn(0x7fe46406dca0 0x7fe46406e080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.1:42238/0 (socket says 172.21.15.1:42238) 2024-05-09T08:03:03.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:03.588+0000 7fe46a600640 1 -- 172.21.15.1:0/1024341774 >> v1:172.21.15.7:6789/0 conn(0x7fe46406dca0 legacy=0x7fe46406e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:03.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:03.588+0000 7fe46a600640 1 -- 172.21.15.1:0/1024341774 >> v1:172.21.15.1:6789/0 conn(0x7fe4640c6ea0 legacy=0x7fe464152410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:03.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:03.588+0000 7fe463e00640 1 -- 172.21.15.1:0/1024341774 <== mon.1 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2599738404 0 0) 0x7fe4640734e0 con 0x7fe46406dca0 2024-05-09T08:03:03.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:03.588+0000 7fe46a600640 1 -- 172.21.15.1:0/1024341774 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe464071e20 con 0x7fe464152b20 2024-05-09T08:03:03.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:03.589+0000 7fe46a600640 1 --2- 172.21.15.1:0/1024341774 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe464152b20 0x7fe464072900 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fe45c0097a0 tx=0x7fe45c02a2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bce2c3baa3b813ff server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:03:03.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:03.589+0000 7fe463e00640 1 -- 172.21.15.1:0/1024341774 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe45c037070 con 0x7fe464152b20 2024-05-09T08:03:03.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:03.590+0000 7fe463e00640 1 -- 172.21.15.1:0/1024341774 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe45c032640 con 0x7fe464152b20 2024-05-09T08:03:03.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:03.590+0000 7fe463e00640 1 -- 172.21.15.1:0/1024341774 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe45c032a10 con 0x7fe464152b20 2024-05-09T08:03:03.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:03.590+0000 7fe46c600640 1 -- 172.21.15.1:0/1024341774 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe464152b20 msgr2=0x7fe464072900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:03.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:03.590+0000 7fe46c600640 1 --2- 172.21.15.1:0/1024341774 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe464152b20 0x7fe464072900 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fe45c0097a0 tx=0x7fe45c02a2c0 comp rx=0 tx=0).stop 2024-05-09T08:03:03.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:03.591+0000 7fe46c600640 1 -- 172.21.15.1:0/1024341774 shutdown_connections 2024-05-09T08:03:03.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:03.591+0000 7fe46c600640 1 --2- 172.21.15.1:0/1024341774 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe464152b20 0x7fe464072900 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:03.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:03.591+0000 7fe46c600640 1 -- 172.21.15.1:0/1024341774 >> 172.21.15.1:0/1024341774 conn(0x7fe4640cc070 msgr2=0x7fe4640cc480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:03:03.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:03.591+0000 7fe46c600640 1 -- 172.21.15.1:0/1024341774 shutdown_connections 2024-05-09T08:03:03.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:03.591+0000 7fe46c600640 1 -- 172.21.15.1:0/1024341774 wait complete. 2024-05-09T08:03:03.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:03.592+0000 7fe46c600640 1 Processor -- start 2024-05-09T08:03:03.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:03.592+0000 7fe46c600640 1 -- start start 2024-05-09T08:03:03.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:03.593+0000 7fe46c600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe46406dca0 0x7fe464141df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:03.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:03.593+0000 7fe46c600640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fe4640c6ea0 0x7fe464142330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:03.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:03.593+0000 7fe46c600640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe464152b20 0x7fe4640c20a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:03.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:03.594+0000 7fe46c600640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fe464072470 con 0x7fe464152b20 2024-05-09T08:03:03.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:03.594+0000 7fe46c600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe46406ece0 con 0x7fe46406dca0 2024-05-09T08:03:03.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:03.594+0000 7fe46c600640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7fe46406ee60 con 0x7fe4640c6ea0 2024-05-09T08:03:03.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:03.594+0000 7fe46a600640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe464152b20 0x7fe4640c20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:03.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:03.594+0000 7fe46a600640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe464152b20 0x7fe4640c20a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38562/0 (socket says 172.21.15.1:38562) 2024-05-09T08:03:03.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:03.594+0000 7fe46a600640 1 -- 172.21.15.1:0/3061230619 learned_addr learned my addr 172.21.15.1:0/3061230619 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:03:03.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:03.594+0000 7fe469c00640 1 --2- 172.21.15.1:0/3061230619 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe46406dca0 0x7fe464141df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:03.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:03.594+0000 7fe469200640 1 --2- 172.21.15.1:0/3061230619 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fe4640c6ea0 0x7fe464142330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:03.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:03.594+0000 7fe46a600640 1 -- 172.21.15.1:0/3061230619 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fe4640c6ea0 msgr2=0x7fe464142330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:03.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:03.594+0000 7fe46a600640 1 --2- 172.21.15.1:0/3061230619 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fe4640c6ea0 0x7fe464142330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:03.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:03.594+0000 7fe46a600640 1 -- 172.21.15.1:0/3061230619 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe46406dca0 msgr2=0x7fe464141df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:03.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:03.594+0000 7fe46a600640 1 --2- 172.21.15.1:0/3061230619 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe46406dca0 0x7fe464141df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:03.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:03.594+0000 7fe46a600640 1 -- 172.21.15.1:0/3061230619 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4641f4b00 con 0x7fe464152b20 2024-05-09T08:03:03.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:03.595+0000 7fe46a600640 1 --2- 172.21.15.1:0/3061230619 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe464152b20 0x7fe4640c20a0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fe45c0097a0 tx=0x7fe45c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:03:03.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:03.595+0000 7fe461600640 1 -- 172.21.15.1:0/3061230619 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe45c041020 con 0x7fe464152b20 2024-05-09T08:03:03.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:03.595+0000 7fe461600640 1 -- 172.21.15.1:0/3061230619 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe45c0308f0 con 0x7fe464152b20 2024-05-09T08:03:03.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:03.595+0000 7fe461600640 1 -- 172.21.15.1:0/3061230619 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe45c037040 con 0x7fe464152b20 2024-05-09T08:03:03.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:03.595+0000 7fe46c600640 1 -- 172.21.15.1:0/3061230619 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4641f4da0 con 0x7fe464152b20 2024-05-09T08:03:03.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:03.595+0000 7fe46c600640 1 -- 172.21.15.1:0/3061230619 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4641f5360 con 0x7fe464152b20 2024-05-09T08:03:03.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:03.596+0000 7fe46c600640 1 -- 172.21.15.1:0/3061230619 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe464070610 con 0x7fe464152b20 2024-05-09T08:03:03.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:03.597+0000 7fe461600640 1 -- 172.21.15.1:0/3061230619 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe45c044020 con 0x7fe464152b20 2024-05-09T08:03:03.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:03.599+0000 7fe461600640 1 --2- 172.21.15.1:0/3061230619 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fe44406be60 0x7fe44406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:03.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:03.599+0000 7fe461600640 1 -- 172.21.15.1:0/3061230619 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7fe45c0abad0 con 0x7fe464152b20 2024-05-09T08:03:03.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:03.599+0000 7fe469c00640 1 --2- 172.21.15.1:0/3061230619 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fe44406be60 0x7fe44406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:03.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:03.600+0000 7fe469c00640 1 --2- 172.21.15.1:0/3061230619 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fe44406be60 0x7fe44406e320 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fe458001920 tx=0x7fe4580018b0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:03:03.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:03.604+0000 7fe461600640 1 -- 172.21.15.1:0/3061230619 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fe464070610 con 0x7fe464152b20 2024-05-09T08:03:03.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:03.852+0000 7fe46c600640 1 -- 172.21.15.1:0/3061230619 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe464074d70 con 0x7fe464152b20 2024-05-09T08:03:03.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:03.853+0000 7fe461600640 1 -- 172.21.15.1:0/3061230619 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+6807 (secure 0 0 0) 0x7fe45c04b030 con 0x7fe464152b20 2024-05-09T08:03:03.871 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T08:03:03.871 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":10,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4220,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/1157980939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":1157980939},{"type":"v1","addr":"172.21.15.7:6837","nonce":1157980939}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4235,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6833/3269798795","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6832","nonce":3269798795},{"type":"v1","addr":"172.21.15.1:6833","nonce":3269798795}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4241,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/725115499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6835","nonce":725115499},{"type":"v1","addr":"172.21.15.1:6837","nonce":725115499}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4276,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6838/1827115274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1827115274},{"type":"v1","addr":"172.21.15.7:6838","nonce":1827115274}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/1032767352","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":1032767352},{"type":"v1","addr":"172.21.15.7:6839","nonce":1032767352}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":10,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-09T08:02:15.332696+0000","modified":"2024-05-09T08:02:49.734561+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4294},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4294":{"gid":4294,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":6,"addr":"172.21.15.1:6836/1509134635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":1509134635},{"type":"v1","addr":"172.21.15.1:6836","nonce":1509134635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4294,"qdb_cluster":[4294]},"id":1}]} 2024-05-09T08:03:03.871 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 10 2024-05-09T08:03:03.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:03.856+0000 7fe46c600640 1 -- 172.21.15.1:0/3061230619 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fe44406be60 msgr2=0x7fe44406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:03.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:03.856+0000 7fe46c600640 1 --2- 172.21.15.1:0/3061230619 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fe44406be60 0x7fe44406e320 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fe458001920 tx=0x7fe4580018b0 comp rx=0 tx=0).stop 2024-05-09T08:03:03.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:03.856+0000 7fe46c600640 1 -- 172.21.15.1:0/3061230619 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe464152b20 msgr2=0x7fe4640c20a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:03.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:03.856+0000 7fe46c600640 1 --2- 172.21.15.1:0/3061230619 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe464152b20 0x7fe4640c20a0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fe45c0097a0 tx=0x7fe45c0046c0 comp rx=0 tx=0).stop 2024-05-09T08:03:03.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:03.857+0000 7fe46c600640 1 -- 172.21.15.1:0/3061230619 shutdown_connections 2024-05-09T08:03:03.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:03.857+0000 7fe46c600640 1 --2- 172.21.15.1:0/3061230619 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fe44406be60 0x7fe44406e320 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:03.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:03.857+0000 7fe46c600640 1 --2- 172.21.15.1:0/3061230619 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe464152b20 0x7fe4640c20a0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:03.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:03.857+0000 7fe46c600640 1 --2- 172.21.15.1:0/3061230619 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fe4640c6ea0 0x7fe464142330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:03.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:03.857+0000 7fe46c600640 1 --2- 172.21.15.1:0/3061230619 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe46406dca0 0x7fe464141df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:03.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:03.857+0000 7fe46c600640 1 -- 172.21.15.1:0/3061230619 >> 172.21.15.1:0/3061230619 conn(0x7fe4640cc070 msgr2=0x7fe4640c7380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:03:03.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:03.857+0000 7fe46c600640 1 -- 172.21.15.1:0/3061230619 shutdown_connections 2024-05-09T08:03:03.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:03.857+0000 7fe46c600640 1 -- 172.21.15.1:0/3061230619 wait complete. 2024-05-09T08:03:03.893 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T08:03:04.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:04.091+0000 7eff2be00640 1 Processor -- start 2024-05-09T08:03:04.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:04.091+0000 7eff2be00640 1 -- start start 2024-05-09T08:03:04.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:04.093+0000 7eff2be00640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7eff2c06d350 0x7eff2c0ca5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:04.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:04.093+0000 7eff2be00640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7eff2c15b550 con 0x7eff2c15ba50 2024-05-09T08:03:04.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:04.093+0000 7eff2be00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7eff2c06c8b0 con 0x7eff2c0cd040 2024-05-09T08:03:04.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:04.093+0000 7eff2be00640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7eff2c06cec0 con 0x7eff2c06d350 2024-05-09T08:03:04.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:04.093+0000 7eff2b400640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7eff2c06d350 0x7eff2c0ca5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:04.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:04.093+0000 7eff2b400640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7eff2c06d350 0x7eff2c0ca5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:39580/0 (socket says 172.21.15.1:39580) 2024-05-09T08:03:04.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:04.093+0000 7eff2b400640 1 -- 172.21.15.1:0/3947739064 learned_addr learned my addr 172.21.15.1:0/3947739064 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:03:04.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:04.093+0000 7eff23e00640 1 --1- 172.21.15.1:0/3947739064 >> v1:172.21.15.1:6789/0 conn(0x7eff2c15ba50 0x7eff2c15a5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:42674/0 (socket says 172.21.15.1:42674) 2024-05-09T08:03:04.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:04.094+0000 7eff2b400640 1 -- 172.21.15.1:0/3947739064 >> v1:172.21.15.7:6789/0 conn(0x7eff2c0cd040 legacy=0x7eff2c06da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:04.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:04.094+0000 7eff2b400640 1 -- 172.21.15.1:0/3947739064 >> v1:172.21.15.1:6789/0 conn(0x7eff2c15ba50 legacy=0x7eff2c15a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:04.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:04.094+0000 7eff2b400640 1 -- 172.21.15.1:0/3947739064 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff2c15acf0 con 0x7eff2c06d350 2024-05-09T08:03:04.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:04.094+0000 7eff2a000640 1 -- 172.21.15.1:0/3947739064 <== mon.1 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2998409999 0 0) 0x7eff2c06c8b0 con 0x7eff2c0cd040 2024-05-09T08:03:04.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:04.095+0000 7eff2b400640 1 --2- 172.21.15.1:0/3947739064 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7eff2c06d350 0x7eff2c0ca5f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7eff24009e50 tx=0x7eff2402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4b3b498715fa840 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:03:04.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:04.095+0000 7eff2a000640 1 -- 172.21.15.1:0/3947739064 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff24036070 con 0x7eff2c06d350 2024-05-09T08:03:04.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:04.095+0000 7eff2a000640 1 -- 172.21.15.1:0/3947739064 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff24031600 con 0x7eff2c06d350 2024-05-09T08:03:04.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:04.095+0000 7eff2a000640 1 -- 172.21.15.1:0/3947739064 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff240319d0 con 0x7eff2c06d350 2024-05-09T08:03:04.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:04.096+0000 7eff2be00640 1 -- 172.21.15.1:0/3947739064 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7eff2c06d350 msgr2=0x7eff2c0ca5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:04.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:04.096+0000 7eff2be00640 1 --2- 172.21.15.1:0/3947739064 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7eff2c06d350 0x7eff2c0ca5f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7eff24009e50 tx=0x7eff2402a6a0 comp rx=0 tx=0).stop 2024-05-09T08:03:04.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:04.096+0000 7eff2be00640 1 -- 172.21.15.1:0/3947739064 shutdown_connections 2024-05-09T08:03:04.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:04.096+0000 7eff2be00640 1 --2- 172.21.15.1:0/3947739064 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7eff2c06d350 0x7eff2c0ca5f0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:04.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:04.096+0000 7eff2be00640 1 -- 172.21.15.1:0/3947739064 >> 172.21.15.1:0/3947739064 conn(0x7eff2c0b1780 msgr2=0x7eff2c0b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:03:04.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:04.096+0000 7eff2be00640 1 -- 172.21.15.1:0/3947739064 shutdown_connections 2024-05-09T08:03:04.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:04.096+0000 7eff2be00640 1 -- 172.21.15.1:0/3947739064 wait complete. 2024-05-09T08:03:04.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:04.098+0000 7eff2be00640 1 Processor -- start 2024-05-09T08:03:04.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:04.098+0000 7eff2be00640 1 -- start start 2024-05-09T08:03:04.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:04.099+0000 7eff2be00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7eff2c06d350 0x7eff2c1586c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:04.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:04.099+0000 7eff2be00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7eff2c0cd040 0x7eff2c0cfb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:04.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:04.099+0000 7eff2be00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7eff2c15ba50 0x7eff2c1e5830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:04.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:04.099+0000 7eff2be00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7eff2c15b270 con 0x7eff2c06d350 2024-05-09T08:03:04.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:04.099+0000 7eff2be00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7eff2c06d7c0 con 0x7eff2c15ba50 2024-05-09T08:03:04.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:04.099+0000 7eff2be00640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7eff2c06cb50 con 0x7eff2c0cd040 2024-05-09T08:03:04.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:04.099+0000 7eff2aa00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7eff2c06d350 0x7eff2c1586c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:04.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:04.099+0000 7eff2b400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7eff2c15ba50 0x7eff2c1e5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:04.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:04.100+0000 7eff2b400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7eff2c15ba50 0x7eff2c1e5830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.1:33490/0 (socket says 172.21.15.1:33490) 2024-05-09T08:03:04.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:04.100+0000 7eff2b400640 1 -- 172.21.15.1:0/1897797931 learned_addr learned my addr 172.21.15.1:0/1897797931 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:03:04.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:04.100+0000 7eff23e00640 1 --2- 172.21.15.1:0/1897797931 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7eff2c0cd040 0x7eff2c0cfb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:04.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:04.100+0000 7eff2b400640 1 -- 172.21.15.1:0/1897797931 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7eff2c0cd040 msgr2=0x7eff2c0cfb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:04.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:04.100+0000 7eff2b400640 1 --2- 172.21.15.1:0/1897797931 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7eff2c0cd040 0x7eff2c0cfb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:04.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:04.100+0000 7eff2b400640 1 -- 172.21.15.1:0/1897797931 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7eff2c06d350 msgr2=0x7eff2c1586c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:04.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:04.100+0000 7eff2b400640 1 --2- 172.21.15.1:0/1897797931 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7eff2c06d350 0x7eff2c1586c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:04.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:04.100+0000 7eff2b400640 1 -- 172.21.15.1:0/1897797931 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff2c1e62e0 con 0x7eff2c15ba50 2024-05-09T08:03:04.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:04.100+0000 7eff2b400640 1 --2- 172.21.15.1:0/1897797931 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7eff2c15ba50 0x7eff2c1e5830 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7eff24034040 tx=0x7eff24004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:03:04.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:04.101+0000 7eff22a00640 1 -- 172.21.15.1:0/1897797931 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff24036050 con 0x7eff2c15ba50 2024-05-09T08:03:04.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:04.101+0000 7eff22a00640 1 -- 172.21.15.1:0/1897797931 <== mon.1 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff24042070 con 0x7eff2c15ba50 2024-05-09T08:03:04.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:04.101+0000 7eff22a00640 1 -- 172.21.15.1:0/1897797931 <== mon.1 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff24009510 con 0x7eff2c15ba50 2024-05-09T08:03:04.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:04.101+0000 7eff2be00640 1 -- 172.21.15.1:0/1897797931 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff2c1e6580 con 0x7eff2c15ba50 2024-05-09T08:03:04.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:04.101+0000 7eff2be00640 1 -- 172.21.15.1:0/1897797931 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff2c0cf360 con 0x7eff2c15ba50 2024-05-09T08:03:04.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:04.102+0000 7eff2be00640 1 -- 172.21.15.1:0/1897797931 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff2c04f660 con 0x7eff2c15ba50 2024-05-09T08:03:04.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:04.103+0000 7eff22a00640 1 -- 172.21.15.1:0/1897797931 <== mon.1 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7eff24007bd0 con 0x7eff2c15ba50 2024-05-09T08:03:04.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:04.104+0000 7eff22a00640 1 --2- 172.21.15.1:0/1897797931 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7eff0806be60 0x7eff0806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:04.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:04.104+0000 7eff22a00640 1 -- 172.21.15.1:0/1897797931 <== mon.1 v2:172.21.15.7:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7eff240ac0b0 con 0x7eff2c15ba50 2024-05-09T08:03:04.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:04.104+0000 7eff2aa00640 1 --2- 172.21.15.1:0/1897797931 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7eff0806be60 0x7eff0806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:04.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:04.105+0000 7eff2aa00640 1 --2- 172.21.15.1:0/1897797931 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7eff0806be60 0x7eff0806e320 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7eff1c000f60 tx=0x7eff1c009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:03:04.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:04.109+0000 7eff22a00640 1 -- 172.21.15.1:0/1897797931 <== mon.1 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7eff24079f80 con 0x7eff2c15ba50 2024-05-09T08:03:04.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:04.313+0000 7eff2be00640 1 -- 172.21.15.1:0/1897797931 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7eff2c06e260 con 0x7eff2c15ba50 2024-05-09T08:03:04.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:04.315+0000 7eff22a00640 1 -- 172.21.15.1:0/1897797931 <== mon.1 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18741 (secure 0 0 0) 0x7eff2407e280 con 0x7eff2c15ba50 2024-05-09T08:03:04.332 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T08:03:04.332 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":27,"fsid":"54329ec8-3270-4043-9a82-c6cd7420aa7f","created":"2024-05-09T08:01:56.666970+0000","modified":"2024-05-09T08:03:02.304156+0000","last_up_change":"2024-05-09T08:02:07.146082+0000","last_in_change":"2024-05-09T08:01:59.008225+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T08:02:09.165511+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T08:02:12.925485+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T08:02:14.367455+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-09T08:02:17.967064+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c064454e-a9b5-4a24-9a77-17f0a4085952","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6800","nonce":1513349547},{"type":"v1","addr":"172.21.15.1:6801","nonce":1513349547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":1513349547},{"type":"v1","addr":"172.21.15.1:6803","nonce":1513349547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":1513349547},{"type":"v1","addr":"172.21.15.1:6807","nonce":1513349547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":1513349547},{"type":"v1","addr":"172.21.15.1:6805","nonce":1513349547}]},"public_addr":"172.21.15.1:6801/1513349547","cluster_addr":"172.21.15.1:6803/1513349547","heartbeat_back_addr":"172.21.15.1:6807/1513349547","heartbeat_front_addr":"172.21.15.1:6805/1513349547","state":["exists","up"]},{"osd":1,"uuid":"1a9ea687-2ff6-407a-949b-0e7e1083af21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":2942255259},{"type":"v1","addr":"172.21.15.1:6817","nonce":2942255259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":2942255259},{"type":"v1","addr":"172.21.15.1:6819","nonce":2942255259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":2942255259},{"type":"v1","addr":"172.21.15.1:6823","nonce":2942255259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6820","nonce":2942255259},{"type":"v1","addr":"172.21.15.1:6821","nonce":2942255259}]},"public_addr":"172.21.15.1:6817/2942255259","cluster_addr":"172.21.15.1:6819/2942255259","heartbeat_back_addr":"172.21.15.1:6823/2942255259","heartbeat_front_addr":"172.21.15.1:6821/2942255259","state":["exists","up"]},{"osd":2,"uuid":"c88af08b-7c9a-4dd8-9640-44138239addb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":2925031811},{"type":"v1","addr":"172.21.15.1:6809","nonce":2925031811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":2925031811},{"type":"v1","addr":"172.21.15.1:6811","nonce":2925031811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":2925031811},{"type":"v1","addr":"172.21.15.1:6815","nonce":2925031811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":2925031811},{"type":"v1","addr":"172.21.15.1:6813","nonce":2925031811}]},"public_addr":"172.21.15.1:6809/2925031811","cluster_addr":"172.21.15.1:6811/2925031811","heartbeat_back_addr":"172.21.15.1:6815/2925031811","heartbeat_front_addr":"172.21.15.1:6813/2925031811","state":["exists","up"]},{"osd":3,"uuid":"c087b7e3-a41d-4333-9365-32e3bd8c8e6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6824","nonce":2497776654},{"type":"v1","addr":"172.21.15.1:6825","nonce":2497776654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":2497776654},{"type":"v1","addr":"172.21.15.1:6827","nonce":2497776654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":2497776654},{"type":"v1","addr":"172.21.15.1:6831","nonce":2497776654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6828","nonce":2497776654},{"type":"v1","addr":"172.21.15.1:6829","nonce":2497776654}]},"public_addr":"172.21.15.1:6825/2497776654","cluster_addr":"172.21.15.1:6827/2497776654","heartbeat_back_addr":"172.21.15.1:6831/2497776654","heartbeat_front_addr":"172.21.15.1:6829/2497776654","state":["exists","up"]},{"osd":4,"uuid":"b4576b9c-871d-47a8-8d36-65af525d0200","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":2957026670},{"type":"v1","addr":"172.21.15.7:6801","nonce":2957026670}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2957026670},{"type":"v1","addr":"172.21.15.7:6803","nonce":2957026670}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":2957026670},{"type":"v1","addr":"172.21.15.7:6807","nonce":2957026670}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2957026670},{"type":"v1","addr":"172.21.15.7:6805","nonce":2957026670}]},"public_addr":"172.21.15.7:6801/2957026670","cluster_addr":"172.21.15.7:6803/2957026670","heartbeat_back_addr":"172.21.15.7:6807/2957026670","heartbeat_front_addr":"172.21.15.7:6805/2957026670","state":["exists","up"]},{"osd":5,"uuid":"902b3d94-72f3-4499-b21a-e32453ef5144","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1670295242},{"type":"v1","addr":"172.21.15.7:6809","nonce":1670295242}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1670295242},{"type":"v1","addr":"172.21.15.7:6811","nonce":1670295242}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1670295242},{"type":"v1","addr":"172.21.15.7:6815","nonce":1670295242}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1670295242},{"type":"v1","addr":"172.21.15.7:6813","nonce":1670295242}]},"public_addr":"172.21.15.7:6809/1670295242","cluster_addr":"172.21.15.7:6811/1670295242","heartbeat_back_addr":"172.21.15.7:6815/1670295242","heartbeat_front_addr":"172.21.15.7:6813/1670295242","state":["exists","up"]},{"osd":6,"uuid":"ead1d771-1728-4f85-adb4-26be21cedff3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":2852086880},{"type":"v1","addr":"172.21.15.7:6817","nonce":2852086880}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":2852086880},{"type":"v1","addr":"172.21.15.7:6819","nonce":2852086880}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":2852086880},{"type":"v1","addr":"172.21.15.7:6823","nonce":2852086880}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":2852086880},{"type":"v1","addr":"172.21.15.7:6821","nonce":2852086880}]},"public_addr":"172.21.15.7:6817/2852086880","cluster_addr":"172.21.15.7:6819/2852086880","heartbeat_back_addr":"172.21.15.7:6823/2852086880","heartbeat_front_addr":"172.21.15.7:6821/2852086880","state":["exists","up"]},{"osd":7,"uuid":"3ab5a201-9394-48a8-8918-e95197893669","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":1442884138},{"type":"v1","addr":"172.21.15.7:6825","nonce":1442884138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1442884138},{"type":"v1","addr":"172.21.15.7:6827","nonce":1442884138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1442884138},{"type":"v1","addr":"172.21.15.7:6831","nonce":1442884138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1442884138},{"type":"v1","addr":"172.21.15.7:6829","nonce":1442884138}]},"public_addr":"172.21.15.7:6825/1442884138","cluster_addr":"172.21.15.7:6827/1442884138","heartbeat_back_addr":"172.21.15.7:6831/1442884138","heartbeat_front_addr":"172.21.15.7:6829/1442884138","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:02:04.615675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:02:04.385974+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:02:04.619217+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:02:04.401681+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:02:04.097081+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:02:04.129981+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:02:04.225327+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:02:04.193351+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.35","mappings":[{"from":5,"to":2}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2},{"from":3,"to":1}]},{"pgid":"3.3f","mappings":[{"from":7,"to":2}]},{"pgid":"4.4","mappings":[{"from":1,"to":5}]},{"pgid":"4.e","mappings":[{"from":1,"to":2}]},{"pgid":"4.f","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T08:03:04.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:04.318+0000 7eff2be00640 1 -- 172.21.15.1:0/1897797931 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7eff0806be60 msgr2=0x7eff0806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:04.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:04.318+0000 7eff2be00640 1 --2- 172.21.15.1:0/1897797931 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7eff0806be60 0x7eff0806e320 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7eff1c000f60 tx=0x7eff1c009040 comp rx=0 tx=0).stop 2024-05-09T08:03:04.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:04.318+0000 7eff2be00640 1 -- 172.21.15.1:0/1897797931 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7eff2c15ba50 msgr2=0x7eff2c1e5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:04.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:04.318+0000 7eff2be00640 1 --2- 172.21.15.1:0/1897797931 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7eff2c15ba50 0x7eff2c1e5830 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7eff24034040 tx=0x7eff24004a40 comp rx=0 tx=0).stop 2024-05-09T08:03:04.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:04.318+0000 7eff2be00640 1 -- 172.21.15.1:0/1897797931 shutdown_connections 2024-05-09T08:03:04.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:04.318+0000 7eff2be00640 1 --2- 172.21.15.1:0/1897797931 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7eff0806be60 0x7eff0806e320 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:04.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:04.318+0000 7eff2be00640 1 --2- 172.21.15.1:0/1897797931 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7eff2c15ba50 0x7eff2c1e5830 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:04.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:04.318+0000 7eff2be00640 1 --2- 172.21.15.1:0/1897797931 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7eff2c0cd040 0x7eff2c0cfb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:04.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:04.318+0000 7eff2be00640 1 --2- 172.21.15.1:0/1897797931 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7eff2c06d350 0x7eff2c1586c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:04.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:04.318+0000 7eff2be00640 1 -- 172.21.15.1:0/1897797931 >> 172.21.15.1:0/1897797931 conn(0x7eff2c0b1780 msgr2=0x7eff2c15be90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:03:04.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:04.318+0000 7eff2be00640 1 -- 172.21.15.1:0/1897797931 shutdown_connections 2024-05-09T08:03:04.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:04.318+0000 7eff2be00640 1 -- 172.21.15.1:0/1897797931 wait complete. 2024-05-09T08:03:04.356 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T08:03:04.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:04.552+0000 7f1644000640 1 Processor -- start 2024-05-09T08:03:04.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:04.552+0000 7f1644000640 1 -- start start 2024-05-09T08:03:04.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:04.554+0000 7f1644000640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f163c0c6ea0 0x7f163c072900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:04.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:04.554+0000 7f1644000640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f163c06ece0 0x7f163c159010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:04.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:04.554+0000 7f1644000640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f163c073200 con 0x7f163c06dca0 2024-05-09T08:03:04.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:04.554+0000 7f1644000640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f163c072e40 con 0x7f163c06ece0 2024-05-09T08:03:04.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:04.554+0000 7f1644000640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f163c0737c0 con 0x7f163c0c6ea0 2024-05-09T08:03:04.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:04.554+0000 7f1641600640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f163c06dca0 0x7f163c06e080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:42682/0 (socket says 172.21.15.1:42682) 2024-05-09T08:03:04.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:04.554+0000 7f1641600640 1 -- 172.21.15.1:0/2500054376 learned_addr learned my addr 172.21.15.1:0/2500054376 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T08:03:04.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:04.554+0000 7f1642000640 1 --2- 172.21.15.1:0/2500054376 >> v2:172.21.15.7:3300/0 conn(0x7f163c06ece0 0x7f163c159010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:04.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:04.554+0000 7f1640c00640 1 --2- 172.21.15.1:0/2500054376 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f163c0c6ea0 0x7f163c072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:04.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:04.555+0000 7f1640c00640 1 -- 172.21.15.1:0/2500054376 >> v2:172.21.15.7:3300/0 conn(0x7f163c06ece0 msgr2=0x7f163c159010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:03:04.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:04.555+0000 7f1640c00640 1 --2- 172.21.15.1:0/2500054376 >> v2:172.21.15.7:3300/0 conn(0x7f163c06ece0 0x7f163c159010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:04.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:04.555+0000 7f1640c00640 1 -- 172.21.15.1:0/2500054376 >> v1:172.21.15.1:6789/0 conn(0x7f163c06dca0 legacy=0x7f163c06e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:04.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:04.555+0000 7f1640c00640 1 -- 172.21.15.1:0/2500054376 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f163c0734e0 con 0x7f163c0c6ea0 2024-05-09T08:03:04.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:04.555+0000 7f163be00640 1 -- 172.21.15.1:0/2500054376 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4159672580 0 0) 0x7f163c073200 con 0x7f163c06dca0 2024-05-09T08:03:04.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:04.555+0000 7f1642000640 1 --2- 172.21.15.1:0/2500054376 >> v2:172.21.15.7:3300/0 conn(0x7f163c06ece0 0x7f163c159010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:03:04.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:04.556+0000 7f1640c00640 1 --2- 172.21.15.1:0/2500054376 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f163c0c6ea0 0x7f163c072900 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f16280097a0 tx=0x7f162802a230 comp rx=0 tx=0).ready entity=mon.2 client_cookie=66ff94d00f092605 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:03:04.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:04.556+0000 7f163be00640 1 -- 172.21.15.1:0/2500054376 <== mon.2 v2:172.21.15.7:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1628037070 con 0x7f163c0c6ea0 2024-05-09T08:03:04.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:04.556+0000 7f163be00640 1 -- 172.21.15.1:0/2500054376 <== mon.2 v2:172.21.15.7:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1628032440 con 0x7f163c0c6ea0 2024-05-09T08:03:04.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:04.556+0000 7f163be00640 1 -- 172.21.15.1:0/2500054376 <== mon.2 v2:172.21.15.7:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1628032810 con 0x7f163c0c6ea0 2024-05-09T08:03:04.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:04.557+0000 7f1644000640 1 -- 172.21.15.1:0/2500054376 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f163c0c6ea0 msgr2=0x7f163c072900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:04.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:04.557+0000 7f1644000640 1 --2- 172.21.15.1:0/2500054376 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f163c0c6ea0 0x7f163c072900 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f16280097a0 tx=0x7f162802a230 comp rx=0 tx=0).stop 2024-05-09T08:03:04.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:04.557+0000 7f1644000640 1 -- 172.21.15.1:0/2500054376 shutdown_connections 2024-05-09T08:03:04.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:04.557+0000 7f1644000640 1 --2- 172.21.15.1:0/2500054376 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f163c0c6ea0 0x7f163c072900 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:04.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:04.557+0000 7f1644000640 1 --2- 172.21.15.1:0/2500054376 >> v2:172.21.15.7:3300/0 conn(0x7f163c06ece0 0x7f163c159010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:04.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:04.557+0000 7f1644000640 1 -- 172.21.15.1:0/2500054376 >> 172.21.15.1:0/2500054376 conn(0x7f163c0cc070 msgr2=0x7f163c0cc480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:03:04.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:04.557+0000 7f1644000640 1 -- 172.21.15.1:0/2500054376 shutdown_connections 2024-05-09T08:03:04.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:04.557+0000 7f1644000640 1 -- 172.21.15.1:0/2500054376 wait complete. 2024-05-09T08:03:04.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:04.559+0000 7f1644000640 1 Processor -- start 2024-05-09T08:03:04.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:04.559+0000 7f1644000640 1 -- start start 2024-05-09T08:03:04.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:04.560+0000 7f1644000640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f163c06dca0 0x7f163c0c9f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:04.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:04.560+0000 7f1644000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f163c06ece0 0x7f163c142c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:04.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:04.560+0000 7f1644000640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f163c0c6ea0 0x7f163c1f1100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:04.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:04.560+0000 7f1644000640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f163c072470 con 0x7f163c0c6ea0 2024-05-09T08:03:04.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:04.560+0000 7f1644000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f163c159550 con 0x7f163c06ece0 2024-05-09T08:03:04.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:04.560+0000 7f1644000640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f163c072100 con 0x7f163c06dca0 2024-05-09T08:03:04.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:04.560+0000 7f1642000640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f163c0c6ea0 0x7f163c1f1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:04.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:04.560+0000 7f1642000640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f163c0c6ea0 0x7f163c1f1100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:39590/0 (socket says 172.21.15.1:39590) 2024-05-09T08:03:04.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:04.560+0000 7f1642000640 1 -- 172.21.15.1:0/2085043846 learned_addr learned my addr 172.21.15.1:0/2085043846 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:03:04.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:04.560+0000 7f1640c00640 1 --2- 172.21.15.1:0/2085043846 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f163c06ece0 0x7f163c142c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:04.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:04.560+0000 7f1641600640 1 --2- 172.21.15.1:0/2085043846 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f163c06dca0 0x7f163c0c9f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:04.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:04.560+0000 7f1642000640 1 -- 172.21.15.1:0/2085043846 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f163c06dca0 msgr2=0x7f163c0c9f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:04.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:04.560+0000 7f1642000640 1 --2- 172.21.15.1:0/2085043846 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f163c06dca0 0x7f163c0c9f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:04.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:04.560+0000 7f1642000640 1 -- 172.21.15.1:0/2085043846 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f163c06ece0 msgr2=0x7f163c142c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:04.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:04.560+0000 7f1642000640 1 --2- 172.21.15.1:0/2085043846 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f163c06ece0 0x7f163c142c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:04.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:04.560+0000 7f1642000640 1 -- 172.21.15.1:0/2085043846 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f163c0b1a90 con 0x7f163c0c6ea0 2024-05-09T08:03:04.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:04.561+0000 7f1642000640 1 --2- 172.21.15.1:0/2085043846 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f163c0c6ea0 0x7f163c1f1100 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f1634009f00 tx=0x7f163400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:03:04.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:04.561+0000 7f1639600640 1 -- 172.21.15.1:0/2085043846 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1634018030 con 0x7f163c0c6ea0 2024-05-09T08:03:04.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:04.561+0000 7f1639600640 1 -- 172.21.15.1:0/2085043846 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f163400ce50 con 0x7f163c0c6ea0 2024-05-09T08:03:04.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:04.561+0000 7f1644000640 1 -- 172.21.15.1:0/2085043846 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f163c0c5a40 con 0x7f163c0c6ea0 2024-05-09T08:03:04.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:04.561+0000 7f1639600640 1 -- 172.21.15.1:0/2085043846 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1634003bb0 con 0x7f163c0c6ea0 2024-05-09T08:03:04.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:04.561+0000 7f1644000640 1 -- 172.21.15.1:0/2085043846 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f163c1474c0 con 0x7f163c0c6ea0 2024-05-09T08:03:04.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:04.562+0000 7f1644000640 1 -- 172.21.15.1:0/2085043846 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f163c074d70 con 0x7f163c0c6ea0 2024-05-09T08:03:04.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:04.563+0000 7f1639600640 1 -- 172.21.15.1:0/2085043846 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1634005d10 con 0x7f163c0c6ea0 2024-05-09T08:03:04.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:04.564+0000 7f1639600640 1 --2- 172.21.15.1:0/2085043846 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f161c06be60 0x7f161c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:04.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:04.565+0000 7f1639600640 1 -- 172.21.15.1:0/2085043846 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7f163408bbb0 con 0x7f163c0c6ea0 2024-05-09T08:03:04.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:04.565+0000 7f1641600640 1 --2- 172.21.15.1:0/2085043846 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f161c06be60 0x7f161c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:04.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:04.565+0000 7f1641600640 1 --2- 172.21.15.1:0/2085043846 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f161c06be60 0x7f161c06e320 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f1630000f60 tx=0x7f1630007d50 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:03:04.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:04.569+0000 7f1639600640 1 -- 172.21.15.1:0/2085043846 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f1634022070 con 0x7f163c0c6ea0 2024-05-09T08:03:04.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:04.810+0000 7f1644000640 1 -- 172.21.15.1:0/2085043846 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f163c155fa0 con 0x7f163c0c6ea0 2024-05-09T08:03:04.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:04.811+0000 7f1639600640 1 -- 172.21.15.1:0/2085043846 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+6807 (secure 0 0 0) 0x7f1634010040 con 0x7f163c0c6ea0 2024-05-09T08:03:04.828 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T08:03:04.828 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":10,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4220,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/1157980939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":1157980939},{"type":"v1","addr":"172.21.15.7:6837","nonce":1157980939}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4235,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6833/3269798795","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6832","nonce":3269798795},{"type":"v1","addr":"172.21.15.1:6833","nonce":3269798795}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4241,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/725115499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6835","nonce":725115499},{"type":"v1","addr":"172.21.15.1:6837","nonce":725115499}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4276,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6838/1827115274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1827115274},{"type":"v1","addr":"172.21.15.7:6838","nonce":1827115274}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/1032767352","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":1032767352},{"type":"v1","addr":"172.21.15.7:6839","nonce":1032767352}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":10,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-09T08:02:15.332696+0000","modified":"2024-05-09T08:02:49.734561+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4294},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4294":{"gid":4294,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":6,"addr":"172.21.15.1:6836/1509134635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":1509134635},{"type":"v1","addr":"172.21.15.1:6836","nonce":1509134635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4294,"qdb_cluster":[4294]},"id":1}]} 2024-05-09T08:03:04.829 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 10 2024-05-09T08:03:04.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:04.814+0000 7f1644000640 1 -- 172.21.15.1:0/2085043846 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f161c06be60 msgr2=0x7f161c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:04.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:04.814+0000 7f1644000640 1 --2- 172.21.15.1:0/2085043846 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f161c06be60 0x7f161c06e320 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f1630000f60 tx=0x7f1630007d50 comp rx=0 tx=0).stop 2024-05-09T08:03:04.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:04.814+0000 7f1644000640 1 -- 172.21.15.1:0/2085043846 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f163c0c6ea0 msgr2=0x7f163c1f1100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:04.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:04.814+0000 7f1644000640 1 --2- 172.21.15.1:0/2085043846 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f163c0c6ea0 0x7f163c1f1100 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f1634009f00 tx=0x7f163400c490 comp rx=0 tx=0).stop 2024-05-09T08:03:04.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:04.814+0000 7f1644000640 1 -- 172.21.15.1:0/2085043846 shutdown_connections 2024-05-09T08:03:04.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:04.814+0000 7f1644000640 1 --2- 172.21.15.1:0/2085043846 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f161c06be60 0x7f161c06e320 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:04.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:04.814+0000 7f1644000640 1 --2- 172.21.15.1:0/2085043846 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f163c06ece0 0x7f163c142c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:04.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:04.814+0000 7f1644000640 1 --2- 172.21.15.1:0/2085043846 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f163c0c6ea0 0x7f163c1f1100 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:04.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:04.814+0000 7f1644000640 1 --2- 172.21.15.1:0/2085043846 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f163c06dca0 0x7f163c0c9f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:04.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:04.814+0000 7f1644000640 1 -- 172.21.15.1:0/2085043846 >> 172.21.15.1:0/2085043846 conn(0x7f163c0cc070 msgr2=0x7f163c14fd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:03:04.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:04.814+0000 7f1644000640 1 -- 172.21.15.1:0/2085043846 shutdown_connections 2024-05-09T08:03:04.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:04.814+0000 7f1644000640 1 -- 172.21.15.1:0/2085043846 wait complete. 2024-05-09T08:03:05.454 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 10, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-09T08:02:15.332696+0000', 'modified': '2024-05-09T08:02:49.734561+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4294}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4294': {'gid': 4294, 'name': 'c', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.1:6836/1509134635', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6834', 'nonce': 1509134635}, {'type': 'v1', 'addr': '172.21.15.1:6836', 'nonce': 1509134635}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4294, 'qdb_cluster': [4294]} 2024-05-09T08:03:05.454 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-09T08:03:05.457 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-05-09T08:03:05.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.660+0000 7f7cd7200640 1 Processor -- start 2024-05-09T08:03:05.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.660+0000 7f7cd7200640 1 -- start start 2024-05-09T08:03:05.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.661+0000 7f7cd7200640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f7cd81513a0 0x7f7cd80c3070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:05.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.661+0000 7f7cd7200640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f7cd80c5df0 0x7f7cd8157240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:05.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.661+0000 7f7cd7200640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7cd80c5350 con 0x7f7cd80c7b10 2024-05-09T08:03:05.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.661+0000 7f7cd7200640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f7cd80c3670 con 0x7f7cd80c5df0 2024-05-09T08:03:05.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.661+0000 7f7cd7200640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f7cd80c3950 con 0x7f7cd81513a0 2024-05-09T08:03:05.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.662+0000 7f7cd6800640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f7cd80c7b10 0x7f7cd80c8600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:42692/0 (socket says 172.21.15.1:42692) 2024-05-09T08:03:05.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.662+0000 7f7cd6800640 1 -- 172.21.15.1:0/201300775 learned_addr learned my addr 172.21.15.1:0/201300775 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T08:03:05.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.662+0000 7f7cd5e00640 1 --2- 172.21.15.1:0/201300775 >> v2:172.21.15.7:3300/0 conn(0x7f7cd81513a0 0x7f7cd80c3070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:05.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.662+0000 7f7ccfe00640 1 --2- 172.21.15.1:0/201300775 >> v2:172.21.15.1:3300/0 conn(0x7f7cd80c5df0 0x7f7cd8157240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:05.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.662+0000 7f7cd5400640 1 -- 172.21.15.1:0/201300775 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2388848644 0 0) 0x7f7cd80c5350 con 0x7f7cd80c7b10 2024-05-09T08:03:05.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.663+0000 7f7cd5400640 1 -- 172.21.15.1:0/201300775 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7cbc003520 con 0x7f7cd80c7b10 2024-05-09T08:03:05.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.663+0000 7f7cd5e00640 1 -- 172.21.15.1:0/201300775 >> v2:172.21.15.1:3300/0 conn(0x7f7cd80c5df0 msgr2=0x7f7cd8157240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:03:05.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.663+0000 7f7cd5e00640 1 --2- 172.21.15.1:0/201300775 >> v2:172.21.15.1:3300/0 conn(0x7f7cd80c5df0 0x7f7cd8157240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:05.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.663+0000 7f7cd5e00640 1 -- 172.21.15.1:0/201300775 >> v1:172.21.15.1:6789/0 conn(0x7f7cd80c7b10 legacy=0x7f7cd80c8600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:05.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.663+0000 7f7cd5e00640 1 -- 172.21.15.1:0/201300775 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cd80c55f0 con 0x7f7cd81513a0 2024-05-09T08:03:05.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.663+0000 7f7ccfe00640 1 --2- 172.21.15.1:0/201300775 >> v2:172.21.15.1:3300/0 conn(0x7f7cd80c5df0 0x7f7cd8157240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:03:05.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.664+0000 7f7cd5e00640 1 --2- 172.21.15.1:0/201300775 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7cd81513a0 0x7f7cd80c3070 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f7cd8070250 tx=0x7f7cc802a1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=97c0cee3079c5dd2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:03:05.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.664+0000 7f7cd5400640 1 -- 172.21.15.1:0/201300775 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cc8037070 con 0x7f7cd81513a0 2024-05-09T08:03:05.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.664+0000 7f7cd5400640 1 -- 172.21.15.1:0/201300775 <== mon.1 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7cc8032440 con 0x7f7cd81513a0 2024-05-09T08:03:05.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.664+0000 7f7cd5400640 1 -- 172.21.15.1:0/201300775 <== mon.1 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cc8032810 con 0x7f7cd81513a0 2024-05-09T08:03:05.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.665+0000 7f7cd7200640 1 -- 172.21.15.1:0/201300775 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7cd81513a0 msgr2=0x7f7cd80c3070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:05.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.665+0000 7f7cd7200640 1 --2- 172.21.15.1:0/201300775 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7cd81513a0 0x7f7cd80c3070 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f7cd8070250 tx=0x7f7cc802a1a0 comp rx=0 tx=0).stop 2024-05-09T08:03:05.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.665+0000 7f7cd7200640 1 -- 172.21.15.1:0/201300775 shutdown_connections 2024-05-09T08:03:05.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.665+0000 7f7cd7200640 1 --2- 172.21.15.1:0/201300775 >> v2:172.21.15.1:3300/0 conn(0x7f7cd80c5df0 0x7f7cd8157240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:05.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.665+0000 7f7cd7200640 1 --2- 172.21.15.1:0/201300775 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7cd81513a0 0x7f7cd80c3070 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:05.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.665+0000 7f7cd7200640 1 -- 172.21.15.1:0/201300775 >> 172.21.15.1:0/201300775 conn(0x7f7cd806b250 msgr2=0x7f7cd806b630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:03:05.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.665+0000 7f7cd7200640 1 -- 172.21.15.1:0/201300775 shutdown_connections 2024-05-09T08:03:05.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.665+0000 7f7cd7200640 1 -- 172.21.15.1:0/201300775 wait complete. 2024-05-09T08:03:05.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.667+0000 7f7cd7200640 1 Processor -- start 2024-05-09T08:03:05.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.667+0000 7f7cd7200640 1 -- start start 2024-05-09T08:03:05.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.668+0000 7f7cd7200640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7cd80c5df0 0x7f7cd80cac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:05.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.668+0000 7f7cd7200640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f7cd80c7b10 0x7f7cd80c0b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:05.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.668+0000 7f7cd7200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7cd81513a0 0x7f7cd80c10c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:05.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.668+0000 7f7cd7200640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f7cd80c3db0 con 0x7f7cd80c5df0 2024-05-09T08:03:05.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.668+0000 7f7cd7200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7cd80c3c30 con 0x7f7cd81513a0 2024-05-09T08:03:05.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.668+0000 7f7cd7200640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f7cd80c8d10 con 0x7f7cd80c7b10 2024-05-09T08:03:05.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.668+0000 7f7cd6800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7cd81513a0 0x7f7cd80c10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:05.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.668+0000 7f7cd6800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7cd81513a0 0x7f7cd80c10c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.1:33516/0 (socket says 172.21.15.1:33516) 2024-05-09T08:03:05.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.668+0000 7f7cd6800640 1 -- 172.21.15.1:0/1546497864 learned_addr learned my addr 172.21.15.1:0/1546497864 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:03:05.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.668+0000 7f7ccfe00640 1 --2- 172.21.15.1:0/1546497864 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f7cd80c7b10 0x7f7cd80c0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:05.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.668+0000 7f7cd5e00640 1 --2- 172.21.15.1:0/1546497864 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7cd80c5df0 0x7f7cd80cac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:05.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.669+0000 7f7cd6800640 1 -- 172.21.15.1:0/1546497864 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f7cd80c7b10 msgr2=0x7f7cd80c0b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:05.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.669+0000 7f7cd6800640 1 --2- 172.21.15.1:0/1546497864 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f7cd80c7b10 0x7f7cd80c0b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:05.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.669+0000 7f7cd6800640 1 -- 172.21.15.1:0/1546497864 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7cd80c5df0 msgr2=0x7f7cd80cac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:05.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.669+0000 7f7cd6800640 1 --2- 172.21.15.1:0/1546497864 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7cd80c5df0 0x7f7cd80cac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:05.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.669+0000 7f7cd6800640 1 -- 172.21.15.1:0/1546497864 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cd81f3d80 con 0x7f7cd81513a0 2024-05-09T08:03:05.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.669+0000 7f7cd6800640 1 --2- 172.21.15.1:0/1546497864 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7cd81513a0 0x7f7cd80c10c0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f7cd000cb40 tx=0x7f7cd000a060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:03:05.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.669+0000 7f7cce000640 1 -- 172.21.15.1:0/1546497864 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cd0017070 con 0x7f7cd81513a0 2024-05-09T08:03:05.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.669+0000 7f7cce000640 1 -- 172.21.15.1:0/1546497864 <== mon.1 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7cd000aca0 con 0x7f7cd81513a0 2024-05-09T08:03:05.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.669+0000 7f7cce000640 1 -- 172.21.15.1:0/1546497864 <== mon.1 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cd0012620 con 0x7f7cd81513a0 2024-05-09T08:03:05.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.669+0000 7f7cd7200640 1 -- 172.21.15.1:0/1546497864 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7cd81f4060 con 0x7f7cd81513a0 2024-05-09T08:03:05.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.670+0000 7f7cd7200640 1 -- 172.21.15.1:0/1546497864 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7cd80bffa0 con 0x7f7cd81513a0 2024-05-09T08:03:05.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.672+0000 7f7cce000640 1 -- 172.21.15.1:0/1546497864 <== mon.1 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7cd001a070 con 0x7f7cd81513a0 2024-05-09T08:03:05.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.673+0000 7f7cce000640 1 --2- 172.21.15.1:0/1546497864 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f7cb406be30 0x7f7cb406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:05.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.673+0000 7f7cce000640 1 -- 172.21.15.1:0/1546497864 <== mon.1 v2:172.21.15.7:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7f7cd008afe0 con 0x7f7cd81513a0 2024-05-09T08:03:05.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.673+0000 7f7cd5e00640 1 --2- 172.21.15.1:0/1546497864 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f7cb406be30 0x7f7cb406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:05.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.674+0000 7f7cdfa29740 1 Processor -- start 2024-05-09T08:03:05.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.674+0000 7f7cd5e00640 1 --2- 172.21.15.1:0/1546497864 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f7cb406be30 0x7f7cb406e2f0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f7cc8034000 tx=0x7f7cc802e000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:03:05.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.674+0000 7f7cdfa29740 1 -- start start 2024-05-09T08:03:05.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.675+0000 7f7cdfa29740 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x55c7c5217dd0 0x55c7c5256d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:05.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.675+0000 7f7cdfa29740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c7c52572d0 0x55c7c52181b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:05.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.675+0000 7f7cdfa29740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c7c5219330 0x55c7c52186f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:05.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.675+0000 7f7cdfa29740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55c7c517a480 con 0x55c7c5219330 2024-05-09T08:03:05.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.675+0000 7f7cdfa29740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55c7c51225c0 con 0x55c7c52572d0 2024-05-09T08:03:05.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.675+0000 7f7cdfa29740 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x55c7c5176440 con 0x55c7c5217dd0 2024-05-09T08:03:05.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.675+0000 7f7cd5e00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x55c7c5217dd0 0x55c7c5256d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:05.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.675+0000 7f7ccfe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c7c52572d0 0x55c7c52181b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:05.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.675+0000 7f7cd6800640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c7c5219330 0x55c7c52186f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:05.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.675+0000 7f7cd5e00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x55c7c5217dd0 0x55c7c5256d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3301/0 says I am v2:172.21.15.1:49536/0 (socket says 172.21.15.1:49536) 2024-05-09T08:03:05.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.675+0000 7f7ccfe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c7c52572d0 0x55c7c52181b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.1:33530/0 (socket says 172.21.15.1:33530) 2024-05-09T08:03:05.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.675+0000 7f7cd5e00640 1 -- 172.21.15.1:0/795404680 learned_addr learned my addr 172.21.15.1:0/795404680 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:03:05.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.676+0000 7f7ccfe00640 1 -- 172.21.15.1:0/795404680 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x55c7c5217dd0 msgr2=0x55c7c5256d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:03:05.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.676+0000 7f7ccfe00640 1 --2- 172.21.15.1:0/795404680 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x55c7c5217dd0 0x55c7c5256d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:05.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.676+0000 7f7ccfe00640 1 -- 172.21.15.1:0/795404680 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c7c5219330 msgr2=0x55c7c52186f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:03:05.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.676+0000 7f7ccfe00640 1 --2- 172.21.15.1:0/795404680 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c7c5219330 0x55c7c52186f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:05.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.676+0000 7f7ccfe00640 1 -- 172.21.15.1:0/795404680 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c7c4e8e2d0 con 0x55c7c52572d0 2024-05-09T08:03:05.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.676+0000 7f7ccfe00640 1 --2- 172.21.15.1:0/795404680 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c7c52572d0 0x55c7c52181b0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f7cc0009b60 tx=0x7f7cc0011690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ccd4ad6c0a3b449 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:03:05.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.676+0000 7f7cc6600640 1 -- 172.21.15.1:0/795404680 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cc0009e60 con 0x55c7c52572d0 2024-05-09T08:03:05.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.676+0000 7f7cc6600640 1 -- 172.21.15.1:0/795404680 <== mon.1 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7cc0014440 con 0x55c7c52572d0 2024-05-09T08:03:05.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.676+0000 7f7cc6600640 1 -- 172.21.15.1:0/795404680 <== mon.1 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cc0014810 con 0x55c7c52572d0 2024-05-09T08:03:05.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.677+0000 7f7cdfa29740 1 -- 172.21.15.1:0/795404680 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c7c52572d0 msgr2=0x55c7c52181b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:05.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.677+0000 7f7cdfa29740 1 --2- 172.21.15.1:0/795404680 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c7c52572d0 0x55c7c52181b0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f7cc0009b60 tx=0x7f7cc0011690 comp rx=0 tx=0).stop 2024-05-09T08:03:05.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.677+0000 7f7cdfa29740 1 -- 172.21.15.1:0/795404680 shutdown_connections 2024-05-09T08:03:05.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.677+0000 7f7cdfa29740 1 --2- 172.21.15.1:0/795404680 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c7c52572d0 0x55c7c52181b0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:05.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.677+0000 7f7cdfa29740 1 --2- 172.21.15.1:0/795404680 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c7c5219330 0x55c7c52186f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:05.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.677+0000 7f7cdfa29740 1 --2- 172.21.15.1:0/795404680 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x55c7c5217dd0 0x55c7c5256d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:05.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.677+0000 7f7cdfa29740 1 -- 172.21.15.1:0/795404680 >> 172.21.15.1:0/795404680 conn(0x55c7c51de8a0 msgr2=0x55c7c525bd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:03:05.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.677+0000 7f7cdfa29740 1 -- 172.21.15.1:0/795404680 shutdown_connections 2024-05-09T08:03:05.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.677+0000 7f7cdfa29740 1 -- 172.21.15.1:0/795404680 wait complete. 2024-05-09T08:03:05.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.678+0000 7f7cdfa29740 1 -- start start 2024-05-09T08:03:05.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.678+0000 7f7cdfa29740 1 Processor -- start 2024-05-09T08:03:05.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.679+0000 7f7cdfa29740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c7c5217dd0 0x55c7c52553c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:05.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.679+0000 7f7cdfa29740 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x55c7c5219330 0x55c7c5242950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:05.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.679+0000 7f7cdfa29740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c7c52572d0 0x55c7c526bb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:05.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.679+0000 7f7cdfa29740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55c7c5216410 con 0x55c7c5217dd0 2024-05-09T08:03:05.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.679+0000 7f7cdfa29740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55c7c5135980 con 0x55c7c52572d0 2024-05-09T08:03:05.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.679+0000 7f7cdfa29740 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x55c7c5219960 con 0x55c7c5219330 2024-05-09T08:03:05.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.679+0000 7f7cdfa29740 10 client.0 _ll_register_callbacks cb 0x55c7c5170fc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T08:03:05.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.679+0000 7f7cd5e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c7c5217dd0 0x55c7c52553c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:05.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.679+0000 7f7cd5e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c7c5217dd0 0x55c7c52553c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:39634/0 (socket says 172.21.15.1:39634) 2024-05-09T08:03:05.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.679+0000 7f7cd5e00640 1 -- 172.21.15.1:0/1565109051 learned_addr learned my addr 172.21.15.1:0/1565109051 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:03:05.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.679+0000 7f7ccfe00640 1 --2- 172.21.15.1:0/1565109051 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x55c7c5219330 0x55c7c5242950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:05.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.679+0000 7f7cd6800640 1 --2- 172.21.15.1:0/1565109051 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c7c52572d0 0x55c7c526bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:05.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.680+0000 7f7cd5e00640 1 -- 172.21.15.1:0/1565109051 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x55c7c5219330 msgr2=0x55c7c5242950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:03:05.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.680+0000 7f7cd5e00640 1 --2- 172.21.15.1:0/1565109051 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x55c7c5219330 0x55c7c5242950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:05.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.680+0000 7f7cd5e00640 1 -- 172.21.15.1:0/1565109051 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c7c52572d0 msgr2=0x55c7c526bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:03:05.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.680+0000 7f7cd5e00640 1 --2- 172.21.15.1:0/1565109051 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c7c52572d0 0x55c7c526bb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:05.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.680+0000 7f7cd5e00640 1 -- 172.21.15.1:0/1565109051 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c7c4f586b0 con 0x55c7c5217dd0 2024-05-09T08:03:05.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.680+0000 7f7cd5e00640 1 --2- 172.21.15.1:0/1565109051 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c7c5217dd0 0x55c7c52553c0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f7cc802a790 tx=0x7f7cc8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8435bdf5b7f095e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:03:05.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.680+0000 7f7cbaa00640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-09T08:03:05.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.680+0000 7f7cbaa00640 1 -- 172.21.15.1:0/1565109051 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cc804f020 con 0x55c7c5217dd0 2024-05-09T08:03:05.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.681+0000 7f7cbaa00640 1 -- 172.21.15.1:0/1565109051 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7f7cc8032da0 con 0x55c7c5217dd0 2024-05-09T08:03:05.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.681+0000 7f7cdfa29740 1 -- 172.21.15.1:0/1565109051 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7cc0014440 con 0x55c7c5217dd0 2024-05-09T08:03:05.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.681+0000 7f7cbaa00640 1 -- 172.21.15.1:0/1565109051 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=10) ==== 24+0+0 (secure 0 0 0) 0x7f7cc8059050 con 0x55c7c5217dd0 2024-05-09T08:03:05.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.681+0000 7f7cdfa29740 10 client.4764 fetch_fsmap learned FSMap version 10 2024-05-09T08:03:05.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.681+0000 7f7cdfa29740 1 -- 172.21.15.1:0/1565109051 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=10,monmap=2+}) -- 0x55c7c525cb30 con 0x55c7c5217dd0 2024-05-09T08:03:05.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.681+0000 7f7cbaa00640 1 -- 172.21.15.1:0/1565109051 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7cc0014440 con 0x55c7c5217dd0 2024-05-09T08:03:05.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.681+0000 7f7cbaa00640 1 -- 172.21.15.1:0/1565109051 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 10) ==== 4712+0+0 (secure 0 0 0) 0x7f7cc805c020 con 0x55c7c5217dd0 2024-05-09T08:03:05.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.681+0000 7f7cdfa29740 10 client.4764 fetch_fsmap finished waiting for FSMap version 10 2024-05-09T08:03:05.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.681+0000 7f7cdfa29740 10 client.4764 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-09T08:03:05.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.681+0000 7f7cdfa29740 20 client.4764 populate_metadata read hostname 'smithi001' 2024-05-09T08:03:05.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.681+0000 7f7cdfa29740 1 --2- 172.21.15.1:0/1565109051 >> [v2:172.21.15.1:6834/1509134635,v1:172.21.15.1:6836/1509134635] conn(0x55c7c523f0d0 0x55c7c523f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:05.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.681+0000 7f7cdfa29740 4 client.4764 mds_command: new command op to 4294 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T08:03:05.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.681+0000 7f7cdfa29740 1 -- 172.21.15.1:0/1565109051 --> [v2:172.21.15.1:6834/1509134635,v1:172.21.15.1:6836/1509134635] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c7c518e060 con 0x55c7c523f0d0 2024-05-09T08:03:05.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.682+0000 7f7ccfe00640 1 --2- 172.21.15.1:0/1565109051 >> [v2:172.21.15.1:6834/1509134635,v1:172.21.15.1:6836/1509134635] conn(0x55c7c523f0d0 0x55c7c523f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:05.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.682+0000 7f7ccfe00640 1 --2- 172.21.15.1:0/1565109051 >> [v2:172.21.15.1:6834/1509134635,v1:172.21.15.1:6836/1509134635] conn(0x55c7c523f0d0 0x55c7c523f780 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e055a6db2d28048d server_cookie=afde436192be689 in_seq=0 out_seq=0 2024-05-09T08:03:05.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.682+0000 7f7cbaa00640 10 client.4764 ms_handle_connect on v2:172.21.15.1:6834/1509134635 2024-05-09T08:03:05.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.683+0000 7f7cbaa00640 1 -- 172.21.15.1:0/1565109051 <== mds.0 v2:172.21.15.1:6834/1509134635 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c7c518e060 con 0x55c7c523f0d0 2024-05-09T08:03:05.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.683+0000 7f7cbaa00640 10 client.4764 handle_command_reply: tid=0 2024-05-09T08:03:05.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.683+0000 7f7cdfa29740 1 client.4764 shutdown 2024-05-09T08:03:05.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.683+0000 7f7cdfa29740 1 -- 172.21.15.1:0/1565109051 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c7c5217dd0 msgr2=0x55c7c52553c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:05.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.683+0000 7f7cdfa29740 1 --2- 172.21.15.1:0/1565109051 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c7c5217dd0 0x55c7c52553c0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f7cc802a790 tx=0x7f7cc8004730 comp rx=0 tx=0).stop 2024-05-09T08:03:05.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.684+0000 7f7cdfa29740 1 -- 172.21.15.1:0/1565109051 shutdown_connections 2024-05-09T08:03:05.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.684+0000 7f7cdfa29740 1 --2- 172.21.15.1:0/1565109051 >> [v2:172.21.15.1:6834/1509134635,v1:172.21.15.1:6836/1509134635] conn(0x55c7c523f0d0 0x55c7c523f780 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:05.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.684+0000 7f7cdfa29740 1 --2- 172.21.15.1:0/1565109051 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c7c52572d0 0x55c7c526bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:05.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.684+0000 7f7cdfa29740 1 --2- 172.21.15.1:0/1565109051 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x55c7c5219330 0x55c7c5242950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:05.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.684+0000 7f7cbaa00640 0 client.4764 ms_handle_reset on v2:172.21.15.1:6834/1509134635 2024-05-09T08:03:05.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.684+0000 7f7cdfa29740 1 --2- 172.21.15.1:0/1565109051 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c7c5217dd0 0x55c7c52553c0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:05.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.684+0000 7f7cdfa29740 1 -- 172.21.15.1:0/1565109051 >> 172.21.15.1:0/1565109051 conn(0x55c7c51de8a0 msgr2=0x55c7c5218a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:03:05.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.684+0000 7f7cdfa29740 1 -- 172.21.15.1:0/1565109051 shutdown_connections 2024-05-09T08:03:05.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.684+0000 7f7cdfa29740 1 -- 172.21.15.1:0/1565109051 wait complete. 2024-05-09T08:03:05.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.684+0000 7f7cdfa29740 20 client.4764 trim_cache size 0 max 16384 2024-05-09T08:03:05.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.704+0000 7f7cdfa29740 1 Processor -- start 2024-05-09T08:03:05.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.704+0000 7f7cdfa29740 1 -- start start 2024-05-09T08:03:05.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.705+0000 7f7cdfa29740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c7c5217dd0 0x55c7c5211390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:05.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.705+0000 7f7cdfa29740 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x55c7c5219330 0x55c7c52118d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:05.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.705+0000 7f7cdfa29740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c7c52572d0 0x55c7c5271890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:05.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.705+0000 7f7cdfa29740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f7cd81fcc60 con 0x55c7c5217dd0 2024-05-09T08:03:05.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.705+0000 7f7cdfa29740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55c7c52696f0 con 0x55c7c52572d0 2024-05-09T08:03:05.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.705+0000 7f7cdfa29740 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x55c7c526c190 con 0x55c7c5219330 2024-05-09T08:03:05.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.705+0000 7f7cd5e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c7c5217dd0 0x55c7c5211390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:05.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.705+0000 7f7ccfe00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x55c7c5219330 0x55c7c52118d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:05.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.706+0000 7f7cd6800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c7c52572d0 0x55c7c5271890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:05.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.706+0000 7f7cd5e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c7c5217dd0 0x55c7c5211390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:39636/0 (socket says 172.21.15.1:39636) 2024-05-09T08:03:05.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.706+0000 7f7ccfe00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x55c7c5219330 0x55c7c52118d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3301/0 says I am v2:172.21.15.1:49554/0 (socket says 172.21.15.1:49554) 2024-05-09T08:03:05.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.706+0000 7f7cd5e00640 1 -- 172.21.15.1:0/3091429730 learned_addr learned my addr 172.21.15.1:0/3091429730 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:03:05.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.706+0000 7f7cd5e00640 1 -- 172.21.15.1:0/3091429730 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x55c7c5219330 msgr2=0x55c7c52118d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:03:05.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.706+0000 7f7cd5e00640 1 --2- 172.21.15.1:0/3091429730 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x55c7c5219330 0x55c7c52118d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:05.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.706+0000 7f7cd5e00640 1 -- 172.21.15.1:0/3091429730 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c7c52572d0 msgr2=0x55c7c5271890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:03:05.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.706+0000 7f7cd5e00640 1 --2- 172.21.15.1:0/3091429730 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c7c52572d0 0x55c7c5271890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:05.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.706+0000 7f7cd5e00640 1 -- 172.21.15.1:0/3091429730 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c7c5252520 con 0x55c7c5217dd0 2024-05-09T08:03:05.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.706+0000 7f7cd5e00640 1 --2- 172.21.15.1:0/3091429730 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c7c5217dd0 0x55c7c5211390 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f7cc8035040 tx=0x7f7cc8048000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ef17006fe20659a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:03:05.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.706+0000 7f7cc5c00640 1 -- 172.21.15.1:0/3091429730 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cc8040020 con 0x55c7c5217dd0 2024-05-09T08:03:05.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.707+0000 7f7cc5c00640 1 -- 172.21.15.1:0/3091429730 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7cc8032ba0 con 0x55c7c5217dd0 2024-05-09T08:03:05.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.707+0000 7f7cc5c00640 1 -- 172.21.15.1:0/3091429730 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cc804f070 con 0x55c7c5217dd0 2024-05-09T08:03:05.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.707+0000 7f7cdfa29740 1 -- 172.21.15.1:0/3091429730 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c7c5217dd0 msgr2=0x55c7c5211390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:05.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.707+0000 7f7cdfa29740 1 --2- 172.21.15.1:0/3091429730 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c7c5217dd0 0x55c7c5211390 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f7cc8035040 tx=0x7f7cc8048000 comp rx=0 tx=0).stop 2024-05-09T08:03:05.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.707+0000 7f7cdfa29740 1 -- 172.21.15.1:0/3091429730 shutdown_connections 2024-05-09T08:03:05.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.707+0000 7f7cdfa29740 1 --2- 172.21.15.1:0/3091429730 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c7c52572d0 0x55c7c5271890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:05.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.707+0000 7f7cdfa29740 1 --2- 172.21.15.1:0/3091429730 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x55c7c5219330 0x55c7c52118d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:05.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.707+0000 7f7cdfa29740 1 --2- 172.21.15.1:0/3091429730 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c7c5217dd0 0x55c7c5211390 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:05.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.707+0000 7f7cdfa29740 1 -- 172.21.15.1:0/3091429730 >> 172.21.15.1:0/3091429730 conn(0x55c7c51de8a0 msgr2=0x55c7c520d7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:03:05.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.707+0000 7f7cdfa29740 1 -- 172.21.15.1:0/3091429730 shutdown_connections 2024-05-09T08:03:05.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.707+0000 7f7cdfa29740 1 -- 172.21.15.1:0/3091429730 wait complete. 2024-05-09T08:03:05.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.708+0000 7f7cdfa29740 1 -- start start 2024-05-09T08:03:05.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.708+0000 7f7cdfa29740 1 Processor -- start 2024-05-09T08:03:05.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.709+0000 7f7cdfa29740 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x55c7c5217dd0 0x55c7c5259170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:05.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.709+0000 7f7cdfa29740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c7c5219330 0x55c7c5269ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:05.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.709+0000 7f7cdfa29740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c7c52572d0 0x55c7c5258550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:05.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.709+0000 7f7cdfa29740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55c7c523f600 con 0x55c7c5219330 2024-05-09T08:03:05.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.709+0000 7f7cdfa29740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55c7c523ef50 con 0x55c7c52572d0 2024-05-09T08:03:05.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.709+0000 7f7cdfa29740 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x55c7c52681f0 con 0x55c7c5217dd0 2024-05-09T08:03:05.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.709+0000 7f7cdfa29740 10 client.0 _ll_register_callbacks cb 0x7f7cc0009060 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T08:03:05.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.709+0000 7f7ccfe00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c7c5219330 0x55c7c5269ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:05.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.709+0000 7f7ccfe00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c7c5219330 0x55c7c5269ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:39650/0 (socket says 172.21.15.1:39650) 2024-05-09T08:03:05.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.709+0000 7f7ccfe00640 1 -- 172.21.15.1:0/1960950239 learned_addr learned my addr 172.21.15.1:0/1960950239 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:03:05.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.709+0000 7f7cd6800640 1 --2- 172.21.15.1:0/1960950239 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c7c52572d0 0x55c7c5258550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:05.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.709+0000 7f7cd5e00640 1 --2- 172.21.15.1:0/1960950239 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x55c7c5217dd0 0x55c7c5259170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:05.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.710+0000 7f7ccfe00640 1 -- 172.21.15.1:0/1960950239 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x55c7c5217dd0 msgr2=0x55c7c5259170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:03:05.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.710+0000 7f7ccfe00640 1 --2- 172.21.15.1:0/1960950239 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x55c7c5217dd0 0x55c7c5259170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:05.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.710+0000 7f7ccfe00640 1 -- 172.21.15.1:0/1960950239 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c7c52572d0 msgr2=0x55c7c5258550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:03:05.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.710+0000 7f7ccfe00640 1 --2- 172.21.15.1:0/1960950239 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c7c52572d0 0x55c7c5258550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:05.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.710+0000 7f7ccfe00640 1 -- 172.21.15.1:0/1960950239 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c7c52677c0 con 0x55c7c5219330 2024-05-09T08:03:05.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.710+0000 7f7cd6800640 1 --2- 172.21.15.1:0/1960950239 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c7c52572d0 0x55c7c5258550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:03:05.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.710+0000 7f7ccfe00640 1 --2- 172.21.15.1:0/1960950239 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c7c5219330 0x55c7c5269ea0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f7cc0011b90 tx=0x7f7cc0014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3c3c5522492c549 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:03:05.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.710+0000 7f7cbaa00640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-09T08:03:05.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.710+0000 7f7cbaa00640 1 -- 172.21.15.1:0/1960950239 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cc0004450 con 0x55c7c5219330 2024-05-09T08:03:05.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.710+0000 7f7cbaa00640 1 -- 172.21.15.1:0/1960950239 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7f7cc00092e0 con 0x55c7c5219330 2024-05-09T08:03:05.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.710+0000 7f7cdfa29740 1 -- 172.21.15.1:0/1960950239 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7cc8032ba0 con 0x55c7c5219330 2024-05-09T08:03:05.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.711+0000 7f7cbaa00640 1 -- 172.21.15.1:0/1960950239 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=10) ==== 24+0+0 (secure 0 0 0) 0x7f7cc0035050 con 0x55c7c5219330 2024-05-09T08:03:05.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.711+0000 7f7cdfa29740 10 client.4770 fetch_fsmap learned FSMap version 10 2024-05-09T08:03:05.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.711+0000 7f7cdfa29740 1 -- 172.21.15.1:0/1960950239 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=10,monmap=2+}) -- 0x55c7c5213960 con 0x55c7c5219330 2024-05-09T08:03:05.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.711+0000 7f7cbaa00640 1 -- 172.21.15.1:0/1960950239 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7cc8032ba0 con 0x55c7c5219330 2024-05-09T08:03:05.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.711+0000 7f7cbaa00640 1 -- 172.21.15.1:0/1960950239 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 10) ==== 4712+0+0 (secure 0 0 0) 0x7f7cc0038020 con 0x55c7c5219330 2024-05-09T08:03:05.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.711+0000 7f7cdfa29740 10 client.4770 fetch_fsmap finished waiting for FSMap version 10 2024-05-09T08:03:05.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.711+0000 7f7cdfa29740 10 client.4770 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-09T08:03:05.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.711+0000 7f7cdfa29740 20 client.4770 populate_metadata read hostname 'smithi001' 2024-05-09T08:03:05.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.711+0000 7f7cdfa29740 1 --2- 172.21.15.1:0/1960950239 >> [v2:172.21.15.1:6834/1509134635,v1:172.21.15.1:6836/1509134635] conn(0x55c7c52de790 0x55c7c52deb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:05.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.711+0000 7f7cdfa29740 4 client.4770 mds_command: new command op to 4294 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-09T08:03:05.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.711+0000 7f7cdfa29740 1 -- 172.21.15.1:0/1960950239 --> [v2:172.21.15.1:6834/1509134635,v1:172.21.15.1:6836/1509134635] -- command(tid 0: {"prefix": "status"}) -- 0x55c7c5187bb0 con 0x55c7c52de790 2024-05-09T08:03:05.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.712+0000 7f7cd5e00640 1 --2- 172.21.15.1:0/1960950239 >> [v2:172.21.15.1:6834/1509134635,v1:172.21.15.1:6836/1509134635] conn(0x55c7c52de790 0x55c7c52deb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:05.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.712+0000 7f7cd5e00640 1 --2- 172.21.15.1:0/1960950239 >> [v2:172.21.15.1:6834/1509134635,v1:172.21.15.1:6836/1509134635] conn(0x55c7c52de790 0x55c7c52deb70 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a9116c0cc9565bb9 server_cookie=41cec12d529f281b in_seq=0 out_seq=0 2024-05-09T08:03:05.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.712+0000 7f7cbaa00640 10 client.4770 ms_handle_connect on v2:172.21.15.1:6834/1509134635 2024-05-09T08:03:05.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.713+0000 7f7cbaa00640 1 -- 172.21.15.1:0/1960950239 <== mds.0 v2:172.21.15.1:6834/1509134635 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x55c7c5187bb0 con 0x55c7c52de790 2024-05-09T08:03:05.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.713+0000 7f7cbaa00640 10 client.4770 handle_command_reply: tid=0 2024-05-09T08:03:05.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.713+0000 7f7cdfa29740 1 client.4770 shutdown 2024-05-09T08:03:05.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.713+0000 7f7cdfa29740 1 -- 172.21.15.1:0/1960950239 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c7c5219330 msgr2=0x55c7c5269ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:05.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.713+0000 7f7cdfa29740 1 --2- 172.21.15.1:0/1960950239 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c7c5219330 0x55c7c5269ea0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f7cc0011b90 tx=0x7f7cc0014000 comp rx=0 tx=0).stop 2024-05-09T08:03:05.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.713+0000 7f7cdfa29740 1 -- 172.21.15.1:0/1960950239 shutdown_connections 2024-05-09T08:03:05.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.713+0000 7f7cdfa29740 1 --2- 172.21.15.1:0/1960950239 >> [v2:172.21.15.1:6834/1509134635,v1:172.21.15.1:6836/1509134635] conn(0x55c7c52de790 0x55c7c52deb70 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:05.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.713+0000 7f7cdfa29740 1 --2- 172.21.15.1:0/1960950239 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c7c52572d0 0x55c7c5258550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:05.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.713+0000 7f7cbaa00640 0 client.4770 ms_handle_reset on v2:172.21.15.1:6834/1509134635 2024-05-09T08:03:05.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.713+0000 7f7cdfa29740 1 --2- 172.21.15.1:0/1960950239 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c7c5219330 0x55c7c5269ea0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:05.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.713+0000 7f7cdfa29740 1 --2- 172.21.15.1:0/1960950239 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x55c7c5217dd0 0x55c7c5259170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:05.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.713+0000 7f7cdfa29740 1 -- 172.21.15.1:0/1960950239 >> 172.21.15.1:0/1960950239 conn(0x55c7c51de8a0 msgr2=0x55c7c520d7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:03:05.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.713+0000 7f7cdfa29740 1 -- 172.21.15.1:0/1960950239 shutdown_connections 2024-05-09T08:03:05.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.713+0000 7f7cdfa29740 1 -- 172.21.15.1:0/1960950239 wait complete. 2024-05-09T08:03:05.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.713+0000 7f7cdfa29740 20 client.4770 trim_cache size 0 max 16384 2024-05-09T08:03:05.734 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-05-09T08:03:05.734 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_fsid": "54329ec8-3270-4043-9a82-c6cd7420aa7f", 2024-05-09T08:03:05.734 INFO:teuthology.orchestra.run.smithi001.stdout: "whoami": 0, 2024-05-09T08:03:05.735 INFO:teuthology.orchestra.run.smithi001.stdout: "id": 4294, 2024-05-09T08:03:05.735 INFO:teuthology.orchestra.run.smithi001.stdout: "want_state": "up:active", 2024-05-09T08:03:05.735 INFO:teuthology.orchestra.run.smithi001.stdout: "state": "up:active", 2024-05-09T08:03:05.735 INFO:teuthology.orchestra.run.smithi001.stdout: "fs_name": "cephfs", 2024-05-09T08:03:05.735 INFO:teuthology.orchestra.run.smithi001.stdout: "rank_uptime": 49.817636681000003, 2024-05-09T08:03:05.735 INFO:teuthology.orchestra.run.smithi001.stdout: "mdsmap_epoch": 10, 2024-05-09T08:03:05.735 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_epoch": 27, 2024-05-09T08:03:05.735 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_epoch_barrier": 25, 2024-05-09T08:03:05.735 INFO:teuthology.orchestra.run.smithi001.stdout: "uptime": 57.878648449000003 2024-05-09T08:03:05.735 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-05-09T08:03:05.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.714+0000 7f7cd7200640 1 -- 172.21.15.1:0/1546497864 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f7cb406be30 msgr2=0x7f7cb406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:05.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.714+0000 7f7cd7200640 1 --2- 172.21.15.1:0/1546497864 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f7cb406be30 0x7f7cb406e2f0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f7cc8034000 tx=0x7f7cc802e000 comp rx=0 tx=0).stop 2024-05-09T08:03:05.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.714+0000 7f7cd7200640 1 -- 172.21.15.1:0/1546497864 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7cd81513a0 msgr2=0x7f7cd80c10c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:05.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.714+0000 7f7cd7200640 1 --2- 172.21.15.1:0/1546497864 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7cd81513a0 0x7f7cd80c10c0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f7cd000cb40 tx=0x7f7cd000a060 comp rx=0 tx=0).stop 2024-05-09T08:03:05.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.714+0000 7f7cd7200640 1 -- 172.21.15.1:0/1546497864 shutdown_connections 2024-05-09T08:03:05.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.714+0000 7f7cd7200640 1 --2- 172.21.15.1:0/1546497864 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f7cb406be30 0x7f7cb406e2f0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:05.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.714+0000 7f7cd7200640 1 --2- 172.21.15.1:0/1546497864 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7cd81513a0 0x7f7cd80c10c0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:05.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.714+0000 7f7cd7200640 1 --2- 172.21.15.1:0/1546497864 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f7cd80c7b10 0x7f7cd80c0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:05.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.714+0000 7f7cd7200640 1 --2- 172.21.15.1:0/1546497864 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7cd80c5df0 0x7f7cd80cac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:05.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.714+0000 7f7cd7200640 1 -- 172.21.15.1:0/1546497864 >> 172.21.15.1:0/1546497864 conn(0x7f7cd806b250 msgr2=0x7f7cd8071e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:03:05.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.714+0000 7f7cd7200640 1 -- 172.21.15.1:0/1546497864 shutdown_connections 2024-05-09T08:03:05.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:05.714+0000 7f7cd7200640 1 -- 172.21.15.1:0/1546497864 wait complete. 2024-05-09T08:03:05.748 INFO:teuthology.run_tasks:Running task sequential... 2024-05-09T08:03:05.761 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-05-09T08:03:05.761 INFO:teuthology.task.sequential:In sequential, running task print... 2024-05-09T08:03:05.762 INFO:teuthology.task.print:Enabling mgr modules 2024-05-09T08:03:05.762 INFO:teuthology.run_tasks:Running task kclient... 2024-05-09T08:03:05.775 INFO:tasks.kclient:Mounting kernel clients... 2024-05-09T08:03:05.775 INFO:tasks.kclient:config is {'client.0': {}} 2024-05-09T08:03:05.775 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T08:03:05.775 DEBUG:teuthology.orchestra.run.smithi001:> ip netns list 2024-05-09T08:03:05.831 INFO:teuthology.orchestra.run.smithi001.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-05-09T08:03:05.833 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T08:03:05.833 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-09T08:03:05.926 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T08:03:05.926 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip link delete ceph-brx 2024-05-09T08:03:06.053 INFO:tasks.kclient:client.0 config is {'syntax': 'v1'} 2024-05-09T08:03:06.053 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-09T08:03:06.053 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-09T08:03:06.054 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-09T08:03:06.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:06.217+0000 7f0e12a00640 1 Processor -- start 2024-05-09T08:03:06.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:06.218+0000 7f0e12a00640 1 -- start start 2024-05-09T08:03:06.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:06.219+0000 7f0e12a00640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f0e0c0c8180 0x7f0e0c0cc730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:06.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:06.219+0000 7f0e12a00640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e0c15c790 con 0x7f0e0c152500 2024-05-09T08:03:06.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:06.219+0000 7f0e12a00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e0c0c6a30 con 0x7f0e0c0c7660 2024-05-09T08:03:06.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:06.219+0000 7f0e12a00640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f0e0c0c5e50 con 0x7f0e0c0c8180 2024-05-09T08:03:06.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:06.219+0000 7f0e10c00640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f0e0c0c8180 0x7f0e0c0cc730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:06.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:06.220+0000 7f0e10c00640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f0e0c0c8180 0x7f0e0c0cc730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:39652/0 (socket says 172.21.15.1:39652) 2024-05-09T08:03:06.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:06.220+0000 7f0e10c00640 1 -- 172.21.15.1:0/2792933566 learned_addr learned my addr 172.21.15.1:0/2792933566 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:03:06.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:06.220+0000 7f0e10c00640 1 -- 172.21.15.1:0/2792933566 >> v1:172.21.15.7:6789/0 conn(0x7f0e0c0c7660 legacy=0x7f0e0c162bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:06.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:06.220+0000 7f0e10c00640 1 -- 172.21.15.1:0/2792933566 >> v1:172.21.15.1:6789/0 conn(0x7f0e0c152500 legacy=0x7f0e0c15c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:06.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:06.220+0000 7f0e10c00640 1 -- 172.21.15.1:0/2792933566 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e0c0c6cf0 con 0x7f0e0c0c8180 2024-05-09T08:03:06.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:06.221+0000 7f0e10c00640 1 --2- 172.21.15.1:0/2792933566 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0e0c0c8180 0x7f0e0c0cc730 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f0df8009870 tx=0x7f0df802a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47d957f27cd5d3e5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:03:06.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:06.221+0000 7f0e0be00640 1 -- 172.21.15.1:0/2792933566 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0df8037070 con 0x7f0e0c0c8180 2024-05-09T08:03:06.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:06.221+0000 7f0e0be00640 1 -- 172.21.15.1:0/2792933566 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0df802ae50 con 0x7f0e0c0c8180 2024-05-09T08:03:06.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:06.221+0000 7f0e0be00640 1 -- 172.21.15.1:0/2792933566 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0df8032860 con 0x7f0e0c0c8180 2024-05-09T08:03:06.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:06.222+0000 7f0e12a00640 1 -- 172.21.15.1:0/2792933566 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0e0c0c8180 msgr2=0x7f0e0c0cc730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:06.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:06.222+0000 7f0e12a00640 1 --2- 172.21.15.1:0/2792933566 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0e0c0c8180 0x7f0e0c0cc730 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f0df8009870 tx=0x7f0df802a300 comp rx=0 tx=0).stop 2024-05-09T08:03:06.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:06.222+0000 7f0e12a00640 1 -- 172.21.15.1:0/2792933566 shutdown_connections 2024-05-09T08:03:06.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:06.222+0000 7f0e12a00640 1 --2- 172.21.15.1:0/2792933566 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0e0c0c8180 0x7f0e0c0cc730 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:06.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:06.222+0000 7f0e12a00640 1 -- 172.21.15.1:0/2792933566 >> 172.21.15.1:0/2792933566 conn(0x7f0e0c0cbc10 msgr2=0x7f0e0c0cc020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:03:06.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:06.222+0000 7f0e12a00640 1 -- 172.21.15.1:0/2792933566 shutdown_connections 2024-05-09T08:03:06.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:06.222+0000 7f0e12a00640 1 -- 172.21.15.1:0/2792933566 wait complete. 2024-05-09T08:03:06.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:06.224+0000 7f0e12a00640 1 Processor -- start 2024-05-09T08:03:06.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:06.224+0000 7f0e12a00640 1 -- start start 2024-05-09T08:03:06.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:06.225+0000 7f0e12a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e0c0c7660 0x7f0e0c0c5010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:06.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:06.225+0000 7f0e12a00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f0e0c0c8180 0x7f0e0c15b030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:06.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:06.225+0000 7f0e12a00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0e0c152500 0x7f0e0c14d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:06.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:06.225+0000 7f0e12a00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f0e0c0c72b0 con 0x7f0e0c152500 2024-05-09T08:03:06.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:06.225+0000 7f0e12a00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0e0c0c22d0 con 0x7f0e0c0c7660 2024-05-09T08:03:06.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:06.225+0000 7f0e12a00640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f0e0c15b880 con 0x7f0e0c0c8180 2024-05-09T08:03:06.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:06.225+0000 7f0e12000640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0e0c152500 0x7f0e0c14d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:06.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:06.225+0000 7f0e12000640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0e0c152500 0x7f0e0c14d790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:39668/0 (socket says 172.21.15.1:39668) 2024-05-09T08:03:06.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:06.225+0000 7f0e12000640 1 -- 172.21.15.1:0/4165021421 learned_addr learned my addr 172.21.15.1:0/4165021421 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:03:06.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:06.226+0000 7f0e10c00640 1 --2- 172.21.15.1:0/4165021421 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f0e0c0c8180 0x7f0e0c15b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:06.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:06.226+0000 7f0e11600640 1 --2- 172.21.15.1:0/4165021421 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e0c0c7660 0x7f0e0c0c5010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:06.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:06.226+0000 7f0e12000640 1 -- 172.21.15.1:0/4165021421 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f0e0c0c8180 msgr2=0x7f0e0c15b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:06.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:06.226+0000 7f0e12000640 1 --2- 172.21.15.1:0/4165021421 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f0e0c0c8180 0x7f0e0c15b030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:06.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:06.226+0000 7f0e12000640 1 -- 172.21.15.1:0/4165021421 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e0c0c7660 msgr2=0x7f0e0c0c5010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:06.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:06.226+0000 7f0e12000640 1 --2- 172.21.15.1:0/4165021421 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e0c0c7660 0x7f0e0c0c5010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:06.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:06.226+0000 7f0e12000640 1 -- 172.21.15.1:0/4165021421 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e0c1f32e0 con 0x7f0e0c152500 2024-05-09T08:03:06.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:06.226+0000 7f0e12000640 1 --2- 172.21.15.1:0/4165021421 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0e0c152500 0x7f0e0c14d790 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f0e0400c7a0 tx=0x7f0e0400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:03:06.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:06.227+0000 7f0e09600640 1 -- 172.21.15.1:0/4165021421 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e04003b70 con 0x7f0e0c152500 2024-05-09T08:03:06.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:06.227+0000 7f0e09600640 1 -- 172.21.15.1:0/4165021421 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e04003d30 con 0x7f0e0c152500 2024-05-09T08:03:06.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:06.227+0000 7f0e09600640 1 -- 172.21.15.1:0/4165021421 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e0400aa90 con 0x7f0e0c152500 2024-05-09T08:03:06.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:06.227+0000 7f0e12a00640 1 -- 172.21.15.1:0/4165021421 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e0c1f35c0 con 0x7f0e0c152500 2024-05-09T08:03:06.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:06.227+0000 7f0e12a00640 1 -- 172.21.15.1:0/4165021421 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e0c1f3b80 con 0x7f0e0c152500 2024-05-09T08:03:06.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:06.228+0000 7f0e12a00640 1 -- 172.21.15.1:0/4165021421 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e0c04f660 con 0x7f0e0c152500 2024-05-09T08:03:06.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:06.229+0000 7f0e09600640 1 -- 172.21.15.1:0/4165021421 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0e0400ac30 con 0x7f0e0c152500 2024-05-09T08:03:06.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:06.230+0000 7f0e09600640 1 --2- 172.21.15.1:0/4165021421 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f0dec06be30 0x7f0dec06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:06.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:06.230+0000 7f0e09600640 1 -- 172.21.15.1:0/4165021421 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7f0e0408b070 con 0x7f0e0c152500 2024-05-09T08:03:06.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:06.230+0000 7f0e11600640 1 --2- 172.21.15.1:0/4165021421 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f0dec06be30 0x7f0dec06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:06.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:06.231+0000 7f0e11600640 1 --2- 172.21.15.1:0/4165021421 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f0dec06be30 0x7f0dec06e2f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f0e000016e0 tx=0x7f0e00009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:03:06.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:06.234+0000 7f0e09600640 1 -- 172.21.15.1:0/4165021421 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f0e04058fd0 con 0x7f0e0c152500 2024-05-09T08:03:06.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:06.482+0000 7f0e12a00640 1 -- 172.21.15.1:0/4165021421 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f0e0c0c2a40 con 0x7f0e0c152500 2024-05-09T08:03:06.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:06.483+0000 7f0e09600640 1 -- 172.21.15.1:0/4165021421 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v10) ==== 53+0+88 (secure 0 0 0) 0x7f0e0405d2d0 con 0x7f0e0c152500 2024-05-09T08:03:06.500 INFO:teuthology.orchestra.run.smithi001.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-05-09T08:03:06.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:06.486+0000 7f0e12a00640 1 -- 172.21.15.1:0/4165021421 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f0dec06be30 msgr2=0x7f0dec06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:06.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:06.486+0000 7f0e12a00640 1 --2- 172.21.15.1:0/4165021421 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f0dec06be30 0x7f0dec06e2f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f0e000016e0 tx=0x7f0e00009040 comp rx=0 tx=0).stop 2024-05-09T08:03:06.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:06.486+0000 7f0e12a00640 1 -- 172.21.15.1:0/4165021421 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0e0c152500 msgr2=0x7f0e0c14d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:06.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:06.486+0000 7f0e12a00640 1 --2- 172.21.15.1:0/4165021421 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0e0c152500 0x7f0e0c14d790 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f0e0400c7a0 tx=0x7f0e0400cc60 comp rx=0 tx=0).stop 2024-05-09T08:03:06.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:06.486+0000 7f0e12a00640 1 -- 172.21.15.1:0/4165021421 shutdown_connections 2024-05-09T08:03:06.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:06.486+0000 7f0e12a00640 1 --2- 172.21.15.1:0/4165021421 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f0dec06be30 0x7f0dec06e2f0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:06.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:06.486+0000 7f0e12a00640 1 --2- 172.21.15.1:0/4165021421 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0e0c152500 0x7f0e0c14d790 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:06.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:06.486+0000 7f0e12a00640 1 --2- 172.21.15.1:0/4165021421 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f0e0c0c8180 0x7f0e0c15b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:06.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:06.486+0000 7f0e12a00640 1 --2- 172.21.15.1:0/4165021421 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e0c0c7660 0x7f0e0c0c5010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:06.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:06.486+0000 7f0e12a00640 1 -- 172.21.15.1:0/4165021421 >> 172.21.15.1:0/4165021421 conn(0x7f0e0c0cbc10 msgr2=0x7f0e0c0cac80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:03:06.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:06.487+0000 7f0e12a00640 1 -- 172.21.15.1:0/4165021421 shutdown_connections 2024-05-09T08:03:06.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:06.487+0000 7f0e12a00640 1 -- 172.21.15.1:0/4165021421 wait complete. 2024-05-09T08:03:06.522 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-09T08:03:06.522 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-09T08:03:06.522 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi001.front.sepia.ceph.com 2024-05-09T08:03:06.522 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-09T08:03:06.522 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-09T08:03:06.522 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-09T08:03:06.522 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-09T08:03:06.522 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-09T08:03:06.522 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-09T08:03:06.523 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T08:03:06.523 DEBUG:teuthology.orchestra.run.smithi001:> ip addr 2024-05-09T08:03:06.575 INFO:teuthology.orchestra.run.smithi001.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-09T08:03:06.575 INFO:teuthology.orchestra.run.smithi001.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-09T08:03:06.576 INFO:teuthology.orchestra.run.smithi001.stdout: inet 127.0.0.1/8 scope host lo 2024-05-09T08:03:06.576 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft forever preferred_lft forever 2024-05-09T08:03:06.576 INFO:teuthology.orchestra.run.smithi001.stdout: inet6 ::1/128 scope host 2024-05-09T08:03:06.576 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft forever preferred_lft forever 2024-05-09T08:03:06.576 INFO:teuthology.orchestra.run.smithi001.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-09T08:03:06.576 INFO:teuthology.orchestra.run.smithi001.stdout: link/ether 0c:c4:7a:6b:f6:7a brd ff:ff:ff:ff:ff:ff 2024-05-09T08:03:06.576 INFO:teuthology.orchestra.run.smithi001.stdout: altname enp7s0f0 2024-05-09T08:03:06.576 INFO:teuthology.orchestra.run.smithi001.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-09T08:03:06.576 INFO:teuthology.orchestra.run.smithi001.stdout: link/ether 0c:c4:7a:6b:f6:7b brd ff:ff:ff:ff:ff:ff 2024-05-09T08:03:06.576 INFO:teuthology.orchestra.run.smithi001.stdout: altname enp7s0f1 2024-05-09T08:03:06.576 INFO:teuthology.orchestra.run.smithi001.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-09T08:03:06.576 INFO:teuthology.orchestra.run.smithi001.stdout: link/ether 0c:c4:7a:bd:15:e8 brd ff:ff:ff:ff:ff:ff 2024-05-09T08:03:06.577 INFO:teuthology.orchestra.run.smithi001.stdout: altname enp1s0f0 2024-05-09T08:03:06.577 INFO:teuthology.orchestra.run.smithi001.stdout: inet 172.21.15.1/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-05-09T08:03:06.577 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft 42603sec preferred_lft 42603sec 2024-05-09T08:03:06.577 INFO:teuthology.orchestra.run.smithi001.stdout: inet6 fe80::ec4:7aff:febd:15e8/64 scope link noprefixroute 2024-05-09T08:03:06.577 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft forever preferred_lft forever 2024-05-09T08:03:06.577 INFO:teuthology.orchestra.run.smithi001.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-09T08:03:06.577 INFO:teuthology.orchestra.run.smithi001.stdout: link/ether 0c:c4:7a:bd:15:e9 brd ff:ff:ff:ff:ff:ff 2024-05-09T08:03:06.577 INFO:teuthology.orchestra.run.smithi001.stdout: altname enp1s0f1 2024-05-09T08:03:06.577 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-05-09T08:03:06.577 DEBUG:teuthology.orchestra.run.smithi001:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T08:03:06.578 DEBUG:teuthology.orchestra.run.smithi001:> set -e 2024-05-09T08:03:06.578 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip link add name ceph-brx type bridge 2024-05-09T08:03:06.578 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip addr flush dev ceph-brx 2024-05-09T08:03:06.578 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip link set ceph-brx up 2024-05-09T08:03:06.578 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-05-09T08:03:06.578 DEBUG:teuthology.orchestra.run.smithi001:> ') 2024-05-09T08:03:06.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:06 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T08:03:06.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T08:03:06.830 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T08:03:06.830 DEBUG:teuthology.orchestra.run.smithi001:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-09T08:03:06.895 INFO:teuthology.orchestra.run.smithi001.stdout:1 2024-05-09T08:03:06.898 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T08:03:06.898 DEBUG:teuthology.orchestra.run.smithi001:> route 2024-05-09T08:03:06.949 INFO:teuthology.orchestra.run.smithi001.stdout:Kernel IP routing table 2024-05-09T08:03:06.949 INFO:teuthology.orchestra.run.smithi001.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-09T08:03:06.949 INFO:teuthology.orchestra.run.smithi001.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-05-09T08:03:06.949 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-05-09T08:03:06.949 INFO:teuthology.orchestra.run.smithi001.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-09T08:03:06.950 DEBUG:teuthology.orchestra.run.smithi001:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T08:03:06.950 DEBUG:teuthology.orchestra.run.smithi001:> set -e 2024-05-09T08:03:06.950 DEBUG:teuthology.orchestra.run.smithi001:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-05-09T08:03:06.950 DEBUG:teuthology.orchestra.run.smithi001:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-05-09T08:03:06.951 DEBUG:teuthology.orchestra.run.smithi001:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-05-09T08:03:06.951 DEBUG:teuthology.orchestra.run.smithi001:> ') 2024-05-09T08:03:07.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:07 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T08:03:07.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T08:03:07.175 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T08:03:07.175 DEBUG:teuthology.orchestra.run.smithi001:> ip netns list 2024-05-09T08:03:07.221 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T08:03:07.221 DEBUG:teuthology.orchestra.run.smithi001:> ip netns list-id 2024-05-09T08:03:07.311 DEBUG:teuthology.orchestra.run.smithi001:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T08:03:07.312 DEBUG:teuthology.orchestra.run.smithi001:> set -e 2024-05-09T08:03:07.312 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-09T08:03:07.312 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-05-09T08:03:07.312 DEBUG:teuthology.orchestra.run.smithi001:> ') 2024-05-09T08:03:07.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:07 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T08:03:07.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T08:03:07.495 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-05-09T08:03:07.495 DEBUG:teuthology.orchestra.run.smithi001:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T08:03:07.495 DEBUG:teuthology.orchestra.run.smithi001:> set -e 2024-05-09T08:03:07.495 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-05-09T08:03:07.495 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-05-09T08:03:07.495 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-05-09T08:03:07.495 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-05-09T08:03:07.495 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-05-09T08:03:07.495 DEBUG:teuthology.orchestra.run.smithi001:> ') 2024-05-09T08:03:07.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:07 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T08:03:07.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T08:03:07.751 DEBUG:teuthology.orchestra.run.smithi001:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T08:03:07.751 DEBUG:teuthology.orchestra.run.smithi001:> set -e 2024-05-09T08:03:07.751 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip link set brx.0 up 2024-05-09T08:03:07.751 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip link set dev brx.0 master ceph-brx 2024-05-09T08:03:07.751 DEBUG:teuthology.orchestra.run.smithi001:> ') 2024-05-09T08:03:07.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:07 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T08:03:07.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T08:03:07.944 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-09T08:03:07.944 DEBUG:teuthology.orchestra.run.smithi001:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-09T08:03:07.986 INFO:teuthology.orchestra.run.smithi001.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-09T08:03:07.987 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-09T08:03:07.987 DEBUG:teuthology.orchestra.run.smithi001:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-09T08:03:08.079 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-05-09T08:03:08.079 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T08:03:08.079 DEBUG:teuthology.orchestra.run.smithi001:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-09T08:03:08.190 INFO:teuthology.orchestra.run.smithi001.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-09T08:03:08.190 INFO:teuthology.orchestra.run.smithi001.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-09T08:03:08.190 INFO:teuthology.orchestra.run.smithi001.stdout:invalid new device string format 2024-05-09T08:03:08.929 INFO:teuthology.orchestra.run.smithi001.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-09T08:03:08.930 INFO:teuthology.orchestra.run.smithi001.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-09T08:03:08.930 INFO:teuthology.orchestra.run.smithi001.stdout:invalid new device string format 2024-05-09T08:03:08.930 INFO:teuthology.orchestra.run.smithi001.stdout:mount.ceph: resolved to: "172.21.15.1:3300,172.21.15.7:3300,172.21.15.7:3301" 2024-05-09T08:03:08.930 INFO:teuthology.orchestra.run.smithi001.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-09T08:03:08.963 INFO:teuthology.orchestra.run.smithi001.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-09T08:03:08.963 INFO:teuthology.orchestra.run.smithi001.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-09T08:03:08.963 INFO:teuthology.orchestra.run.smithi001.stdout:invalid new device string format 2024-05-09T08:03:08.963 INFO:teuthology.orchestra.run.smithi001.stdout:mount.ceph: resolved to: "172.21.15.1:3300,172.21.15.7:3300,172.21.15.7:3301" 2024-05-09T08:03:08.963 INFO:teuthology.orchestra.run.smithi001.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-09T08:03:08.964 INFO:teuthology.orchestra.run.smithi001.stdout:mount.ceph: resolved to: "172.21.15.1:6789,172.21.15.7:6789,172.21.15.7:6790" 2024-05-09T08:03:08.964 INFO:teuthology.orchestra.run.smithi001.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.1:6789,172.21.15.7:6789,172.21.15.7:6790:/ 2024-05-09T08:03:08.964 INFO:teuthology.orchestra.run.smithi001.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=54329ec8-3270-4043-9a82-c6cd7420aa7f" will pass to kernel 2024-05-09T08:03:08.970 INFO:teuthology.orchestra.run.smithi001.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-05-09T08:03:08.970 INFO:teuthology.orchestra.run.smithi001.stdout: You just mounted a file system that supports labels which does not 2024-05-09T08:03:08.970 INFO:teuthology.orchestra.run.smithi001.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-05-09T08:03:08.970 INFO:teuthology.orchestra.run.smithi001.stdout: applications will generate AVC messages and not be allowed access to 2024-05-09T08:03:08.970 INFO:teuthology.orchestra.run.smithi001.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-05-09T08:03:08.975 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-09T08:03:08.975 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T08:03:08.975 DEBUG:teuthology.orchestra.run.smithi001:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-09T08:03:09.061 DEBUG:teuthology.orchestra.run.smithi001:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-09T08:03:09.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:09 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-09T08:03:09.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T08:03:09.193 INFO:teuthology.orchestra.run.smithi001.stdout:client4903 2024-05-09T08:03:09.193 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-05-09T08:03:09.193 DEBUG:teuthology.orchestra.run.smithi001:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T08:03:09.249 DEBUG:teuthology.orchestra.run.smithi001:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-09T08:03:09.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:09 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-09T08:03:09.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T08:03:09.375 INFO:teuthology.orchestra.run.smithi001.stdout:client4903 2024-05-09T08:03:09.375 DEBUG:teuthology.orchestra.run.smithi001:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-09T08:03:09.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:09 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-09T08:03:09.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T08:03:09.497 INFO:teuthology.orchestra.run.smithi001.stdout:client4903 2024-05-09T08:03:09.497 DEBUG:teuthology.orchestra.run.smithi001:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/54329ec8-3270-4043-9a82-c6cd7420aa7f.client4903/status') 2024-05-09T08:03:09.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:09 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T08:03:09.639 INFO:teuthology.orchestra.run.smithi001.stdout:instance: client.4903 (3)172.21.15.1:0/1236213124 2024-05-09T08:03:09.639 INFO:teuthology.orchestra.run.smithi001.stdout:blocklisted: false 2024-05-09T08:03:09.639 INFO:teuthology.orchestra.run.smithi001.stderr:0+1 records in 2024-05-09T08:03:09.639 INFO:teuthology.orchestra.run.smithi001.stderr:0+1 records out 2024-05-09T08:03:09.640 INFO:teuthology.orchestra.run.smithi001.stderr:69 bytes copied, 8.9545e-05 s, 771 kB/s 2024-05-09T08:03:09.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T08:03:09.653 INFO:teuthology.run_tasks:Running task mon_thrash... 2024-05-09T08:03:09.671 INFO:tasks.mon_thrash:Beginning mon_thrash... 2024-05-09T08:03:09.671 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T08:03:09.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:09.867+0000 7f54cce00640 1 Processor -- start 2024-05-09T08:03:09.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:09.868+0000 7f54cce00640 1 -- start start 2024-05-09T08:03:09.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:09.869+0000 7f54cce00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f54c81548e0 0x7f54c80cc400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:09.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:09.869+0000 7f54cce00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f54c80cb760 0x7f54c815ab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:09.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:09.869+0000 7f54cce00640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f54c815e6f0 con 0x7f54c80cbf20 2024-05-09T08:03:09.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:09.869+0000 7f54cce00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f54c815ec60 con 0x7f54c80cb760 2024-05-09T08:03:09.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:09.869+0000 7f54cce00640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f54c815e980 con 0x7f54c81548e0 2024-05-09T08:03:09.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:09.869+0000 7f54c5400640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f54c80cbf20 0x7f54c808bff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:42716/0 (socket says 172.21.15.1:42716) 2024-05-09T08:03:09.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:09.869+0000 7f54c5400640 1 -- 172.21.15.1:0/2530535580 learned_addr learned my addr 172.21.15.1:0/2530535580 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T08:03:09.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:09.869+0000 7f54c5e00640 1 --2- 172.21.15.1:0/2530535580 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f54c81548e0 0x7f54c80cc400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:09.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:09.869+0000 7f54c6800640 1 --2- 172.21.15.1:0/2530535580 >> v2:172.21.15.7:3300/0 conn(0x7f54c80cb760 0x7f54c815ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:09.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:09.870+0000 7f54c6800640 1 -- 172.21.15.1:0/2530535580 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f54c81548e0 msgr2=0x7f54c80cc400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:03:09.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:09.870+0000 7f54c6800640 1 --2- 172.21.15.1:0/2530535580 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f54c81548e0 0x7f54c80cc400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:09.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:09.870+0000 7f54c6800640 1 -- 172.21.15.1:0/2530535580 >> v1:172.21.15.1:6789/0 conn(0x7f54c80cbf20 legacy=0x7f54c808bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:09.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:09.870+0000 7f54c6800640 1 -- 172.21.15.1:0/2530535580 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54c80caa10 con 0x7f54c80cb760 2024-05-09T08:03:09.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:09.870+0000 7f54c4a00640 1 -- 172.21.15.1:0/2530535580 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3516273868 0 0) 0x7f54c815e6f0 con 0x7f54c80cbf20 2024-05-09T08:03:09.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:09.871+0000 7f54c6800640 1 --2- 172.21.15.1:0/2530535580 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54c80cb760 0x7f54c815ab80 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f54c0009d90 tx=0x7f54c002a5d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=10e7f02012ec1c3e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:03:09.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:09.871+0000 7f54c4a00640 1 -- 172.21.15.1:0/2530535580 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54c0031600 con 0x7f54c80cb760 2024-05-09T08:03:09.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:09.871+0000 7f54c4a00640 1 -- 172.21.15.1:0/2530535580 <== mon.1 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54c00317a0 con 0x7f54c80cb760 2024-05-09T08:03:09.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:09.871+0000 7f54c4a00640 1 -- 172.21.15.1:0/2530535580 <== mon.1 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54c0031b70 con 0x7f54c80cb760 2024-05-09T08:03:09.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:09.872+0000 7f54cce00640 1 -- 172.21.15.1:0/2530535580 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54c80cb760 msgr2=0x7f54c815ab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:09.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:09.872+0000 7f54cce00640 1 --2- 172.21.15.1:0/2530535580 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54c80cb760 0x7f54c815ab80 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f54c0009d90 tx=0x7f54c002a5d0 comp rx=0 tx=0).stop 2024-05-09T08:03:09.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:09.872+0000 7f54cce00640 1 -- 172.21.15.1:0/2530535580 shutdown_connections 2024-05-09T08:03:09.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:09.872+0000 7f54cce00640 1 --2- 172.21.15.1:0/2530535580 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54c80cb760 0x7f54c815ab80 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:09.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:09.872+0000 7f54cce00640 1 --2- 172.21.15.1:0/2530535580 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f54c81548e0 0x7f54c80cc400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:09.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:09.872+0000 7f54cce00640 1 -- 172.21.15.1:0/2530535580 >> 172.21.15.1:0/2530535580 conn(0x7f54c8073010 msgr2=0x7f54c806c880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:03:09.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:09.872+0000 7f54cce00640 1 -- 172.21.15.1:0/2530535580 shutdown_connections 2024-05-09T08:03:09.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:09.872+0000 7f54cce00640 1 -- 172.21.15.1:0/2530535580 wait complete. 2024-05-09T08:03:09.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:09.874+0000 7f54cce00640 1 Processor -- start 2024-05-09T08:03:09.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:09.874+0000 7f54cce00640 1 -- start start 2024-05-09T08:03:09.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:09.875+0000 7f54cce00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54c80cb760 0x7f54c80b4500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:09.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:09.875+0000 7f54cce00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f54c80cbf20 0x7f54c81f3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:09.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:09.875+0000 7f54cce00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f54c81548e0 0x7f54c81f4000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:09.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:09.875+0000 7f54cce00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f54c8154e40 con 0x7f54c80cbf20 2024-05-09T08:03:09.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:09.875+0000 7f54cce00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f54c8154cc0 con 0x7f54c80cb760 2024-05-09T08:03:09.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:09.875+0000 7f54cce00640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f54c808c700 con 0x7f54c81548e0 2024-05-09T08:03:09.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:09.875+0000 7f54c5400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f54c80cbf20 0x7f54c81f3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:09.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:09.875+0000 7f54c5400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f54c80cbf20 0x7f54c81f3ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:39676/0 (socket says 172.21.15.1:39676) 2024-05-09T08:03:09.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:09.875+0000 7f54c5400640 1 -- 172.21.15.1:0/3593643741 learned_addr learned my addr 172.21.15.1:0/3593643741 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:03:09.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:09.875+0000 7f54c5e00640 1 --2- 172.21.15.1:0/3593643741 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54c80cb760 0x7f54c80b4500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:09.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:09.875+0000 7f54c6800640 1 --2- 172.21.15.1:0/3593643741 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f54c81548e0 0x7f54c81f4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:09.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:09.876+0000 7f54c5e00640 1 -- 172.21.15.1:0/3593643741 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f54c81548e0 msgr2=0x7f54c81f4000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:09.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:09.876+0000 7f54c5e00640 1 --2- 172.21.15.1:0/3593643741 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f54c81548e0 0x7f54c81f4000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:09.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:09.876+0000 7f54c5e00640 1 -- 172.21.15.1:0/3593643741 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f54c80cbf20 msgr2=0x7f54c81f3ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:09.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:09.876+0000 7f54c5e00640 1 --2- 172.21.15.1:0/3593643741 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f54c80cbf20 0x7f54c81f3ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:09.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:09.876+0000 7f54c5e00640 1 -- 172.21.15.1:0/3593643741 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54c814caa0 con 0x7f54c80cb760 2024-05-09T08:03:09.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:09.876+0000 7f54c5e00640 1 --2- 172.21.15.1:0/3593643741 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54c80cb760 0x7f54c80b4500 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f54b800e8b0 tx=0x7f54b800ed70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:03:09.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:09.876+0000 7f54be000640 1 -- 172.21.15.1:0/3593643741 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54b8017070 con 0x7f54c80cb760 2024-05-09T08:03:09.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:09.876+0000 7f54be000640 1 -- 172.21.15.1:0/3593643741 <== mon.1 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54b800c940 con 0x7f54c80cb760 2024-05-09T08:03:09.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:09.876+0000 7f54be000640 1 -- 172.21.15.1:0/3593643741 <== mon.1 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54b800cd30 con 0x7f54c80cb760 2024-05-09T08:03:09.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:09.876+0000 7f54cce00640 1 -- 172.21.15.1:0/3593643741 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54c814b960 con 0x7f54c80cb760 2024-05-09T08:03:09.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:09.876+0000 7f54cce00640 1 -- 172.21.15.1:0/3593643741 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54c80c9ba0 con 0x7f54c80cb760 2024-05-09T08:03:09.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:09.877+0000 7f54cce00640 1 -- 172.21.15.1:0/3593643741 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54c80ca390 con 0x7f54c80cb760 2024-05-09T08:03:09.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:09.878+0000 7f54be000640 1 -- 172.21.15.1:0/3593643741 <== mon.1 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f54b801a070 con 0x7f54c80cb760 2024-05-09T08:03:09.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:09.879+0000 7f54be000640 1 --2- 172.21.15.1:0/3593643741 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f54a406be30 0x7f54a406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:09.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:09.879+0000 7f54be000640 1 -- 172.21.15.1:0/3593643741 <== mon.1 v2:172.21.15.7:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7f54b808b170 con 0x7f54c80cb760 2024-05-09T08:03:09.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:09.880+0000 7f54c5400640 1 --2- 172.21.15.1:0/3593643741 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f54a406be30 0x7f54a406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:09.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:09.880+0000 7f54c5400640 1 --2- 172.21.15.1:0/3593643741 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f54a406be30 0x7f54a406e2f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f54c80d0f40 tx=0x7f54b0007e70 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:03:09.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:09.884+0000 7f54be000640 1 -- 172.21.15.1:0/3593643741 <== mon.1 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f54b8059040 con 0x7f54c80cb760 2024-05-09T08:03:10.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:10.090+0000 7f54cce00640 1 -- 172.21.15.1:0/3593643741 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f54c815e680 con 0x7f54c80cb760 2024-05-09T08:03:10.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:10.093+0000 7f54be000640 1 -- 172.21.15.1:0/3593643741 <== mon.1 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18741 (secure 0 0 0) 0x7f54b805d340 con 0x7f54c80cb760 2024-05-09T08:03:10.110 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T08:03:10.110 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":27,"fsid":"54329ec8-3270-4043-9a82-c6cd7420aa7f","created":"2024-05-09T08:01:56.666970+0000","modified":"2024-05-09T08:03:02.304156+0000","last_up_change":"2024-05-09T08:02:07.146082+0000","last_in_change":"2024-05-09T08:01:59.008225+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T08:02:09.165511+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T08:02:12.925485+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T08:02:14.367455+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-09T08:02:17.967064+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c064454e-a9b5-4a24-9a77-17f0a4085952","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6800","nonce":1513349547},{"type":"v1","addr":"172.21.15.1:6801","nonce":1513349547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":1513349547},{"type":"v1","addr":"172.21.15.1:6803","nonce":1513349547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":1513349547},{"type":"v1","addr":"172.21.15.1:6807","nonce":1513349547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":1513349547},{"type":"v1","addr":"172.21.15.1:6805","nonce":1513349547}]},"public_addr":"172.21.15.1:6801/1513349547","cluster_addr":"172.21.15.1:6803/1513349547","heartbeat_back_addr":"172.21.15.1:6807/1513349547","heartbeat_front_addr":"172.21.15.1:6805/1513349547","state":["exists","up"]},{"osd":1,"uuid":"1a9ea687-2ff6-407a-949b-0e7e1083af21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":2942255259},{"type":"v1","addr":"172.21.15.1:6817","nonce":2942255259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":2942255259},{"type":"v1","addr":"172.21.15.1:6819","nonce":2942255259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":2942255259},{"type":"v1","addr":"172.21.15.1:6823","nonce":2942255259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6820","nonce":2942255259},{"type":"v1","addr":"172.21.15.1:6821","nonce":2942255259}]},"public_addr":"172.21.15.1:6817/2942255259","cluster_addr":"172.21.15.1:6819/2942255259","heartbeat_back_addr":"172.21.15.1:6823/2942255259","heartbeat_front_addr":"172.21.15.1:6821/2942255259","state":["exists","up"]},{"osd":2,"uuid":"c88af08b-7c9a-4dd8-9640-44138239addb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":2925031811},{"type":"v1","addr":"172.21.15.1:6809","nonce":2925031811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":2925031811},{"type":"v1","addr":"172.21.15.1:6811","nonce":2925031811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":2925031811},{"type":"v1","addr":"172.21.15.1:6815","nonce":2925031811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":2925031811},{"type":"v1","addr":"172.21.15.1:6813","nonce":2925031811}]},"public_addr":"172.21.15.1:6809/2925031811","cluster_addr":"172.21.15.1:6811/2925031811","heartbeat_back_addr":"172.21.15.1:6815/2925031811","heartbeat_front_addr":"172.21.15.1:6813/2925031811","state":["exists","up"]},{"osd":3,"uuid":"c087b7e3-a41d-4333-9365-32e3bd8c8e6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6824","nonce":2497776654},{"type":"v1","addr":"172.21.15.1:6825","nonce":2497776654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":2497776654},{"type":"v1","addr":"172.21.15.1:6827","nonce":2497776654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":2497776654},{"type":"v1","addr":"172.21.15.1:6831","nonce":2497776654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6828","nonce":2497776654},{"type":"v1","addr":"172.21.15.1:6829","nonce":2497776654}]},"public_addr":"172.21.15.1:6825/2497776654","cluster_addr":"172.21.15.1:6827/2497776654","heartbeat_back_addr":"172.21.15.1:6831/2497776654","heartbeat_front_addr":"172.21.15.1:6829/2497776654","state":["exists","up"]},{"osd":4,"uuid":"b4576b9c-871d-47a8-8d36-65af525d0200","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":2957026670},{"type":"v1","addr":"172.21.15.7:6801","nonce":2957026670}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2957026670},{"type":"v1","addr":"172.21.15.7:6803","nonce":2957026670}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":2957026670},{"type":"v1","addr":"172.21.15.7:6807","nonce":2957026670}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2957026670},{"type":"v1","addr":"172.21.15.7:6805","nonce":2957026670}]},"public_addr":"172.21.15.7:6801/2957026670","cluster_addr":"172.21.15.7:6803/2957026670","heartbeat_back_addr":"172.21.15.7:6807/2957026670","heartbeat_front_addr":"172.21.15.7:6805/2957026670","state":["exists","up"]},{"osd":5,"uuid":"902b3d94-72f3-4499-b21a-e32453ef5144","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1670295242},{"type":"v1","addr":"172.21.15.7:6809","nonce":1670295242}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1670295242},{"type":"v1","addr":"172.21.15.7:6811","nonce":1670295242}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1670295242},{"type":"v1","addr":"172.21.15.7:6815","nonce":1670295242}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1670295242},{"type":"v1","addr":"172.21.15.7:6813","nonce":1670295242}]},"public_addr":"172.21.15.7:6809/1670295242","cluster_addr":"172.21.15.7:6811/1670295242","heartbeat_back_addr":"172.21.15.7:6815/1670295242","heartbeat_front_addr":"172.21.15.7:6813/1670295242","state":["exists","up"]},{"osd":6,"uuid":"ead1d771-1728-4f85-adb4-26be21cedff3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":2852086880},{"type":"v1","addr":"172.21.15.7:6817","nonce":2852086880}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":2852086880},{"type":"v1","addr":"172.21.15.7:6819","nonce":2852086880}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":2852086880},{"type":"v1","addr":"172.21.15.7:6823","nonce":2852086880}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":2852086880},{"type":"v1","addr":"172.21.15.7:6821","nonce":2852086880}]},"public_addr":"172.21.15.7:6817/2852086880","cluster_addr":"172.21.15.7:6819/2852086880","heartbeat_back_addr":"172.21.15.7:6823/2852086880","heartbeat_front_addr":"172.21.15.7:6821/2852086880","state":["exists","up"]},{"osd":7,"uuid":"3ab5a201-9394-48a8-8918-e95197893669","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":1442884138},{"type":"v1","addr":"172.21.15.7:6825","nonce":1442884138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1442884138},{"type":"v1","addr":"172.21.15.7:6827","nonce":1442884138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1442884138},{"type":"v1","addr":"172.21.15.7:6831","nonce":1442884138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1442884138},{"type":"v1","addr":"172.21.15.7:6829","nonce":1442884138}]},"public_addr":"172.21.15.7:6825/1442884138","cluster_addr":"172.21.15.7:6827/1442884138","heartbeat_back_addr":"172.21.15.7:6831/1442884138","heartbeat_front_addr":"172.21.15.7:6829/1442884138","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:02:04.615675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:02:04.385974+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:02:04.619217+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:02:04.401681+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:02:04.097081+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:02:04.129981+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:02:04.225327+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:02:04.193351+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.35","mappings":[{"from":5,"to":2}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2},{"from":3,"to":1}]},{"pgid":"3.3f","mappings":[{"from":7,"to":2}]},{"pgid":"4.4","mappings":[{"from":1,"to":5}]},{"pgid":"4.e","mappings":[{"from":1,"to":2}]},{"pgid":"4.f","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T08:03:10.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:10.095+0000 7f54cce00640 1 -- 172.21.15.1:0/3593643741 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f54a406be30 msgr2=0x7f54a406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:10.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:10.095+0000 7f54cce00640 1 --2- 172.21.15.1:0/3593643741 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f54a406be30 0x7f54a406e2f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f54c80d0f40 tx=0x7f54b0007e70 comp rx=0 tx=0).stop 2024-05-09T08:03:10.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:10.095+0000 7f54cce00640 1 -- 172.21.15.1:0/3593643741 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54c80cb760 msgr2=0x7f54c80b4500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:10.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:10.095+0000 7f54cce00640 1 --2- 172.21.15.1:0/3593643741 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54c80cb760 0x7f54c80b4500 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f54b800e8b0 tx=0x7f54b800ed70 comp rx=0 tx=0).stop 2024-05-09T08:03:10.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:10.095+0000 7f54cce00640 1 -- 172.21.15.1:0/3593643741 shutdown_connections 2024-05-09T08:03:10.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:10.095+0000 7f54cce00640 1 --2- 172.21.15.1:0/3593643741 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f54a406be30 0x7f54a406e2f0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:10.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:10.095+0000 7f54cce00640 1 --2- 172.21.15.1:0/3593643741 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f54c81548e0 0x7f54c81f4000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:10.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:10.095+0000 7f54cce00640 1 --2- 172.21.15.1:0/3593643741 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f54c80cbf20 0x7f54c81f3ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:10.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:10.095+0000 7f54cce00640 1 --2- 172.21.15.1:0/3593643741 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54c80cb760 0x7f54c80b4500 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:10.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:10.095+0000 7f54cce00640 1 -- 172.21.15.1:0/3593643741 >> 172.21.15.1:0/3593643741 conn(0x7f54c8073010 msgr2=0x7f54c80c60d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:03:10.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:10.096+0000 7f54cce00640 1 -- 172.21.15.1:0/3593643741 shutdown_connections 2024-05-09T08:03:10.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:10.096+0000 7f54cce00640 1 -- 172.21.15.1:0/3593643741 wait complete. 2024-05-09T08:03:10.133 INFO:tasks.mon_thrash.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T08:02:09.165511+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-09T08:02:12.925485+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-09T08:02:14.367455+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-09T08:02:17.967064+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-09T08:03:10.133 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-09T08:03:10.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:10.320+0000 7fe352e00640 1 Processor -- start 2024-05-09T08:03:10.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:10.321+0000 7fe352e00640 1 -- start start 2024-05-09T08:03:10.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:10.322+0000 7fe352e00640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fe34c0cac60 0x7fe34c0cda00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:10.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:10.322+0000 7fe352e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe34c15ac80 0x7fe34c0cdf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:10.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:10.322+0000 7fe352e00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe34c0ccb20 con 0x7fe34c0cc5f0 2024-05-09T08:03:10.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:10.322+0000 7fe352e00640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7fe34c0cce00 con 0x7fe34c0cac60 2024-05-09T08:03:10.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:10.322+0000 7fe352e00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fe34c0cf120 con 0x7fe34c15ac80 2024-05-09T08:03:10.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:10.323+0000 7fe351000640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fe34c0cc5f0 0x7fe34c15a570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.1:50074/0 (socket says 172.21.15.1:50074) 2024-05-09T08:03:10.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:10.323+0000 7fe351000640 1 -- 172.21.15.1:0/3801411317 learned_addr learned my addr 172.21.15.1:0/3801411317 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T08:03:10.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:10.323+0000 7fe352400640 1 --2- 172.21.15.1:0/3801411317 >> v2:172.21.15.7:3300/0 conn(0x7fe34c15ac80 0x7fe34c0cdf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:10.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:10.323+0000 7fe351a00640 1 --2- 172.21.15.1:0/3801411317 >> v2:172.21.15.1:3300/0 conn(0x7fe34c0cac60 0x7fe34c0cda00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:10.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:10.323+0000 7fe34be00640 1 -- 172.21.15.1:0/3801411317 <== mon.1 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1711979121 0 0) 0x7fe34c0ccb20 con 0x7fe34c0cc5f0 2024-05-09T08:03:10.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:10.323+0000 7fe34be00640 1 -- 172.21.15.1:0/3801411317 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe334003520 con 0x7fe34c0cc5f0 2024-05-09T08:03:10.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:10.323+0000 7fe352400640 1 -- 172.21.15.1:0/3801411317 >> v2:172.21.15.1:3300/0 conn(0x7fe34c0cac60 msgr2=0x7fe34c0cda00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:03:10.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:10.323+0000 7fe352400640 1 --2- 172.21.15.1:0/3801411317 >> v2:172.21.15.1:3300/0 conn(0x7fe34c0cac60 0x7fe34c0cda00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:10.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:10.323+0000 7fe352400640 1 -- 172.21.15.1:0/3801411317 >> v1:172.21.15.7:6789/0 conn(0x7fe34c0cc5f0 legacy=0x7fe34c15a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:10.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:10.323+0000 7fe352400640 1 -- 172.21.15.1:0/3801411317 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe34c0cf400 con 0x7fe34c15ac80 2024-05-09T08:03:10.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:10.325+0000 7fe352400640 1 --2- 172.21.15.1:0/3801411317 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe34c15ac80 0x7fe34c0cdf40 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fe34c08cfb0 tx=0x7fe34402a250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5fd1ab0f7d25c9dd server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:03:10.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:10.325+0000 7fe34be00640 1 -- 172.21.15.1:0/3801411317 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe344037070 con 0x7fe34c15ac80 2024-05-09T08:03:10.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:10.325+0000 7fe34be00640 1 -- 172.21.15.1:0/3801411317 <== mon.1 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe344032440 con 0x7fe34c15ac80 2024-05-09T08:03:10.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:10.325+0000 7fe34be00640 1 -- 172.21.15.1:0/3801411317 <== mon.1 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe344032810 con 0x7fe34c15ac80 2024-05-09T08:03:10.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:10.326+0000 7fe352e00640 1 -- 172.21.15.1:0/3801411317 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe34c15ac80 msgr2=0x7fe34c0cdf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:10.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:10.326+0000 7fe352e00640 1 --2- 172.21.15.1:0/3801411317 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe34c15ac80 0x7fe34c0cdf40 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fe34c08cfb0 tx=0x7fe34402a250 comp rx=0 tx=0).stop 2024-05-09T08:03:10.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:10.326+0000 7fe352e00640 1 -- 172.21.15.1:0/3801411317 shutdown_connections 2024-05-09T08:03:10.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:10.326+0000 7fe352e00640 1 --2- 172.21.15.1:0/3801411317 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe34c15ac80 0x7fe34c0cdf40 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:10.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:10.326+0000 7fe352e00640 1 --2- 172.21.15.1:0/3801411317 >> v2:172.21.15.1:3300/0 conn(0x7fe34c0cac60 0x7fe34c0cda00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:10.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:10.326+0000 7fe352e00640 1 -- 172.21.15.1:0/3801411317 >> 172.21.15.1:0/3801411317 conn(0x7fe34c0cfc20 msgr2=0x7fe34c0d0030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:03:10.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:10.326+0000 7fe352e00640 1 -- 172.21.15.1:0/3801411317 shutdown_connections 2024-05-09T08:03:10.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:10.326+0000 7fe352e00640 1 -- 172.21.15.1:0/3801411317 wait complete. 2024-05-09T08:03:10.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:10.328+0000 7fe352e00640 1 Processor -- start 2024-05-09T08:03:10.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:10.328+0000 7fe352e00640 1 -- start start 2024-05-09T08:03:10.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:10.329+0000 7fe352e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe34c0cac60 0x7fe34c14e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:10.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:10.329+0000 7fe352e00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fe34c0cc5f0 0x7fe34c1498b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:10.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:10.329+0000 7fe352e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe34c15ac80 0x7fe34c0b13a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:10.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:10.329+0000 7fe352e00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fe34c0ce4e0 con 0x7fe34c15ac80 2024-05-09T08:03:10.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:10.329+0000 7fe352e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe34c0cd550 con 0x7fe34c0cac60 2024-05-09T08:03:10.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:10.329+0000 7fe352e00640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7fe34c0ce660 con 0x7fe34c0cc5f0 2024-05-09T08:03:10.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:10.329+0000 7fe351a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe34c0cac60 0x7fe34c14e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:10.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:10.329+0000 7fe352400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe34c15ac80 0x7fe34c0b13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:10.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:10.329+0000 7fe351000640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fe34c0cc5f0 0x7fe34c1498b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:10.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:10.329+0000 7fe351a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe34c0cac60 0x7fe34c14e790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.1:33616/0 (socket says 172.21.15.1:33616) 2024-05-09T08:03:10.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:10.329+0000 7fe351a00640 1 -- 172.21.15.1:0/3577647730 learned_addr learned my addr 172.21.15.1:0/3577647730 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:03:10.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:10.329+0000 7fe351a00640 1 -- 172.21.15.1:0/3577647730 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fe34c0cc5f0 msgr2=0x7fe34c1498b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:10.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:10.329+0000 7fe351a00640 1 --2- 172.21.15.1:0/3577647730 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fe34c0cc5f0 0x7fe34c1498b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:10.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:10.329+0000 7fe351a00640 1 -- 172.21.15.1:0/3577647730 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe34c15ac80 msgr2=0x7fe34c0b13a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:10.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:10.329+0000 7fe351a00640 1 --2- 172.21.15.1:0/3577647730 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe34c15ac80 0x7fe34c0b13a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:10.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:10.329+0000 7fe351a00640 1 -- 172.21.15.1:0/3577647730 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe34c14d8e0 con 0x7fe34c0cac60 2024-05-09T08:03:10.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:10.330+0000 7fe351a00640 1 --2- 172.21.15.1:0/3577647730 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe34c0cac60 0x7fe34c14e790 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fe34000efc0 tx=0x7fe34000c490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:03:10.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:10.330+0000 7fe349600640 1 -- 172.21.15.1:0/3577647730 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe340018020 con 0x7fe34c0cac60 2024-05-09T08:03:10.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:10.330+0000 7fe349600640 1 -- 172.21.15.1:0/3577647730 <== mon.1 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe340014480 con 0x7fe34c0cac60 2024-05-09T08:03:10.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:10.330+0000 7fe349600640 1 -- 172.21.15.1:0/3577647730 <== mon.1 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe340014850 con 0x7fe34c0cac60 2024-05-09T08:03:10.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:10.330+0000 7fe352e00640 1 -- 172.21.15.1:0/3577647730 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe34c14f2f0 con 0x7fe34c0cac60 2024-05-09T08:03:10.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:10.330+0000 7fe352e00640 1 -- 172.21.15.1:0/3577647730 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe34c14f8b0 con 0x7fe34c0cac60 2024-05-09T08:03:10.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:10.331+0000 7fe352e00640 1 -- 172.21.15.1:0/3577647730 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe34c14fc90 con 0x7fe34c0cac60 2024-05-09T08:03:10.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:10.333+0000 7fe349600640 1 -- 172.21.15.1:0/3577647730 <== mon.1 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe34001b020 con 0x7fe34c0cac60 2024-05-09T08:03:10.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:10.333+0000 7fe349600640 1 --2- 172.21.15.1:0/3577647730 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fe32c06be30 0x7fe32c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:10.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:10.334+0000 7fe349600640 1 -- 172.21.15.1:0/3577647730 <== mon.1 v2:172.21.15.7:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7fe34008bd80 con 0x7fe34c0cac60 2024-05-09T08:03:10.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:10.334+0000 7fe351000640 1 --2- 172.21.15.1:0/3577647730 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fe32c06be30 0x7fe32c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:10.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:10.334+0000 7fe351000640 1 --2- 172.21.15.1:0/3577647730 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fe32c06be30 0x7fe32c06e2f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fe34c0c9010 tx=0x7fe338009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:03:10.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:10.338+0000 7fe349600640 1 -- 172.21.15.1:0/3577647730 <== mon.1 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fe34c14fc90 con 0x7fe34c0cac60 2024-05-09T08:03:10.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:10.553+0000 7fe352e00640 1 -- 172.21.15.1:0/3577647730 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fe34c04f660 con 0x7fe34c0cac60 2024-05-09T08:03:10.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:10.554+0000 7fe349600640 1 -- 172.21.15.1:0/3577647730 <== mon.1 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) ==== 93+0+10 (secure 0 0 0) 0x7fe340022030 con 0x7fe34c0cac60 2024-05-09T08:03:10.571 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 1 2024-05-09T08:03:10.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:10.556+0000 7fe352e00640 1 -- 172.21.15.1:0/3577647730 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fe32c06be30 msgr2=0x7fe32c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:10.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:10.556+0000 7fe352e00640 1 --2- 172.21.15.1:0/3577647730 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fe32c06be30 0x7fe32c06e2f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fe34c0c9010 tx=0x7fe338009040 comp rx=0 tx=0).stop 2024-05-09T08:03:10.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:10.556+0000 7fe352e00640 1 -- 172.21.15.1:0/3577647730 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe34c0cac60 msgr2=0x7fe34c14e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:10.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:10.556+0000 7fe352e00640 1 --2- 172.21.15.1:0/3577647730 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe34c0cac60 0x7fe34c14e790 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fe34000efc0 tx=0x7fe34000c490 comp rx=0 tx=0).stop 2024-05-09T08:03:10.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:10.557+0000 7fe352e00640 1 -- 172.21.15.1:0/3577647730 shutdown_connections 2024-05-09T08:03:10.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:10.557+0000 7fe352e00640 1 --2- 172.21.15.1:0/3577647730 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fe32c06be30 0x7fe32c06e2f0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:10.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:10.557+0000 7fe352e00640 1 --2- 172.21.15.1:0/3577647730 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe34c15ac80 0x7fe34c0b13a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:10.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:10.557+0000 7fe352e00640 1 --2- 172.21.15.1:0/3577647730 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fe34c0cc5f0 0x7fe34c1498b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:10.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:10.557+0000 7fe352e00640 1 --2- 172.21.15.1:0/3577647730 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe34c0cac60 0x7fe34c14e790 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:10.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:10.557+0000 7fe352e00640 1 -- 172.21.15.1:0/3577647730 >> 172.21.15.1:0/3577647730 conn(0x7fe34c0cfc20 msgr2=0x7fe34c0d0000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:03:10.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:10.557+0000 7fe352e00640 1 -- 172.21.15.1:0/3577647730 shutdown_connections 2024-05-09T08:03:10.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:10.557+0000 7fe352e00640 1 -- 172.21.15.1:0/3577647730 wait complete. 2024-05-09T08:03:10.594 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-09T08:03:10.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:10.782+0000 7f0ec5e00640 1 Processor -- start 2024-05-09T08:03:10.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:10.782+0000 7f0ec5e00640 1 -- start start 2024-05-09T08:03:10.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:10.783+0000 7f0ec5e00640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f0ec00c54f0 0x7f0ec006f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:10.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:10.784+0000 7f0ec5e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f0ec00c58d0 0x7f0ec01589e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:10.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:10.784+0000 7f0ec5e00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f0ec00c6470 0x7f0ec0158f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:10.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:10.784+0000 7f0ec5e00640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f0ec00726e0 con 0x7f0ec00c54f0 2024-05-09T08:03:10.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:10.784+0000 7f0ec5e00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f0ec00718e0 con 0x7f0ec00c58d0 2024-05-09T08:03:10.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:10.784+0000 7f0ec5e00640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f0ec0071b60 con 0x7f0ec00c6470 2024-05-09T08:03:10.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:10.784+0000 7f0ec4a00640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f0ec00c54f0 0x7f0ec006f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:10.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:10.784+0000 7f0ec4a00640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f0ec00c54f0 0x7f0ec006f930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:39702/0 (socket says 172.21.15.1:39702) 2024-05-09T08:03:10.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:10.784+0000 7f0ec4a00640 1 -- 172.21.15.1:0/2448576708 learned_addr learned my addr 172.21.15.1:0/2448576708 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:03:10.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:10.784+0000 7f0ebfe00640 1 --2- 172.21.15.1:0/2448576708 >> v2:172.21.15.7:3300/0 conn(0x7f0ec00c58d0 0x7f0ec01589e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:10.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:10.784+0000 7f0ec5400640 1 --2- 172.21.15.1:0/2448576708 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f0ec00c6470 0x7f0ec0158f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:10.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:10.784+0000 7f0ec4a00640 1 -- 172.21.15.1:0/2448576708 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f0ec00c6470 msgr2=0x7f0ec0158f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:03:10.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:10.784+0000 7f0ec4a00640 1 --2- 172.21.15.1:0/2448576708 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f0ec00c6470 0x7f0ec0158f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:10.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:10.784+0000 7f0ec4a00640 1 -- 172.21.15.1:0/2448576708 >> v2:172.21.15.7:3300/0 conn(0x7f0ec00c58d0 msgr2=0x7f0ec01589e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:03:10.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:10.785+0000 7f0ec4a00640 1 --2- 172.21.15.1:0/2448576708 >> v2:172.21.15.7:3300/0 conn(0x7f0ec00c58d0 0x7f0ec01589e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:10.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:10.785+0000 7f0ec4a00640 1 -- 172.21.15.1:0/2448576708 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ec0071e40 con 0x7f0ec00c54f0 2024-05-09T08:03:10.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:10.786+0000 7f0ec4a00640 1 --2- 172.21.15.1:0/2448576708 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0ec00c54f0 0x7f0ec006f930 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f0ec00b2200 tx=0x7f0eb402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34fb580beeb7db1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:03:10.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:10.786+0000 7f0ebf400640 1 -- 172.21.15.1:0/2448576708 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eb4037070 con 0x7f0ec00c54f0 2024-05-09T08:03:10.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:10.786+0000 7f0ebf400640 1 -- 172.21.15.1:0/2448576708 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0eb4032440 con 0x7f0ec00c54f0 2024-05-09T08:03:10.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:10.786+0000 7f0ebf400640 1 -- 172.21.15.1:0/2448576708 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eb4032810 con 0x7f0ec00c54f0 2024-05-09T08:03:10.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:10.787+0000 7f0ec5e00640 1 -- 172.21.15.1:0/2448576708 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0ec00c54f0 msgr2=0x7f0ec006f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:10.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:10.787+0000 7f0ec5e00640 1 --2- 172.21.15.1:0/2448576708 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0ec00c54f0 0x7f0ec006f930 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f0ec00b2200 tx=0x7f0eb402a1e0 comp rx=0 tx=0).stop 2024-05-09T08:03:10.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:10.787+0000 7f0ec5e00640 1 -- 172.21.15.1:0/2448576708 shutdown_connections 2024-05-09T08:03:10.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:10.787+0000 7f0ec5e00640 1 --2- 172.21.15.1:0/2448576708 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f0ec00c6470 0x7f0ec0158f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:10.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:10.787+0000 7f0ec5e00640 1 --2- 172.21.15.1:0/2448576708 >> v2:172.21.15.7:3300/0 conn(0x7f0ec00c58d0 0x7f0ec01589e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:10.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:10.787+0000 7f0ec5e00640 1 --2- 172.21.15.1:0/2448576708 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0ec00c54f0 0x7f0ec006f930 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:10.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:10.787+0000 7f0ec5e00640 1 -- 172.21.15.1:0/2448576708 >> 172.21.15.1:0/2448576708 conn(0x7f0ec006b250 msgr2=0x7f0ec006b630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:03:10.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:10.787+0000 7f0ec5e00640 1 -- 172.21.15.1:0/2448576708 shutdown_connections 2024-05-09T08:03:10.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:10.787+0000 7f0ec5e00640 1 -- 172.21.15.1:0/2448576708 wait complete. 2024-05-09T08:03:10.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:10.789+0000 7f0ec5e00640 1 Processor -- start 2024-05-09T08:03:10.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:10.789+0000 7f0ec5e00640 1 -- start start 2024-05-09T08:03:10.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:10.790+0000 7f0ec5e00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f0ec00c54f0 0x7f0ec00bef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:10.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:10.790+0000 7f0ec5e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ec00c58d0 0x7f0ec0147340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:10.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:10.790+0000 7f0ec5e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0ec00c6470 0x7f0ec0147880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:10.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:10.790+0000 7f0ec5e00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f0ec01584c0 con 0x7f0ec00c6470 2024-05-09T08:03:10.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:10.790+0000 7f0ec5e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0ec0072400 con 0x7f0ec00c58d0 2024-05-09T08:03:10.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:10.790+0000 7f0ec5e00640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f0ec006fe70 con 0x7f0ec00c54f0 2024-05-09T08:03:10.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:10.790+0000 7f0ec5400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0ec00c6470 0x7f0ec0147880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:10.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:10.790+0000 7f0ec5400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0ec00c6470 0x7f0ec0147880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:39716/0 (socket says 172.21.15.1:39716) 2024-05-09T08:03:10.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:10.790+0000 7f0ec5400640 1 -- 172.21.15.1:0/168269386 learned_addr learned my addr 172.21.15.1:0/168269386 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:03:10.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:10.790+0000 7f0ec4a00640 1 --2- 172.21.15.1:0/168269386 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f0ec00c54f0 0x7f0ec00bef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:10.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:10.790+0000 7f0ebfe00640 1 --2- 172.21.15.1:0/168269386 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ec00c58d0 0x7f0ec0147340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:10.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:10.791+0000 7f0ec5400640 1 -- 172.21.15.1:0/168269386 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f0ec00c54f0 msgr2=0x7f0ec00bef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:10.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:10.791+0000 7f0ec5400640 1 --2- 172.21.15.1:0/168269386 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f0ec00c54f0 0x7f0ec00bef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:10.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:10.791+0000 7f0ec5400640 1 -- 172.21.15.1:0/168269386 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ec00c58d0 msgr2=0x7f0ec0147340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:10.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:10.791+0000 7f0ec5400640 1 --2- 172.21.15.1:0/168269386 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ec00c58d0 0x7f0ec0147340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:10.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:10.791+0000 7f0ec5400640 1 -- 172.21.15.1:0/168269386 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ec014d780 con 0x7f0ec00c6470 2024-05-09T08:03:10.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:10.791+0000 7f0ec5400640 1 --2- 172.21.15.1:0/168269386 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0ec00c6470 0x7f0ec0147880 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f0eb800efc0 tx=0x7f0eb800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:03:10.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:10.791+0000 7f0ebcc00640 1 -- 172.21.15.1:0/168269386 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eb8018020 con 0x7f0ec00c6470 2024-05-09T08:03:10.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:10.791+0000 7f0ebcc00640 1 -- 172.21.15.1:0/168269386 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0eb8014480 con 0x7f0ec00c6470 2024-05-09T08:03:10.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:10.791+0000 7f0ebcc00640 1 -- 172.21.15.1:0/168269386 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eb8014870 con 0x7f0ec00c6470 2024-05-09T08:03:10.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:10.791+0000 7f0ec5e00640 1 -- 172.21.15.1:0/168269386 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ec01f44a0 con 0x7f0ec00c6470 2024-05-09T08:03:10.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:10.792+0000 7f0ec5e00640 1 -- 172.21.15.1:0/168269386 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ec014bf30 con 0x7f0ec00c6470 2024-05-09T08:03:10.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:10.793+0000 7f0ec5e00640 1 -- 172.21.15.1:0/168269386 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ec01f42b0 con 0x7f0ec00c6470 2024-05-09T08:03:10.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:10.794+0000 7f0ebcc00640 1 -- 172.21.15.1:0/168269386 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0eb8005d10 con 0x7f0ec00c6470 2024-05-09T08:03:10.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:10.795+0000 7f0ebcc00640 1 --2- 172.21.15.1:0/168269386 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f0ea006be30 0x7f0ea006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:10.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:10.795+0000 7f0ebcc00640 1 -- 172.21.15.1:0/168269386 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7f0eb808bf20 con 0x7f0ec00c6470 2024-05-09T08:03:10.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:10.795+0000 7f0ec4a00640 1 --2- 172.21.15.1:0/168269386 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f0ea006be30 0x7f0ea006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:10.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:10.796+0000 7f0ec4a00640 1 --2- 172.21.15.1:0/168269386 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f0ea006be30 0x7f0ea006e2f0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f0eb4034000 tx=0x7f0eb402e000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:03:10.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:10.800+0000 7f0ebcc00640 1 -- 172.21.15.1:0/168269386 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f0eb8022030 con 0x7f0ec00c6470 2024-05-09T08:03:11.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:11.008+0000 7f0ec5e00640 1 -- 172.21.15.1:0/168269386 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f0ec004f660 con 0x7f0ec00c6470 2024-05-09T08:03:11.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:11.009+0000 7f0ebcc00640 1 -- 172.21.15.1:0/168269386 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) ==== 104+0+11 (secure 0 0 0) 0x7f0eb8010040 con 0x7f0ec00c6470 2024-05-09T08:03:11.026 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 32 2024-05-09T08:03:11.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:11.012+0000 7f0ec5e00640 1 -- 172.21.15.1:0/168269386 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f0ea006be30 msgr2=0x7f0ea006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:11.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:11.012+0000 7f0ec5e00640 1 --2- 172.21.15.1:0/168269386 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f0ea006be30 0x7f0ea006e2f0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f0eb4034000 tx=0x7f0eb402e000 comp rx=0 tx=0).stop 2024-05-09T08:03:11.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:11.012+0000 7f0ec5e00640 1 -- 172.21.15.1:0/168269386 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0ec00c6470 msgr2=0x7f0ec0147880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:11.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:11.012+0000 7f0ec5e00640 1 --2- 172.21.15.1:0/168269386 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0ec00c6470 0x7f0ec0147880 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f0eb800efc0 tx=0x7f0eb800c490 comp rx=0 tx=0).stop 2024-05-09T08:03:11.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:11.012+0000 7f0ec5e00640 1 -- 172.21.15.1:0/168269386 shutdown_connections 2024-05-09T08:03:11.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:11.012+0000 7f0ec5e00640 1 --2- 172.21.15.1:0/168269386 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f0ea006be30 0x7f0ea006e2f0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:11.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:11.012+0000 7f0ec5e00640 1 --2- 172.21.15.1:0/168269386 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ec00c58d0 0x7f0ec0147340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:11.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:11.012+0000 7f0ec5e00640 1 --2- 172.21.15.1:0/168269386 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0ec00c6470 0x7f0ec0147880 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:11.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:11.012+0000 7f0ec5e00640 1 --2- 172.21.15.1:0/168269386 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f0ec00c54f0 0x7f0ec00bef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:11.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:11.012+0000 7f0ec5e00640 1 -- 172.21.15.1:0/168269386 >> 172.21.15.1:0/168269386 conn(0x7f0ec006b250 msgr2=0x7f0ec014f7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:03:11.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:11.012+0000 7f0ec5e00640 1 -- 172.21.15.1:0/168269386 shutdown_connections 2024-05-09T08:03:11.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:11.012+0000 7f0ec5e00640 1 -- 172.21.15.1:0/168269386 wait complete. 2024-05-09T08:03:11.048 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-09T08:03:11.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:11.248+0000 7f9bc8a00640 1 Processor -- start 2024-05-09T08:03:11.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:11.248+0000 7f9bc8a00640 1 -- start start 2024-05-09T08:03:11.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:11.250+0000 7f9bc8a00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f9bc406b250 0x7f9bc40c8680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:11.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:11.250+0000 7f9bc8a00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f9bc406f4c0 0x7f9bc406f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:11.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:11.250+0000 7f9bc8a00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9bc40c7300 con 0x7f9bc4154480 2024-05-09T08:03:11.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:11.250+0000 7f9bc8a00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f9bc40c75e0 con 0x7f9bc406b250 2024-05-09T08:03:11.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:11.250+0000 7f9bc8a00640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f9bc40c78c0 con 0x7f9bc406f4c0 2024-05-09T08:03:11.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:11.251+0000 7f9bc3e00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f9bc406f4c0 0x7f9bc406f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:11.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:11.251+0000 7f9bc2a00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f9bc406b250 0x7f9bc40c8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:11.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:11.251+0000 7f9bc3400640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f9bc4154480 0x7f9bc415aed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.1:50090/0 (socket says 172.21.15.1:50090) 2024-05-09T08:03:11.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:11.251+0000 7f9bc3400640 1 -- 172.21.15.1:0/2079612556 learned_addr learned my addr 172.21.15.1:0/2079612556 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T08:03:11.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:11.251+0000 7f9bc2a00640 1 -- 172.21.15.1:0/2079612556 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f9bc406f4c0 msgr2=0x7f9bc406f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:03:11.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:11.251+0000 7f9bc2a00640 1 --2- 172.21.15.1:0/2079612556 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f9bc406f4c0 0x7f9bc406f8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:11.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:11.251+0000 7f9bc2a00640 1 -- 172.21.15.1:0/2079612556 >> v1:172.21.15.7:6789/0 conn(0x7f9bc4154480 legacy=0x7f9bc415aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:11.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:11.251+0000 7f9bc2a00640 1 -- 172.21.15.1:0/2079612556 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bc40c7ba0 con 0x7f9bc406b250 2024-05-09T08:03:11.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:11.251+0000 7f9bc2000640 1 -- 172.21.15.1:0/2079612556 <== mon.1 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1091264270 0 0) 0x7f9bc40c7300 con 0x7f9bc4154480 2024-05-09T08:03:11.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:11.252+0000 7f9bc2a00640 1 --2- 172.21.15.1:0/2079612556 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9bc406b250 0x7f9bc40c8680 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f9bb00097a0 tx=0x7f9bb002a210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d058fe01c3b85c2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:03:11.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:11.252+0000 7f9bc2000640 1 -- 172.21.15.1:0/2079612556 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bb0037070 con 0x7f9bc406b250 2024-05-09T08:03:11.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:11.252+0000 7f9bc2000640 1 -- 172.21.15.1:0/2079612556 <== mon.1 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9bb0032440 con 0x7f9bc406b250 2024-05-09T08:03:11.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:11.252+0000 7f9bc2000640 1 -- 172.21.15.1:0/2079612556 <== mon.1 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bb0032810 con 0x7f9bc406b250 2024-05-09T08:03:11.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:11.253+0000 7f9bc8a00640 1 -- 172.21.15.1:0/2079612556 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9bc406b250 msgr2=0x7f9bc40c8680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:11.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:11.253+0000 7f9bc8a00640 1 --2- 172.21.15.1:0/2079612556 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9bc406b250 0x7f9bc40c8680 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f9bb00097a0 tx=0x7f9bb002a210 comp rx=0 tx=0).stop 2024-05-09T08:03:11.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:11.253+0000 7f9bc8a00640 1 -- 172.21.15.1:0/2079612556 shutdown_connections 2024-05-09T08:03:11.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:11.253+0000 7f9bc8a00640 1 --2- 172.21.15.1:0/2079612556 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f9bc406f4c0 0x7f9bc406f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:11.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:11.253+0000 7f9bc8a00640 1 --2- 172.21.15.1:0/2079612556 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9bc406b250 0x7f9bc40c8680 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:11.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:11.253+0000 7f9bc8a00640 1 -- 172.21.15.1:0/2079612556 >> 172.21.15.1:0/2079612556 conn(0x7f9bc40c8e30 msgr2=0x7f9bc40c9210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:03:11.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:11.253+0000 7f9bc8a00640 1 -- 172.21.15.1:0/2079612556 shutdown_connections 2024-05-09T08:03:11.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:11.253+0000 7f9bc8a00640 1 -- 172.21.15.1:0/2079612556 wait complete. 2024-05-09T08:03:11.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:11.255+0000 7f9bc8a00640 1 Processor -- start 2024-05-09T08:03:11.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:11.255+0000 7f9bc8a00640 1 -- start start 2024-05-09T08:03:11.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:11.256+0000 7f9bc8a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9bc406b250 0x7f9bc40cb5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:11.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:11.256+0000 7f9bc8a00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f9bc406f4c0 0x7f9bc41f41b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:11.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:11.256+0000 7f9bc8a00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9bc4154480 0x7f9bc41f0c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:11.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:11.256+0000 7f9bc8a00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f9bc4154860 con 0x7f9bc4154480 2024-05-09T08:03:11.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:11.256+0000 7f9bc8a00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9bc40c8160 con 0x7f9bc406b250 2024-05-09T08:03:11.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:11.256+0000 7f9bc8a00640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f9bc40673b0 con 0x7f9bc406f4c0 2024-05-09T08:03:11.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:11.256+0000 7f9bc3e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9bc4154480 0x7f9bc41f0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:11.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:11.256+0000 7f9bc3400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9bc406b250 0x7f9bc40cb5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:11.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:11.256+0000 7f9bc2a00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f9bc406f4c0 0x7f9bc41f41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:11.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:11.256+0000 7f9bc3400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9bc406b250 0x7f9bc40cb5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.1:33648/0 (socket says 172.21.15.1:33648) 2024-05-09T08:03:11.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:11.256+0000 7f9bc3400640 1 -- 172.21.15.1:0/2470466516 learned_addr learned my addr 172.21.15.1:0/2470466516 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:03:11.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:11.257+0000 7f9bc3400640 1 -- 172.21.15.1:0/2470466516 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f9bc406f4c0 msgr2=0x7f9bc41f41b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:11.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:11.257+0000 7f9bc3400640 1 --2- 172.21.15.1:0/2470466516 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f9bc406f4c0 0x7f9bc41f41b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:11.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:11.257+0000 7f9bc3400640 1 -- 172.21.15.1:0/2470466516 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9bc4154480 msgr2=0x7f9bc41f0c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:11.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:11.257+0000 7f9bc3400640 1 --2- 172.21.15.1:0/2470466516 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9bc4154480 0x7f9bc41f0c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:11.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:11.257+0000 7f9bc3400640 1 -- 172.21.15.1:0/2470466516 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bc40cbb20 con 0x7f9bc406b250 2024-05-09T08:03:11.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:11.257+0000 7f9bc2a00640 1 --2- 172.21.15.1:0/2470466516 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f9bc406f4c0 0x7f9bc41f41b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:03:11.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:11.257+0000 7f9bc3400640 1 --2- 172.21.15.1:0/2470466516 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9bc406b250 0x7f9bc40cb5e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f9bb400c430 tx=0x7f9bb400c8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:03:11.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:11.257+0000 7f9bbb400640 1 -- 172.21.15.1:0/2470466516 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bb40074e0 con 0x7f9bc406b250 2024-05-09T08:03:11.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:11.257+0000 7f9bbb400640 1 -- 172.21.15.1:0/2470466516 <== mon.1 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9bb40047e0 con 0x7f9bc406b250 2024-05-09T08:03:11.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:11.257+0000 7f9bbb400640 1 -- 172.21.15.1:0/2470466516 <== mon.1 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bb4004bd0 con 0x7f9bc406b250 2024-05-09T08:03:11.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:11.257+0000 7f9bc8a00640 1 -- 172.21.15.1:0/2470466516 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9bc4147f00 con 0x7f9bc406b250 2024-05-09T08:03:11.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:11.258+0000 7f9bc8a00640 1 -- 172.21.15.1:0/2470466516 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9bc4147940 con 0x7f9bc406b250 2024-05-09T08:03:11.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:11.259+0000 7f9bc8a00640 1 -- 172.21.15.1:0/2470466516 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9bc404f660 con 0x7f9bc406b250 2024-05-09T08:03:11.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:11.260+0000 7f9bbb400640 1 -- 172.21.15.1:0/2470466516 <== mon.1 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9bb4011070 con 0x7f9bc406b250 2024-05-09T08:03:11.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:11.261+0000 7f9bbb400640 1 --2- 172.21.15.1:0/2470466516 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f9ba406be30 0x7f9ba406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:11.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:11.261+0000 7f9bbb400640 1 -- 172.21.15.1:0/2470466516 <== mon.1 v2:172.21.15.7:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7f9bb408add0 con 0x7f9bc406b250 2024-05-09T08:03:11.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:11.261+0000 7f9bc2a00640 1 --2- 172.21.15.1:0/2470466516 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f9ba406be30 0x7f9ba406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:11.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:11.262+0000 7f9bc2a00640 1 --2- 172.21.15.1:0/2470466516 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f9ba406be30 0x7f9ba406e2f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f9bb0034000 tx=0x7f9bb002e000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:03:11.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:11.266+0000 7f9bbb400640 1 -- 172.21.15.1:0/2470466516 <== mon.1 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f9bb4058ca0 con 0x7f9bc406b250 2024-05-09T08:03:11.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:11.473+0000 7f9bc8a00640 1 -- 172.21.15.1:0/2470466516 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f9bc40c57d0 con 0x7f9bc406b250 2024-05-09T08:03:11.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:11.475+0000 7f9bbb400640 1 -- 172.21.15.1:0/2470466516 <== mon.1 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) ==== 100+0+11 (secure 0 0 0) 0x7f9bb405cfa0 con 0x7f9bc406b250 2024-05-09T08:03:11.492 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 64 2024-05-09T08:03:11.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:11.477+0000 7f9bc8a00640 1 -- 172.21.15.1:0/2470466516 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f9ba406be30 msgr2=0x7f9ba406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:11.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:11.477+0000 7f9bc8a00640 1 --2- 172.21.15.1:0/2470466516 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f9ba406be30 0x7f9ba406e2f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f9bb0034000 tx=0x7f9bb002e000 comp rx=0 tx=0).stop 2024-05-09T08:03:11.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:11.477+0000 7f9bc8a00640 1 -- 172.21.15.1:0/2470466516 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9bc406b250 msgr2=0x7f9bc40cb5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:11.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:11.477+0000 7f9bc8a00640 1 --2- 172.21.15.1:0/2470466516 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9bc406b250 0x7f9bc40cb5e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f9bb400c430 tx=0x7f9bb400c8f0 comp rx=0 tx=0).stop 2024-05-09T08:03:11.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:11.477+0000 7f9bc8a00640 1 -- 172.21.15.1:0/2470466516 shutdown_connections 2024-05-09T08:03:11.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:11.477+0000 7f9bc8a00640 1 --2- 172.21.15.1:0/2470466516 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f9ba406be30 0x7f9ba406e2f0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:11.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:11.477+0000 7f9bc8a00640 1 --2- 172.21.15.1:0/2470466516 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9bc4154480 0x7f9bc41f0c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:11.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:11.477+0000 7f9bc8a00640 1 --2- 172.21.15.1:0/2470466516 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f9bc406f4c0 0x7f9bc41f41b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:11.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:11.477+0000 7f9bc8a00640 1 --2- 172.21.15.1:0/2470466516 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9bc406b250 0x7f9bc40cb5e0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:11.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:11.477+0000 7f9bc8a00640 1 -- 172.21.15.1:0/2470466516 >> 172.21.15.1:0/2470466516 conn(0x7f9bc40c8e30 msgr2=0x7f9bc406c260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:03:11.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:11.478+0000 7f9bc8a00640 1 -- 172.21.15.1:0/2470466516 shutdown_connections 2024-05-09T08:03:11.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:11.478+0000 7f9bc8a00640 1 -- 172.21.15.1:0/2470466516 wait complete. 2024-05-09T08:03:11.514 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-09T08:03:11.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:11.715+0000 7f4e80c00640 1 Processor -- start 2024-05-09T08:03:11.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:11.715+0000 7f4e80c00640 1 -- start start 2024-05-09T08:03:11.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:11.717+0000 7f4e80c00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f4e7c0c5e40 0x7f4e7c0c9570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:11.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:11.717+0000 7f4e80c00640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4e7c071c80 con 0x7f4e7c15c360 2024-05-09T08:03:11.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:11.717+0000 7f4e80c00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4e7c071f20 con 0x7f4e7c072c20 2024-05-09T08:03:11.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:11.717+0000 7f4e80c00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f4e7c15b980 con 0x7f4e7c0c5e40 2024-05-09T08:03:11.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:11.717+0000 7f4e79e00640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f4e7c15c360 0x7f4e7c0cc5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:42730/0 (socket says 172.21.15.1:42730) 2024-05-09T08:03:11.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:11.717+0000 7f4e79e00640 1 -- 172.21.15.1:0/1733248518 learned_addr learned my addr 172.21.15.1:0/1733248518 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T08:03:11.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:11.718+0000 7f4e79400640 1 --2- 172.21.15.1:0/1733248518 >> v2:172.21.15.7:3300/0 conn(0x7f4e7c0c5e40 0x7f4e7c0c9570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:11.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:11.718+0000 7f4e78a00640 1 -- 172.21.15.1:0/1733248518 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 235603606 0 0) 0x7f4e7c071c80 con 0x7f4e7c15c360 2024-05-09T08:03:11.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:11.718+0000 7f4e78a00640 1 -- 172.21.15.1:0/1733248518 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4e5c003520 con 0x7f4e7c15c360 2024-05-09T08:03:11.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:11.718+0000 7f4e78a00640 1 -- 172.21.15.1:0/1733248518 <== mon.1 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3534680035 0 0) 0x7f4e7c071f20 con 0x7f4e7c072c20 2024-05-09T08:03:11.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:11.718+0000 7f4e78a00640 1 -- 172.21.15.1:0/1733248518 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4e7c071c80 con 0x7f4e7c072c20 2024-05-09T08:03:11.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:11.718+0000 7f4e79400640 1 -- 172.21.15.1:0/1733248518 >> v1:172.21.15.7:6789/0 conn(0x7f4e7c072c20 legacy=0x7f4e7c073050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:11.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:11.718+0000 7f4e79400640 1 -- 172.21.15.1:0/1733248518 >> v1:172.21.15.1:6789/0 conn(0x7f4e7c15c360 legacy=0x7f4e7c0cc5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:11.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:11.718+0000 7f4e79400640 1 -- 172.21.15.1:0/1733248518 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e7c0721e0 con 0x7f4e7c0c5e40 2024-05-09T08:03:11.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:11.719+0000 7f4e79400640 1 --2- 172.21.15.1:0/1733248518 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e7c0c5e40 0x7f4e7c0c9570 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f4e640097a0 tx=0x7f4e6402a110 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ae005046752e078d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:03:11.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:11.720+0000 7f4e78a00640 1 -- 172.21.15.1:0/1733248518 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e64037070 con 0x7f4e7c0c5e40 2024-05-09T08:03:11.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:11.720+0000 7f4e78a00640 1 -- 172.21.15.1:0/1733248518 <== mon.1 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e64032440 con 0x7f4e7c0c5e40 2024-05-09T08:03:11.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:11.720+0000 7f4e78a00640 1 -- 172.21.15.1:0/1733248518 <== mon.1 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e6402ae60 con 0x7f4e7c0c5e40 2024-05-09T08:03:11.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:11.720+0000 7f4e80c00640 1 -- 172.21.15.1:0/1733248518 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e7c0c5e40 msgr2=0x7f4e7c0c9570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:11.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:11.720+0000 7f4e80c00640 1 --2- 172.21.15.1:0/1733248518 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e7c0c5e40 0x7f4e7c0c9570 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f4e640097a0 tx=0x7f4e6402a110 comp rx=0 tx=0).stop 2024-05-09T08:03:11.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:11.720+0000 7f4e80c00640 1 -- 172.21.15.1:0/1733248518 shutdown_connections 2024-05-09T08:03:11.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:11.720+0000 7f4e80c00640 1 --2- 172.21.15.1:0/1733248518 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e7c0c5e40 0x7f4e7c0c9570 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:11.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:11.720+0000 7f4e80c00640 1 -- 172.21.15.1:0/1733248518 >> 172.21.15.1:0/1733248518 conn(0x7f4e7c06dad0 msgr2=0x7f4e7c06deb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:03:11.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:11.720+0000 7f4e80c00640 1 -- 172.21.15.1:0/1733248518 shutdown_connections 2024-05-09T08:03:11.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:11.720+0000 7f4e80c00640 1 -- 172.21.15.1:0/1733248518 wait complete. 2024-05-09T08:03:11.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:11.722+0000 7f4e80c00640 1 Processor -- start 2024-05-09T08:03:11.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:11.722+0000 7f4e80c00640 1 -- start start 2024-05-09T08:03:11.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:11.723+0000 7f4e80c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e7c072c20 0x7f4e7c157b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:11.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:11.723+0000 7f4e80c00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4e7c0c5e40 0x7f4e7c14de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:11.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:11.723+0000 7f4e80c00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f4e7c15c360 0x7f4e7c156960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:11.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:11.723+0000 7f4e80c00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f4e7c0727a0 con 0x7f4e7c0c5e40 2024-05-09T08:03:11.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:11.723+0000 7f4e80c00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f4e7c073760 con 0x7f4e7c072c20 2024-05-09T08:03:11.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:11.723+0000 7f4e80c00640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f4e7c0cc0d0 con 0x7f4e7c15c360 2024-05-09T08:03:11.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:11.724+0000 7f4e79400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4e7c0c5e40 0x7f4e7c14de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:11.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:11.724+0000 7f4e79e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e7c072c20 0x7f4e7c157b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:11.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:11.724+0000 7f4e79400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4e7c0c5e40 0x7f4e7c14de40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:39742/0 (socket says 172.21.15.1:39742) 2024-05-09T08:03:11.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:11.724+0000 7f4e79400640 1 -- 172.21.15.1:0/1284660355 learned_addr learned my addr 172.21.15.1:0/1284660355 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:03:11.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:11.724+0000 7f4e7a800640 1 --2- 172.21.15.1:0/1284660355 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f4e7c15c360 0x7f4e7c156960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:11.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:11.724+0000 7f4e79e00640 1 -- 172.21.15.1:0/1284660355 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f4e7c15c360 msgr2=0x7f4e7c156960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:11.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:11.724+0000 7f4e79e00640 1 --2- 172.21.15.1:0/1284660355 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f4e7c15c360 0x7f4e7c156960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:11.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:11.724+0000 7f4e79e00640 1 -- 172.21.15.1:0/1284660355 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4e7c0c5e40 msgr2=0x7f4e7c14de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:11.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:11.724+0000 7f4e79e00640 1 --2- 172.21.15.1:0/1284660355 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4e7c0c5e40 0x7f4e7c14de40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:11.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:11.724+0000 7f4e79e00640 1 -- 172.21.15.1:0/1284660355 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e7c0c3050 con 0x7f4e7c072c20 2024-05-09T08:03:11.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:11.724+0000 7f4e79400640 1 --2- 172.21.15.1:0/1284660355 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4e7c0c5e40 0x7f4e7c14de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:03:11.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:11.724+0000 7f4e79e00640 1 --2- 172.21.15.1:0/1284660355 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e7c072c20 0x7f4e7c157b00 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f4e7c071e50 tx=0x7f4e6c00c890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:03:11.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:11.725+0000 7f4e72000640 1 -- 172.21.15.1:0/1284660355 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e6c017070 con 0x7f4e7c072c20 2024-05-09T08:03:11.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:11.725+0000 7f4e72000640 1 -- 172.21.15.1:0/1284660355 <== mon.1 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e6c005ce0 con 0x7f4e7c072c20 2024-05-09T08:03:11.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:11.725+0000 7f4e72000640 1 -- 172.21.15.1:0/1284660355 <== mon.1 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e6c007310 con 0x7f4e7c072c20 2024-05-09T08:03:11.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:11.725+0000 7f4e80c00640 1 -- 172.21.15.1:0/1284660355 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e7c1561a0 con 0x7f4e7c072c20 2024-05-09T08:03:11.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:11.725+0000 7f4e80c00640 1 -- 172.21.15.1:0/1284660355 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e7c0c2d70 con 0x7f4e7c072c20 2024-05-09T08:03:11.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:11.726+0000 7f4e80c00640 1 -- 172.21.15.1:0/1284660355 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e7c0706f0 con 0x7f4e7c072c20 2024-05-09T08:03:11.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:11.727+0000 7f4e72000640 1 -- 172.21.15.1:0/1284660355 <== mon.1 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4e6c01a070 con 0x7f4e7c072c20 2024-05-09T08:03:11.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:11.728+0000 7f4e72000640 1 --2- 172.21.15.1:0/1284660355 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f4e5806be80 0x7f4e5806e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:11.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:11.728+0000 7f4e72000640 1 -- 172.21.15.1:0/1284660355 <== mon.1 v2:172.21.15.7:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7f4e6c08a4a0 con 0x7f4e7c072c20 2024-05-09T08:03:11.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:11.728+0000 7f4e79400640 1 --2- 172.21.15.1:0/1284660355 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f4e5806be80 0x7f4e5806e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:11.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:11.729+0000 7f4e79400640 1 --2- 172.21.15.1:0/1284660355 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f4e5806be80 0x7f4e5806e340 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f4e6402a150 tx=0x7f4e6402e000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:03:11.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:11.733+0000 7f4e72000640 1 -- 172.21.15.1:0/1284660355 <== mon.1 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f4e7c0706f0 con 0x7f4e7c072c20 2024-05-09T08:03:11.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:11.941+0000 7f4e80c00640 1 -- 172.21.15.1:0/1284660355 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f4e7c074d70 con 0x7f4e7c072c20 2024-05-09T08:03:11.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:11.942+0000 7f4e72000640 1 -- 172.21.15.1:0/1284660355 <== mon.1 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) ==== 103+0+11 (secure 0 0 0) 0x7f4e6c058540 con 0x7f4e7c072c20 2024-05-09T08:03:11.959 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 32 2024-05-09T08:03:11.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:11.944+0000 7f4e80c00640 1 -- 172.21.15.1:0/1284660355 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f4e5806be80 msgr2=0x7f4e5806e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:11.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:11.944+0000 7f4e80c00640 1 --2- 172.21.15.1:0/1284660355 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f4e5806be80 0x7f4e5806e340 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f4e6402a150 tx=0x7f4e6402e000 comp rx=0 tx=0).stop 2024-05-09T08:03:11.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:11.944+0000 7f4e80c00640 1 -- 172.21.15.1:0/1284660355 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e7c072c20 msgr2=0x7f4e7c157b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:11.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:11.944+0000 7f4e80c00640 1 --2- 172.21.15.1:0/1284660355 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e7c072c20 0x7f4e7c157b00 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f4e7c071e50 tx=0x7f4e6c00c890 comp rx=0 tx=0).stop 2024-05-09T08:03:11.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:11.945+0000 7f4e80c00640 1 -- 172.21.15.1:0/1284660355 shutdown_connections 2024-05-09T08:03:11.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:11.945+0000 7f4e80c00640 1 --2- 172.21.15.1:0/1284660355 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f4e5806be80 0x7f4e5806e340 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:11.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:11.945+0000 7f4e80c00640 1 --2- 172.21.15.1:0/1284660355 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f4e7c15c360 0x7f4e7c156960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:11.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:11.945+0000 7f4e80c00640 1 --2- 172.21.15.1:0/1284660355 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4e7c0c5e40 0x7f4e7c14de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:11.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:11.945+0000 7f4e80c00640 1 --2- 172.21.15.1:0/1284660355 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e7c072c20 0x7f4e7c157b00 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:11.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:11.945+0000 7f4e80c00640 1 -- 172.21.15.1:0/1284660355 >> 172.21.15.1:0/1284660355 conn(0x7f4e7c06dad0 msgr2=0x7f4e7c073050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:03:11.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:11.945+0000 7f4e80c00640 1 -- 172.21.15.1:0/1284660355 shutdown_connections 2024-05-09T08:03:11.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:11.945+0000 7f4e80c00640 1 -- 172.21.15.1:0/1284660355 wait complete. 2024-05-09T08:03:11.982 INFO:tasks.mon_thrash.ceph_manager:waiting for clean 2024-05-09T08:03:11.982 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-09T08:03:12.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:12.177+0000 7f5d4b200640 1 Processor -- start 2024-05-09T08:03:12.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:12.178+0000 7f5d4b200640 1 -- start start 2024-05-09T08:03:12.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:12.179+0000 7f5d4b200640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f5d4c15ba50 0x7f5d4c06d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:12.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:12.179+0000 7f5d4b200640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f5d4c15a5e0 0x7f5d4c0ca5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:12.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:12.179+0000 7f5d4b200640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5d4c15b550 con 0x7f5d4c0cd040 2024-05-09T08:03:12.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:12.179+0000 7f5d4b200640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f5d4c06cec0 con 0x7f5d4c15ba50 2024-05-09T08:03:12.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:12.179+0000 7f5d4b200640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f5d4c06c8b0 con 0x7f5d4c15a5e0 2024-05-09T08:03:12.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:12.180+0000 7f5d49400640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f5d4c15ba50 0x7f5d4c06d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:12.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:12.180+0000 7f5d49400640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f5d4c15ba50 0x7f5d4c06d350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:39754/0 (socket says 172.21.15.1:39754) 2024-05-09T08:03:12.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:12.180+0000 7f5d49400640 1 -- 172.21.15.1:0/3002627169 learned_addr learned my addr 172.21.15.1:0/3002627169 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:03:12.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:12.180+0000 7f5d4a800640 1 --2- 172.21.15.1:0/3002627169 >> v2:172.21.15.7:3300/0 conn(0x7f5d4c15a5e0 0x7f5d4c0ca5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:12.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:12.180+0000 7f5d49400640 1 -- 172.21.15.1:0/3002627169 >> v2:172.21.15.7:3300/0 conn(0x7f5d4c15a5e0 msgr2=0x7f5d4c0ca5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:03:12.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:12.180+0000 7f5d49400640 1 --2- 172.21.15.1:0/3002627169 >> v2:172.21.15.7:3300/0 conn(0x7f5d4c15a5e0 0x7f5d4c0ca5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:12.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:12.180+0000 7f5d49400640 1 -- 172.21.15.1:0/3002627169 >> v1:172.21.15.7:6789/0 conn(0x7f5d4c0cd040 legacy=0x7f5d4c06da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:12.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:12.180+0000 7f5d49400640 1 -- 172.21.15.1:0/3002627169 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d4c15af90 con 0x7f5d4c15ba50 2024-05-09T08:03:12.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:12.180+0000 7f5d48a00640 1 -- 172.21.15.1:0/3002627169 <== mon.1 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4108471518 0 0) 0x7f5d4c15b550 con 0x7f5d4c0cd040 2024-05-09T08:03:12.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:12.181+0000 7f5d49400640 1 --2- 172.21.15.1:0/3002627169 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5d4c15ba50 0x7f5d4c06d350 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f5d4c0af6a0 tx=0x7f5d3402a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ceedd5ff85679db server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:03:12.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:12.182+0000 7f5d48a00640 1 -- 172.21.15.1:0/3002627169 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d34036070 con 0x7f5d4c15ba50 2024-05-09T08:03:12.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:12.182+0000 7f5d48a00640 1 -- 172.21.15.1:0/3002627169 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d34031600 con 0x7f5d4c15ba50 2024-05-09T08:03:12.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:12.182+0000 7f5d48a00640 1 -- 172.21.15.1:0/3002627169 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d340319d0 con 0x7f5d4c15ba50 2024-05-09T08:03:12.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:12.182+0000 7f5d4b200640 1 -- 172.21.15.1:0/3002627169 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5d4c15ba50 msgr2=0x7f5d4c06d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:12.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:12.182+0000 7f5d4b200640 1 --2- 172.21.15.1:0/3002627169 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5d4c15ba50 0x7f5d4c06d350 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f5d4c0af6a0 tx=0x7f5d3402a610 comp rx=0 tx=0).stop 2024-05-09T08:03:12.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:12.183+0000 7f5d4b200640 1 -- 172.21.15.1:0/3002627169 shutdown_connections 2024-05-09T08:03:12.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:12.183+0000 7f5d4b200640 1 --2- 172.21.15.1:0/3002627169 >> v2:172.21.15.7:3300/0 conn(0x7f5d4c15a5e0 0x7f5d4c0ca5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:12.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:12.183+0000 7f5d4b200640 1 --2- 172.21.15.1:0/3002627169 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5d4c15ba50 0x7f5d4c06d350 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:12.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:12.183+0000 7f5d4b200640 1 -- 172.21.15.1:0/3002627169 >> 172.21.15.1:0/3002627169 conn(0x7f5d4c0b1780 msgr2=0x7f5d4c0b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:03:12.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:12.183+0000 7f5d4b200640 1 -- 172.21.15.1:0/3002627169 shutdown_connections 2024-05-09T08:03:12.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:12.183+0000 7f5d4b200640 1 -- 172.21.15.1:0/3002627169 wait complete. 2024-05-09T08:03:12.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:12.184+0000 7f5d4b200640 1 Processor -- start 2024-05-09T08:03:12.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:12.185+0000 7f5d4b200640 1 -- start start 2024-05-09T08:03:12.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:12.185+0000 7f5d4b200640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f5d4c0cd040 0x7f5d4c1f3dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:12.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:12.185+0000 7f5d4b200640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5d4c15a5e0 0x7f5d4c1f4300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:12.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:12.185+0000 7f5d4b200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d4c15ba50 0x7f5d4c1f05e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:12.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:12.185+0000 7f5d4b200640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f5d4c15aa50 con 0x7f5d4c15a5e0 2024-05-09T08:03:12.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:12.185+0000 7f5d4b200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5d4c06cb50 con 0x7f5d4c15ba50 2024-05-09T08:03:12.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:12.185+0000 7f5d4b200640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f5d4c15abd0 con 0x7f5d4c0cd040 2024-05-09T08:03:12.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:12.186+0000 7f5d49400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5d4c15a5e0 0x7f5d4c1f4300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:12.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:12.186+0000 7f5d49400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5d4c15a5e0 0x7f5d4c1f4300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:39770/0 (socket says 172.21.15.1:39770) 2024-05-09T08:03:12.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:12.186+0000 7f5d49400640 1 -- 172.21.15.1:0/2522366071 learned_addr learned my addr 172.21.15.1:0/2522366071 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:03:12.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:12.186+0000 7f5d4a800640 1 --2- 172.21.15.1:0/2522366071 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d4c15ba50 0x7f5d4c1f05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:12.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:12.186+0000 7f5d49e00640 1 --2- 172.21.15.1:0/2522366071 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f5d4c0cd040 0x7f5d4c1f3dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:12.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:12.186+0000 7f5d49400640 1 -- 172.21.15.1:0/2522366071 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f5d4c0cd040 msgr2=0x7f5d4c1f3dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:12.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:12.186+0000 7f5d49400640 1 --2- 172.21.15.1:0/2522366071 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f5d4c0cd040 0x7f5d4c1f3dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:12.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:12.186+0000 7f5d49400640 1 -- 172.21.15.1:0/2522366071 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d4c15ba50 msgr2=0x7f5d4c1f05e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:12.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:12.186+0000 7f5d49400640 1 --2- 172.21.15.1:0/2522366071 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d4c15ba50 0x7f5d4c1f05e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:12.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:12.186+0000 7f5d49400640 1 -- 172.21.15.1:0/2522366071 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d4c0c5f60 con 0x7f5d4c15a5e0 2024-05-09T08:03:12.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:12.187+0000 7f5d49400640 1 --2- 172.21.15.1:0/2522366071 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5d4c15a5e0 0x7f5d4c1f4300 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f5d34034040 tx=0x7f5d34004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:03:12.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:12.187+0000 7f5d42000640 1 -- 172.21.15.1:0/2522366071 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d34041020 con 0x7f5d4c15a5e0 2024-05-09T08:03:12.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:12.187+0000 7f5d42000640 1 -- 172.21.15.1:0/2522366071 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d34037040 con 0x7f5d4c15a5e0 2024-05-09T08:03:12.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:12.187+0000 7f5d4b200640 1 -- 172.21.15.1:0/2522366071 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d4c0b57a0 con 0x7f5d4c15a5e0 2024-05-09T08:03:12.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:12.187+0000 7f5d42000640 1 -- 172.21.15.1:0/2522366071 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d34036050 con 0x7f5d4c15a5e0 2024-05-09T08:03:12.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:12.187+0000 7f5d4b200640 1 -- 172.21.15.1:0/2522366071 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d4c0c81e0 con 0x7f5d4c15a5e0 2024-05-09T08:03:12.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:12.188+0000 7f5d4b200640 1 -- 172.21.15.1:0/2522366071 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d4c0c3460 con 0x7f5d4c15a5e0 2024-05-09T08:03:12.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:12.189+0000 7f5d42000640 1 -- 172.21.15.1:0/2522366071 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5d34007c50 con 0x7f5d4c15a5e0 2024-05-09T08:03:12.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:12.190+0000 7f5d42000640 1 --2- 172.21.15.1:0/2522366071 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f5d2806be30 0x7f5d2806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:12.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:12.190+0000 7f5d42000640 1 -- 172.21.15.1:0/2522366071 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7f5d340ab8a0 con 0x7f5d4c15a5e0 2024-05-09T08:03:12.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:12.190+0000 7f5d49e00640 1 --2- 172.21.15.1:0/2522366071 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f5d2806be30 0x7f5d2806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:12.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:12.191+0000 7f5d49e00640 1 --2- 172.21.15.1:0/2522366071 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f5d2806be30 0x7f5d2806e2f0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f5d3c000f60 tx=0x7f5d3c009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:03:12.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:12.195+0000 7f5d42000640 1 -- 172.21.15.1:0/2522366071 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f5d3404b060 con 0x7f5d4c15a5e0 2024-05-09T08:03:12.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:12.403+0000 7f5d4b200640 1 -- 172.21.15.1:0/2522366071 --> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5d4c06e260 con 0x7f5d2806be30 2024-05-09T08:03:12.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:12.414+0000 7f5d42000640 1 -- 172.21.15.1:0/2522366071 <== mgr.4114 v2:172.21.15.7:6832/3562612464 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332534 (secure 0 0 0) 0x7f5d4c06e260 con 0x7f5d2806be30 2024-05-09T08:03:12.437 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T08:03:12.439 INFO:teuthology.orchestra.run.smithi001.stderr:dumped all 2024-05-09T08:03:12.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:12.421+0000 7f5d4b200640 1 -- 172.21.15.1:0/2522366071 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f5d2806be30 msgr2=0x7f5d2806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:12.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:12.421+0000 7f5d4b200640 1 --2- 172.21.15.1:0/2522366071 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f5d2806be30 0x7f5d2806e2f0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f5d3c000f60 tx=0x7f5d3c009040 comp rx=0 tx=0).stop 2024-05-09T08:03:12.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:12.421+0000 7f5d4b200640 1 -- 172.21.15.1:0/2522366071 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5d4c15a5e0 msgr2=0x7f5d4c1f4300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:12.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:12.421+0000 7f5d4b200640 1 --2- 172.21.15.1:0/2522366071 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5d4c15a5e0 0x7f5d4c1f4300 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f5d34034040 tx=0x7f5d34004a40 comp rx=0 tx=0).stop 2024-05-09T08:03:12.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:12.422+0000 7f5d4b200640 1 -- 172.21.15.1:0/2522366071 shutdown_connections 2024-05-09T08:03:12.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:12.422+0000 7f5d4b200640 1 --2- 172.21.15.1:0/2522366071 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f5d2806be30 0x7f5d2806e2f0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:12.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:12.422+0000 7f5d4b200640 1 --2- 172.21.15.1:0/2522366071 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d4c15ba50 0x7f5d4c1f05e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:12.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:12.422+0000 7f5d4b200640 1 --2- 172.21.15.1:0/2522366071 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5d4c15a5e0 0x7f5d4c1f4300 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:12.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:12.422+0000 7f5d4b200640 1 --2- 172.21.15.1:0/2522366071 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f5d4c0cd040 0x7f5d4c1f3dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:12.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:12.422+0000 7f5d4b200640 1 -- 172.21.15.1:0/2522366071 >> 172.21.15.1:0/2522366071 conn(0x7f5d4c0b1780 msgr2=0x7f5d4c0ca0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:03:12.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:12.422+0000 7f5d4b200640 1 -- 172.21.15.1:0/2522366071 shutdown_connections 2024-05-09T08:03:12.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:12.422+0000 7f5d4b200640 1 -- 172.21.15.1:0/2522366071 wait complete. 2024-05-09T08:03:12.458 INFO:teuthology.orchestra.run.smithi001.stdout:{"pg_ready":true,"pg_map":{"version":52,"stamp":"2024-05-09T08:03:11.110878+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":208,"num_write_kb":2435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160,"ondisk_log_size":160,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":244612,"kb_used_data":3496,"kb_used_omap":12,"kb_used_meta":241011,"kb_avail":754730108,"statfs":{"total":773094113280,"available":772843630592,"internally_reserved":0,"allocated":3579904,"data_stored":2141768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":246795826},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.805863"},"pg_stats":[{"pgid":"2.1f","version":"24'4","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.314662+0000","last_change":"2024-05-09T08:02:14.875108+0000","last_active":"2024-05-09T08:03:02.314662+0000","last_peered":"2024-05-09T08:03:02.314662+0000","last_clean":"2024-05-09T08:03:02.314662+0000","last_became_active":"2024-05-09T08:02:14.874841+0000","last_became_peered":"2024-05-09T08:02:14.874841+0000","last_unstale":"2024-05-09T08:03:02.314662+0000","last_undegraded":"2024-05-09T08:03:02.314662+0000","last_fullsized":"2024-05-09T08:03:02.314662+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:00:40.599475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.514890+0000","last_change":"2024-05-09T08:02:15.899818+0000","last_active":"2024-05-09T08:03:02.514890+0000","last_peered":"2024-05-09T08:03:02.514890+0000","last_clean":"2024-05-09T08:03:02.514890+0000","last_became_active":"2024-05-09T08:02:15.899479+0000","last_became_peered":"2024-05-09T08:02:15.899479+0000","last_unstale":"2024-05-09T08:03:02.514890+0000","last_undegraded":"2024-05-09T08:03:02.514890+0000","last_fullsized":"2024-05-09T08:03:02.514890+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:18:36.309627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.513647+0000","last_change":"2024-05-09T08:02:20.057944+0000","last_active":"2024-05-09T08:03:02.513647+0000","last_peered":"2024-05-09T08:03:02.513647+0000","last_clean":"2024-05-09T08:03:02.513647+0000","last_became_active":"2024-05-09T08:02:20.057016+0000","last_became_peered":"2024-05-09T08:02:20.057016+0000","last_unstale":"2024-05-09T08:03:02.513647+0000","last_undegraded":"2024-05-09T08:03:02.513647+0000","last_fullsized":"2024-05-09T08:03:02.513647+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:17:01.514168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"24'1","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.515170+0000","last_change":"2024-05-09T08:02:14.886137+0000","last_active":"2024-05-09T08:03:02.515170+0000","last_peered":"2024-05-09T08:03:02.515170+0000","last_clean":"2024-05-09T08:03:02.515170+0000","last_became_active":"2024-05-09T08:02:14.885830+0000","last_became_peered":"2024-05-09T08:02:14.885830+0000","last_unstale":"2024-05-09T08:03:02.515170+0000","last_undegraded":"2024-05-09T08:03:02.515170+0000","last_fullsized":"2024-05-09T08:03:02.515170+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:46:11.160815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.312424+0000","last_change":"2024-05-09T08:02:15.906280+0000","last_active":"2024-05-09T08:03:02.312424+0000","last_peered":"2024-05-09T08:03:02.312424+0000","last_clean":"2024-05-09T08:03:02.312424+0000","last_became_active":"2024-05-09T08:02:15.905590+0000","last_became_peered":"2024-05-09T08:02:15.905590+0000","last_unstale":"2024-05-09T08:03:02.312424+0000","last_undegraded":"2024-05-09T08:03:02.312424+0000","last_fullsized":"2024-05-09T08:03:02.312424+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:20:07.368327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.313525+0000","last_change":"2024-05-09T08:02:19.954548+0000","last_active":"2024-05-09T08:03:02.313525+0000","last_peered":"2024-05-09T08:03:02.313525+0000","last_clean":"2024-05-09T08:03:02.313525+0000","last_became_active":"2024-05-09T08:02:19.954161+0000","last_became_peered":"2024-05-09T08:02:19.954161+0000","last_unstale":"2024-05-09T08:03:02.313525+0000","last_undegraded":"2024-05-09T08:03:02.313525+0000","last_fullsized":"2024-05-09T08:03:02.313525+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:53:27.246334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"24'2","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.606002+0000","last_change":"2024-05-09T08:02:14.867128+0000","last_active":"2024-05-09T08:03:02.606002+0000","last_peered":"2024-05-09T08:03:02.606002+0000","last_clean":"2024-05-09T08:03:02.606002+0000","last_became_active":"2024-05-09T08:02:14.866809+0000","last_became_peered":"2024-05-09T08:02:14.866809+0000","last_unstale":"2024-05-09T08:03:02.606002+0000","last_undegraded":"2024-05-09T08:03:02.606002+0000","last_fullsized":"2024-05-09T08:03:02.606002+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:57:06.093718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.313344+0000","last_change":"2024-05-09T08:02:15.888431+0000","last_active":"2024-05-09T08:03:02.313344+0000","last_peered":"2024-05-09T08:03:02.313344+0000","last_clean":"2024-05-09T08:03:02.313344+0000","last_became_active":"2024-05-09T08:02:15.887509+0000","last_became_peered":"2024-05-09T08:02:15.887509+0000","last_unstale":"2024-05-09T08:03:02.313344+0000","last_undegraded":"2024-05-09T08:03:02.313344+0000","last_fullsized":"2024-05-09T08:03:02.313344+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:25:01.238816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.314249+0000","last_change":"2024-05-09T08:02:19.953600+0000","last_active":"2024-05-09T08:03:02.314249+0000","last_peered":"2024-05-09T08:03:02.314249+0000","last_clean":"2024-05-09T08:03:02.314249+0000","last_became_active":"2024-05-09T08:02:19.952275+0000","last_became_peered":"2024-05-09T08:02:19.952275+0000","last_unstale":"2024-05-09T08:03:02.314249+0000","last_undegraded":"2024-05-09T08:03:02.314249+0000","last_fullsized":"2024-05-09T08:03:02.314249+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:25:05.306541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"24'2","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.605642+0000","last_change":"2024-05-09T08:02:14.870819+0000","last_active":"2024-05-09T08:03:02.605642+0000","last_peered":"2024-05-09T08:03:02.605642+0000","last_clean":"2024-05-09T08:03:02.605642+0000","last_became_active":"2024-05-09T08:02:14.870476+0000","last_became_peered":"2024-05-09T08:02:14.870476+0000","last_unstale":"2024-05-09T08:03:02.605642+0000","last_undegraded":"2024-05-09T08:03:02.605642+0000","last_fullsized":"2024-05-09T08:03:02.605642+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:43:01.061801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.313275+0000","last_change":"2024-05-09T08:02:15.886471+0000","last_active":"2024-05-09T08:03:02.313275+0000","last_peered":"2024-05-09T08:03:02.313275+0000","last_clean":"2024-05-09T08:03:02.313275+0000","last_became_active":"2024-05-09T08:02:15.886168+0000","last_became_peered":"2024-05-09T08:02:15.886168+0000","last_unstale":"2024-05-09T08:03:02.313275+0000","last_undegraded":"2024-05-09T08:03:02.313275+0000","last_fullsized":"2024-05-09T08:03:02.313275+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:30:33.351158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.313990+0000","last_change":"2024-05-09T08:02:19.947347+0000","last_active":"2024-05-09T08:03:02.313990+0000","last_peered":"2024-05-09T08:03:02.313990+0000","last_clean":"2024-05-09T08:03:02.313990+0000","last_became_active":"2024-05-09T08:02:19.946906+0000","last_became_peered":"2024-05-09T08:02:19.946906+0000","last_unstale":"2024-05-09T08:03:02.313990+0000","last_undegraded":"2024-05-09T08:03:02.313990+0000","last_fullsized":"2024-05-09T08:03:02.313990+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:30:37.418883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.310048+0000","last_change":"2024-05-09T08:02:15.880765+0000","last_active":"2024-05-09T08:03:02.310048+0000","last_peered":"2024-05-09T08:03:02.310048+0000","last_clean":"2024-05-09T08:03:02.310048+0000","last_became_active":"2024-05-09T08:02:15.880439+0000","last_became_peered":"2024-05-09T08:02:15.880439+0000","last_unstale":"2024-05-09T08:03:02.310048+0000","last_undegraded":"2024-05-09T08:03:02.310048+0000","last_fullsized":"2024-05-09T08:03:02.310048+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:29:59.441314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"24'2","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.605857+0000","last_change":"2024-05-09T08:02:14.867587+0000","last_active":"2024-05-09T08:03:02.605857+0000","last_peered":"2024-05-09T08:03:02.605857+0000","last_clean":"2024-05-09T08:03:02.605857+0000","last_became_active":"2024-05-09T08:02:14.867291+0000","last_became_peered":"2024-05-09T08:02:14.867291+0000","last_unstale":"2024-05-09T08:03:02.605857+0000","last_undegraded":"2024-05-09T08:03:02.605857+0000","last_fullsized":"2024-05-09T08:03:02.605857+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:20:08.921752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.314846+0000","last_change":"2024-05-09T08:02:19.954902+0000","last_active":"2024-05-09T08:03:02.314846+0000","last_peered":"2024-05-09T08:03:02.314846+0000","last_clean":"2024-05-09T08:03:02.314846+0000","last_became_active":"2024-05-09T08:02:19.954007+0000","last_became_peered":"2024-05-09T08:02:19.954007+0000","last_unstale":"2024-05-09T08:03:02.314846+0000","last_undegraded":"2024-05-09T08:03:02.314846+0000","last_fullsized":"2024-05-09T08:03:02.314846+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:41:31.470546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.514342+0000","last_change":"2024-05-09T08:02:15.915155+0000","last_active":"2024-05-09T08:03:02.514342+0000","last_peered":"2024-05-09T08:03:02.514342+0000","last_clean":"2024-05-09T08:03:02.514342+0000","last_became_active":"2024-05-09T08:02:15.914852+0000","last_became_peered":"2024-05-09T08:02:15.914852+0000","last_unstale":"2024-05-09T08:03:02.514342+0000","last_undegraded":"2024-05-09T08:03:02.514342+0000","last_fullsized":"2024-05-09T08:03:02.514342+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:53:23.178610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.611056+0000","last_change":"2024-05-09T08:02:14.881770+0000","last_active":"2024-05-09T08:03:02.611056+0000","last_peered":"2024-05-09T08:03:02.611056+0000","last_clean":"2024-05-09T08:03:02.611056+0000","last_became_active":"2024-05-09T08:02:14.881437+0000","last_became_peered":"2024-05-09T08:02:14.881437+0000","last_unstale":"2024-05-09T08:03:02.611056+0000","last_undegraded":"2024-05-09T08:03:02.611056+0000","last_fullsized":"2024-05-09T08:03:02.611056+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:25:23.306827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.314596+0000","last_change":"2024-05-09T08:02:19.955928+0000","last_active":"2024-05-09T08:03:02.314596+0000","last_peered":"2024-05-09T08:03:02.314596+0000","last_clean":"2024-05-09T08:03:02.314596+0000","last_became_active":"2024-05-09T08:02:19.955432+0000","last_became_peered":"2024-05-09T08:02:19.955432+0000","last_unstale":"2024-05-09T08:03:02.314596+0000","last_undegraded":"2024-05-09T08:03:02.314596+0000","last_fullsized":"2024-05-09T08:03:02.314596+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:53:15.458120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.513937+0000","last_change":"2024-05-09T08:02:15.913311+0000","last_active":"2024-05-09T08:03:02.513937+0000","last_peered":"2024-05-09T08:03:02.513937+0000","last_clean":"2024-05-09T08:03:02.513937+0000","last_became_active":"2024-05-09T08:02:15.912484+0000","last_became_peered":"2024-05-09T08:02:15.912484+0000","last_unstale":"2024-05-09T08:03:02.513937+0000","last_undegraded":"2024-05-09T08:03:02.513937+0000","last_fullsized":"2024-05-09T08:03:02.513937+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:43:06.938904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.611451+0000","last_change":"2024-05-09T08:02:14.868884+0000","last_active":"2024-05-09T08:03:02.611451+0000","last_peered":"2024-05-09T08:03:02.611451+0000","last_clean":"2024-05-09T08:03:02.611451+0000","last_became_active":"2024-05-09T08:02:14.868553+0000","last_became_peered":"2024-05-09T08:02:14.868553+0000","last_unstale":"2024-05-09T08:03:02.611451+0000","last_undegraded":"2024-05-09T08:03:02.611451+0000","last_fullsized":"2024-05-09T08:03:02.611451+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:32:48.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.613297+0000","last_change":"2024-05-09T08:03:02.609342+0000","last_active":"2024-05-09T08:03:02.613297+0000","last_peered":"2024-05-09T08:03:02.613297+0000","last_clean":"2024-05-09T08:03:02.613297+0000","last_became_active":"2024-05-09T08:03:02.607564+0000","last_became_peered":"2024-05-09T08:03:02.607564+0000","last_unstale":"2024-05-09T08:03:02.613297+0000","last_undegraded":"2024-05-09T08:03:02.613297+0000","last_fullsized":"2024-05-09T08:03:02.613297+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:23:44.316545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4,7],"acting":[2,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.314326+0000","last_change":"2024-05-09T08:02:15.891115+0000","last_active":"2024-05-09T08:03:02.314326+0000","last_peered":"2024-05-09T08:03:02.314326+0000","last_clean":"2024-05-09T08:03:02.314326+0000","last_became_active":"2024-05-09T08:02:15.890806+0000","last_became_peered":"2024-05-09T08:02:15.890806+0000","last_unstale":"2024-05-09T08:03:02.314326+0000","last_undegraded":"2024-05-09T08:03:02.314326+0000","last_fullsized":"2024-05-09T08:03:02.314326+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:57:07.107359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.605148+0000","last_change":"2024-05-09T08:02:14.874987+0000","last_active":"2024-05-09T08:03:02.605148+0000","last_peered":"2024-05-09T08:03:02.605148+0000","last_clean":"2024-05-09T08:03:02.605148+0000","last_became_active":"2024-05-09T08:02:14.874549+0000","last_became_peered":"2024-05-09T08:02:14.874549+0000","last_unstale":"2024-05-09T08:03:02.605148+0000","last_undegraded":"2024-05-09T08:03:02.605148+0000","last_fullsized":"2024-05-09T08:03:02.605148+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:57:10.338510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.516739+0000","last_change":"2024-05-09T08:03:02.516739+0000","last_active":"2024-05-09T08:03:02.516739+0000","last_peered":"2024-05-09T08:03:02.516739+0000","last_clean":"2024-05-09T08:03:02.516739+0000","last_became_active":"2024-05-09T08:03:02.516161+0000","last_became_peered":"2024-05-09T08:03:02.516161+0000","last_unstale":"2024-05-09T08:03:02.516739+0000","last_undegraded":"2024-05-09T08:03:02.516739+0000","last_fullsized":"2024-05-09T08:03:02.516739+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:10:24.797526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,2],"acting":[4,3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.313602+0000","last_change":"2024-05-09T08:02:15.913011+0000","last_active":"2024-05-09T08:03:02.313602+0000","last_peered":"2024-05-09T08:03:02.313602+0000","last_clean":"2024-05-09T08:03:02.313602+0000","last_became_active":"2024-05-09T08:02:15.912590+0000","last_became_peered":"2024-05-09T08:02:15.912590+0000","last_unstale":"2024-05-09T08:03:02.313602+0000","last_undegraded":"2024-05-09T08:03:02.313602+0000","last_fullsized":"2024-05-09T08:03:02.313602+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:43:06.938904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"24'2","reported_seq":28,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.310929+0000","last_change":"2024-05-09T08:02:14.864763+0000","last_active":"2024-05-09T08:03:02.310929+0000","last_peered":"2024-05-09T08:03:02.310929+0000","last_clean":"2024-05-09T08:03:02.310929+0000","last_became_active":"2024-05-09T08:02:14.864470+0000","last_became_peered":"2024-05-09T08:02:14.864470+0000","last_unstale":"2024-05-09T08:03:02.310929+0000","last_undegraded":"2024-05-09T08:03:02.310929+0000","last_fullsized":"2024-05-09T08:03:02.310929+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:00:40.599475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.313868+0000","last_change":"2024-05-09T08:02:20.054896+0000","last_active":"2024-05-09T08:03:02.313868+0000","last_peered":"2024-05-09T08:03:02.313868+0000","last_clean":"2024-05-09T08:03:02.313868+0000","last_became_active":"2024-05-09T08:02:20.054424+0000","last_became_peered":"2024-05-09T08:02:20.054424+0000","last_unstale":"2024-05-09T08:03:02.313868+0000","last_undegraded":"2024-05-09T08:03:02.313868+0000","last_fullsized":"2024-05-09T08:03:02.313868+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:23:44.316545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.515468+0000","last_change":"2024-05-09T08:02:15.905448+0000","last_active":"2024-05-09T08:03:02.515468+0000","last_peered":"2024-05-09T08:03:02.515468+0000","last_clean":"2024-05-09T08:03:02.515468+0000","last_became_active":"2024-05-09T08:02:15.905131+0000","last_became_peered":"2024-05-09T08:02:15.905131+0000","last_unstale":"2024-05-09T08:03:02.515468+0000","last_undegraded":"2024-05-09T08:03:02.515468+0000","last_fullsized":"2024-05-09T08:03:02.515468+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:25:01.238816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"25'1","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.611654+0000","last_change":"2024-05-09T08:02:14.868410+0000","last_active":"2024-05-09T08:03:02.611654+0000","last_peered":"2024-05-09T08:03:02.611654+0000","last_clean":"2024-05-09T08:03:02.611654+0000","last_became_active":"2024-05-09T08:02:14.865621+0000","last_became_peered":"2024-05-09T08:02:14.865621+0000","last_unstale":"2024-05-09T08:03:02.611654+0000","last_undegraded":"2024-05-09T08:03:02.611654+0000","last_fullsized":"2024-05-09T08:03:02.611654+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:43:01.061801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.513488+0000","last_change":"2024-05-09T08:02:20.057603+0000","last_active":"2024-05-09T08:03:02.513488+0000","last_peered":"2024-05-09T08:03:02.513488+0000","last_clean":"2024-05-09T08:03:02.513488+0000","last_became_active":"2024-05-09T08:02:20.055365+0000","last_became_peered":"2024-05-09T08:02:20.055365+0000","last_unstale":"2024-05-09T08:03:02.513488+0000","last_undegraded":"2024-05-09T08:03:02.513488+0000","last_fullsized":"2024-05-09T08:03:02.513488+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:48:38.283562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.613397+0000","last_change":"2024-05-09T08:02:15.890980+0000","last_active":"2024-05-09T08:03:02.613397+0000","last_peered":"2024-05-09T08:03:02.613397+0000","last_clean":"2024-05-09T08:03:02.613397+0000","last_became_active":"2024-05-09T08:02:15.890346+0000","last_became_peered":"2024-05-09T08:02:15.890346+0000","last_unstale":"2024-05-09T08:03:02.613397+0000","last_undegraded":"2024-05-09T08:03:02.613397+0000","last_fullsized":"2024-05-09T08:03:02.613397+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:29:59.441314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.606148+0000","last_change":"2024-05-09T08:02:14.876206+0000","last_active":"2024-05-09T08:03:02.606148+0000","last_peered":"2024-05-09T08:03:02.606148+0000","last_clean":"2024-05-09T08:03:02.606148+0000","last_became_active":"2024-05-09T08:02:14.875894+0000","last_became_peered":"2024-05-09T08:02:14.875894+0000","last_unstale":"2024-05-09T08:03:02.606148+0000","last_undegraded":"2024-05-09T08:03:02.606148+0000","last_fullsized":"2024-05-09T08:03:02.606148+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:49:53.882513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.312428+0000","last_change":"2024-05-09T08:02:20.055222+0000","last_active":"2024-05-09T08:03:02.312428+0000","last_peered":"2024-05-09T08:03:02.312428+0000","last_clean":"2024-05-09T08:03:02.312428+0000","last_became_active":"2024-05-09T08:02:20.053802+0000","last_became_peered":"2024-05-09T08:02:20.053802+0000","last_unstale":"2024-05-09T08:03:02.312428+0000","last_undegraded":"2024-05-09T08:03:02.312428+0000","last_fullsized":"2024-05-09T08:03:02.312428+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:42:00.939587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.313378+0000","last_change":"2024-05-09T08:02:15.904528+0000","last_active":"2024-05-09T08:03:02.313378+0000","last_peered":"2024-05-09T08:03:02.313378+0000","last_clean":"2024-05-09T08:03:02.313378+0000","last_became_active":"2024-05-09T08:02:15.904049+0000","last_became_peered":"2024-05-09T08:02:15.904049+0000","last_unstale":"2024-05-09T08:03:02.313378+0000","last_undegraded":"2024-05-09T08:03:02.313378+0000","last_fullsized":"2024-05-09T08:03:02.313378+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:53:11.390395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.612917+0000","last_change":"2024-05-09T08:02:14.872897+0000","last_active":"2024-05-09T08:03:02.612917+0000","last_peered":"2024-05-09T08:03:02.612917+0000","last_clean":"2024-05-09T08:03:02.612917+0000","last_became_active":"2024-05-09T08:02:14.872096+0000","last_became_peered":"2024-05-09T08:02:14.872096+0000","last_unstale":"2024-05-09T08:03:02.612917+0000","last_undegraded":"2024-05-09T08:03:02.612917+0000","last_fullsized":"2024-05-09T08:03:02.612917+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:00:40.599475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.610605+0000","last_change":"2024-05-09T08:02:19.963716+0000","last_active":"2024-05-09T08:03:02.610605+0000","last_peered":"2024-05-09T08:03:02.610605+0000","last_clean":"2024-05-09T08:03:02.610605+0000","last_became_active":"2024-05-09T08:02:19.962922+0000","last_became_peered":"2024-05-09T08:02:19.962922+0000","last_unstale":"2024-05-09T08:03:02.610605+0000","last_undegraded":"2024-05-09T08:03:02.610605+0000","last_fullsized":"2024-05-09T08:03:02.610605+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:42:00.939587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.314904+0000","last_change":"2024-05-09T08:02:15.901450+0000","last_active":"2024-05-09T08:03:02.314904+0000","last_peered":"2024-05-09T08:03:02.314904+0000","last_clean":"2024-05-09T08:03:02.314904+0000","last_became_active":"2024-05-09T08:02:15.901180+0000","last_became_peered":"2024-05-09T08:02:15.901180+0000","last_unstale":"2024-05-09T08:03:02.314904+0000","last_undegraded":"2024-05-09T08:03:02.314904+0000","last_fullsized":"2024-05-09T08:03:02.314904+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:43:02.075442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.313301+0000","last_change":"2024-05-09T08:02:14.876518+0000","last_active":"2024-05-09T08:03:02.313301+0000","last_peered":"2024-05-09T08:03:02.313301+0000","last_clean":"2024-05-09T08:03:02.313301+0000","last_became_active":"2024-05-09T08:02:14.876175+0000","last_became_peered":"2024-05-09T08:02:14.876175+0000","last_unstale":"2024-05-09T08:03:02.313301+0000","last_undegraded":"2024-05-09T08:03:02.313301+0000","last_fullsized":"2024-05-09T08:03:02.313301+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:00:40.599475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.517496+0000","last_change":"2024-05-09T08:03:02.516595+0000","last_active":"2024-05-09T08:03:02.517496+0000","last_peered":"2024-05-09T08:03:02.517496+0000","last_clean":"2024-05-09T08:03:02.517496+0000","last_became_active":"2024-05-09T08:03:02.516104+0000","last_became_peered":"2024-05-09T08:03:02.516104+0000","last_unstale":"2024-05-09T08:03:02.517496+0000","last_undegraded":"2024-05-09T08:03:02.517496+0000","last_fullsized":"2024-05-09T08:03:02.517496+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:47:30.081323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3,6],"acting":[5,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.610756+0000","last_change":"2024-05-09T08:02:15.891774+0000","last_active":"2024-05-09T08:03:02.610756+0000","last_peered":"2024-05-09T08:03:02.610756+0000","last_clean":"2024-05-09T08:03:02.610756+0000","last_became_active":"2024-05-09T08:02:15.891407+0000","last_became_peered":"2024-05-09T08:02:15.891407+0000","last_unstale":"2024-05-09T08:03:02.610756+0000","last_undegraded":"2024-05-09T08:03:02.610756+0000","last_fullsized":"2024-05-09T08:03:02.610756+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:05:02.181367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.319052+0000","last_change":"2024-05-09T08:02:14.883655+0000","last_active":"2024-05-09T08:03:02.319052+0000","last_peered":"2024-05-09T08:03:02.319052+0000","last_clean":"2024-05-09T08:03:02.319052+0000","last_became_active":"2024-05-09T08:02:14.882359+0000","last_became_peered":"2024-05-09T08:02:14.882359+0000","last_unstale":"2024-05-09T08:03:02.319052+0000","last_undegraded":"2024-05-09T08:03:02.319052+0000","last_fullsized":"2024-05-09T08:03:02.319052+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:46:11.160815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.611626+0000","last_change":"2024-05-09T08:02:19.953319+0000","last_active":"2024-05-09T08:03:02.611626+0000","last_peered":"2024-05-09T08:03:02.611626+0000","last_clean":"2024-05-09T08:03:02.611626+0000","last_became_active":"2024-05-09T08:02:19.952740+0000","last_became_peered":"2024-05-09T08:02:19.952740+0000","last_unstale":"2024-05-09T08:03:02.611626+0000","last_undegraded":"2024-05-09T08:03:02.611626+0000","last_fullsized":"2024-05-09T08:03:02.611626+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:25:28.804048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.311259+0000","last_change":"2024-05-09T08:02:15.913828+0000","last_active":"2024-05-09T08:03:02.311259+0000","last_peered":"2024-05-09T08:03:02.311259+0000","last_clean":"2024-05-09T08:03:02.311259+0000","last_became_active":"2024-05-09T08:02:15.913313+0000","last_became_peered":"2024-05-09T08:02:15.913313+0000","last_unstale":"2024-05-09T08:03:02.311259+0000","last_undegraded":"2024-05-09T08:03:02.311259+0000","last_fullsized":"2024-05-09T08:03:02.311259+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:20:09.935394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.605151+0000","last_change":"2024-05-09T08:02:14.876438+0000","last_active":"2024-05-09T08:03:02.605151+0000","last_peered":"2024-05-09T08:03:02.605151+0000","last_clean":"2024-05-09T08:03:02.605151+0000","last_became_active":"2024-05-09T08:02:14.876146+0000","last_became_peered":"2024-05-09T08:02:14.876146+0000","last_unstale":"2024-05-09T08:03:02.605151+0000","last_undegraded":"2024-05-09T08:03:02.605151+0000","last_fullsized":"2024-05-09T08:03:02.605151+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:36:49.629991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.312518+0000","last_change":"2024-05-09T08:02:20.051067+0000","last_active":"2024-05-09T08:03:02.312518+0000","last_peered":"2024-05-09T08:03:02.312518+0000","last_clean":"2024-05-09T08:03:02.312518+0000","last_became_active":"2024-05-09T08:02:20.050593+0000","last_became_peered":"2024-05-09T08:02:20.050593+0000","last_unstale":"2024-05-09T08:03:02.312518+0000","last_undegraded":"2024-05-09T08:03:02.312518+0000","last_fullsized":"2024-05-09T08:03:02.312518+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:25:28.804048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.513850+0000","last_change":"2024-05-09T08:02:15.899137+0000","last_active":"2024-05-09T08:03:02.513850+0000","last_peered":"2024-05-09T08:03:02.513850+0000","last_clean":"2024-05-09T08:03:02.513850+0000","last_became_active":"2024-05-09T08:02:15.898766+0000","last_became_peered":"2024-05-09T08:02:15.898766+0000","last_unstale":"2024-05-09T08:03:02.513850+0000","last_undegraded":"2024-05-09T08:03:02.513850+0000","last_fullsized":"2024-05-09T08:03:02.513850+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:47:32.605288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.312721+0000","last_change":"2024-05-09T08:02:14.868850+0000","last_active":"2024-05-09T08:03:02.312721+0000","last_peered":"2024-05-09T08:03:02.312721+0000","last_clean":"2024-05-09T08:03:02.312721+0000","last_became_active":"2024-05-09T08:02:14.868384+0000","last_became_peered":"2024-05-09T08:02:14.868384+0000","last_unstale":"2024-05-09T08:03:02.312721+0000","last_undegraded":"2024-05-09T08:03:02.312721+0000","last_fullsized":"2024-05-09T08:03:02.312721+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:14:52.920972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":181,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.605375+0000","last_change":"2024-05-09T08:02:11.559804+0000","last_active":"2024-05-09T08:03:02.605375+0000","last_peered":"2024-05-09T08:03:02.605375+0000","last_clean":"2024-05-09T08:03:02.605375+0000","last_became_active":"2024-05-09T08:02:11.558761+0000","last_became_peered":"2024-05-09T08:02:11.558761+0000","last_unstale":"2024-05-09T08:03:02.605375+0000","last_undegraded":"2024-05-09T08:03:02.605375+0000","last_fullsized":"2024-05-09T08:03:02.605375+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:09.800143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:09.800143+0000","last_clean_scrub_stamp":"2024-05-09T08:02:09.800143+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:18:35.964149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.310459+0000","last_change":"2024-05-09T08:02:20.054746+0000","last_active":"2024-05-09T08:03:02.310459+0000","last_peered":"2024-05-09T08:03:02.310459+0000","last_clean":"2024-05-09T08:03:02.310459+0000","last_became_active":"2024-05-09T08:02:20.054186+0000","last_became_peered":"2024-05-09T08:02:20.054186+0000","last_unstale":"2024-05-09T08:03:02.310459+0000","last_undegraded":"2024-05-09T08:03:02.310459+0000","last_fullsized":"2024-05-09T08:03:02.310459+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:53:15.458120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.605779+0000","last_change":"2024-05-09T08:02:15.889657+0000","last_active":"2024-05-09T08:03:02.605779+0000","last_peered":"2024-05-09T08:03:02.605779+0000","last_clean":"2024-05-09T08:03:02.605779+0000","last_became_active":"2024-05-09T08:02:15.889381+0000","last_became_peered":"2024-05-09T08:02:15.889381+0000","last_unstale":"2024-05-09T08:03:02.605779+0000","last_undegraded":"2024-05-09T08:03:02.605779+0000","last_fullsized":"2024-05-09T08:03:02.605779+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:59:14.957806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.318796+0000","last_change":"2024-05-09T08:02:14.868892+0000","last_active":"2024-05-09T08:03:02.318796+0000","last_peered":"2024-05-09T08:03:02.318796+0000","last_clean":"2024-05-09T08:03:02.318796+0000","last_became_active":"2024-05-09T08:02:14.868513+0000","last_became_peered":"2024-05-09T08:02:14.868513+0000","last_unstale":"2024-05-09T08:03:02.318796+0000","last_undegraded":"2024-05-09T08:03:02.318796+0000","last_fullsized":"2024-05-09T08:03:02.318796+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:19:31.913236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.309986+0000","last_change":"2024-05-09T08:02:20.053097+0000","last_active":"2024-05-09T08:03:02.309986+0000","last_peered":"2024-05-09T08:03:02.309986+0000","last_clean":"2024-05-09T08:03:02.309986+0000","last_became_active":"2024-05-09T08:02:20.052653+0000","last_became_peered":"2024-05-09T08:02:20.052653+0000","last_unstale":"2024-05-09T08:03:02.309986+0000","last_undegraded":"2024-05-09T08:03:02.309986+0000","last_fullsized":"2024-05-09T08:03:02.309986+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:25:05.306541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.313567+0000","last_change":"2024-05-09T08:02:15.899059+0000","last_active":"2024-05-09T08:03:02.313567+0000","last_peered":"2024-05-09T08:03:02.313567+0000","last_clean":"2024-05-09T08:03:02.313567+0000","last_became_active":"2024-05-09T08:02:15.898692+0000","last_became_peered":"2024-05-09T08:02:15.898692+0000","last_unstale":"2024-05-09T08:03:02.313567+0000","last_undegraded":"2024-05-09T08:03:02.313567+0000","last_fullsized":"2024-05-09T08:03:02.313567+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:52:36.461913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"24'1","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.611806+0000","last_change":"2024-05-09T08:02:14.873071+0000","last_active":"2024-05-09T08:03:02.611806+0000","last_peered":"2024-05-09T08:03:02.611806+0000","last_clean":"2024-05-09T08:03:02.611806+0000","last_became_active":"2024-05-09T08:02:14.872556+0000","last_became_peered":"2024-05-09T08:02:14.872556+0000","last_unstale":"2024-05-09T08:03:02.611806+0000","last_undegraded":"2024-05-09T08:03:02.611806+0000","last_fullsized":"2024-05-09T08:03:02.611806+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:57:06.093718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.313822+0000","last_change":"2024-05-09T08:02:19.960414+0000","last_active":"2024-05-09T08:03:02.313822+0000","last_peered":"2024-05-09T08:03:02.313822+0000","last_clean":"2024-05-09T08:03:02.313822+0000","last_became_active":"2024-05-09T08:02:19.957941+0000","last_became_peered":"2024-05-09T08:02:19.957941+0000","last_unstale":"2024-05-09T08:03:02.313822+0000","last_undegraded":"2024-05-09T08:03:02.313822+0000","last_fullsized":"2024-05-09T08:03:02.313822+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:41:31.470546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.606213+0000","last_change":"2024-05-09T08:02:15.884395+0000","last_active":"2024-05-09T08:03:02.606213+0000","last_peered":"2024-05-09T08:03:02.606213+0000","last_clean":"2024-05-09T08:03:02.606213+0000","last_became_active":"2024-05-09T08:02:15.883788+0000","last_became_peered":"2024-05-09T08:02:15.883788+0000","last_unstale":"2024-05-09T08:03:02.606213+0000","last_undegraded":"2024-05-09T08:03:02.606213+0000","last_fullsized":"2024-05-09T08:03:02.606213+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:48:34.215837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"24'2","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.319378+0000","last_change":"2024-05-09T08:02:14.885044+0000","last_active":"2024-05-09T08:03:02.319378+0000","last_peered":"2024-05-09T08:03:02.319378+0000","last_clean":"2024-05-09T08:03:02.319378+0000","last_became_active":"2024-05-09T08:02:14.884706+0000","last_became_peered":"2024-05-09T08:02:14.884706+0000","last_unstale":"2024-05-09T08:03:02.319378+0000","last_undegraded":"2024-05-09T08:03:02.319378+0000","last_fullsized":"2024-05-09T08:03:02.319378+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:14:52.920972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.312303+0000","last_change":"2024-05-09T08:02:20.050865+0000","last_active":"2024-05-09T08:03:02.312303+0000","last_peered":"2024-05-09T08:03:02.312303+0000","last_clean":"2024-05-09T08:03:02.312303+0000","last_became_active":"2024-05-09T08:02:20.050270+0000","last_became_peered":"2024-05-09T08:02:20.050270+0000","last_unstale":"2024-05-09T08:03:02.312303+0000","last_undegraded":"2024-05-09T08:03:02.312303+0000","last_fullsized":"2024-05-09T08:03:02.312303+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:48:38.283562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.605983+0000","last_change":"2024-05-09T08:02:15.884737+0000","last_active":"2024-05-09T08:03:02.605983+0000","last_peered":"2024-05-09T08:03:02.605983+0000","last_clean":"2024-05-09T08:03:02.605983+0000","last_became_active":"2024-05-09T08:02:15.884465+0000","last_became_peered":"2024-05-09T08:02:15.884465+0000","last_unstale":"2024-05-09T08:03:02.605983+0000","last_undegraded":"2024-05-09T08:03:02.605983+0000","last_fullsized":"2024-05-09T08:03:02.605983+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:52:36.461913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"24'1","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.313984+0000","last_change":"2024-05-09T08:02:14.875003+0000","last_active":"2024-05-09T08:03:02.313984+0000","last_peered":"2024-05-09T08:03:02.313984+0000","last_clean":"2024-05-09T08:03:02.313984+0000","last_became_active":"2024-05-09T08:02:14.874665+0000","last_became_peered":"2024-05-09T08:02:14.874665+0000","last_unstale":"2024-05-09T08:03:02.313984+0000","last_undegraded":"2024-05-09T08:03:02.313984+0000","last_fullsized":"2024-05-09T08:03:02.313984+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:18:39.991319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.313729+0000","last_change":"2024-05-09T08:02:19.952692+0000","last_active":"2024-05-09T08:03:02.313729+0000","last_peered":"2024-05-09T08:03:02.313729+0000","last_clean":"2024-05-09T08:03:02.313729+0000","last_became_active":"2024-05-09T08:02:19.952224+0000","last_became_peered":"2024-05-09T08:02:19.952224+0000","last_unstale":"2024-05-09T08:03:02.313729+0000","last_undegraded":"2024-05-09T08:03:02.313729+0000","last_fullsized":"2024-05-09T08:03:02.313729+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:47:36.673013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.605340+0000","last_change":"2024-05-09T08:02:15.883408+0000","last_active":"2024-05-09T08:03:02.605340+0000","last_peered":"2024-05-09T08:03:02.605340+0000","last_clean":"2024-05-09T08:03:02.605340+0000","last_became_active":"2024-05-09T08:02:15.883103+0000","last_became_peered":"2024-05-09T08:02:15.883103+0000","last_unstale":"2024-05-09T08:03:02.605340+0000","last_undegraded":"2024-05-09T08:03:02.605340+0000","last_fullsized":"2024-05-09T08:03:02.605340+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:53:11.390395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"24'8","reported_seq":33,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.317771+0000","last_change":"2024-05-09T08:02:14.873184+0000","last_active":"2024-05-09T08:03:02.317771+0000","last_peered":"2024-05-09T08:03:02.317771+0000","last_clean":"2024-05-09T08:03:02.317771+0000","last_became_active":"2024-05-09T08:02:14.871055+0000","last_became_peered":"2024-05-09T08:02:14.871055+0000","last_unstale":"2024-05-09T08:03:02.317771+0000","last_undegraded":"2024-05-09T08:03:02.317771+0000","last_fullsized":"2024-05-09T08:03:02.317771+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:19:56.315972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.312232+0000","last_change":"2024-05-09T08:02:19.950823+0000","last_active":"2024-05-09T08:03:02.312232+0000","last_peered":"2024-05-09T08:03:02.312232+0000","last_clean":"2024-05-09T08:03:02.312232+0000","last_became_active":"2024-05-09T08:02:19.950352+0000","last_became_peered":"2024-05-09T08:02:19.950352+0000","last_unstale":"2024-05-09T08:03:02.312232+0000","last_undegraded":"2024-05-09T08:03:02.312232+0000","last_fullsized":"2024-05-09T08:03:02.312232+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:59:19.025530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.310150+0000","last_change":"2024-05-09T08:02:15.899115+0000","last_active":"2024-05-09T08:03:02.310150+0000","last_peered":"2024-05-09T08:03:02.310150+0000","last_clean":"2024-05-09T08:03:02.310150+0000","last_became_active":"2024-05-09T08:02:15.898811+0000","last_became_peered":"2024-05-09T08:02:15.898811+0000","last_unstale":"2024-05-09T08:03:02.310150+0000","last_undegraded":"2024-05-09T08:03:02.310150+0000","last_fullsized":"2024-05-09T08:03:02.310150+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:43:06.938904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.310107+0000","last_change":"2024-05-09T08:02:14.860818+0000","last_active":"2024-05-09T08:03:02.310107+0000","last_peered":"2024-05-09T08:03:02.310107+0000","last_clean":"2024-05-09T08:03:02.310107+0000","last_became_active":"2024-05-09T08:02:14.860518+0000","last_became_peered":"2024-05-09T08:02:14.860518+0000","last_unstale":"2024-05-09T08:03:02.310107+0000","last_undegraded":"2024-05-09T08:03:02.310107+0000","last_fullsized":"2024-05-09T08:03:02.310107+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:25:23.306827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.515154+0000","last_change":"2024-05-09T08:02:20.053251+0000","last_active":"2024-05-09T08:03:02.515154+0000","last_peered":"2024-05-09T08:03:02.515154+0000","last_clean":"2024-05-09T08:03:02.515154+0000","last_became_active":"2024-05-09T08:02:20.052783+0000","last_became_peered":"2024-05-09T08:02:20.052783+0000","last_unstale":"2024-05-09T08:03:02.515154+0000","last_undegraded":"2024-05-09T08:03:02.515154+0000","last_fullsized":"2024-05-09T08:03:02.515154+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:16:41.863341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.605776+0000","last_change":"2024-05-09T08:02:15.883972+0000","last_active":"2024-05-09T08:03:02.605776+0000","last_peered":"2024-05-09T08:03:02.605776+0000","last_clean":"2024-05-09T08:03:02.605776+0000","last_became_active":"2024-05-09T08:02:15.883613+0000","last_became_peered":"2024-05-09T08:02:15.883613+0000","last_unstale":"2024-05-09T08:03:02.605776+0000","last_undegraded":"2024-05-09T08:03:02.605776+0000","last_fullsized":"2024-05-09T08:03:02.605776+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:41:27.402821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.313157+0000","last_change":"2024-05-09T08:02:14.883366+0000","last_active":"2024-05-09T08:03:02.313157+0000","last_peered":"2024-05-09T08:03:02.313157+0000","last_clean":"2024-05-09T08:03:02.313157+0000","last_became_active":"2024-05-09T08:02:14.883135+0000","last_became_peered":"2024-05-09T08:02:14.883135+0000","last_unstale":"2024-05-09T08:03:02.313157+0000","last_undegraded":"2024-05-09T08:03:02.313157+0000","last_fullsized":"2024-05-09T08:03:02.313157+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:32:48.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.514492+0000","last_change":"2024-05-09T08:02:20.050888+0000","last_active":"2024-05-09T08:03:02.514492+0000","last_peered":"2024-05-09T08:03:02.514492+0000","last_clean":"2024-05-09T08:03:02.514492+0000","last_became_active":"2024-05-09T08:02:20.050390+0000","last_became_peered":"2024-05-09T08:02:20.050390+0000","last_unstale":"2024-05-09T08:03:02.514492+0000","last_undegraded":"2024-05-09T08:03:02.514492+0000","last_fullsized":"2024-05-09T08:03:02.514492+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:10:24.797526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.313551+0000","last_change":"2024-05-09T08:02:15.899096+0000","last_active":"2024-05-09T08:03:02.313551+0000","last_peered":"2024-05-09T08:03:02.313551+0000","last_clean":"2024-05-09T08:03:02.313551+0000","last_became_active":"2024-05-09T08:02:15.898811+0000","last_became_peered":"2024-05-09T08:02:15.898811+0000","last_unstale":"2024-05-09T08:03:02.313551+0000","last_undegraded":"2024-05-09T08:03:02.313551+0000","last_fullsized":"2024-05-09T08:03:02.313551+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:28:51.229262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"24'2","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.314204+0000","last_change":"2024-05-09T08:02:14.868657+0000","last_active":"2024-05-09T08:03:02.314204+0000","last_peered":"2024-05-09T08:03:02.314204+0000","last_clean":"2024-05-09T08:03:02.314204+0000","last_became_active":"2024-05-09T08:02:14.868385+0000","last_became_peered":"2024-05-09T08:02:14.868385+0000","last_unstale":"2024-05-09T08:03:02.314204+0000","last_undegraded":"2024-05-09T08:03:02.314204+0000","last_fullsized":"2024-05-09T08:03:02.314204+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:46:11.160815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.313146+0000","last_change":"2024-05-09T08:02:20.053155+0000","last_active":"2024-05-09T08:03:02.313146+0000","last_peered":"2024-05-09T08:03:02.313146+0000","last_clean":"2024-05-09T08:03:02.313146+0000","last_became_active":"2024-05-09T08:02:20.052629+0000","last_became_peered":"2024-05-09T08:02:20.052629+0000","last_unstale":"2024-05-09T08:03:02.313146+0000","last_undegraded":"2024-05-09T08:03:02.313146+0000","last_fullsized":"2024-05-09T08:03:02.313146+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:35:54.628874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.605788+0000","last_change":"2024-05-09T08:02:15.905406+0000","last_active":"2024-05-09T08:03:02.605788+0000","last_peered":"2024-05-09T08:03:02.605788+0000","last_clean":"2024-05-09T08:03:02.605788+0000","last_became_active":"2024-05-09T08:02:15.905076+0000","last_became_peered":"2024-05-09T08:02:15.905076+0000","last_unstale":"2024-05-09T08:03:02.605788+0000","last_undegraded":"2024-05-09T08:03:02.605788+0000","last_fullsized":"2024-05-09T08:03:02.605788+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:25:01.238816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"25'10","reported_seq":36,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.310546+0000","last_change":"2024-05-09T08:02:14.880410+0000","last_active":"2024-05-09T08:03:02.310546+0000","last_peered":"2024-05-09T08:03:02.310546+0000","last_clean":"2024-05-09T08:03:02.310546+0000","last_became_active":"2024-05-09T08:02:14.880045+0000","last_became_peered":"2024-05-09T08:02:14.880045+0000","last_unstale":"2024-05-09T08:03:02.310546+0000","last_undegraded":"2024-05-09T08:03:02.310546+0000","last_fullsized":"2024-05-09T08:03:02.310546+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:19:56.315972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.610797+0000","last_change":"2024-05-09T08:02:19.963378+0000","last_active":"2024-05-09T08:03:02.610797+0000","last_peered":"2024-05-09T08:03:02.610797+0000","last_clean":"2024-05-09T08:03:02.610797+0000","last_became_active":"2024-05-09T08:02:19.958964+0000","last_became_peered":"2024-05-09T08:02:19.958964+0000","last_unstale":"2024-05-09T08:03:02.610797+0000","last_undegraded":"2024-05-09T08:03:02.610797+0000","last_fullsized":"2024-05-09T08:03:02.610797+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:41:31.470546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.314817+0000","last_change":"2024-05-09T08:02:15.888428+0000","last_active":"2024-05-09T08:03:02.314817+0000","last_peered":"2024-05-09T08:03:02.314817+0000","last_clean":"2024-05-09T08:03:02.314817+0000","last_became_active":"2024-05-09T08:02:15.888071+0000","last_became_peered":"2024-05-09T08:02:15.888071+0000","last_unstale":"2024-05-09T08:03:02.314817+0000","last_undegraded":"2024-05-09T08:03:02.314817+0000","last_fullsized":"2024-05-09T08:03:02.314817+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:36:50.643632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"24'3","reported_seq":28,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.611487+0000","last_change":"2024-05-09T08:02:14.872697+0000","last_active":"2024-05-09T08:03:02.611487+0000","last_peered":"2024-05-09T08:03:02.611487+0000","last_clean":"2024-05-09T08:03:02.611487+0000","last_became_active":"2024-05-09T08:02:14.871945+0000","last_became_peered":"2024-05-09T08:02:14.871945+0000","last_unstale":"2024-05-09T08:03:02.611487+0000","last_undegraded":"2024-05-09T08:03:02.611487+0000","last_fullsized":"2024-05-09T08:03:02.611487+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:19:56.315972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.314051+0000","last_change":"2024-05-09T08:02:19.953635+0000","last_active":"2024-05-09T08:03:02.314051+0000","last_peered":"2024-05-09T08:03:02.314051+0000","last_clean":"2024-05-09T08:03:02.314051+0000","last_became_active":"2024-05-09T08:02:19.953166+0000","last_became_peered":"2024-05-09T08:02:19.953166+0000","last_unstale":"2024-05-09T08:03:02.314051+0000","last_undegraded":"2024-05-09T08:03:02.314051+0000","last_fullsized":"2024-05-09T08:03:02.314051+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:18:40.377352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.314372+0000","last_change":"2024-05-09T08:02:15.905285+0000","last_active":"2024-05-09T08:03:02.314372+0000","last_peered":"2024-05-09T08:03:02.314372+0000","last_clean":"2024-05-09T08:03:02.314372+0000","last_became_active":"2024-05-09T08:02:15.904898+0000","last_became_peered":"2024-05-09T08:02:15.904898+0000","last_unstale":"2024-05-09T08:03:02.314372+0000","last_undegraded":"2024-05-09T08:03:02.314372+0000","last_fullsized":"2024-05-09T08:03:02.314372+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:47:32.605288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.313275+0000","last_change":"2024-05-09T08:02:14.868682+0000","last_active":"2024-05-09T08:03:02.313275+0000","last_peered":"2024-05-09T08:03:02.313275+0000","last_clean":"2024-05-09T08:03:02.313275+0000","last_became_active":"2024-05-09T08:02:14.868294+0000","last_became_peered":"2024-05-09T08:02:14.868294+0000","last_unstale":"2024-05-09T08:03:02.313275+0000","last_undegraded":"2024-05-09T08:03:02.313275+0000","last_fullsized":"2024-05-09T08:03:02.313275+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:57:06.093718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.513640+0000","last_change":"2024-05-09T08:02:19.956115+0000","last_active":"2024-05-09T08:03:02.513640+0000","last_peered":"2024-05-09T08:03:02.513640+0000","last_clean":"2024-05-09T08:03:02.513640+0000","last_became_active":"2024-05-09T08:02:19.955662+0000","last_became_peered":"2024-05-09T08:02:19.955662+0000","last_unstale":"2024-05-09T08:03:02.513640+0000","last_undegraded":"2024-05-09T08:03:02.513640+0000","last_fullsized":"2024-05-09T08:03:02.513640+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:15:28.286330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.313165+0000","last_change":"2024-05-09T08:02:15.888406+0000","last_active":"2024-05-09T08:03:02.313165+0000","last_peered":"2024-05-09T08:03:02.313165+0000","last_clean":"2024-05-09T08:03:02.313165+0000","last_became_active":"2024-05-09T08:02:15.888090+0000","last_became_peered":"2024-05-09T08:02:15.888090+0000","last_unstale":"2024-05-09T08:03:02.313165+0000","last_undegraded":"2024-05-09T08:03:02.313165+0000","last_fullsized":"2024-05-09T08:03:02.313165+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:53:23.178610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.310785+0000","last_change":"2024-05-09T08:02:14.862463+0000","last_active":"2024-05-09T08:03:02.310785+0000","last_peered":"2024-05-09T08:03:02.310785+0000","last_clean":"2024-05-09T08:03:02.310785+0000","last_became_active":"2024-05-09T08:02:14.862104+0000","last_became_peered":"2024-05-09T08:02:14.862104+0000","last_unstale":"2024-05-09T08:03:02.310785+0000","last_undegraded":"2024-05-09T08:03:02.310785+0000","last_fullsized":"2024-05-09T08:03:02.310785+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:32:48.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.513779+0000","last_change":"2024-05-09T08:02:20.053600+0000","last_active":"2024-05-09T08:03:02.513779+0000","last_peered":"2024-05-09T08:03:02.513779+0000","last_clean":"2024-05-09T08:03:02.513779+0000","last_became_active":"2024-05-09T08:02:20.052485+0000","last_became_peered":"2024-05-09T08:02:20.052485+0000","last_unstale":"2024-05-09T08:03:02.513779+0000","last_undegraded":"2024-05-09T08:03:02.513779+0000","last_fullsized":"2024-05-09T08:03:02.513779+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:35:54.628874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.311909+0000","last_change":"2024-05-09T08:02:20.054016+0000","last_active":"2024-05-09T08:03:02.311909+0000","last_peered":"2024-05-09T08:03:02.311909+0000","last_clean":"2024-05-09T08:03:02.311909+0000","last_became_active":"2024-05-09T08:02:20.053250+0000","last_became_peered":"2024-05-09T08:02:20.053250+0000","last_unstale":"2024-05-09T08:03:02.311909+0000","last_undegraded":"2024-05-09T08:03:02.311909+0000","last_fullsized":"2024-05-09T08:03:02.311909+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:52:40.529638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.610879+0000","last_change":"2024-05-09T08:02:15.907080+0000","last_active":"2024-05-09T08:03:02.610879+0000","last_peered":"2024-05-09T08:03:02.610879+0000","last_clean":"2024-05-09T08:03:02.610879+0000","last_became_active":"2024-05-09T08:02:15.906600+0000","last_became_peered":"2024-05-09T08:02:15.906600+0000","last_unstale":"2024-05-09T08:03:02.610879+0000","last_undegraded":"2024-05-09T08:03:02.610879+0000","last_fullsized":"2024-05-09T08:03:02.610879+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:43:06.938904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.312696+0000","last_change":"2024-05-09T08:02:14.870990+0000","last_active":"2024-05-09T08:03:02.312696+0000","last_peered":"2024-05-09T08:03:02.312696+0000","last_clean":"2024-05-09T08:03:02.312696+0000","last_became_active":"2024-05-09T08:02:14.869818+0000","last_became_peered":"2024-05-09T08:02:14.869818+0000","last_unstale":"2024-05-09T08:03:02.312696+0000","last_undegraded":"2024-05-09T08:03:02.312696+0000","last_fullsized":"2024-05-09T08:03:02.312696+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:25:23.306827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.310516+0000","last_change":"2024-05-09T08:02:19.952582+0000","last_active":"2024-05-09T08:03:02.310516+0000","last_peered":"2024-05-09T08:03:02.310516+0000","last_clean":"2024-05-09T08:03:02.310516+0000","last_became_active":"2024-05-09T08:02:19.952117+0000","last_became_peered":"2024-05-09T08:02:19.952117+0000","last_unstale":"2024-05-09T08:03:02.310516+0000","last_undegraded":"2024-05-09T08:03:02.310516+0000","last_fullsized":"2024-05-09T08:03:02.310516+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:42:00.939587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.513811+0000","last_change":"2024-05-09T08:02:15.913109+0000","last_active":"2024-05-09T08:03:02.513811+0000","last_peered":"2024-05-09T08:03:02.513811+0000","last_clean":"2024-05-09T08:03:02.513811+0000","last_became_active":"2024-05-09T08:02:15.912883+0000","last_became_peered":"2024-05-09T08:02:15.912883+0000","last_unstale":"2024-05-09T08:03:02.513811+0000","last_undegraded":"2024-05-09T08:03:02.513811+0000","last_fullsized":"2024-05-09T08:03:02.513811+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:29:59.441314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.514075+0000","last_change":"2024-05-09T08:02:14.885256+0000","last_active":"2024-05-09T08:03:02.514075+0000","last_peered":"2024-05-09T08:03:02.514075+0000","last_clean":"2024-05-09T08:03:02.514075+0000","last_became_active":"2024-05-09T08:02:14.884965+0000","last_became_peered":"2024-05-09T08:02:14.884965+0000","last_unstale":"2024-05-09T08:03:02.514075+0000","last_undegraded":"2024-05-09T08:03:02.514075+0000","last_fullsized":"2024-05-09T08:03:02.514075+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:14:52.920972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"24'1","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.310239+0000","last_change":"2024-05-09T08:02:14.860339+0000","last_active":"2024-05-09T08:03:02.310239+0000","last_peered":"2024-05-09T08:03:02.310239+0000","last_clean":"2024-05-09T08:03:02.310239+0000","last_became_active":"2024-05-09T08:02:14.859917+0000","last_became_peered":"2024-05-09T08:02:14.859917+0000","last_unstale":"2024-05-09T08:03:02.310239+0000","last_undegraded":"2024-05-09T08:03:02.310239+0000","last_fullsized":"2024-05-09T08:03:02.310239+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:14:52.920972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.314048+0000","last_change":"2024-05-09T08:02:15.897001+0000","last_active":"2024-05-09T08:03:02.314048+0000","last_peered":"2024-05-09T08:03:02.314048+0000","last_clean":"2024-05-09T08:03:02.314048+0000","last_became_active":"2024-05-09T08:02:15.896708+0000","last_became_peered":"2024-05-09T08:02:15.896708+0000","last_unstale":"2024-05-09T08:03:02.314048+0000","last_undegraded":"2024-05-09T08:03:02.314048+0000","last_fullsized":"2024-05-09T08:03:02.314048+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:29:59.441314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.318372+0000","last_change":"2024-05-09T08:02:19.955369+0000","last_active":"2024-05-09T08:03:02.318372+0000","last_peered":"2024-05-09T08:03:02.318372+0000","last_clean":"2024-05-09T08:03:02.318372+0000","last_became_active":"2024-05-09T08:02:19.954829+0000","last_became_peered":"2024-05-09T08:02:19.954829+0000","last_unstale":"2024-05-09T08:03:02.318372+0000","last_undegraded":"2024-05-09T08:03:02.318372+0000","last_fullsized":"2024-05-09T08:03:02.318372+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:49:58.963879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"24'1","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.514396+0000","last_change":"2024-05-09T08:02:14.883856+0000","last_active":"2024-05-09T08:03:02.514396+0000","last_peered":"2024-05-09T08:03:02.514396+0000","last_clean":"2024-05-09T08:03:02.514396+0000","last_became_active":"2024-05-09T08:02:14.882610+0000","last_became_peered":"2024-05-09T08:02:14.882610+0000","last_unstale":"2024-05-09T08:03:02.514396+0000","last_undegraded":"2024-05-09T08:03:02.514396+0000","last_fullsized":"2024-05-09T08:03:02.514396+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:19:31.913236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.314155+0000","last_change":"2024-05-09T08:02:15.889974+0000","last_active":"2024-05-09T08:03:02.314155+0000","last_peered":"2024-05-09T08:03:02.314155+0000","last_clean":"2024-05-09T08:03:02.314155+0000","last_became_active":"2024-05-09T08:02:15.889650+0000","last_became_peered":"2024-05-09T08:02:15.889650+0000","last_unstale":"2024-05-09T08:03:02.314155+0000","last_undegraded":"2024-05-09T08:03:02.314155+0000","last_fullsized":"2024-05-09T08:03:02.314155+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:00:41.613116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.514719+0000","last_change":"2024-05-09T08:02:19.955362+0000","last_active":"2024-05-09T08:03:02.514719+0000","last_peered":"2024-05-09T08:03:02.514719+0000","last_clean":"2024-05-09T08:03:02.514719+0000","last_became_active":"2024-05-09T08:02:19.954947+0000","last_became_peered":"2024-05-09T08:02:19.954947+0000","last_unstale":"2024-05-09T08:03:02.514719+0000","last_undegraded":"2024-05-09T08:03:02.514719+0000","last_fullsized":"2024-05-09T08:03:02.514719+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:43:30.549966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.309981+0000","last_change":"2024-05-09T08:02:15.909379+0000","last_active":"2024-05-09T08:03:02.309981+0000","last_peered":"2024-05-09T08:03:02.309981+0000","last_clean":"2024-05-09T08:03:02.309981+0000","last_became_active":"2024-05-09T08:02:15.909086+0000","last_became_peered":"2024-05-09T08:02:15.909086+0000","last_unstale":"2024-05-09T08:03:02.309981+0000","last_undegraded":"2024-05-09T08:03:02.309981+0000","last_fullsized":"2024-05-09T08:03:02.309981+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:05:02.181367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.318734+0000","last_change":"2024-05-09T08:02:15.891242+0000","last_active":"2024-05-09T08:03:02.318734+0000","last_peered":"2024-05-09T08:03:02.318734+0000","last_clean":"2024-05-09T08:03:02.318734+0000","last_became_active":"2024-05-09T08:02:15.890640+0000","last_became_peered":"2024-05-09T08:02:15.890640+0000","last_unstale":"2024-05-09T08:03:02.318734+0000","last_undegraded":"2024-05-09T08:03:02.318734+0000","last_fullsized":"2024-05-09T08:03:02.318734+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:57:07.107359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.312946+0000","last_change":"2024-05-09T08:02:15.899001+0000","last_active":"2024-05-09T08:03:02.312946+0000","last_peered":"2024-05-09T08:03:02.312946+0000","last_clean":"2024-05-09T08:03:02.312946+0000","last_became_active":"2024-05-09T08:02:15.898695+0000","last_became_peered":"2024-05-09T08:02:15.898695+0000","last_unstale":"2024-05-09T08:03:02.312946+0000","last_undegraded":"2024-05-09T08:03:02.312946+0000","last_fullsized":"2024-05-09T08:03:02.312946+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:41:27.402821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.313935+0000","last_change":"2024-05-09T08:02:15.907374+0000","last_active":"2024-05-09T08:03:02.313935+0000","last_peered":"2024-05-09T08:03:02.313935+0000","last_clean":"2024-05-09T08:03:02.313935+0000","last_became_active":"2024-05-09T08:02:15.906767+0000","last_became_peered":"2024-05-09T08:02:15.906767+0000","last_unstale":"2024-05-09T08:03:02.313935+0000","last_undegraded":"2024-05-09T08:03:02.313935+0000","last_fullsized":"2024-05-09T08:03:02.313935+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:05:02.181367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.514680+0000","last_change":"2024-05-09T08:02:15.905326+0000","last_active":"2024-05-09T08:03:02.514680+0000","last_peered":"2024-05-09T08:03:02.514680+0000","last_clean":"2024-05-09T08:03:02.514680+0000","last_became_active":"2024-05-09T08:02:15.905083+0000","last_became_peered":"2024-05-09T08:02:15.905083+0000","last_unstale":"2024-05-09T08:03:02.514680+0000","last_undegraded":"2024-05-09T08:03:02.514680+0000","last_fullsized":"2024-05-09T08:03:02.514680+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:23:40.248821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.612136+0000","last_change":"2024-05-09T08:02:15.907753+0000","last_active":"2024-05-09T08:03:02.612136+0000","last_peered":"2024-05-09T08:03:02.612136+0000","last_clean":"2024-05-09T08:03:02.612136+0000","last_became_active":"2024-05-09T08:02:15.907480+0000","last_became_peered":"2024-05-09T08:02:15.907480+0000","last_unstale":"2024-05-09T08:03:02.612136+0000","last_undegraded":"2024-05-09T08:03:02.612136+0000","last_fullsized":"2024-05-09T08:03:02.612136+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:20:07.368327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.311045+0000","last_change":"2024-05-09T08:02:15.888605+0000","last_active":"2024-05-09T08:03:02.311045+0000","last_peered":"2024-05-09T08:03:02.311045+0000","last_clean":"2024-05-09T08:03:02.311045+0000","last_became_active":"2024-05-09T08:02:15.887981+0000","last_became_peered":"2024-05-09T08:02:15.887981+0000","last_unstale":"2024-05-09T08:03:02.311045+0000","last_undegraded":"2024-05-09T08:03:02.311045+0000","last_fullsized":"2024-05-09T08:03:02.311045+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:47:32.605288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.515394+0000","last_change":"2024-05-09T08:02:15.904744+0000","last_active":"2024-05-09T08:03:02.515394+0000","last_peered":"2024-05-09T08:03:02.515394+0000","last_clean":"2024-05-09T08:03:02.515394+0000","last_became_active":"2024-05-09T08:02:15.904334+0000","last_became_peered":"2024-05-09T08:02:15.904334+0000","last_unstale":"2024-05-09T08:03:02.515394+0000","last_undegraded":"2024-05-09T08:03:02.515394+0000","last_fullsized":"2024-05-09T08:03:02.515394+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:30:33.351158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.312480+0000","last_change":"2024-05-09T08:02:15.904835+0000","last_active":"2024-05-09T08:03:02.312480+0000","last_peered":"2024-05-09T08:03:02.312480+0000","last_clean":"2024-05-09T08:03:02.312480+0000","last_became_active":"2024-05-09T08:02:15.904416+0000","last_became_peered":"2024-05-09T08:02:15.904416+0000","last_unstale":"2024-05-09T08:03:02.312480+0000","last_undegraded":"2024-05-09T08:03:02.312480+0000","last_fullsized":"2024-05-09T08:03:02.312480+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:47:32.605288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.513916+0000","last_change":"2024-05-09T08:02:15.898324+0000","last_active":"2024-05-09T08:03:02.513916+0000","last_peered":"2024-05-09T08:03:02.513916+0000","last_clean":"2024-05-09T08:03:02.513916+0000","last_became_active":"2024-05-09T08:02:15.897981+0000","last_became_peered":"2024-05-09T08:02:15.897981+0000","last_unstale":"2024-05-09T08:03:02.513916+0000","last_undegraded":"2024-05-09T08:03:02.513916+0000","last_fullsized":"2024-05-09T08:03:02.513916+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:01:50.341133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.312828+0000","last_change":"2024-05-09T08:02:15.890945+0000","last_active":"2024-05-09T08:03:02.312828+0000","last_peered":"2024-05-09T08:03:02.312828+0000","last_clean":"2024-05-09T08:03:02.312828+0000","last_became_active":"2024-05-09T08:02:15.890647+0000","last_became_peered":"2024-05-09T08:02:15.890647+0000","last_unstale":"2024-05-09T08:03:02.312828+0000","last_undegraded":"2024-05-09T08:03:02.312828+0000","last_fullsized":"2024-05-09T08:03:02.312828+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:41:56.871863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.312546+0000","last_change":"2024-05-09T08:02:15.913728+0000","last_active":"2024-05-09T08:03:02.312546+0000","last_peered":"2024-05-09T08:03:02.312546+0000","last_clean":"2024-05-09T08:03:02.312546+0000","last_became_active":"2024-05-09T08:02:15.909818+0000","last_became_peered":"2024-05-09T08:02:15.909818+0000","last_unstale":"2024-05-09T08:03:02.312546+0000","last_undegraded":"2024-05-09T08:03:02.312546+0000","last_fullsized":"2024-05-09T08:03:02.312546+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:41:55.788033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.310499+0000","last_change":"2024-05-09T08:02:15.885025+0000","last_active":"2024-05-09T08:03:02.310499+0000","last_peered":"2024-05-09T08:03:02.310499+0000","last_clean":"2024-05-09T08:03:02.310499+0000","last_became_active":"2024-05-09T08:02:15.884517+0000","last_became_peered":"2024-05-09T08:02:15.884517+0000","last_unstale":"2024-05-09T08:03:02.310499+0000","last_undegraded":"2024-05-09T08:03:02.310499+0000","last_fullsized":"2024-05-09T08:03:02.310499+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:01:50.341133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.606110+0000","last_change":"2024-05-09T08:02:15.890306+0000","last_active":"2024-05-09T08:03:02.606110+0000","last_peered":"2024-05-09T08:03:02.606110+0000","last_clean":"2024-05-09T08:03:02.606110+0000","last_became_active":"2024-05-09T08:02:15.889998+0000","last_became_peered":"2024-05-09T08:02:15.889998+0000","last_unstale":"2024-05-09T08:03:02.606110+0000","last_undegraded":"2024-05-09T08:03:02.606110+0000","last_fullsized":"2024-05-09T08:03:02.606110+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:16:57.446444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.319098+0000","last_change":"2024-05-09T08:02:15.886417+0000","last_active":"2024-05-09T08:03:02.319098+0000","last_peered":"2024-05-09T08:03:02.319098+0000","last_clean":"2024-05-09T08:03:02.319098+0000","last_became_active":"2024-05-09T08:02:15.885525+0000","last_became_peered":"2024-05-09T08:02:15.885525+0000","last_unstale":"2024-05-09T08:03:02.319098+0000","last_undegraded":"2024-05-09T08:03:02.319098+0000","last_fullsized":"2024-05-09T08:03:02.319098+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:43:02.075442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.313589+0000","last_change":"2024-05-09T08:02:15.893172+0000","last_active":"2024-05-09T08:03:02.313589+0000","last_peered":"2024-05-09T08:03:02.313589+0000","last_clean":"2024-05-09T08:03:02.313589+0000","last_became_active":"2024-05-09T08:02:15.892886+0000","last_became_peered":"2024-05-09T08:02:15.892886+0000","last_unstale":"2024-05-09T08:03:02.313589+0000","last_undegraded":"2024-05-09T08:03:02.313589+0000","last_fullsized":"2024-05-09T08:03:02.313589+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:59:14.957806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.312550+0000","last_change":"2024-05-09T08:02:15.908659+0000","last_active":"2024-05-09T08:03:02.312550+0000","last_peered":"2024-05-09T08:03:02.312550+0000","last_clean":"2024-05-09T08:03:02.312550+0000","last_became_active":"2024-05-09T08:02:15.908307+0000","last_became_peered":"2024-05-09T08:02:15.908307+0000","last_unstale":"2024-05-09T08:03:02.312550+0000","last_undegraded":"2024-05-09T08:03:02.312550+0000","last_fullsized":"2024-05-09T08:03:02.312550+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:20:07.368327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.611181+0000","last_change":"2024-05-09T08:02:15.885800+0000","last_active":"2024-05-09T08:03:02.611181+0000","last_peered":"2024-05-09T08:03:02.611181+0000","last_clean":"2024-05-09T08:03:02.611181+0000","last_became_active":"2024-05-09T08:02:15.885185+0000","last_became_peered":"2024-05-09T08:02:15.885185+0000","last_unstale":"2024-05-09T08:03:02.611181+0000","last_undegraded":"2024-05-09T08:03:02.611181+0000","last_fullsized":"2024-05-09T08:03:02.611181+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:01:50.341133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.313497+0000","last_change":"2024-05-09T08:02:15.911496+0000","last_active":"2024-05-09T08:03:02.313497+0000","last_peered":"2024-05-09T08:03:02.313497+0000","last_clean":"2024-05-09T08:03:02.313497+0000","last_became_active":"2024-05-09T08:02:15.911142+0000","last_became_peered":"2024-05-09T08:02:15.911142+0000","last_unstale":"2024-05-09T08:03:02.313497+0000","last_undegraded":"2024-05-09T08:03:02.313497+0000","last_fullsized":"2024-05-09T08:03:02.313497+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:27:59.315750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.514303+0000","last_change":"2024-05-09T08:02:15.898847+0000","last_active":"2024-05-09T08:03:02.514303+0000","last_peered":"2024-05-09T08:03:02.514303+0000","last_clean":"2024-05-09T08:03:02.514303+0000","last_became_active":"2024-05-09T08:02:15.898480+0000","last_became_peered":"2024-05-09T08:02:15.898480+0000","last_unstale":"2024-05-09T08:03:02.514303+0000","last_undegraded":"2024-05-09T08:03:02.514303+0000","last_fullsized":"2024-05-09T08:03:02.514303+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:20:07.368327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.314781+0000","last_change":"2024-05-09T08:02:15.883117+0000","last_active":"2024-05-09T08:03:02.314781+0000","last_peered":"2024-05-09T08:03:02.314781+0000","last_clean":"2024-05-09T08:03:02.314781+0000","last_became_active":"2024-05-09T08:02:15.882141+0000","last_became_peered":"2024-05-09T08:02:15.882141+0000","last_unstale":"2024-05-09T08:03:02.314781+0000","last_undegraded":"2024-05-09T08:03:02.314781+0000","last_fullsized":"2024-05-09T08:03:02.314781+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:57:11.352151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.322462+0000","last_change":"2024-05-09T08:03:02.322462+0000","last_active":"2024-05-09T08:03:02.322462+0000","last_peered":"2024-05-09T08:03:02.322462+0000","last_clean":"2024-05-09T08:03:02.322462+0000","last_became_active":"2024-05-09T08:03:02.322073+0000","last_became_peered":"2024-05-09T08:03:02.322073+0000","last_unstale":"2024-05-09T08:03:02.322462+0000","last_undegraded":"2024-05-09T08:03:02.322462+0000","last_fullsized":"2024-05-09T08:03:02.322462+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:16:37.795617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.310964+0000","last_change":"2024-05-09T08:02:15.886482+0000","last_active":"2024-05-09T08:03:02.310964+0000","last_peered":"2024-05-09T08:03:02.310964+0000","last_clean":"2024-05-09T08:03:02.310964+0000","last_became_active":"2024-05-09T08:02:15.886183+0000","last_became_peered":"2024-05-09T08:02:15.886183+0000","last_unstale":"2024-05-09T08:03:02.310964+0000","last_undegraded":"2024-05-09T08:03:02.310964+0000","last_fullsized":"2024-05-09T08:03:02.310964+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:23:40.248821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.606193+0000","last_change":"2024-05-09T08:02:15.887506+0000","last_active":"2024-05-09T08:03:02.606193+0000","last_peered":"2024-05-09T08:03:02.606193+0000","last_clean":"2024-05-09T08:03:02.606193+0000","last_became_active":"2024-05-09T08:02:15.887225+0000","last_became_peered":"2024-05-09T08:02:15.887225+0000","last_unstale":"2024-05-09T08:03:02.606193+0000","last_undegraded":"2024-05-09T08:03:02.606193+0000","last_fullsized":"2024-05-09T08:03:02.606193+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:25:24.736323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.513523+0000","last_change":"2024-05-09T08:02:15.902065+0000","last_active":"2024-05-09T08:03:02.513523+0000","last_peered":"2024-05-09T08:03:02.513523+0000","last_clean":"2024-05-09T08:03:02.513523+0000","last_became_active":"2024-05-09T08:02:15.901766+0000","last_became_peered":"2024-05-09T08:02:15.901766+0000","last_unstale":"2024-05-09T08:03:02.513523+0000","last_undegraded":"2024-05-09T08:03:02.513523+0000","last_fullsized":"2024-05-09T08:03:02.513523+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:05:02.181367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.605331+0000","last_change":"2024-05-09T08:02:15.911058+0000","last_active":"2024-05-09T08:03:02.605331+0000","last_peered":"2024-05-09T08:03:02.605331+0000","last_clean":"2024-05-09T08:03:02.605331+0000","last_became_active":"2024-05-09T08:02:15.910716+0000","last_became_peered":"2024-05-09T08:02:15.910716+0000","last_unstale":"2024-05-09T08:03:02.605331+0000","last_undegraded":"2024-05-09T08:03:02.605331+0000","last_fullsized":"2024-05-09T08:03:02.605331+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:41:56.871863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.312711+0000","last_change":"2024-05-09T08:02:15.908653+0000","last_active":"2024-05-09T08:03:02.312711+0000","last_peered":"2024-05-09T08:03:02.312711+0000","last_clean":"2024-05-09T08:03:02.312711+0000","last_became_active":"2024-05-09T08:02:15.908290+0000","last_became_peered":"2024-05-09T08:02:15.908290+0000","last_unstale":"2024-05-09T08:03:02.312711+0000","last_undegraded":"2024-05-09T08:03:02.312711+0000","last_fullsized":"2024-05-09T08:03:02.312711+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:18:36.309627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.605649+0000","last_change":"2024-05-09T08:02:15.896652+0000","last_active":"2024-05-09T08:03:02.605649+0000","last_peered":"2024-05-09T08:03:02.605649+0000","last_clean":"2024-05-09T08:03:02.605649+0000","last_became_active":"2024-05-09T08:02:15.896384+0000","last_became_peered":"2024-05-09T08:02:15.896384+0000","last_unstale":"2024-05-09T08:03:02.605649+0000","last_undegraded":"2024-05-09T08:03:02.605649+0000","last_fullsized":"2024-05-09T08:03:02.605649+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:18:36.309627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":10,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.617215+0000","last_change":"2024-05-09T08:03:02.615230+0000","last_active":"2024-05-09T08:03:02.617215+0000","last_peered":"2024-05-09T08:03:02.617215+0000","last_clean":"2024-05-09T08:03:02.617215+0000","last_became_active":"2024-05-09T08:03:02.614387+0000","last_became_peered":"2024-05-09T08:03:02.614387+0000","last_unstale":"2024-05-09T08:03:02.617215+0000","last_undegraded":"2024-05-09T08:03:02.617215+0000","last_fullsized":"2024-05-09T08:03:02.617215+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:53:23.178610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.312999+0000","last_change":"2024-05-09T08:02:15.905078+0000","last_active":"2024-05-09T08:03:02.312999+0000","last_peered":"2024-05-09T08:03:02.312999+0000","last_clean":"2024-05-09T08:03:02.312999+0000","last_became_active":"2024-05-09T08:02:15.904736+0000","last_became_peered":"2024-05-09T08:02:15.904736+0000","last_unstale":"2024-05-09T08:03:02.312999+0000","last_undegraded":"2024-05-09T08:03:02.312999+0000","last_fullsized":"2024-05-09T08:03:02.312999+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:01:50.341133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.611372+0000","last_change":"2024-05-09T08:02:15.897743+0000","last_active":"2024-05-09T08:03:02.611372+0000","last_peered":"2024-05-09T08:03:02.611372+0000","last_clean":"2024-05-09T08:03:02.611372+0000","last_became_active":"2024-05-09T08:02:15.897367+0000","last_became_peered":"2024-05-09T08:02:15.897367+0000","last_unstale":"2024-05-09T08:03:02.611372+0000","last_undegraded":"2024-05-09T08:03:02.611372+0000","last_fullsized":"2024-05-09T08:03:02.611372+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:47:32.605288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":10,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.325400+0000","last_change":"2024-05-09T08:03:02.323519+0000","last_active":"2024-05-09T08:03:02.325400+0000","last_peered":"2024-05-09T08:03:02.325400+0000","last_clean":"2024-05-09T08:03:02.325400+0000","last_became_active":"2024-05-09T08:03:02.322397+0000","last_became_peered":"2024-05-09T08:03:02.322397+0000","last_unstale":"2024-05-09T08:03:02.325400+0000","last_undegraded":"2024-05-09T08:03:02.325400+0000","last_fullsized":"2024-05-09T08:03:02.325400+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:47:32.605288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574862,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32048,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339792,"statfs":{"total":96636764160,"available":96603947008,"internally_reserved":0,"allocated":888832,"data_stored":709919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574862,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31472,"kb_used_data":296,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340368,"statfs":{"total":96636764160,"available":96604536832,"internally_reserved":0,"allocated":303104,"data_stored":122811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574862,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31464,"kb_used_data":288,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340376,"statfs":{"total":96636764160,"available":96604545024,"internally_reserved":0,"allocated":294912,"data_stored":119059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574863,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31476,"kb_used_data":296,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340364,"statfs":{"total":96636764160,"available":96604532736,"internally_reserved":0,"allocated":303104,"data_stored":119095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574863,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27312,"kb_used_data":296,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344528,"statfs":{"total":96636764160,"available":96608796672,"internally_reserved":0,"allocated":303104,"data_stored":122811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574863,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31468,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340372,"statfs":{"total":96636764160,"available":96604540928,"internally_reserved":0,"allocated":290816,"data_stored":118969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574864,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27312,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344528,"statfs":{"total":96636764160,"available":96608796672,"internally_reserved":0,"allocated":299008,"data_stored":119129,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574863,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32060,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339780,"statfs":{"total":96636764160,"available":96603934720,"internally_reserved":0,"allocated":897024,"data_stored":709975,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T08:03:12.464 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-09T08:03:12.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:12.701+0000 7f7dd1800640 1 Processor -- start 2024-05-09T08:03:12.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:12.702+0000 7f7dd1800640 1 -- start start 2024-05-09T08:03:12.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:12.704+0000 7f7dd1800640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f7dcc15ba50 0x7f7dcc06d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:12.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:12.704+0000 7f7dd1800640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f7dcc15a5e0 0x7f7dcc0ca5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:12.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:12.704+0000 7f7dd1800640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7dcc15b550 con 0x7f7dcc0cd040 2024-05-09T08:03:12.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:12.704+0000 7f7dd1800640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f7dcc06cec0 con 0x7f7dcc15a5e0 2024-05-09T08:03:12.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:12.704+0000 7f7dd1800640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f7dcc06c8b0 con 0x7f7dcc15ba50 2024-05-09T08:03:12.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:12.704+0000 7f7dcbe00640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f7dcc0cd040 0x7f7dcc06da60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:42742/0 (socket says 172.21.15.1:42742) 2024-05-09T08:03:12.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:12.704+0000 7f7dcbe00640 1 -- 172.21.15.1:0/686428549 learned_addr learned my addr 172.21.15.1:0/686428549 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T08:03:12.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:12.704+0000 7f7dd0e00640 1 --2- 172.21.15.1:0/686428549 >> v2:172.21.15.1:3300/0 conn(0x7f7dcc15a5e0 0x7f7dcc0ca5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:12.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:12.705+0000 7f7dc3e00640 1 --2- 172.21.15.1:0/686428549 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f7dcc15ba50 0x7f7dcc06d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:12.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:12.705+0000 7f7dd0e00640 1 -- 172.21.15.1:0/686428549 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f7dcc15ba50 msgr2=0x7f7dcc06d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:03:12.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:12.705+0000 7f7dd0e00640 1 --2- 172.21.15.1:0/686428549 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f7dcc15ba50 0x7f7dcc06d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:12.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:12.705+0000 7f7dd0e00640 1 -- 172.21.15.1:0/686428549 >> v1:172.21.15.1:6789/0 conn(0x7f7dcc0cd040 legacy=0x7f7dcc06da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:12.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:12.705+0000 7f7dcb400640 1 -- 172.21.15.1:0/686428549 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 569140311 0 0) 0x7f7dcc15b550 con 0x7f7dcc0cd040 2024-05-09T08:03:12.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:12.705+0000 7f7dd0e00640 1 -- 172.21.15.1:0/686428549 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7dcc15af90 con 0x7f7dcc15a5e0 2024-05-09T08:03:12.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:12.705+0000 7f7dc3e00640 1 --2- 172.21.15.1:0/686428549 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f7dcc15ba50 0x7f7dcc06d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T08:03:12.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:12.706+0000 7f7dd0e00640 1 --2- 172.21.15.1:0/686428549 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7dcc15a5e0 0x7f7dcc0ca5f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f7dc4009d90 tx=0x7f7dc402a530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=551ad4255451e13c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:03:12.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:12.706+0000 7f7dcb400640 1 -- 172.21.15.1:0/686428549 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dc4007740 con 0x7f7dcc15a5e0 2024-05-09T08:03:12.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:12.706+0000 7f7dcb400640 1 -- 172.21.15.1:0/686428549 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7dc40078e0 con 0x7f7dcc15a5e0 2024-05-09T08:03:12.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:12.706+0000 7f7dcb400640 1 -- 172.21.15.1:0/686428549 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dc4007cb0 con 0x7f7dcc15a5e0 2024-05-09T08:03:12.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:12.707+0000 7f7dd1800640 1 -- 172.21.15.1:0/686428549 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7dcc15a5e0 msgr2=0x7f7dcc0ca5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:12.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:12.707+0000 7f7dd1800640 1 --2- 172.21.15.1:0/686428549 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7dcc15a5e0 0x7f7dcc0ca5f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f7dc4009d90 tx=0x7f7dc402a530 comp rx=0 tx=0).stop 2024-05-09T08:03:12.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:12.707+0000 7f7dd1800640 1 -- 172.21.15.1:0/686428549 shutdown_connections 2024-05-09T08:03:12.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:12.707+0000 7f7dd1800640 1 --2- 172.21.15.1:0/686428549 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f7dcc15ba50 0x7f7dcc06d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:12.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:12.707+0000 7f7dd1800640 1 --2- 172.21.15.1:0/686428549 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7dcc15a5e0 0x7f7dcc0ca5f0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:12.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:12.707+0000 7f7dd1800640 1 -- 172.21.15.1:0/686428549 >> 172.21.15.1:0/686428549 conn(0x7f7dcc0b1780 msgr2=0x7f7dcc0b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:03:12.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:12.707+0000 7f7dd1800640 1 -- 172.21.15.1:0/686428549 shutdown_connections 2024-05-09T08:03:12.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:12.707+0000 7f7dd1800640 1 -- 172.21.15.1:0/686428549 wait complete. 2024-05-09T08:03:12.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:12.709+0000 7f7dd1800640 1 Processor -- start 2024-05-09T08:03:12.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:12.709+0000 7f7dd1800640 1 -- start start 2024-05-09T08:03:12.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:12.710+0000 7f7dd1800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7dcc0cd040 0x7f7dcc1f4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:12.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:12.710+0000 7f7dd1800640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7dcc15a5e0 0x7f7dcc157090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:12.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:12.710+0000 7f7dd1800640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f7dcc15ba50 0x7f7dcc153e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:12.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:12.710+0000 7f7dd1800640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f7dcc15aa70 con 0x7f7dcc15a5e0 2024-05-09T08:03:12.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:12.710+0000 7f7dd1800640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7dcc06cb50 con 0x7f7dcc0cd040 2024-05-09T08:03:12.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:12.710+0000 7f7dd1800640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f7dcc15abf0 con 0x7f7dcc15ba50 2024-05-09T08:03:12.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:12.710+0000 7f7dd0e00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f7dcc15ba50 0x7f7dcc153e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:12.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:12.710+0000 7f7dcbe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7dcc0cd040 0x7f7dcc1f4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:12.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:12.710+0000 7f7dd0e00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f7dcc15ba50 0x7f7dcc153e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3301/0 says I am v2:172.21.15.1:49670/0 (socket says 172.21.15.1:49670) 2024-05-09T08:03:12.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:12.710+0000 7f7dd0e00640 1 -- 172.21.15.1:0/901521176 learned_addr learned my addr 172.21.15.1:0/901521176 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:03:12.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:12.711+0000 7f7dc3e00640 1 --2- 172.21.15.1:0/901521176 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7dcc15a5e0 0x7f7dcc157090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:12.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:12.711+0000 7f7dd0e00640 1 -- 172.21.15.1:0/901521176 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7dcc0cd040 msgr2=0x7f7dcc1f4720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:12.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:12.711+0000 7f7dd0e00640 1 --2- 172.21.15.1:0/901521176 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7dcc0cd040 0x7f7dcc1f4720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:12.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:12.711+0000 7f7dd0e00640 1 -- 172.21.15.1:0/901521176 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7dcc15a5e0 msgr2=0x7f7dcc157090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:12.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:12.711+0000 7f7dd0e00640 1 --2- 172.21.15.1:0/901521176 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7dcc15a5e0 0x7f7dcc157090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:12.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:12.711+0000 7f7dd0e00640 1 -- 172.21.15.1:0/901521176 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7dcc0c8450 con 0x7f7dcc15ba50 2024-05-09T08:03:12.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:12.711+0000 7f7dcbe00640 1 --2- 172.21.15.1:0/901521176 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7dcc0cd040 0x7f7dcc1f4720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:03:12.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:12.711+0000 7f7dc3e00640 1 --2- 172.21.15.1:0/901521176 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7dcc15a5e0 0x7f7dcc157090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T08:03:12.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:12.711+0000 7f7dd0e00640 1 --2- 172.21.15.1:0/901521176 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f7dcc15ba50 0x7f7dcc153e50 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f7dc402ab20 tx=0x7f7dc4004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:03:12.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:12.712+0000 7f7dc8c00640 1 -- 172.21.15.1:0/901521176 <== mon.2 v2:172.21.15.7:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dc403f070 con 0x7f7dcc15ba50 2024-05-09T08:03:12.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:12.712+0000 7f7dc8c00640 1 -- 172.21.15.1:0/901521176 <== mon.2 v2:172.21.15.7:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7dc4031c90 con 0x7f7dcc15ba50 2024-05-09T08:03:12.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:12.712+0000 7f7dd1800640 1 -- 172.21.15.1:0/901521176 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7dcc0c8730 con 0x7f7dcc15ba50 2024-05-09T08:03:12.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:12.712+0000 7f7dc8c00640 1 -- 172.21.15.1:0/901521176 <== mon.2 v2:172.21.15.7:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dc4044680 con 0x7f7dcc15ba50 2024-05-09T08:03:12.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:12.712+0000 7f7dd1800640 1 -- 172.21.15.1:0/901521176 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f7dcc1f3f20 con 0x7f7dcc15ba50 2024-05-09T08:03:12.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:12.713+0000 7f7dd1800640 1 -- 172.21.15.1:0/901521176 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7dcc04f660 con 0x7f7dcc15ba50 2024-05-09T08:03:12.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:12.714+0000 7f7dc8c00640 1 -- 172.21.15.1:0/901521176 <== mon.2 v2:172.21.15.7:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7dc403b410 con 0x7f7dcc15ba50 2024-05-09T08:03:12.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:12.715+0000 7f7dc8c00640 1 --2- 172.21.15.1:0/901521176 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f7da806be60 0x7f7da806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:12.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:12.715+0000 7f7dc8c00640 1 -- 172.21.15.1:0/901521176 <== mon.2 v2:172.21.15.7:3301/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7f7dc40aca80 con 0x7f7dcc15ba50 2024-05-09T08:03:12.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:12.715+0000 7f7dcbe00640 1 --2- 172.21.15.1:0/901521176 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f7da806be60 0x7f7da806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:12.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:12.716+0000 7f7dcbe00640 1 --2- 172.21.15.1:0/901521176 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f7da806be60 0x7f7da806e320 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f7dbc007fd0 tx=0x7f7dbc007de0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:03:12.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:12.720+0000 7f7dc8c00640 1 -- 172.21.15.1:0/901521176 <== mon.2 v2:172.21.15.7:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f7dc40aee20 con 0x7f7dcc15ba50 2024-05-09T08:03:12.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:12.920+0000 7f7dd1800640 1 -- 172.21.15.1:0/901521176 --> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7dcc1f4c60 con 0x7f7da806be60 2024-05-09T08:03:12.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:12.929+0000 7f7dc8c00640 1 -- 172.21.15.1:0/901521176 <== mgr.4114 v2:172.21.15.7:6832/3562612464 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332534 (secure 0 0 0) 0x7f7dbc001c10 con 0x7f7da806be60 2024-05-09T08:03:12.950 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T08:03:12.953 INFO:teuthology.orchestra.run.smithi001.stderr:dumped all 2024-05-09T08:03:12.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:12.937+0000 7f7dd1800640 1 -- 172.21.15.1:0/901521176 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f7da806be60 msgr2=0x7f7da806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:12.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:12.937+0000 7f7dd1800640 1 --2- 172.21.15.1:0/901521176 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f7da806be60 0x7f7da806e320 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f7dbc007fd0 tx=0x7f7dbc007de0 comp rx=0 tx=0).stop 2024-05-09T08:03:12.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:12.937+0000 7f7dd1800640 1 -- 172.21.15.1:0/901521176 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f7dcc15ba50 msgr2=0x7f7dcc153e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:12.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:12.937+0000 7f7dd1800640 1 --2- 172.21.15.1:0/901521176 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f7dcc15ba50 0x7f7dcc153e50 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f7dc402ab20 tx=0x7f7dc4004a40 comp rx=0 tx=0).stop 2024-05-09T08:03:12.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:12.937+0000 7f7dd1800640 1 -- 172.21.15.1:0/901521176 shutdown_connections 2024-05-09T08:03:12.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:12.937+0000 7f7dd1800640 1 --2- 172.21.15.1:0/901521176 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f7da806be60 0x7f7da806e320 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:12.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:12.937+0000 7f7dd1800640 1 --2- 172.21.15.1:0/901521176 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f7dcc15ba50 0x7f7dcc153e50 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:12.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:12.937+0000 7f7dd1800640 1 --2- 172.21.15.1:0/901521176 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7dcc15a5e0 0x7f7dcc157090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:12.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:12.937+0000 7f7dd1800640 1 --2- 172.21.15.1:0/901521176 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7dcc0cd040 0x7f7dcc1f4720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:12.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:12.937+0000 7f7dd1800640 1 -- 172.21.15.1:0/901521176 >> 172.21.15.1:0/901521176 conn(0x7f7dcc0b1780 msgr2=0x7f7dcc0ca270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:03:12.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:12.937+0000 7f7dd1800640 1 -- 172.21.15.1:0/901521176 shutdown_connections 2024-05-09T08:03:12.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:12.937+0000 7f7dd1800640 1 -- 172.21.15.1:0/901521176 wait complete. 2024-05-09T08:03:12.975 INFO:teuthology.orchestra.run.smithi001.stdout:{"pg_ready":true,"pg_map":{"version":52,"stamp":"2024-05-09T08:03:11.110878+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":208,"num_write_kb":2435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160,"ondisk_log_size":160,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":244612,"kb_used_data":3496,"kb_used_omap":12,"kb_used_meta":241011,"kb_avail":754730108,"statfs":{"total":773094113280,"available":772843630592,"internally_reserved":0,"allocated":3579904,"data_stored":2141768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":246795826},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.805863"},"pg_stats":[{"pgid":"2.1f","version":"24'4","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.314662+0000","last_change":"2024-05-09T08:02:14.875108+0000","last_active":"2024-05-09T08:03:02.314662+0000","last_peered":"2024-05-09T08:03:02.314662+0000","last_clean":"2024-05-09T08:03:02.314662+0000","last_became_active":"2024-05-09T08:02:14.874841+0000","last_became_peered":"2024-05-09T08:02:14.874841+0000","last_unstale":"2024-05-09T08:03:02.314662+0000","last_undegraded":"2024-05-09T08:03:02.314662+0000","last_fullsized":"2024-05-09T08:03:02.314662+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:00:40.599475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.514890+0000","last_change":"2024-05-09T08:02:15.899818+0000","last_active":"2024-05-09T08:03:02.514890+0000","last_peered":"2024-05-09T08:03:02.514890+0000","last_clean":"2024-05-09T08:03:02.514890+0000","last_became_active":"2024-05-09T08:02:15.899479+0000","last_became_peered":"2024-05-09T08:02:15.899479+0000","last_unstale":"2024-05-09T08:03:02.514890+0000","last_undegraded":"2024-05-09T08:03:02.514890+0000","last_fullsized":"2024-05-09T08:03:02.514890+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:18:36.309627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.513647+0000","last_change":"2024-05-09T08:02:20.057944+0000","last_active":"2024-05-09T08:03:02.513647+0000","last_peered":"2024-05-09T08:03:02.513647+0000","last_clean":"2024-05-09T08:03:02.513647+0000","last_became_active":"2024-05-09T08:02:20.057016+0000","last_became_peered":"2024-05-09T08:02:20.057016+0000","last_unstale":"2024-05-09T08:03:02.513647+0000","last_undegraded":"2024-05-09T08:03:02.513647+0000","last_fullsized":"2024-05-09T08:03:02.513647+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:17:01.514168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"24'1","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.515170+0000","last_change":"2024-05-09T08:02:14.886137+0000","last_active":"2024-05-09T08:03:02.515170+0000","last_peered":"2024-05-09T08:03:02.515170+0000","last_clean":"2024-05-09T08:03:02.515170+0000","last_became_active":"2024-05-09T08:02:14.885830+0000","last_became_peered":"2024-05-09T08:02:14.885830+0000","last_unstale":"2024-05-09T08:03:02.515170+0000","last_undegraded":"2024-05-09T08:03:02.515170+0000","last_fullsized":"2024-05-09T08:03:02.515170+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:46:11.160815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.312424+0000","last_change":"2024-05-09T08:02:15.906280+0000","last_active":"2024-05-09T08:03:02.312424+0000","last_peered":"2024-05-09T08:03:02.312424+0000","last_clean":"2024-05-09T08:03:02.312424+0000","last_became_active":"2024-05-09T08:02:15.905590+0000","last_became_peered":"2024-05-09T08:02:15.905590+0000","last_unstale":"2024-05-09T08:03:02.312424+0000","last_undegraded":"2024-05-09T08:03:02.312424+0000","last_fullsized":"2024-05-09T08:03:02.312424+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:20:07.368327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.313525+0000","last_change":"2024-05-09T08:02:19.954548+0000","last_active":"2024-05-09T08:03:02.313525+0000","last_peered":"2024-05-09T08:03:02.313525+0000","last_clean":"2024-05-09T08:03:02.313525+0000","last_became_active":"2024-05-09T08:02:19.954161+0000","last_became_peered":"2024-05-09T08:02:19.954161+0000","last_unstale":"2024-05-09T08:03:02.313525+0000","last_undegraded":"2024-05-09T08:03:02.313525+0000","last_fullsized":"2024-05-09T08:03:02.313525+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:53:27.246334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"24'2","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.606002+0000","last_change":"2024-05-09T08:02:14.867128+0000","last_active":"2024-05-09T08:03:02.606002+0000","last_peered":"2024-05-09T08:03:02.606002+0000","last_clean":"2024-05-09T08:03:02.606002+0000","last_became_active":"2024-05-09T08:02:14.866809+0000","last_became_peered":"2024-05-09T08:02:14.866809+0000","last_unstale":"2024-05-09T08:03:02.606002+0000","last_undegraded":"2024-05-09T08:03:02.606002+0000","last_fullsized":"2024-05-09T08:03:02.606002+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:57:06.093718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.313344+0000","last_change":"2024-05-09T08:02:15.888431+0000","last_active":"2024-05-09T08:03:02.313344+0000","last_peered":"2024-05-09T08:03:02.313344+0000","last_clean":"2024-05-09T08:03:02.313344+0000","last_became_active":"2024-05-09T08:02:15.887509+0000","last_became_peered":"2024-05-09T08:02:15.887509+0000","last_unstale":"2024-05-09T08:03:02.313344+0000","last_undegraded":"2024-05-09T08:03:02.313344+0000","last_fullsized":"2024-05-09T08:03:02.313344+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:25:01.238816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.314249+0000","last_change":"2024-05-09T08:02:19.953600+0000","last_active":"2024-05-09T08:03:02.314249+0000","last_peered":"2024-05-09T08:03:02.314249+0000","last_clean":"2024-05-09T08:03:02.314249+0000","last_became_active":"2024-05-09T08:02:19.952275+0000","last_became_peered":"2024-05-09T08:02:19.952275+0000","last_unstale":"2024-05-09T08:03:02.314249+0000","last_undegraded":"2024-05-09T08:03:02.314249+0000","last_fullsized":"2024-05-09T08:03:02.314249+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:25:05.306541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"24'2","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.605642+0000","last_change":"2024-05-09T08:02:14.870819+0000","last_active":"2024-05-09T08:03:02.605642+0000","last_peered":"2024-05-09T08:03:02.605642+0000","last_clean":"2024-05-09T08:03:02.605642+0000","last_became_active":"2024-05-09T08:02:14.870476+0000","last_became_peered":"2024-05-09T08:02:14.870476+0000","last_unstale":"2024-05-09T08:03:02.605642+0000","last_undegraded":"2024-05-09T08:03:02.605642+0000","last_fullsized":"2024-05-09T08:03:02.605642+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:43:01.061801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.313275+0000","last_change":"2024-05-09T08:02:15.886471+0000","last_active":"2024-05-09T08:03:02.313275+0000","last_peered":"2024-05-09T08:03:02.313275+0000","last_clean":"2024-05-09T08:03:02.313275+0000","last_became_active":"2024-05-09T08:02:15.886168+0000","last_became_peered":"2024-05-09T08:02:15.886168+0000","last_unstale":"2024-05-09T08:03:02.313275+0000","last_undegraded":"2024-05-09T08:03:02.313275+0000","last_fullsized":"2024-05-09T08:03:02.313275+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:30:33.351158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.313990+0000","last_change":"2024-05-09T08:02:19.947347+0000","last_active":"2024-05-09T08:03:02.313990+0000","last_peered":"2024-05-09T08:03:02.313990+0000","last_clean":"2024-05-09T08:03:02.313990+0000","last_became_active":"2024-05-09T08:02:19.946906+0000","last_became_peered":"2024-05-09T08:02:19.946906+0000","last_unstale":"2024-05-09T08:03:02.313990+0000","last_undegraded":"2024-05-09T08:03:02.313990+0000","last_fullsized":"2024-05-09T08:03:02.313990+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:30:37.418883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.310048+0000","last_change":"2024-05-09T08:02:15.880765+0000","last_active":"2024-05-09T08:03:02.310048+0000","last_peered":"2024-05-09T08:03:02.310048+0000","last_clean":"2024-05-09T08:03:02.310048+0000","last_became_active":"2024-05-09T08:02:15.880439+0000","last_became_peered":"2024-05-09T08:02:15.880439+0000","last_unstale":"2024-05-09T08:03:02.310048+0000","last_undegraded":"2024-05-09T08:03:02.310048+0000","last_fullsized":"2024-05-09T08:03:02.310048+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:29:59.441314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"24'2","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.605857+0000","last_change":"2024-05-09T08:02:14.867587+0000","last_active":"2024-05-09T08:03:02.605857+0000","last_peered":"2024-05-09T08:03:02.605857+0000","last_clean":"2024-05-09T08:03:02.605857+0000","last_became_active":"2024-05-09T08:02:14.867291+0000","last_became_peered":"2024-05-09T08:02:14.867291+0000","last_unstale":"2024-05-09T08:03:02.605857+0000","last_undegraded":"2024-05-09T08:03:02.605857+0000","last_fullsized":"2024-05-09T08:03:02.605857+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:20:08.921752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.314846+0000","last_change":"2024-05-09T08:02:19.954902+0000","last_active":"2024-05-09T08:03:02.314846+0000","last_peered":"2024-05-09T08:03:02.314846+0000","last_clean":"2024-05-09T08:03:02.314846+0000","last_became_active":"2024-05-09T08:02:19.954007+0000","last_became_peered":"2024-05-09T08:02:19.954007+0000","last_unstale":"2024-05-09T08:03:02.314846+0000","last_undegraded":"2024-05-09T08:03:02.314846+0000","last_fullsized":"2024-05-09T08:03:02.314846+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:41:31.470546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.514342+0000","last_change":"2024-05-09T08:02:15.915155+0000","last_active":"2024-05-09T08:03:02.514342+0000","last_peered":"2024-05-09T08:03:02.514342+0000","last_clean":"2024-05-09T08:03:02.514342+0000","last_became_active":"2024-05-09T08:02:15.914852+0000","last_became_peered":"2024-05-09T08:02:15.914852+0000","last_unstale":"2024-05-09T08:03:02.514342+0000","last_undegraded":"2024-05-09T08:03:02.514342+0000","last_fullsized":"2024-05-09T08:03:02.514342+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:53:23.178610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.611056+0000","last_change":"2024-05-09T08:02:14.881770+0000","last_active":"2024-05-09T08:03:02.611056+0000","last_peered":"2024-05-09T08:03:02.611056+0000","last_clean":"2024-05-09T08:03:02.611056+0000","last_became_active":"2024-05-09T08:02:14.881437+0000","last_became_peered":"2024-05-09T08:02:14.881437+0000","last_unstale":"2024-05-09T08:03:02.611056+0000","last_undegraded":"2024-05-09T08:03:02.611056+0000","last_fullsized":"2024-05-09T08:03:02.611056+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:25:23.306827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.314596+0000","last_change":"2024-05-09T08:02:19.955928+0000","last_active":"2024-05-09T08:03:02.314596+0000","last_peered":"2024-05-09T08:03:02.314596+0000","last_clean":"2024-05-09T08:03:02.314596+0000","last_became_active":"2024-05-09T08:02:19.955432+0000","last_became_peered":"2024-05-09T08:02:19.955432+0000","last_unstale":"2024-05-09T08:03:02.314596+0000","last_undegraded":"2024-05-09T08:03:02.314596+0000","last_fullsized":"2024-05-09T08:03:02.314596+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:53:15.458120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.513937+0000","last_change":"2024-05-09T08:02:15.913311+0000","last_active":"2024-05-09T08:03:02.513937+0000","last_peered":"2024-05-09T08:03:02.513937+0000","last_clean":"2024-05-09T08:03:02.513937+0000","last_became_active":"2024-05-09T08:02:15.912484+0000","last_became_peered":"2024-05-09T08:02:15.912484+0000","last_unstale":"2024-05-09T08:03:02.513937+0000","last_undegraded":"2024-05-09T08:03:02.513937+0000","last_fullsized":"2024-05-09T08:03:02.513937+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:43:06.938904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.611451+0000","last_change":"2024-05-09T08:02:14.868884+0000","last_active":"2024-05-09T08:03:02.611451+0000","last_peered":"2024-05-09T08:03:02.611451+0000","last_clean":"2024-05-09T08:03:02.611451+0000","last_became_active":"2024-05-09T08:02:14.868553+0000","last_became_peered":"2024-05-09T08:02:14.868553+0000","last_unstale":"2024-05-09T08:03:02.611451+0000","last_undegraded":"2024-05-09T08:03:02.611451+0000","last_fullsized":"2024-05-09T08:03:02.611451+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:32:48.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.613297+0000","last_change":"2024-05-09T08:03:02.609342+0000","last_active":"2024-05-09T08:03:02.613297+0000","last_peered":"2024-05-09T08:03:02.613297+0000","last_clean":"2024-05-09T08:03:02.613297+0000","last_became_active":"2024-05-09T08:03:02.607564+0000","last_became_peered":"2024-05-09T08:03:02.607564+0000","last_unstale":"2024-05-09T08:03:02.613297+0000","last_undegraded":"2024-05-09T08:03:02.613297+0000","last_fullsized":"2024-05-09T08:03:02.613297+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:23:44.316545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4,7],"acting":[2,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.314326+0000","last_change":"2024-05-09T08:02:15.891115+0000","last_active":"2024-05-09T08:03:02.314326+0000","last_peered":"2024-05-09T08:03:02.314326+0000","last_clean":"2024-05-09T08:03:02.314326+0000","last_became_active":"2024-05-09T08:02:15.890806+0000","last_became_peered":"2024-05-09T08:02:15.890806+0000","last_unstale":"2024-05-09T08:03:02.314326+0000","last_undegraded":"2024-05-09T08:03:02.314326+0000","last_fullsized":"2024-05-09T08:03:02.314326+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:57:07.107359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.605148+0000","last_change":"2024-05-09T08:02:14.874987+0000","last_active":"2024-05-09T08:03:02.605148+0000","last_peered":"2024-05-09T08:03:02.605148+0000","last_clean":"2024-05-09T08:03:02.605148+0000","last_became_active":"2024-05-09T08:02:14.874549+0000","last_became_peered":"2024-05-09T08:02:14.874549+0000","last_unstale":"2024-05-09T08:03:02.605148+0000","last_undegraded":"2024-05-09T08:03:02.605148+0000","last_fullsized":"2024-05-09T08:03:02.605148+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:57:10.338510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.516739+0000","last_change":"2024-05-09T08:03:02.516739+0000","last_active":"2024-05-09T08:03:02.516739+0000","last_peered":"2024-05-09T08:03:02.516739+0000","last_clean":"2024-05-09T08:03:02.516739+0000","last_became_active":"2024-05-09T08:03:02.516161+0000","last_became_peered":"2024-05-09T08:03:02.516161+0000","last_unstale":"2024-05-09T08:03:02.516739+0000","last_undegraded":"2024-05-09T08:03:02.516739+0000","last_fullsized":"2024-05-09T08:03:02.516739+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:10:24.797526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,2],"acting":[4,3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.313602+0000","last_change":"2024-05-09T08:02:15.913011+0000","last_active":"2024-05-09T08:03:02.313602+0000","last_peered":"2024-05-09T08:03:02.313602+0000","last_clean":"2024-05-09T08:03:02.313602+0000","last_became_active":"2024-05-09T08:02:15.912590+0000","last_became_peered":"2024-05-09T08:02:15.912590+0000","last_unstale":"2024-05-09T08:03:02.313602+0000","last_undegraded":"2024-05-09T08:03:02.313602+0000","last_fullsized":"2024-05-09T08:03:02.313602+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:43:06.938904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"24'2","reported_seq":28,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.310929+0000","last_change":"2024-05-09T08:02:14.864763+0000","last_active":"2024-05-09T08:03:02.310929+0000","last_peered":"2024-05-09T08:03:02.310929+0000","last_clean":"2024-05-09T08:03:02.310929+0000","last_became_active":"2024-05-09T08:02:14.864470+0000","last_became_peered":"2024-05-09T08:02:14.864470+0000","last_unstale":"2024-05-09T08:03:02.310929+0000","last_undegraded":"2024-05-09T08:03:02.310929+0000","last_fullsized":"2024-05-09T08:03:02.310929+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:00:40.599475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.313868+0000","last_change":"2024-05-09T08:02:20.054896+0000","last_active":"2024-05-09T08:03:02.313868+0000","last_peered":"2024-05-09T08:03:02.313868+0000","last_clean":"2024-05-09T08:03:02.313868+0000","last_became_active":"2024-05-09T08:02:20.054424+0000","last_became_peered":"2024-05-09T08:02:20.054424+0000","last_unstale":"2024-05-09T08:03:02.313868+0000","last_undegraded":"2024-05-09T08:03:02.313868+0000","last_fullsized":"2024-05-09T08:03:02.313868+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:23:44.316545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.515468+0000","last_change":"2024-05-09T08:02:15.905448+0000","last_active":"2024-05-09T08:03:02.515468+0000","last_peered":"2024-05-09T08:03:02.515468+0000","last_clean":"2024-05-09T08:03:02.515468+0000","last_became_active":"2024-05-09T08:02:15.905131+0000","last_became_peered":"2024-05-09T08:02:15.905131+0000","last_unstale":"2024-05-09T08:03:02.515468+0000","last_undegraded":"2024-05-09T08:03:02.515468+0000","last_fullsized":"2024-05-09T08:03:02.515468+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:25:01.238816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"25'1","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.611654+0000","last_change":"2024-05-09T08:02:14.868410+0000","last_active":"2024-05-09T08:03:02.611654+0000","last_peered":"2024-05-09T08:03:02.611654+0000","last_clean":"2024-05-09T08:03:02.611654+0000","last_became_active":"2024-05-09T08:02:14.865621+0000","last_became_peered":"2024-05-09T08:02:14.865621+0000","last_unstale":"2024-05-09T08:03:02.611654+0000","last_undegraded":"2024-05-09T08:03:02.611654+0000","last_fullsized":"2024-05-09T08:03:02.611654+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:43:01.061801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.513488+0000","last_change":"2024-05-09T08:02:20.057603+0000","last_active":"2024-05-09T08:03:02.513488+0000","last_peered":"2024-05-09T08:03:02.513488+0000","last_clean":"2024-05-09T08:03:02.513488+0000","last_became_active":"2024-05-09T08:02:20.055365+0000","last_became_peered":"2024-05-09T08:02:20.055365+0000","last_unstale":"2024-05-09T08:03:02.513488+0000","last_undegraded":"2024-05-09T08:03:02.513488+0000","last_fullsized":"2024-05-09T08:03:02.513488+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:48:38.283562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.613397+0000","last_change":"2024-05-09T08:02:15.890980+0000","last_active":"2024-05-09T08:03:02.613397+0000","last_peered":"2024-05-09T08:03:02.613397+0000","last_clean":"2024-05-09T08:03:02.613397+0000","last_became_active":"2024-05-09T08:02:15.890346+0000","last_became_peered":"2024-05-09T08:02:15.890346+0000","last_unstale":"2024-05-09T08:03:02.613397+0000","last_undegraded":"2024-05-09T08:03:02.613397+0000","last_fullsized":"2024-05-09T08:03:02.613397+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:29:59.441314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.606148+0000","last_change":"2024-05-09T08:02:14.876206+0000","last_active":"2024-05-09T08:03:02.606148+0000","last_peered":"2024-05-09T08:03:02.606148+0000","last_clean":"2024-05-09T08:03:02.606148+0000","last_became_active":"2024-05-09T08:02:14.875894+0000","last_became_peered":"2024-05-09T08:02:14.875894+0000","last_unstale":"2024-05-09T08:03:02.606148+0000","last_undegraded":"2024-05-09T08:03:02.606148+0000","last_fullsized":"2024-05-09T08:03:02.606148+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:49:53.882513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.312428+0000","last_change":"2024-05-09T08:02:20.055222+0000","last_active":"2024-05-09T08:03:02.312428+0000","last_peered":"2024-05-09T08:03:02.312428+0000","last_clean":"2024-05-09T08:03:02.312428+0000","last_became_active":"2024-05-09T08:02:20.053802+0000","last_became_peered":"2024-05-09T08:02:20.053802+0000","last_unstale":"2024-05-09T08:03:02.312428+0000","last_undegraded":"2024-05-09T08:03:02.312428+0000","last_fullsized":"2024-05-09T08:03:02.312428+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:42:00.939587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.313378+0000","last_change":"2024-05-09T08:02:15.904528+0000","last_active":"2024-05-09T08:03:02.313378+0000","last_peered":"2024-05-09T08:03:02.313378+0000","last_clean":"2024-05-09T08:03:02.313378+0000","last_became_active":"2024-05-09T08:02:15.904049+0000","last_became_peered":"2024-05-09T08:02:15.904049+0000","last_unstale":"2024-05-09T08:03:02.313378+0000","last_undegraded":"2024-05-09T08:03:02.313378+0000","last_fullsized":"2024-05-09T08:03:02.313378+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:53:11.390395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.612917+0000","last_change":"2024-05-09T08:02:14.872897+0000","last_active":"2024-05-09T08:03:02.612917+0000","last_peered":"2024-05-09T08:03:02.612917+0000","last_clean":"2024-05-09T08:03:02.612917+0000","last_became_active":"2024-05-09T08:02:14.872096+0000","last_became_peered":"2024-05-09T08:02:14.872096+0000","last_unstale":"2024-05-09T08:03:02.612917+0000","last_undegraded":"2024-05-09T08:03:02.612917+0000","last_fullsized":"2024-05-09T08:03:02.612917+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:00:40.599475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.610605+0000","last_change":"2024-05-09T08:02:19.963716+0000","last_active":"2024-05-09T08:03:02.610605+0000","last_peered":"2024-05-09T08:03:02.610605+0000","last_clean":"2024-05-09T08:03:02.610605+0000","last_became_active":"2024-05-09T08:02:19.962922+0000","last_became_peered":"2024-05-09T08:02:19.962922+0000","last_unstale":"2024-05-09T08:03:02.610605+0000","last_undegraded":"2024-05-09T08:03:02.610605+0000","last_fullsized":"2024-05-09T08:03:02.610605+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:42:00.939587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.314904+0000","last_change":"2024-05-09T08:02:15.901450+0000","last_active":"2024-05-09T08:03:02.314904+0000","last_peered":"2024-05-09T08:03:02.314904+0000","last_clean":"2024-05-09T08:03:02.314904+0000","last_became_active":"2024-05-09T08:02:15.901180+0000","last_became_peered":"2024-05-09T08:02:15.901180+0000","last_unstale":"2024-05-09T08:03:02.314904+0000","last_undegraded":"2024-05-09T08:03:02.314904+0000","last_fullsized":"2024-05-09T08:03:02.314904+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:43:02.075442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.313301+0000","last_change":"2024-05-09T08:02:14.876518+0000","last_active":"2024-05-09T08:03:02.313301+0000","last_peered":"2024-05-09T08:03:02.313301+0000","last_clean":"2024-05-09T08:03:02.313301+0000","last_became_active":"2024-05-09T08:02:14.876175+0000","last_became_peered":"2024-05-09T08:02:14.876175+0000","last_unstale":"2024-05-09T08:03:02.313301+0000","last_undegraded":"2024-05-09T08:03:02.313301+0000","last_fullsized":"2024-05-09T08:03:02.313301+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:00:40.599475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.517496+0000","last_change":"2024-05-09T08:03:02.516595+0000","last_active":"2024-05-09T08:03:02.517496+0000","last_peered":"2024-05-09T08:03:02.517496+0000","last_clean":"2024-05-09T08:03:02.517496+0000","last_became_active":"2024-05-09T08:03:02.516104+0000","last_became_peered":"2024-05-09T08:03:02.516104+0000","last_unstale":"2024-05-09T08:03:02.517496+0000","last_undegraded":"2024-05-09T08:03:02.517496+0000","last_fullsized":"2024-05-09T08:03:02.517496+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:47:30.081323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3,6],"acting":[5,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.610756+0000","last_change":"2024-05-09T08:02:15.891774+0000","last_active":"2024-05-09T08:03:02.610756+0000","last_peered":"2024-05-09T08:03:02.610756+0000","last_clean":"2024-05-09T08:03:02.610756+0000","last_became_active":"2024-05-09T08:02:15.891407+0000","last_became_peered":"2024-05-09T08:02:15.891407+0000","last_unstale":"2024-05-09T08:03:02.610756+0000","last_undegraded":"2024-05-09T08:03:02.610756+0000","last_fullsized":"2024-05-09T08:03:02.610756+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:05:02.181367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.319052+0000","last_change":"2024-05-09T08:02:14.883655+0000","last_active":"2024-05-09T08:03:02.319052+0000","last_peered":"2024-05-09T08:03:02.319052+0000","last_clean":"2024-05-09T08:03:02.319052+0000","last_became_active":"2024-05-09T08:02:14.882359+0000","last_became_peered":"2024-05-09T08:02:14.882359+0000","last_unstale":"2024-05-09T08:03:02.319052+0000","last_undegraded":"2024-05-09T08:03:02.319052+0000","last_fullsized":"2024-05-09T08:03:02.319052+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:46:11.160815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.611626+0000","last_change":"2024-05-09T08:02:19.953319+0000","last_active":"2024-05-09T08:03:02.611626+0000","last_peered":"2024-05-09T08:03:02.611626+0000","last_clean":"2024-05-09T08:03:02.611626+0000","last_became_active":"2024-05-09T08:02:19.952740+0000","last_became_peered":"2024-05-09T08:02:19.952740+0000","last_unstale":"2024-05-09T08:03:02.611626+0000","last_undegraded":"2024-05-09T08:03:02.611626+0000","last_fullsized":"2024-05-09T08:03:02.611626+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:25:28.804048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.311259+0000","last_change":"2024-05-09T08:02:15.913828+0000","last_active":"2024-05-09T08:03:02.311259+0000","last_peered":"2024-05-09T08:03:02.311259+0000","last_clean":"2024-05-09T08:03:02.311259+0000","last_became_active":"2024-05-09T08:02:15.913313+0000","last_became_peered":"2024-05-09T08:02:15.913313+0000","last_unstale":"2024-05-09T08:03:02.311259+0000","last_undegraded":"2024-05-09T08:03:02.311259+0000","last_fullsized":"2024-05-09T08:03:02.311259+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:20:09.935394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.605151+0000","last_change":"2024-05-09T08:02:14.876438+0000","last_active":"2024-05-09T08:03:02.605151+0000","last_peered":"2024-05-09T08:03:02.605151+0000","last_clean":"2024-05-09T08:03:02.605151+0000","last_became_active":"2024-05-09T08:02:14.876146+0000","last_became_peered":"2024-05-09T08:02:14.876146+0000","last_unstale":"2024-05-09T08:03:02.605151+0000","last_undegraded":"2024-05-09T08:03:02.605151+0000","last_fullsized":"2024-05-09T08:03:02.605151+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:36:49.629991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.312518+0000","last_change":"2024-05-09T08:02:20.051067+0000","last_active":"2024-05-09T08:03:02.312518+0000","last_peered":"2024-05-09T08:03:02.312518+0000","last_clean":"2024-05-09T08:03:02.312518+0000","last_became_active":"2024-05-09T08:02:20.050593+0000","last_became_peered":"2024-05-09T08:02:20.050593+0000","last_unstale":"2024-05-09T08:03:02.312518+0000","last_undegraded":"2024-05-09T08:03:02.312518+0000","last_fullsized":"2024-05-09T08:03:02.312518+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:25:28.804048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.513850+0000","last_change":"2024-05-09T08:02:15.899137+0000","last_active":"2024-05-09T08:03:02.513850+0000","last_peered":"2024-05-09T08:03:02.513850+0000","last_clean":"2024-05-09T08:03:02.513850+0000","last_became_active":"2024-05-09T08:02:15.898766+0000","last_became_peered":"2024-05-09T08:02:15.898766+0000","last_unstale":"2024-05-09T08:03:02.513850+0000","last_undegraded":"2024-05-09T08:03:02.513850+0000","last_fullsized":"2024-05-09T08:03:02.513850+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:47:32.605288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.312721+0000","last_change":"2024-05-09T08:02:14.868850+0000","last_active":"2024-05-09T08:03:02.312721+0000","last_peered":"2024-05-09T08:03:02.312721+0000","last_clean":"2024-05-09T08:03:02.312721+0000","last_became_active":"2024-05-09T08:02:14.868384+0000","last_became_peered":"2024-05-09T08:02:14.868384+0000","last_unstale":"2024-05-09T08:03:02.312721+0000","last_undegraded":"2024-05-09T08:03:02.312721+0000","last_fullsized":"2024-05-09T08:03:02.312721+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:14:52.920972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":181,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.605375+0000","last_change":"2024-05-09T08:02:11.559804+0000","last_active":"2024-05-09T08:03:02.605375+0000","last_peered":"2024-05-09T08:03:02.605375+0000","last_clean":"2024-05-09T08:03:02.605375+0000","last_became_active":"2024-05-09T08:02:11.558761+0000","last_became_peered":"2024-05-09T08:02:11.558761+0000","last_unstale":"2024-05-09T08:03:02.605375+0000","last_undegraded":"2024-05-09T08:03:02.605375+0000","last_fullsized":"2024-05-09T08:03:02.605375+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:09.800143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:09.800143+0000","last_clean_scrub_stamp":"2024-05-09T08:02:09.800143+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:18:35.964149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.310459+0000","last_change":"2024-05-09T08:02:20.054746+0000","last_active":"2024-05-09T08:03:02.310459+0000","last_peered":"2024-05-09T08:03:02.310459+0000","last_clean":"2024-05-09T08:03:02.310459+0000","last_became_active":"2024-05-09T08:02:20.054186+0000","last_became_peered":"2024-05-09T08:02:20.054186+0000","last_unstale":"2024-05-09T08:03:02.310459+0000","last_undegraded":"2024-05-09T08:03:02.310459+0000","last_fullsized":"2024-05-09T08:03:02.310459+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:53:15.458120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.605779+0000","last_change":"2024-05-09T08:02:15.889657+0000","last_active":"2024-05-09T08:03:02.605779+0000","last_peered":"2024-05-09T08:03:02.605779+0000","last_clean":"2024-05-09T08:03:02.605779+0000","last_became_active":"2024-05-09T08:02:15.889381+0000","last_became_peered":"2024-05-09T08:02:15.889381+0000","last_unstale":"2024-05-09T08:03:02.605779+0000","last_undegraded":"2024-05-09T08:03:02.605779+0000","last_fullsized":"2024-05-09T08:03:02.605779+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:59:14.957806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.318796+0000","last_change":"2024-05-09T08:02:14.868892+0000","last_active":"2024-05-09T08:03:02.318796+0000","last_peered":"2024-05-09T08:03:02.318796+0000","last_clean":"2024-05-09T08:03:02.318796+0000","last_became_active":"2024-05-09T08:02:14.868513+0000","last_became_peered":"2024-05-09T08:02:14.868513+0000","last_unstale":"2024-05-09T08:03:02.318796+0000","last_undegraded":"2024-05-09T08:03:02.318796+0000","last_fullsized":"2024-05-09T08:03:02.318796+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:19:31.913236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.309986+0000","last_change":"2024-05-09T08:02:20.053097+0000","last_active":"2024-05-09T08:03:02.309986+0000","last_peered":"2024-05-09T08:03:02.309986+0000","last_clean":"2024-05-09T08:03:02.309986+0000","last_became_active":"2024-05-09T08:02:20.052653+0000","last_became_peered":"2024-05-09T08:02:20.052653+0000","last_unstale":"2024-05-09T08:03:02.309986+0000","last_undegraded":"2024-05-09T08:03:02.309986+0000","last_fullsized":"2024-05-09T08:03:02.309986+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:25:05.306541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.313567+0000","last_change":"2024-05-09T08:02:15.899059+0000","last_active":"2024-05-09T08:03:02.313567+0000","last_peered":"2024-05-09T08:03:02.313567+0000","last_clean":"2024-05-09T08:03:02.313567+0000","last_became_active":"2024-05-09T08:02:15.898692+0000","last_became_peered":"2024-05-09T08:02:15.898692+0000","last_unstale":"2024-05-09T08:03:02.313567+0000","last_undegraded":"2024-05-09T08:03:02.313567+0000","last_fullsized":"2024-05-09T08:03:02.313567+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:52:36.461913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"24'1","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.611806+0000","last_change":"2024-05-09T08:02:14.873071+0000","last_active":"2024-05-09T08:03:02.611806+0000","last_peered":"2024-05-09T08:03:02.611806+0000","last_clean":"2024-05-09T08:03:02.611806+0000","last_became_active":"2024-05-09T08:02:14.872556+0000","last_became_peered":"2024-05-09T08:02:14.872556+0000","last_unstale":"2024-05-09T08:03:02.611806+0000","last_undegraded":"2024-05-09T08:03:02.611806+0000","last_fullsized":"2024-05-09T08:03:02.611806+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:57:06.093718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.313822+0000","last_change":"2024-05-09T08:02:19.960414+0000","last_active":"2024-05-09T08:03:02.313822+0000","last_peered":"2024-05-09T08:03:02.313822+0000","last_clean":"2024-05-09T08:03:02.313822+0000","last_became_active":"2024-05-09T08:02:19.957941+0000","last_became_peered":"2024-05-09T08:02:19.957941+0000","last_unstale":"2024-05-09T08:03:02.313822+0000","last_undegraded":"2024-05-09T08:03:02.313822+0000","last_fullsized":"2024-05-09T08:03:02.313822+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:41:31.470546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.606213+0000","last_change":"2024-05-09T08:02:15.884395+0000","last_active":"2024-05-09T08:03:02.606213+0000","last_peered":"2024-05-09T08:03:02.606213+0000","last_clean":"2024-05-09T08:03:02.606213+0000","last_became_active":"2024-05-09T08:02:15.883788+0000","last_became_peered":"2024-05-09T08:02:15.883788+0000","last_unstale":"2024-05-09T08:03:02.606213+0000","last_undegraded":"2024-05-09T08:03:02.606213+0000","last_fullsized":"2024-05-09T08:03:02.606213+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:48:34.215837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"24'2","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.319378+0000","last_change":"2024-05-09T08:02:14.885044+0000","last_active":"2024-05-09T08:03:02.319378+0000","last_peered":"2024-05-09T08:03:02.319378+0000","last_clean":"2024-05-09T08:03:02.319378+0000","last_became_active":"2024-05-09T08:02:14.884706+0000","last_became_peered":"2024-05-09T08:02:14.884706+0000","last_unstale":"2024-05-09T08:03:02.319378+0000","last_undegraded":"2024-05-09T08:03:02.319378+0000","last_fullsized":"2024-05-09T08:03:02.319378+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:14:52.920972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.312303+0000","last_change":"2024-05-09T08:02:20.050865+0000","last_active":"2024-05-09T08:03:02.312303+0000","last_peered":"2024-05-09T08:03:02.312303+0000","last_clean":"2024-05-09T08:03:02.312303+0000","last_became_active":"2024-05-09T08:02:20.050270+0000","last_became_peered":"2024-05-09T08:02:20.050270+0000","last_unstale":"2024-05-09T08:03:02.312303+0000","last_undegraded":"2024-05-09T08:03:02.312303+0000","last_fullsized":"2024-05-09T08:03:02.312303+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:48:38.283562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.605983+0000","last_change":"2024-05-09T08:02:15.884737+0000","last_active":"2024-05-09T08:03:02.605983+0000","last_peered":"2024-05-09T08:03:02.605983+0000","last_clean":"2024-05-09T08:03:02.605983+0000","last_became_active":"2024-05-09T08:02:15.884465+0000","last_became_peered":"2024-05-09T08:02:15.884465+0000","last_unstale":"2024-05-09T08:03:02.605983+0000","last_undegraded":"2024-05-09T08:03:02.605983+0000","last_fullsized":"2024-05-09T08:03:02.605983+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:52:36.461913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"24'1","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.313984+0000","last_change":"2024-05-09T08:02:14.875003+0000","last_active":"2024-05-09T08:03:02.313984+0000","last_peered":"2024-05-09T08:03:02.313984+0000","last_clean":"2024-05-09T08:03:02.313984+0000","last_became_active":"2024-05-09T08:02:14.874665+0000","last_became_peered":"2024-05-09T08:02:14.874665+0000","last_unstale":"2024-05-09T08:03:02.313984+0000","last_undegraded":"2024-05-09T08:03:02.313984+0000","last_fullsized":"2024-05-09T08:03:02.313984+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:18:39.991319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.313729+0000","last_change":"2024-05-09T08:02:19.952692+0000","last_active":"2024-05-09T08:03:02.313729+0000","last_peered":"2024-05-09T08:03:02.313729+0000","last_clean":"2024-05-09T08:03:02.313729+0000","last_became_active":"2024-05-09T08:02:19.952224+0000","last_became_peered":"2024-05-09T08:02:19.952224+0000","last_unstale":"2024-05-09T08:03:02.313729+0000","last_undegraded":"2024-05-09T08:03:02.313729+0000","last_fullsized":"2024-05-09T08:03:02.313729+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:47:36.673013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.605340+0000","last_change":"2024-05-09T08:02:15.883408+0000","last_active":"2024-05-09T08:03:02.605340+0000","last_peered":"2024-05-09T08:03:02.605340+0000","last_clean":"2024-05-09T08:03:02.605340+0000","last_became_active":"2024-05-09T08:02:15.883103+0000","last_became_peered":"2024-05-09T08:02:15.883103+0000","last_unstale":"2024-05-09T08:03:02.605340+0000","last_undegraded":"2024-05-09T08:03:02.605340+0000","last_fullsized":"2024-05-09T08:03:02.605340+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:53:11.390395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"24'8","reported_seq":33,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.317771+0000","last_change":"2024-05-09T08:02:14.873184+0000","last_active":"2024-05-09T08:03:02.317771+0000","last_peered":"2024-05-09T08:03:02.317771+0000","last_clean":"2024-05-09T08:03:02.317771+0000","last_became_active":"2024-05-09T08:02:14.871055+0000","last_became_peered":"2024-05-09T08:02:14.871055+0000","last_unstale":"2024-05-09T08:03:02.317771+0000","last_undegraded":"2024-05-09T08:03:02.317771+0000","last_fullsized":"2024-05-09T08:03:02.317771+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:19:56.315972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.312232+0000","last_change":"2024-05-09T08:02:19.950823+0000","last_active":"2024-05-09T08:03:02.312232+0000","last_peered":"2024-05-09T08:03:02.312232+0000","last_clean":"2024-05-09T08:03:02.312232+0000","last_became_active":"2024-05-09T08:02:19.950352+0000","last_became_peered":"2024-05-09T08:02:19.950352+0000","last_unstale":"2024-05-09T08:03:02.312232+0000","last_undegraded":"2024-05-09T08:03:02.312232+0000","last_fullsized":"2024-05-09T08:03:02.312232+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:59:19.025530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.310150+0000","last_change":"2024-05-09T08:02:15.899115+0000","last_active":"2024-05-09T08:03:02.310150+0000","last_peered":"2024-05-09T08:03:02.310150+0000","last_clean":"2024-05-09T08:03:02.310150+0000","last_became_active":"2024-05-09T08:02:15.898811+0000","last_became_peered":"2024-05-09T08:02:15.898811+0000","last_unstale":"2024-05-09T08:03:02.310150+0000","last_undegraded":"2024-05-09T08:03:02.310150+0000","last_fullsized":"2024-05-09T08:03:02.310150+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:43:06.938904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.310107+0000","last_change":"2024-05-09T08:02:14.860818+0000","last_active":"2024-05-09T08:03:02.310107+0000","last_peered":"2024-05-09T08:03:02.310107+0000","last_clean":"2024-05-09T08:03:02.310107+0000","last_became_active":"2024-05-09T08:02:14.860518+0000","last_became_peered":"2024-05-09T08:02:14.860518+0000","last_unstale":"2024-05-09T08:03:02.310107+0000","last_undegraded":"2024-05-09T08:03:02.310107+0000","last_fullsized":"2024-05-09T08:03:02.310107+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:25:23.306827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.515154+0000","last_change":"2024-05-09T08:02:20.053251+0000","last_active":"2024-05-09T08:03:02.515154+0000","last_peered":"2024-05-09T08:03:02.515154+0000","last_clean":"2024-05-09T08:03:02.515154+0000","last_became_active":"2024-05-09T08:02:20.052783+0000","last_became_peered":"2024-05-09T08:02:20.052783+0000","last_unstale":"2024-05-09T08:03:02.515154+0000","last_undegraded":"2024-05-09T08:03:02.515154+0000","last_fullsized":"2024-05-09T08:03:02.515154+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:16:41.863341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.605776+0000","last_change":"2024-05-09T08:02:15.883972+0000","last_active":"2024-05-09T08:03:02.605776+0000","last_peered":"2024-05-09T08:03:02.605776+0000","last_clean":"2024-05-09T08:03:02.605776+0000","last_became_active":"2024-05-09T08:02:15.883613+0000","last_became_peered":"2024-05-09T08:02:15.883613+0000","last_unstale":"2024-05-09T08:03:02.605776+0000","last_undegraded":"2024-05-09T08:03:02.605776+0000","last_fullsized":"2024-05-09T08:03:02.605776+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:41:27.402821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.313157+0000","last_change":"2024-05-09T08:02:14.883366+0000","last_active":"2024-05-09T08:03:02.313157+0000","last_peered":"2024-05-09T08:03:02.313157+0000","last_clean":"2024-05-09T08:03:02.313157+0000","last_became_active":"2024-05-09T08:02:14.883135+0000","last_became_peered":"2024-05-09T08:02:14.883135+0000","last_unstale":"2024-05-09T08:03:02.313157+0000","last_undegraded":"2024-05-09T08:03:02.313157+0000","last_fullsized":"2024-05-09T08:03:02.313157+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:32:48.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.514492+0000","last_change":"2024-05-09T08:02:20.050888+0000","last_active":"2024-05-09T08:03:02.514492+0000","last_peered":"2024-05-09T08:03:02.514492+0000","last_clean":"2024-05-09T08:03:02.514492+0000","last_became_active":"2024-05-09T08:02:20.050390+0000","last_became_peered":"2024-05-09T08:02:20.050390+0000","last_unstale":"2024-05-09T08:03:02.514492+0000","last_undegraded":"2024-05-09T08:03:02.514492+0000","last_fullsized":"2024-05-09T08:03:02.514492+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:10:24.797526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.313551+0000","last_change":"2024-05-09T08:02:15.899096+0000","last_active":"2024-05-09T08:03:02.313551+0000","last_peered":"2024-05-09T08:03:02.313551+0000","last_clean":"2024-05-09T08:03:02.313551+0000","last_became_active":"2024-05-09T08:02:15.898811+0000","last_became_peered":"2024-05-09T08:02:15.898811+0000","last_unstale":"2024-05-09T08:03:02.313551+0000","last_undegraded":"2024-05-09T08:03:02.313551+0000","last_fullsized":"2024-05-09T08:03:02.313551+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:28:51.229262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"24'2","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.314204+0000","last_change":"2024-05-09T08:02:14.868657+0000","last_active":"2024-05-09T08:03:02.314204+0000","last_peered":"2024-05-09T08:03:02.314204+0000","last_clean":"2024-05-09T08:03:02.314204+0000","last_became_active":"2024-05-09T08:02:14.868385+0000","last_became_peered":"2024-05-09T08:02:14.868385+0000","last_unstale":"2024-05-09T08:03:02.314204+0000","last_undegraded":"2024-05-09T08:03:02.314204+0000","last_fullsized":"2024-05-09T08:03:02.314204+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:46:11.160815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.313146+0000","last_change":"2024-05-09T08:02:20.053155+0000","last_active":"2024-05-09T08:03:02.313146+0000","last_peered":"2024-05-09T08:03:02.313146+0000","last_clean":"2024-05-09T08:03:02.313146+0000","last_became_active":"2024-05-09T08:02:20.052629+0000","last_became_peered":"2024-05-09T08:02:20.052629+0000","last_unstale":"2024-05-09T08:03:02.313146+0000","last_undegraded":"2024-05-09T08:03:02.313146+0000","last_fullsized":"2024-05-09T08:03:02.313146+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:35:54.628874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.605788+0000","last_change":"2024-05-09T08:02:15.905406+0000","last_active":"2024-05-09T08:03:02.605788+0000","last_peered":"2024-05-09T08:03:02.605788+0000","last_clean":"2024-05-09T08:03:02.605788+0000","last_became_active":"2024-05-09T08:02:15.905076+0000","last_became_peered":"2024-05-09T08:02:15.905076+0000","last_unstale":"2024-05-09T08:03:02.605788+0000","last_undegraded":"2024-05-09T08:03:02.605788+0000","last_fullsized":"2024-05-09T08:03:02.605788+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:25:01.238816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"25'10","reported_seq":36,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.310546+0000","last_change":"2024-05-09T08:02:14.880410+0000","last_active":"2024-05-09T08:03:02.310546+0000","last_peered":"2024-05-09T08:03:02.310546+0000","last_clean":"2024-05-09T08:03:02.310546+0000","last_became_active":"2024-05-09T08:02:14.880045+0000","last_became_peered":"2024-05-09T08:02:14.880045+0000","last_unstale":"2024-05-09T08:03:02.310546+0000","last_undegraded":"2024-05-09T08:03:02.310546+0000","last_fullsized":"2024-05-09T08:03:02.310546+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:19:56.315972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.610797+0000","last_change":"2024-05-09T08:02:19.963378+0000","last_active":"2024-05-09T08:03:02.610797+0000","last_peered":"2024-05-09T08:03:02.610797+0000","last_clean":"2024-05-09T08:03:02.610797+0000","last_became_active":"2024-05-09T08:02:19.958964+0000","last_became_peered":"2024-05-09T08:02:19.958964+0000","last_unstale":"2024-05-09T08:03:02.610797+0000","last_undegraded":"2024-05-09T08:03:02.610797+0000","last_fullsized":"2024-05-09T08:03:02.610797+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:41:31.470546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.314817+0000","last_change":"2024-05-09T08:02:15.888428+0000","last_active":"2024-05-09T08:03:02.314817+0000","last_peered":"2024-05-09T08:03:02.314817+0000","last_clean":"2024-05-09T08:03:02.314817+0000","last_became_active":"2024-05-09T08:02:15.888071+0000","last_became_peered":"2024-05-09T08:02:15.888071+0000","last_unstale":"2024-05-09T08:03:02.314817+0000","last_undegraded":"2024-05-09T08:03:02.314817+0000","last_fullsized":"2024-05-09T08:03:02.314817+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:36:50.643632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"24'3","reported_seq":28,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.611487+0000","last_change":"2024-05-09T08:02:14.872697+0000","last_active":"2024-05-09T08:03:02.611487+0000","last_peered":"2024-05-09T08:03:02.611487+0000","last_clean":"2024-05-09T08:03:02.611487+0000","last_became_active":"2024-05-09T08:02:14.871945+0000","last_became_peered":"2024-05-09T08:02:14.871945+0000","last_unstale":"2024-05-09T08:03:02.611487+0000","last_undegraded":"2024-05-09T08:03:02.611487+0000","last_fullsized":"2024-05-09T08:03:02.611487+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:19:56.315972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.314051+0000","last_change":"2024-05-09T08:02:19.953635+0000","last_active":"2024-05-09T08:03:02.314051+0000","last_peered":"2024-05-09T08:03:02.314051+0000","last_clean":"2024-05-09T08:03:02.314051+0000","last_became_active":"2024-05-09T08:02:19.953166+0000","last_became_peered":"2024-05-09T08:02:19.953166+0000","last_unstale":"2024-05-09T08:03:02.314051+0000","last_undegraded":"2024-05-09T08:03:02.314051+0000","last_fullsized":"2024-05-09T08:03:02.314051+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:18:40.377352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.314372+0000","last_change":"2024-05-09T08:02:15.905285+0000","last_active":"2024-05-09T08:03:02.314372+0000","last_peered":"2024-05-09T08:03:02.314372+0000","last_clean":"2024-05-09T08:03:02.314372+0000","last_became_active":"2024-05-09T08:02:15.904898+0000","last_became_peered":"2024-05-09T08:02:15.904898+0000","last_unstale":"2024-05-09T08:03:02.314372+0000","last_undegraded":"2024-05-09T08:03:02.314372+0000","last_fullsized":"2024-05-09T08:03:02.314372+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:47:32.605288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.313275+0000","last_change":"2024-05-09T08:02:14.868682+0000","last_active":"2024-05-09T08:03:02.313275+0000","last_peered":"2024-05-09T08:03:02.313275+0000","last_clean":"2024-05-09T08:03:02.313275+0000","last_became_active":"2024-05-09T08:02:14.868294+0000","last_became_peered":"2024-05-09T08:02:14.868294+0000","last_unstale":"2024-05-09T08:03:02.313275+0000","last_undegraded":"2024-05-09T08:03:02.313275+0000","last_fullsized":"2024-05-09T08:03:02.313275+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:57:06.093718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.513640+0000","last_change":"2024-05-09T08:02:19.956115+0000","last_active":"2024-05-09T08:03:02.513640+0000","last_peered":"2024-05-09T08:03:02.513640+0000","last_clean":"2024-05-09T08:03:02.513640+0000","last_became_active":"2024-05-09T08:02:19.955662+0000","last_became_peered":"2024-05-09T08:02:19.955662+0000","last_unstale":"2024-05-09T08:03:02.513640+0000","last_undegraded":"2024-05-09T08:03:02.513640+0000","last_fullsized":"2024-05-09T08:03:02.513640+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:15:28.286330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.313165+0000","last_change":"2024-05-09T08:02:15.888406+0000","last_active":"2024-05-09T08:03:02.313165+0000","last_peered":"2024-05-09T08:03:02.313165+0000","last_clean":"2024-05-09T08:03:02.313165+0000","last_became_active":"2024-05-09T08:02:15.888090+0000","last_became_peered":"2024-05-09T08:02:15.888090+0000","last_unstale":"2024-05-09T08:03:02.313165+0000","last_undegraded":"2024-05-09T08:03:02.313165+0000","last_fullsized":"2024-05-09T08:03:02.313165+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:53:23.178610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.310785+0000","last_change":"2024-05-09T08:02:14.862463+0000","last_active":"2024-05-09T08:03:02.310785+0000","last_peered":"2024-05-09T08:03:02.310785+0000","last_clean":"2024-05-09T08:03:02.310785+0000","last_became_active":"2024-05-09T08:02:14.862104+0000","last_became_peered":"2024-05-09T08:02:14.862104+0000","last_unstale":"2024-05-09T08:03:02.310785+0000","last_undegraded":"2024-05-09T08:03:02.310785+0000","last_fullsized":"2024-05-09T08:03:02.310785+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:32:48.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.513779+0000","last_change":"2024-05-09T08:02:20.053600+0000","last_active":"2024-05-09T08:03:02.513779+0000","last_peered":"2024-05-09T08:03:02.513779+0000","last_clean":"2024-05-09T08:03:02.513779+0000","last_became_active":"2024-05-09T08:02:20.052485+0000","last_became_peered":"2024-05-09T08:02:20.052485+0000","last_unstale":"2024-05-09T08:03:02.513779+0000","last_undegraded":"2024-05-09T08:03:02.513779+0000","last_fullsized":"2024-05-09T08:03:02.513779+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:35:54.628874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.311909+0000","last_change":"2024-05-09T08:02:20.054016+0000","last_active":"2024-05-09T08:03:02.311909+0000","last_peered":"2024-05-09T08:03:02.311909+0000","last_clean":"2024-05-09T08:03:02.311909+0000","last_became_active":"2024-05-09T08:02:20.053250+0000","last_became_peered":"2024-05-09T08:02:20.053250+0000","last_unstale":"2024-05-09T08:03:02.311909+0000","last_undegraded":"2024-05-09T08:03:02.311909+0000","last_fullsized":"2024-05-09T08:03:02.311909+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:52:40.529638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.610879+0000","last_change":"2024-05-09T08:02:15.907080+0000","last_active":"2024-05-09T08:03:02.610879+0000","last_peered":"2024-05-09T08:03:02.610879+0000","last_clean":"2024-05-09T08:03:02.610879+0000","last_became_active":"2024-05-09T08:02:15.906600+0000","last_became_peered":"2024-05-09T08:02:15.906600+0000","last_unstale":"2024-05-09T08:03:02.610879+0000","last_undegraded":"2024-05-09T08:03:02.610879+0000","last_fullsized":"2024-05-09T08:03:02.610879+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:43:06.938904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.312696+0000","last_change":"2024-05-09T08:02:14.870990+0000","last_active":"2024-05-09T08:03:02.312696+0000","last_peered":"2024-05-09T08:03:02.312696+0000","last_clean":"2024-05-09T08:03:02.312696+0000","last_became_active":"2024-05-09T08:02:14.869818+0000","last_became_peered":"2024-05-09T08:02:14.869818+0000","last_unstale":"2024-05-09T08:03:02.312696+0000","last_undegraded":"2024-05-09T08:03:02.312696+0000","last_fullsized":"2024-05-09T08:03:02.312696+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:25:23.306827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.310516+0000","last_change":"2024-05-09T08:02:19.952582+0000","last_active":"2024-05-09T08:03:02.310516+0000","last_peered":"2024-05-09T08:03:02.310516+0000","last_clean":"2024-05-09T08:03:02.310516+0000","last_became_active":"2024-05-09T08:02:19.952117+0000","last_became_peered":"2024-05-09T08:02:19.952117+0000","last_unstale":"2024-05-09T08:03:02.310516+0000","last_undegraded":"2024-05-09T08:03:02.310516+0000","last_fullsized":"2024-05-09T08:03:02.310516+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:42:00.939587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.513811+0000","last_change":"2024-05-09T08:02:15.913109+0000","last_active":"2024-05-09T08:03:02.513811+0000","last_peered":"2024-05-09T08:03:02.513811+0000","last_clean":"2024-05-09T08:03:02.513811+0000","last_became_active":"2024-05-09T08:02:15.912883+0000","last_became_peered":"2024-05-09T08:02:15.912883+0000","last_unstale":"2024-05-09T08:03:02.513811+0000","last_undegraded":"2024-05-09T08:03:02.513811+0000","last_fullsized":"2024-05-09T08:03:02.513811+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:29:59.441314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.514075+0000","last_change":"2024-05-09T08:02:14.885256+0000","last_active":"2024-05-09T08:03:02.514075+0000","last_peered":"2024-05-09T08:03:02.514075+0000","last_clean":"2024-05-09T08:03:02.514075+0000","last_became_active":"2024-05-09T08:02:14.884965+0000","last_became_peered":"2024-05-09T08:02:14.884965+0000","last_unstale":"2024-05-09T08:03:02.514075+0000","last_undegraded":"2024-05-09T08:03:02.514075+0000","last_fullsized":"2024-05-09T08:03:02.514075+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:14:52.920972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"24'1","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.310239+0000","last_change":"2024-05-09T08:02:14.860339+0000","last_active":"2024-05-09T08:03:02.310239+0000","last_peered":"2024-05-09T08:03:02.310239+0000","last_clean":"2024-05-09T08:03:02.310239+0000","last_became_active":"2024-05-09T08:02:14.859917+0000","last_became_peered":"2024-05-09T08:02:14.859917+0000","last_unstale":"2024-05-09T08:03:02.310239+0000","last_undegraded":"2024-05-09T08:03:02.310239+0000","last_fullsized":"2024-05-09T08:03:02.310239+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:14:52.920972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.314048+0000","last_change":"2024-05-09T08:02:15.897001+0000","last_active":"2024-05-09T08:03:02.314048+0000","last_peered":"2024-05-09T08:03:02.314048+0000","last_clean":"2024-05-09T08:03:02.314048+0000","last_became_active":"2024-05-09T08:02:15.896708+0000","last_became_peered":"2024-05-09T08:02:15.896708+0000","last_unstale":"2024-05-09T08:03:02.314048+0000","last_undegraded":"2024-05-09T08:03:02.314048+0000","last_fullsized":"2024-05-09T08:03:02.314048+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:29:59.441314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.318372+0000","last_change":"2024-05-09T08:02:19.955369+0000","last_active":"2024-05-09T08:03:02.318372+0000","last_peered":"2024-05-09T08:03:02.318372+0000","last_clean":"2024-05-09T08:03:02.318372+0000","last_became_active":"2024-05-09T08:02:19.954829+0000","last_became_peered":"2024-05-09T08:02:19.954829+0000","last_unstale":"2024-05-09T08:03:02.318372+0000","last_undegraded":"2024-05-09T08:03:02.318372+0000","last_fullsized":"2024-05-09T08:03:02.318372+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:49:58.963879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"24'1","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.514396+0000","last_change":"2024-05-09T08:02:14.883856+0000","last_active":"2024-05-09T08:03:02.514396+0000","last_peered":"2024-05-09T08:03:02.514396+0000","last_clean":"2024-05-09T08:03:02.514396+0000","last_became_active":"2024-05-09T08:02:14.882610+0000","last_became_peered":"2024-05-09T08:02:14.882610+0000","last_unstale":"2024-05-09T08:03:02.514396+0000","last_undegraded":"2024-05-09T08:03:02.514396+0000","last_fullsized":"2024-05-09T08:03:02.514396+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:19:31.913236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.314155+0000","last_change":"2024-05-09T08:02:15.889974+0000","last_active":"2024-05-09T08:03:02.314155+0000","last_peered":"2024-05-09T08:03:02.314155+0000","last_clean":"2024-05-09T08:03:02.314155+0000","last_became_active":"2024-05-09T08:02:15.889650+0000","last_became_peered":"2024-05-09T08:02:15.889650+0000","last_unstale":"2024-05-09T08:03:02.314155+0000","last_undegraded":"2024-05-09T08:03:02.314155+0000","last_fullsized":"2024-05-09T08:03:02.314155+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:00:41.613116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.514719+0000","last_change":"2024-05-09T08:02:19.955362+0000","last_active":"2024-05-09T08:03:02.514719+0000","last_peered":"2024-05-09T08:03:02.514719+0000","last_clean":"2024-05-09T08:03:02.514719+0000","last_became_active":"2024-05-09T08:02:19.954947+0000","last_became_peered":"2024-05-09T08:02:19.954947+0000","last_unstale":"2024-05-09T08:03:02.514719+0000","last_undegraded":"2024-05-09T08:03:02.514719+0000","last_fullsized":"2024-05-09T08:03:02.514719+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:43:30.549966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.309981+0000","last_change":"2024-05-09T08:02:15.909379+0000","last_active":"2024-05-09T08:03:02.309981+0000","last_peered":"2024-05-09T08:03:02.309981+0000","last_clean":"2024-05-09T08:03:02.309981+0000","last_became_active":"2024-05-09T08:02:15.909086+0000","last_became_peered":"2024-05-09T08:02:15.909086+0000","last_unstale":"2024-05-09T08:03:02.309981+0000","last_undegraded":"2024-05-09T08:03:02.309981+0000","last_fullsized":"2024-05-09T08:03:02.309981+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:05:02.181367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.318734+0000","last_change":"2024-05-09T08:02:15.891242+0000","last_active":"2024-05-09T08:03:02.318734+0000","last_peered":"2024-05-09T08:03:02.318734+0000","last_clean":"2024-05-09T08:03:02.318734+0000","last_became_active":"2024-05-09T08:02:15.890640+0000","last_became_peered":"2024-05-09T08:02:15.890640+0000","last_unstale":"2024-05-09T08:03:02.318734+0000","last_undegraded":"2024-05-09T08:03:02.318734+0000","last_fullsized":"2024-05-09T08:03:02.318734+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:57:07.107359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.312946+0000","last_change":"2024-05-09T08:02:15.899001+0000","last_active":"2024-05-09T08:03:02.312946+0000","last_peered":"2024-05-09T08:03:02.312946+0000","last_clean":"2024-05-09T08:03:02.312946+0000","last_became_active":"2024-05-09T08:02:15.898695+0000","last_became_peered":"2024-05-09T08:02:15.898695+0000","last_unstale":"2024-05-09T08:03:02.312946+0000","last_undegraded":"2024-05-09T08:03:02.312946+0000","last_fullsized":"2024-05-09T08:03:02.312946+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:41:27.402821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.313935+0000","last_change":"2024-05-09T08:02:15.907374+0000","last_active":"2024-05-09T08:03:02.313935+0000","last_peered":"2024-05-09T08:03:02.313935+0000","last_clean":"2024-05-09T08:03:02.313935+0000","last_became_active":"2024-05-09T08:02:15.906767+0000","last_became_peered":"2024-05-09T08:02:15.906767+0000","last_unstale":"2024-05-09T08:03:02.313935+0000","last_undegraded":"2024-05-09T08:03:02.313935+0000","last_fullsized":"2024-05-09T08:03:02.313935+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:05:02.181367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.514680+0000","last_change":"2024-05-09T08:02:15.905326+0000","last_active":"2024-05-09T08:03:02.514680+0000","last_peered":"2024-05-09T08:03:02.514680+0000","last_clean":"2024-05-09T08:03:02.514680+0000","last_became_active":"2024-05-09T08:02:15.905083+0000","last_became_peered":"2024-05-09T08:02:15.905083+0000","last_unstale":"2024-05-09T08:03:02.514680+0000","last_undegraded":"2024-05-09T08:03:02.514680+0000","last_fullsized":"2024-05-09T08:03:02.514680+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:23:40.248821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.612136+0000","last_change":"2024-05-09T08:02:15.907753+0000","last_active":"2024-05-09T08:03:02.612136+0000","last_peered":"2024-05-09T08:03:02.612136+0000","last_clean":"2024-05-09T08:03:02.612136+0000","last_became_active":"2024-05-09T08:02:15.907480+0000","last_became_peered":"2024-05-09T08:02:15.907480+0000","last_unstale":"2024-05-09T08:03:02.612136+0000","last_undegraded":"2024-05-09T08:03:02.612136+0000","last_fullsized":"2024-05-09T08:03:02.612136+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:20:07.368327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.311045+0000","last_change":"2024-05-09T08:02:15.888605+0000","last_active":"2024-05-09T08:03:02.311045+0000","last_peered":"2024-05-09T08:03:02.311045+0000","last_clean":"2024-05-09T08:03:02.311045+0000","last_became_active":"2024-05-09T08:02:15.887981+0000","last_became_peered":"2024-05-09T08:02:15.887981+0000","last_unstale":"2024-05-09T08:03:02.311045+0000","last_undegraded":"2024-05-09T08:03:02.311045+0000","last_fullsized":"2024-05-09T08:03:02.311045+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:47:32.605288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.515394+0000","last_change":"2024-05-09T08:02:15.904744+0000","last_active":"2024-05-09T08:03:02.515394+0000","last_peered":"2024-05-09T08:03:02.515394+0000","last_clean":"2024-05-09T08:03:02.515394+0000","last_became_active":"2024-05-09T08:02:15.904334+0000","last_became_peered":"2024-05-09T08:02:15.904334+0000","last_unstale":"2024-05-09T08:03:02.515394+0000","last_undegraded":"2024-05-09T08:03:02.515394+0000","last_fullsized":"2024-05-09T08:03:02.515394+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:30:33.351158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.312480+0000","last_change":"2024-05-09T08:02:15.904835+0000","last_active":"2024-05-09T08:03:02.312480+0000","last_peered":"2024-05-09T08:03:02.312480+0000","last_clean":"2024-05-09T08:03:02.312480+0000","last_became_active":"2024-05-09T08:02:15.904416+0000","last_became_peered":"2024-05-09T08:02:15.904416+0000","last_unstale":"2024-05-09T08:03:02.312480+0000","last_undegraded":"2024-05-09T08:03:02.312480+0000","last_fullsized":"2024-05-09T08:03:02.312480+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:47:32.605288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.513916+0000","last_change":"2024-05-09T08:02:15.898324+0000","last_active":"2024-05-09T08:03:02.513916+0000","last_peered":"2024-05-09T08:03:02.513916+0000","last_clean":"2024-05-09T08:03:02.513916+0000","last_became_active":"2024-05-09T08:02:15.897981+0000","last_became_peered":"2024-05-09T08:02:15.897981+0000","last_unstale":"2024-05-09T08:03:02.513916+0000","last_undegraded":"2024-05-09T08:03:02.513916+0000","last_fullsized":"2024-05-09T08:03:02.513916+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:01:50.341133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.312828+0000","last_change":"2024-05-09T08:02:15.890945+0000","last_active":"2024-05-09T08:03:02.312828+0000","last_peered":"2024-05-09T08:03:02.312828+0000","last_clean":"2024-05-09T08:03:02.312828+0000","last_became_active":"2024-05-09T08:02:15.890647+0000","last_became_peered":"2024-05-09T08:02:15.890647+0000","last_unstale":"2024-05-09T08:03:02.312828+0000","last_undegraded":"2024-05-09T08:03:02.312828+0000","last_fullsized":"2024-05-09T08:03:02.312828+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:41:56.871863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.312546+0000","last_change":"2024-05-09T08:02:15.913728+0000","last_active":"2024-05-09T08:03:02.312546+0000","last_peered":"2024-05-09T08:03:02.312546+0000","last_clean":"2024-05-09T08:03:02.312546+0000","last_became_active":"2024-05-09T08:02:15.909818+0000","last_became_peered":"2024-05-09T08:02:15.909818+0000","last_unstale":"2024-05-09T08:03:02.312546+0000","last_undegraded":"2024-05-09T08:03:02.312546+0000","last_fullsized":"2024-05-09T08:03:02.312546+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:41:55.788033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.310499+0000","last_change":"2024-05-09T08:02:15.885025+0000","last_active":"2024-05-09T08:03:02.310499+0000","last_peered":"2024-05-09T08:03:02.310499+0000","last_clean":"2024-05-09T08:03:02.310499+0000","last_became_active":"2024-05-09T08:02:15.884517+0000","last_became_peered":"2024-05-09T08:02:15.884517+0000","last_unstale":"2024-05-09T08:03:02.310499+0000","last_undegraded":"2024-05-09T08:03:02.310499+0000","last_fullsized":"2024-05-09T08:03:02.310499+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:01:50.341133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.606110+0000","last_change":"2024-05-09T08:02:15.890306+0000","last_active":"2024-05-09T08:03:02.606110+0000","last_peered":"2024-05-09T08:03:02.606110+0000","last_clean":"2024-05-09T08:03:02.606110+0000","last_became_active":"2024-05-09T08:02:15.889998+0000","last_became_peered":"2024-05-09T08:02:15.889998+0000","last_unstale":"2024-05-09T08:03:02.606110+0000","last_undegraded":"2024-05-09T08:03:02.606110+0000","last_fullsized":"2024-05-09T08:03:02.606110+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:16:57.446444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.319098+0000","last_change":"2024-05-09T08:02:15.886417+0000","last_active":"2024-05-09T08:03:02.319098+0000","last_peered":"2024-05-09T08:03:02.319098+0000","last_clean":"2024-05-09T08:03:02.319098+0000","last_became_active":"2024-05-09T08:02:15.885525+0000","last_became_peered":"2024-05-09T08:02:15.885525+0000","last_unstale":"2024-05-09T08:03:02.319098+0000","last_undegraded":"2024-05-09T08:03:02.319098+0000","last_fullsized":"2024-05-09T08:03:02.319098+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:43:02.075442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.313589+0000","last_change":"2024-05-09T08:02:15.893172+0000","last_active":"2024-05-09T08:03:02.313589+0000","last_peered":"2024-05-09T08:03:02.313589+0000","last_clean":"2024-05-09T08:03:02.313589+0000","last_became_active":"2024-05-09T08:02:15.892886+0000","last_became_peered":"2024-05-09T08:02:15.892886+0000","last_unstale":"2024-05-09T08:03:02.313589+0000","last_undegraded":"2024-05-09T08:03:02.313589+0000","last_fullsized":"2024-05-09T08:03:02.313589+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:59:14.957806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.312550+0000","last_change":"2024-05-09T08:02:15.908659+0000","last_active":"2024-05-09T08:03:02.312550+0000","last_peered":"2024-05-09T08:03:02.312550+0000","last_clean":"2024-05-09T08:03:02.312550+0000","last_became_active":"2024-05-09T08:02:15.908307+0000","last_became_peered":"2024-05-09T08:02:15.908307+0000","last_unstale":"2024-05-09T08:03:02.312550+0000","last_undegraded":"2024-05-09T08:03:02.312550+0000","last_fullsized":"2024-05-09T08:03:02.312550+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:20:07.368327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.611181+0000","last_change":"2024-05-09T08:02:15.885800+0000","last_active":"2024-05-09T08:03:02.611181+0000","last_peered":"2024-05-09T08:03:02.611181+0000","last_clean":"2024-05-09T08:03:02.611181+0000","last_became_active":"2024-05-09T08:02:15.885185+0000","last_became_peered":"2024-05-09T08:02:15.885185+0000","last_unstale":"2024-05-09T08:03:02.611181+0000","last_undegraded":"2024-05-09T08:03:02.611181+0000","last_fullsized":"2024-05-09T08:03:02.611181+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:01:50.341133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.313497+0000","last_change":"2024-05-09T08:02:15.911496+0000","last_active":"2024-05-09T08:03:02.313497+0000","last_peered":"2024-05-09T08:03:02.313497+0000","last_clean":"2024-05-09T08:03:02.313497+0000","last_became_active":"2024-05-09T08:02:15.911142+0000","last_became_peered":"2024-05-09T08:02:15.911142+0000","last_unstale":"2024-05-09T08:03:02.313497+0000","last_undegraded":"2024-05-09T08:03:02.313497+0000","last_fullsized":"2024-05-09T08:03:02.313497+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:27:59.315750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.514303+0000","last_change":"2024-05-09T08:02:15.898847+0000","last_active":"2024-05-09T08:03:02.514303+0000","last_peered":"2024-05-09T08:03:02.514303+0000","last_clean":"2024-05-09T08:03:02.514303+0000","last_became_active":"2024-05-09T08:02:15.898480+0000","last_became_peered":"2024-05-09T08:02:15.898480+0000","last_unstale":"2024-05-09T08:03:02.514303+0000","last_undegraded":"2024-05-09T08:03:02.514303+0000","last_fullsized":"2024-05-09T08:03:02.514303+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:20:07.368327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.314781+0000","last_change":"2024-05-09T08:02:15.883117+0000","last_active":"2024-05-09T08:03:02.314781+0000","last_peered":"2024-05-09T08:03:02.314781+0000","last_clean":"2024-05-09T08:03:02.314781+0000","last_became_active":"2024-05-09T08:02:15.882141+0000","last_became_peered":"2024-05-09T08:02:15.882141+0000","last_unstale":"2024-05-09T08:03:02.314781+0000","last_undegraded":"2024-05-09T08:03:02.314781+0000","last_fullsized":"2024-05-09T08:03:02.314781+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:57:11.352151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.322462+0000","last_change":"2024-05-09T08:03:02.322462+0000","last_active":"2024-05-09T08:03:02.322462+0000","last_peered":"2024-05-09T08:03:02.322462+0000","last_clean":"2024-05-09T08:03:02.322462+0000","last_became_active":"2024-05-09T08:03:02.322073+0000","last_became_peered":"2024-05-09T08:03:02.322073+0000","last_unstale":"2024-05-09T08:03:02.322462+0000","last_undegraded":"2024-05-09T08:03:02.322462+0000","last_fullsized":"2024-05-09T08:03:02.322462+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:16:37.795617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.310964+0000","last_change":"2024-05-09T08:02:15.886482+0000","last_active":"2024-05-09T08:03:02.310964+0000","last_peered":"2024-05-09T08:03:02.310964+0000","last_clean":"2024-05-09T08:03:02.310964+0000","last_became_active":"2024-05-09T08:02:15.886183+0000","last_became_peered":"2024-05-09T08:02:15.886183+0000","last_unstale":"2024-05-09T08:03:02.310964+0000","last_undegraded":"2024-05-09T08:03:02.310964+0000","last_fullsized":"2024-05-09T08:03:02.310964+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:23:40.248821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.606193+0000","last_change":"2024-05-09T08:02:15.887506+0000","last_active":"2024-05-09T08:03:02.606193+0000","last_peered":"2024-05-09T08:03:02.606193+0000","last_clean":"2024-05-09T08:03:02.606193+0000","last_became_active":"2024-05-09T08:02:15.887225+0000","last_became_peered":"2024-05-09T08:02:15.887225+0000","last_unstale":"2024-05-09T08:03:02.606193+0000","last_undegraded":"2024-05-09T08:03:02.606193+0000","last_fullsized":"2024-05-09T08:03:02.606193+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:25:24.736323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.513523+0000","last_change":"2024-05-09T08:02:15.902065+0000","last_active":"2024-05-09T08:03:02.513523+0000","last_peered":"2024-05-09T08:03:02.513523+0000","last_clean":"2024-05-09T08:03:02.513523+0000","last_became_active":"2024-05-09T08:02:15.901766+0000","last_became_peered":"2024-05-09T08:02:15.901766+0000","last_unstale":"2024-05-09T08:03:02.513523+0000","last_undegraded":"2024-05-09T08:03:02.513523+0000","last_fullsized":"2024-05-09T08:03:02.513523+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:05:02.181367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.605331+0000","last_change":"2024-05-09T08:02:15.911058+0000","last_active":"2024-05-09T08:03:02.605331+0000","last_peered":"2024-05-09T08:03:02.605331+0000","last_clean":"2024-05-09T08:03:02.605331+0000","last_became_active":"2024-05-09T08:02:15.910716+0000","last_became_peered":"2024-05-09T08:02:15.910716+0000","last_unstale":"2024-05-09T08:03:02.605331+0000","last_undegraded":"2024-05-09T08:03:02.605331+0000","last_fullsized":"2024-05-09T08:03:02.605331+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:41:56.871863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.312711+0000","last_change":"2024-05-09T08:02:15.908653+0000","last_active":"2024-05-09T08:03:02.312711+0000","last_peered":"2024-05-09T08:03:02.312711+0000","last_clean":"2024-05-09T08:03:02.312711+0000","last_became_active":"2024-05-09T08:02:15.908290+0000","last_became_peered":"2024-05-09T08:02:15.908290+0000","last_unstale":"2024-05-09T08:03:02.312711+0000","last_undegraded":"2024-05-09T08:03:02.312711+0000","last_fullsized":"2024-05-09T08:03:02.312711+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:18:36.309627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.605649+0000","last_change":"2024-05-09T08:02:15.896652+0000","last_active":"2024-05-09T08:03:02.605649+0000","last_peered":"2024-05-09T08:03:02.605649+0000","last_clean":"2024-05-09T08:03:02.605649+0000","last_became_active":"2024-05-09T08:02:15.896384+0000","last_became_peered":"2024-05-09T08:02:15.896384+0000","last_unstale":"2024-05-09T08:03:02.605649+0000","last_undegraded":"2024-05-09T08:03:02.605649+0000","last_fullsized":"2024-05-09T08:03:02.605649+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:18:36.309627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":10,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.617215+0000","last_change":"2024-05-09T08:03:02.615230+0000","last_active":"2024-05-09T08:03:02.617215+0000","last_peered":"2024-05-09T08:03:02.617215+0000","last_clean":"2024-05-09T08:03:02.617215+0000","last_became_active":"2024-05-09T08:03:02.614387+0000","last_became_peered":"2024-05-09T08:03:02.614387+0000","last_unstale":"2024-05-09T08:03:02.617215+0000","last_undegraded":"2024-05-09T08:03:02.617215+0000","last_fullsized":"2024-05-09T08:03:02.617215+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:53:23.178610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.312999+0000","last_change":"2024-05-09T08:02:15.905078+0000","last_active":"2024-05-09T08:03:02.312999+0000","last_peered":"2024-05-09T08:03:02.312999+0000","last_clean":"2024-05-09T08:03:02.312999+0000","last_became_active":"2024-05-09T08:02:15.904736+0000","last_became_peered":"2024-05-09T08:02:15.904736+0000","last_unstale":"2024-05-09T08:03:02.312999+0000","last_undegraded":"2024-05-09T08:03:02.312999+0000","last_fullsized":"2024-05-09T08:03:02.312999+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:01:50.341133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.611372+0000","last_change":"2024-05-09T08:02:15.897743+0000","last_active":"2024-05-09T08:03:02.611372+0000","last_peered":"2024-05-09T08:03:02.611372+0000","last_clean":"2024-05-09T08:03:02.611372+0000","last_became_active":"2024-05-09T08:02:15.897367+0000","last_became_peered":"2024-05-09T08:02:15.897367+0000","last_unstale":"2024-05-09T08:03:02.611372+0000","last_undegraded":"2024-05-09T08:03:02.611372+0000","last_fullsized":"2024-05-09T08:03:02.611372+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:47:32.605288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":10,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:03:02.325400+0000","last_change":"2024-05-09T08:03:02.323519+0000","last_active":"2024-05-09T08:03:02.325400+0000","last_peered":"2024-05-09T08:03:02.325400+0000","last_clean":"2024-05-09T08:03:02.325400+0000","last_became_active":"2024-05-09T08:03:02.322397+0000","last_became_peered":"2024-05-09T08:03:02.322397+0000","last_unstale":"2024-05-09T08:03:02.325400+0000","last_undegraded":"2024-05-09T08:03:02.325400+0000","last_fullsized":"2024-05-09T08:03:02.325400+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:47:32.605288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574862,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32048,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339792,"statfs":{"total":96636764160,"available":96603947008,"internally_reserved":0,"allocated":888832,"data_stored":709919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574862,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31472,"kb_used_data":296,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340368,"statfs":{"total":96636764160,"available":96604536832,"internally_reserved":0,"allocated":303104,"data_stored":122811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574862,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31464,"kb_used_data":288,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340376,"statfs":{"total":96636764160,"available":96604545024,"internally_reserved":0,"allocated":294912,"data_stored":119059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574863,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31476,"kb_used_data":296,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340364,"statfs":{"total":96636764160,"available":96604532736,"internally_reserved":0,"allocated":303104,"data_stored":119095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574863,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27312,"kb_used_data":296,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344528,"statfs":{"total":96636764160,"available":96608796672,"internally_reserved":0,"allocated":303104,"data_stored":122811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574863,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31468,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340372,"statfs":{"total":96636764160,"available":96604540928,"internally_reserved":0,"allocated":290816,"data_stored":118969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574864,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27312,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344528,"statfs":{"total":96636764160,"available":96608796672,"internally_reserved":0,"allocated":299008,"data_stored":119129,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574863,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32060,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339780,"statfs":{"total":96636764160,"available":96603934720,"internally_reserved":0,"allocated":897024,"data_stored":709975,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T08:03:12.980 INFO:tasks.mon_thrash.ceph_manager:clean! 2024-05-09T08:03:12.980 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T08:03:13.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:13.216+0000 7fa76ea00640 1 Processor -- start 2024-05-09T08:03:13.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:13.216+0000 7fa76ea00640 1 -- start start 2024-05-09T08:03:13.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:13.218+0000 7fa76ea00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa7680c6ea0 0x7fa768072900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:13.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:13.218+0000 7fa76ea00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fa76806ece0 0x7fa768159010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:13.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:13.218+0000 7fa76ea00640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa768073200 con 0x7fa76806dca0 2024-05-09T08:03:13.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:13.218+0000 7fa76ea00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fa768072e40 con 0x7fa7680c6ea0 2024-05-09T08:03:13.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:13.218+0000 7fa76ea00640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7fa7680737c0 con 0x7fa76806ece0 2024-05-09T08:03:13.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:13.218+0000 7fa767400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa7680c6ea0 0x7fa768072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:13.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:13.218+0000 7fa767e00640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7fa76806dca0 0x7fa76806e080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:42754/0 (socket says 172.21.15.1:42754) 2024-05-09T08:03:13.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:13.218+0000 7fa767e00640 1 -- 172.21.15.1:0/2046797909 learned_addr learned my addr 172.21.15.1:0/2046797909 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T08:03:13.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:13.218+0000 7fa76ca00640 1 --2- 172.21.15.1:0/2046797909 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fa76806ece0 0x7fa768159010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:13.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:13.219+0000 7fa767400640 1 -- 172.21.15.1:0/2046797909 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fa76806ece0 msgr2=0x7fa768159010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:03:13.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:13.219+0000 7fa767400640 1 --2- 172.21.15.1:0/2046797909 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fa76806ece0 0x7fa768159010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:13.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:13.219+0000 7fa767400640 1 -- 172.21.15.1:0/2046797909 >> v1:172.21.15.1:6789/0 conn(0x7fa76806dca0 legacy=0x7fa76806e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:13.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:13.219+0000 7fa767400640 1 -- 172.21.15.1:0/2046797909 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7680734e0 con 0x7fa7680c6ea0 2024-05-09T08:03:13.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:13.219+0000 7fa76ca00640 1 --2- 172.21.15.1:0/2046797909 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fa76806ece0 0x7fa768159010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:03:13.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:13.220+0000 7fa767400640 1 --2- 172.21.15.1:0/2046797909 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa7680c6ea0 0x7fa768072900 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fa7540097a0 tx=0x7fa75402a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ee0a0e51743f9e5d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:03:13.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:13.220+0000 7fa766a00640 1 -- 172.21.15.1:0/2046797909 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa754037070 con 0x7fa7680c6ea0 2024-05-09T08:03:13.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:13.220+0000 7fa766a00640 1 -- 172.21.15.1:0/2046797909 <== mon.1 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa754032440 con 0x7fa7680c6ea0 2024-05-09T08:03:13.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:13.220+0000 7fa766a00640 1 -- 172.21.15.1:0/2046797909 <== mon.1 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa754032810 con 0x7fa7680c6ea0 2024-05-09T08:03:13.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:13.221+0000 7fa76ea00640 1 -- 172.21.15.1:0/2046797909 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa7680c6ea0 msgr2=0x7fa768072900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:13.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:13.221+0000 7fa76ea00640 1 --2- 172.21.15.1:0/2046797909 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa7680c6ea0 0x7fa768072900 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fa7540097a0 tx=0x7fa75402a180 comp rx=0 tx=0).stop 2024-05-09T08:03:13.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:13.221+0000 7fa76ea00640 1 -- 172.21.15.1:0/2046797909 shutdown_connections 2024-05-09T08:03:13.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:13.221+0000 7fa76ea00640 1 --2- 172.21.15.1:0/2046797909 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fa76806ece0 0x7fa768159010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:13.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:13.221+0000 7fa76ea00640 1 --2- 172.21.15.1:0/2046797909 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa7680c6ea0 0x7fa768072900 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:13.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:13.221+0000 7fa76ea00640 1 -- 172.21.15.1:0/2046797909 >> 172.21.15.1:0/2046797909 conn(0x7fa7680cc070 msgr2=0x7fa7680cc480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:03:13.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:13.221+0000 7fa76ea00640 1 -- 172.21.15.1:0/2046797909 shutdown_connections 2024-05-09T08:03:13.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:13.221+0000 7fa76ea00640 1 -- 172.21.15.1:0/2046797909 wait complete. 2024-05-09T08:03:13.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:13.223+0000 7fa76ea00640 1 Processor -- start 2024-05-09T08:03:13.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:13.223+0000 7fa76ea00640 1 -- start start 2024-05-09T08:03:13.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:13.224+0000 7fa76ea00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa76806dca0 0x7fa7681f12c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:13.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:13.224+0000 7fa76ea00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa76806ece0 0x7fa76814f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:13.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:13.224+0000 7fa76ea00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fa7680c6ea0 0x7fa7680c2350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:13.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:13.224+0000 7fa76ea00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fa768072470 con 0x7fa76806dca0 2024-05-09T08:03:13.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:13.224+0000 7fa76ea00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa768159550 con 0x7fa76806ece0 2024-05-09T08:03:13.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:13.224+0000 7fa76ea00640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7fa768072100 con 0x7fa7680c6ea0 2024-05-09T08:03:13.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:13.224+0000 7fa767e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa76806dca0 0x7fa7681f12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:13.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:13.224+0000 7fa767e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa76806dca0 0x7fa7681f12c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:39794/0 (socket says 172.21.15.1:39794) 2024-05-09T08:03:13.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:13.224+0000 7fa767e00640 1 -- 172.21.15.1:0/2595938464 learned_addr learned my addr 172.21.15.1:0/2595938464 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:03:13.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:13.224+0000 7fa767400640 1 --2- 172.21.15.1:0/2595938464 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa76806ece0 0x7fa76814f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:13.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:13.224+0000 7fa76ca00640 1 --2- 172.21.15.1:0/2595938464 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fa7680c6ea0 0x7fa7680c2350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:13.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:13.225+0000 7fa76ca00640 1 -- 172.21.15.1:0/2595938464 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa76806ece0 msgr2=0x7fa76814f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:13.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:13.225+0000 7fa76ca00640 1 --2- 172.21.15.1:0/2595938464 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa76806ece0 0x7fa76814f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:13.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:13.225+0000 7fa76ca00640 1 -- 172.21.15.1:0/2595938464 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa76806dca0 msgr2=0x7fa7681f12c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:13.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:13.225+0000 7fa76ca00640 1 --2- 172.21.15.1:0/2595938464 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa76806dca0 0x7fa7681f12c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:13.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:13.225+0000 7fa76ca00640 1 -- 172.21.15.1:0/2595938464 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa76814ce00 con 0x7fa7680c6ea0 2024-05-09T08:03:13.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:13.225+0000 7fa767400640 1 --2- 172.21.15.1:0/2595938464 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa76806ece0 0x7fa76814f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:03:13.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:13.225+0000 7fa767e00640 1 --2- 172.21.15.1:0/2595938464 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa76806dca0 0x7fa7681f12c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:03:13.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:13.225+0000 7fa76ca00640 1 --2- 172.21.15.1:0/2595938464 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fa7680c6ea0 0x7fa7680c2350 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fa76000efc0 tx=0x7fa76000c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:03:13.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:13.225+0000 7fa75fe00640 1 -- 172.21.15.1:0/2595938464 <== mon.2 v2:172.21.15.7:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa760018030 con 0x7fa7680c6ea0 2024-05-09T08:03:13.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:13.225+0000 7fa75fe00640 1 -- 172.21.15.1:0/2595938464 <== mon.2 v2:172.21.15.7:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa76000ccd0 con 0x7fa7680c6ea0 2024-05-09T08:03:13.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:13.225+0000 7fa75fe00640 1 -- 172.21.15.1:0/2595938464 <== mon.2 v2:172.21.15.7:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa760003bb0 con 0x7fa7680c6ea0 2024-05-09T08:03:13.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:13.225+0000 7fa76ea00640 1 -- 172.21.15.1:0/2595938464 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa76814d0c0 con 0x7fa7680c6ea0 2024-05-09T08:03:13.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:13.225+0000 7fa76ea00640 1 -- 172.21.15.1:0/2595938464 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fa76814d680 con 0x7fa7680c6ea0 2024-05-09T08:03:13.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:13.227+0000 7fa76ea00640 1 -- 172.21.15.1:0/2595938464 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7680c87b0 con 0x7fa7680c6ea0 2024-05-09T08:03:13.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:13.228+0000 7fa75fe00640 1 -- 172.21.15.1:0/2595938464 <== mon.2 v2:172.21.15.7:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa76001b070 con 0x7fa7680c6ea0 2024-05-09T08:03:13.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:13.229+0000 7fa75fe00640 1 --2- 172.21.15.1:0/2595938464 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fa74806be60 0x7fa74806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:13.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:13.229+0000 7fa75fe00640 1 -- 172.21.15.1:0/2595938464 <== mon.2 v2:172.21.15.7:3301/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7fa76008bad0 con 0x7fa7680c6ea0 2024-05-09T08:03:13.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:13.229+0000 7fa767e00640 1 --2- 172.21.15.1:0/2595938464 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fa74806be60 0x7fa74806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:13.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:13.230+0000 7fa767e00640 1 --2- 172.21.15.1:0/2595938464 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fa74806be60 0x7fa74806e320 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fa758004e60 tx=0x7fa758007f10 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:03:13.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:13.234+0000 7fa75fe00640 1 -- 172.21.15.1:0/2595938464 <== mon.2 v2:172.21.15.7:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fa7680c87b0 con 0x7fa7680c6ea0 2024-05-09T08:03:13.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:13.439+0000 7fa76ea00640 1 -- 172.21.15.1:0/2595938464 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa768072ba0 con 0x7fa7680c6ea0 2024-05-09T08:03:13.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:13.441+0000 7fa75fe00640 1 -- 172.21.15.1:0/2595938464 <== mon.2 v2:172.21.15.7:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18741 (secure 0 0 0) 0x7fa768072ba0 con 0x7fa7680c6ea0 2024-05-09T08:03:13.458 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T08:03:13.458 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":27,"fsid":"54329ec8-3270-4043-9a82-c6cd7420aa7f","created":"2024-05-09T08:01:56.666970+0000","modified":"2024-05-09T08:03:02.304156+0000","last_up_change":"2024-05-09T08:02:07.146082+0000","last_in_change":"2024-05-09T08:01:59.008225+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T08:02:09.165511+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T08:02:12.925485+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T08:02:14.367455+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-09T08:02:17.967064+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c064454e-a9b5-4a24-9a77-17f0a4085952","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6800","nonce":1513349547},{"type":"v1","addr":"172.21.15.1:6801","nonce":1513349547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":1513349547},{"type":"v1","addr":"172.21.15.1:6803","nonce":1513349547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":1513349547},{"type":"v1","addr":"172.21.15.1:6807","nonce":1513349547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":1513349547},{"type":"v1","addr":"172.21.15.1:6805","nonce":1513349547}]},"public_addr":"172.21.15.1:6801/1513349547","cluster_addr":"172.21.15.1:6803/1513349547","heartbeat_back_addr":"172.21.15.1:6807/1513349547","heartbeat_front_addr":"172.21.15.1:6805/1513349547","state":["exists","up"]},{"osd":1,"uuid":"1a9ea687-2ff6-407a-949b-0e7e1083af21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":2942255259},{"type":"v1","addr":"172.21.15.1:6817","nonce":2942255259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":2942255259},{"type":"v1","addr":"172.21.15.1:6819","nonce":2942255259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":2942255259},{"type":"v1","addr":"172.21.15.1:6823","nonce":2942255259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6820","nonce":2942255259},{"type":"v1","addr":"172.21.15.1:6821","nonce":2942255259}]},"public_addr":"172.21.15.1:6817/2942255259","cluster_addr":"172.21.15.1:6819/2942255259","heartbeat_back_addr":"172.21.15.1:6823/2942255259","heartbeat_front_addr":"172.21.15.1:6821/2942255259","state":["exists","up"]},{"osd":2,"uuid":"c88af08b-7c9a-4dd8-9640-44138239addb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":2925031811},{"type":"v1","addr":"172.21.15.1:6809","nonce":2925031811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":2925031811},{"type":"v1","addr":"172.21.15.1:6811","nonce":2925031811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":2925031811},{"type":"v1","addr":"172.21.15.1:6815","nonce":2925031811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":2925031811},{"type":"v1","addr":"172.21.15.1:6813","nonce":2925031811}]},"public_addr":"172.21.15.1:6809/2925031811","cluster_addr":"172.21.15.1:6811/2925031811","heartbeat_back_addr":"172.21.15.1:6815/2925031811","heartbeat_front_addr":"172.21.15.1:6813/2925031811","state":["exists","up"]},{"osd":3,"uuid":"c087b7e3-a41d-4333-9365-32e3bd8c8e6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6824","nonce":2497776654},{"type":"v1","addr":"172.21.15.1:6825","nonce":2497776654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":2497776654},{"type":"v1","addr":"172.21.15.1:6827","nonce":2497776654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":2497776654},{"type":"v1","addr":"172.21.15.1:6831","nonce":2497776654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6828","nonce":2497776654},{"type":"v1","addr":"172.21.15.1:6829","nonce":2497776654}]},"public_addr":"172.21.15.1:6825/2497776654","cluster_addr":"172.21.15.1:6827/2497776654","heartbeat_back_addr":"172.21.15.1:6831/2497776654","heartbeat_front_addr":"172.21.15.1:6829/2497776654","state":["exists","up"]},{"osd":4,"uuid":"b4576b9c-871d-47a8-8d36-65af525d0200","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":2957026670},{"type":"v1","addr":"172.21.15.7:6801","nonce":2957026670}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2957026670},{"type":"v1","addr":"172.21.15.7:6803","nonce":2957026670}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":2957026670},{"type":"v1","addr":"172.21.15.7:6807","nonce":2957026670}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2957026670},{"type":"v1","addr":"172.21.15.7:6805","nonce":2957026670}]},"public_addr":"172.21.15.7:6801/2957026670","cluster_addr":"172.21.15.7:6803/2957026670","heartbeat_back_addr":"172.21.15.7:6807/2957026670","heartbeat_front_addr":"172.21.15.7:6805/2957026670","state":["exists","up"]},{"osd":5,"uuid":"902b3d94-72f3-4499-b21a-e32453ef5144","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1670295242},{"type":"v1","addr":"172.21.15.7:6809","nonce":1670295242}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1670295242},{"type":"v1","addr":"172.21.15.7:6811","nonce":1670295242}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1670295242},{"type":"v1","addr":"172.21.15.7:6815","nonce":1670295242}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1670295242},{"type":"v1","addr":"172.21.15.7:6813","nonce":1670295242}]},"public_addr":"172.21.15.7:6809/1670295242","cluster_addr":"172.21.15.7:6811/1670295242","heartbeat_back_addr":"172.21.15.7:6815/1670295242","heartbeat_front_addr":"172.21.15.7:6813/1670295242","state":["exists","up"]},{"osd":6,"uuid":"ead1d771-1728-4f85-adb4-26be21cedff3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":2852086880},{"type":"v1","addr":"172.21.15.7:6817","nonce":2852086880}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":2852086880},{"type":"v1","addr":"172.21.15.7:6819","nonce":2852086880}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":2852086880},{"type":"v1","addr":"172.21.15.7:6823","nonce":2852086880}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":2852086880},{"type":"v1","addr":"172.21.15.7:6821","nonce":2852086880}]},"public_addr":"172.21.15.7:6817/2852086880","cluster_addr":"172.21.15.7:6819/2852086880","heartbeat_back_addr":"172.21.15.7:6823/2852086880","heartbeat_front_addr":"172.21.15.7:6821/2852086880","state":["exists","up"]},{"osd":7,"uuid":"3ab5a201-9394-48a8-8918-e95197893669","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":1442884138},{"type":"v1","addr":"172.21.15.7:6825","nonce":1442884138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1442884138},{"type":"v1","addr":"172.21.15.7:6827","nonce":1442884138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1442884138},{"type":"v1","addr":"172.21.15.7:6831","nonce":1442884138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1442884138},{"type":"v1","addr":"172.21.15.7:6829","nonce":1442884138}]},"public_addr":"172.21.15.7:6825/1442884138","cluster_addr":"172.21.15.7:6827/1442884138","heartbeat_back_addr":"172.21.15.7:6831/1442884138","heartbeat_front_addr":"172.21.15.7:6829/1442884138","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:02:04.615675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:02:04.385974+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:02:04.619217+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:02:04.401681+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:02:04.097081+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:02:04.129981+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:02:04.225327+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:02:04.193351+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.35","mappings":[{"from":5,"to":2}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2},{"from":3,"to":1}]},{"pgid":"3.3f","mappings":[{"from":7,"to":2}]},{"pgid":"4.4","mappings":[{"from":1,"to":5}]},{"pgid":"4.e","mappings":[{"from":1,"to":2}]},{"pgid":"4.f","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T08:03:13.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:13.443+0000 7fa76ea00640 1 -- 172.21.15.1:0/2595938464 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fa74806be60 msgr2=0x7fa74806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:13.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:13.443+0000 7fa76ea00640 1 --2- 172.21.15.1:0/2595938464 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fa74806be60 0x7fa74806e320 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fa758004e60 tx=0x7fa758007f10 comp rx=0 tx=0).stop 2024-05-09T08:03:13.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:13.443+0000 7fa76ea00640 1 -- 172.21.15.1:0/2595938464 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fa7680c6ea0 msgr2=0x7fa7680c2350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:13.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:13.443+0000 7fa76ea00640 1 --2- 172.21.15.1:0/2595938464 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fa7680c6ea0 0x7fa7680c2350 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fa76000efc0 tx=0x7fa76000c490 comp rx=0 tx=0).stop 2024-05-09T08:03:13.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:13.444+0000 7fa76ea00640 1 -- 172.21.15.1:0/2595938464 shutdown_connections 2024-05-09T08:03:13.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:13.444+0000 7fa76ea00640 1 --2- 172.21.15.1:0/2595938464 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fa74806be60 0x7fa74806e320 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:13.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:13.444+0000 7fa76ea00640 1 --2- 172.21.15.1:0/2595938464 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa76806ece0 0x7fa76814f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:13.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:13.444+0000 7fa76ea00640 1 --2- 172.21.15.1:0/2595938464 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fa7680c6ea0 0x7fa7680c2350 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:13.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:13.444+0000 7fa76ea00640 1 --2- 172.21.15.1:0/2595938464 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa76806dca0 0x7fa7681f12c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:13.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:13.444+0000 7fa76ea00640 1 -- 172.21.15.1:0/2595938464 >> 172.21.15.1:0/2595938464 conn(0x7fa7680cc070 msgr2=0x7fa768151810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:03:13.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:13.444+0000 7fa76ea00640 1 -- 172.21.15.1:0/2595938464 shutdown_connections 2024-05-09T08:03:13.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:13.444+0000 7fa76ea00640 1 -- 172.21.15.1:0/2595938464 wait complete. 2024-05-09T08:03:13.481 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T08:02:09.165511+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-09T08:02:12.925485+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-09T08:02:14.367455+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-09T08:02:17.967064+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-09T08:03:13.481 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-09T08:03:13.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:13.683+0000 7f4f36e00640 1 Processor -- start 2024-05-09T08:03:13.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:13.684+0000 7f4f36e00640 1 -- start start 2024-05-09T08:03:13.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:13.686+0000 7f4f36e00640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f4f301543b0 0x7f4f300c84e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:13.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:13.686+0000 7f4f36e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f4f30154790 0x7f4f3015acd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:13.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:13.686+0000 7f4f36e00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4f3006c490 con 0x7f4f300cdf90 2024-05-09T08:03:13.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:13.686+0000 7f4f36e00640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f4f3006d0a0 con 0x7f4f301543b0 2024-05-09T08:03:13.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:13.686+0000 7f4f36e00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f4f3015b210 con 0x7f4f30154790 2024-05-09T08:03:13.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:13.686+0000 7f4f35000640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f4f30154790 0x7f4f3015acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:13.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:13.686+0000 7f4f35000640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f4f30154790 0x7f4f3015acd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.1:33734/0 (socket says 172.21.15.1:33734) 2024-05-09T08:03:13.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:13.686+0000 7f4f35000640 1 -- 172.21.15.1:0/2886199181 learned_addr learned my addr 172.21.15.1:0/2886199181 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:03:13.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:13.686+0000 7f4f36400640 1 --1- 172.21.15.1:0/2886199181 >> v1:172.21.15.7:6789/0 conn(0x7f4f300cdf90 0x7f4f300c2ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.1:50128/0 (socket says 172.21.15.1:50128) 2024-05-09T08:03:13.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:13.686+0000 7f4f35a00640 1 --2- 172.21.15.1:0/2886199181 >> v2:172.21.15.1:3300/0 conn(0x7f4f301543b0 0x7f4f300c84e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:13.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:13.687+0000 7f4f35000640 1 -- 172.21.15.1:0/2886199181 >> v2:172.21.15.1:3300/0 conn(0x7f4f301543b0 msgr2=0x7f4f300c84e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:03:13.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:13.687+0000 7f4f35000640 1 --2- 172.21.15.1:0/2886199181 >> v2:172.21.15.1:3300/0 conn(0x7f4f301543b0 0x7f4f300c84e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:13.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:13.687+0000 7f4f35000640 1 -- 172.21.15.1:0/2886199181 >> v1:172.21.15.7:6789/0 conn(0x7f4f300cdf90 legacy=0x7f4f300c2ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:13.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:13.687+0000 7f4f35000640 1 -- 172.21.15.1:0/2886199181 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f3006c770 con 0x7f4f30154790 2024-05-09T08:03:13.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:13.688+0000 7f4f35000640 1 --2- 172.21.15.1:0/2886199181 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4f30154790 0x7f4f3015acd0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f4f300cb9b0 tx=0x7f4f1c02a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=22aae56d78ce0103 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:03:13.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:13.688+0000 7f4f2fe00640 1 -- 172.21.15.1:0/2886199181 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f1c037070 con 0x7f4f30154790 2024-05-09T08:03:13.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:13.688+0000 7f4f2fe00640 1 -- 172.21.15.1:0/2886199181 <== mon.1 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f1c032440 con 0x7f4f30154790 2024-05-09T08:03:13.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:13.688+0000 7f4f2fe00640 1 -- 172.21.15.1:0/2886199181 <== mon.1 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f1c032810 con 0x7f4f30154790 2024-05-09T08:03:13.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:13.689+0000 7f4f36e00640 1 -- 172.21.15.1:0/2886199181 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4f30154790 msgr2=0x7f4f3015acd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:13.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:13.689+0000 7f4f36e00640 1 --2- 172.21.15.1:0/2886199181 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4f30154790 0x7f4f3015acd0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f4f300cb9b0 tx=0x7f4f1c02a270 comp rx=0 tx=0).stop 2024-05-09T08:03:13.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:13.689+0000 7f4f36e00640 1 -- 172.21.15.1:0/2886199181 shutdown_connections 2024-05-09T08:03:13.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:13.689+0000 7f4f36e00640 1 --2- 172.21.15.1:0/2886199181 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4f30154790 0x7f4f3015acd0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:13.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:13.689+0000 7f4f36e00640 1 --2- 172.21.15.1:0/2886199181 >> v2:172.21.15.1:3300/0 conn(0x7f4f301543b0 0x7f4f300c84e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:13.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:13.689+0000 7f4f36e00640 1 -- 172.21.15.1:0/2886199181 >> 172.21.15.1:0/2886199181 conn(0x7f4f300672f0 msgr2=0x7f4f30067700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:03:13.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:13.689+0000 7f4f36e00640 1 -- 172.21.15.1:0/2886199181 shutdown_connections 2024-05-09T08:03:13.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:13.689+0000 7f4f36e00640 1 -- 172.21.15.1:0/2886199181 wait complete. 2024-05-09T08:03:13.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:13.691+0000 7f4f36e00640 1 Processor -- start 2024-05-09T08:03:13.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:13.691+0000 7f4f36e00640 1 -- start start 2024-05-09T08:03:13.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:13.693+0000 7f4f36e00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f4f300cdf90 0x7f4f300c9860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:13.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:13.693+0000 7f4f36e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4f301543b0 0x7f4f300ccca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:13.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:13.693+0000 7f4f36e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4f30154790 0x7f4f301f3610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:13.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:13.693+0000 7f4f36e00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f4f3006cd30 con 0x7f4f30154790 2024-05-09T08:03:13.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:13.693+0000 7f4f36e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f4f300c8b70 con 0x7f4f301543b0 2024-05-09T08:03:13.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:13.693+0000 7f4f36e00640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f4f30070b60 con 0x7f4f300cdf90 2024-05-09T08:03:13.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:13.693+0000 7f4f36400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4f30154790 0x7f4f301f3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:13.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:13.693+0000 7f4f35000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4f301543b0 0x7f4f300ccca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:13.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:13.693+0000 7f4f35a00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f4f300cdf90 0x7f4f300c9860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:13.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:13.693+0000 7f4f35a00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f4f300cdf90 0x7f4f300c9860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3301/0 says I am v2:172.21.15.1:49688/0 (socket says 172.21.15.1:49688) 2024-05-09T08:03:13.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:13.693+0000 7f4f35a00640 1 -- 172.21.15.1:0/3269528404 learned_addr learned my addr 172.21.15.1:0/3269528404 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:03:13.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:13.693+0000 7f4f35000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4f301543b0 0x7f4f300ccca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.1:33738/0 (socket says 172.21.15.1:33738) 2024-05-09T08:03:13.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:13.694+0000 7f4f35000640 1 -- 172.21.15.1:0/3269528404 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f4f300cdf90 msgr2=0x7f4f300c9860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:13.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:13.694+0000 7f4f35000640 1 --2- 172.21.15.1:0/3269528404 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f4f300cdf90 0x7f4f300c9860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:13.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:13.694+0000 7f4f35000640 1 -- 172.21.15.1:0/3269528404 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4f30154790 msgr2=0x7f4f301f3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:13.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:13.694+0000 7f4f35000640 1 --2- 172.21.15.1:0/3269528404 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4f30154790 0x7f4f301f3610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:13.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:13.694+0000 7f4f35000640 1 -- 172.21.15.1:0/3269528404 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f300ca680 con 0x7f4f301543b0 2024-05-09T08:03:13.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:13.694+0000 7f4f35a00640 1 --2- 172.21.15.1:0/3269528404 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f4f300cdf90 0x7f4f300c9860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:03:13.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:13.694+0000 7f4f35000640 1 --2- 172.21.15.1:0/3269528404 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4f301543b0 0x7f4f300ccca0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f4f300c9240 tx=0x7f4f1c0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:03:13.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:13.695+0000 7f4f2d600640 1 -- 172.21.15.1:0/3269528404 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f1c041070 con 0x7f4f301543b0 2024-05-09T08:03:13.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:13.695+0000 7f4f2d600640 1 -- 172.21.15.1:0/3269528404 <== mon.1 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f1c030680 con 0x7f4f301543b0 2024-05-09T08:03:13.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:13.695+0000 7f4f2d600640 1 -- 172.21.15.1:0/3269528404 <== mon.1 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f1c037040 con 0x7f4f301543b0 2024-05-09T08:03:13.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:13.695+0000 7f4f36e00640 1 -- 172.21.15.1:0/3269528404 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f30141380 con 0x7f4f301543b0 2024-05-09T08:03:13.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:13.695+0000 7f4f36e00640 1 -- 172.21.15.1:0/3269528404 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f300c4650 con 0x7f4f301543b0 2024-05-09T08:03:13.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:13.696+0000 7f4f36e00640 1 -- 172.21.15.1:0/3269528404 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f3004f660 con 0x7f4f301543b0 2024-05-09T08:03:13.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:13.697+0000 7f4f2d600640 1 -- 172.21.15.1:0/3269528404 <== mon.1 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4f1c030a70 con 0x7f4f301543b0 2024-05-09T08:03:13.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:13.698+0000 7f4f2d600640 1 --2- 172.21.15.1:0/3269528404 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f4f1006be30 0x7f4f1006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:13.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:13.698+0000 7f4f2d600640 1 -- 172.21.15.1:0/3269528404 <== mon.1 v2:172.21.15.7:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7f4f1c0ab3c0 con 0x7f4f301543b0 2024-05-09T08:03:13.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:13.699+0000 7f4f35a00640 1 --2- 172.21.15.1:0/3269528404 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f4f1006be30 0x7f4f1006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:13.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:13.699+0000 7f4f35a00640 1 --2- 172.21.15.1:0/3269528404 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f4f1006be30 0x7f4f1006e2f0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f4f200112c0 tx=0x7f4f20009190 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:03:13.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:13.703+0000 7f4f2d600640 1 -- 172.21.15.1:0/3269528404 <== mon.1 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f4f1c079320 con 0x7f4f301543b0 2024-05-09T08:03:13.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:13.907+0000 7f4f36e00640 1 -- 172.21.15.1:0/3269528404 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f4f300c9da0 con 0x7f4f301543b0 2024-05-09T08:03:13.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:13.907+0000 7f4f2d600640 1 -- 172.21.15.1:0/3269528404 <== mon.1 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) ==== 93+0+10 (secure 0 0 0) 0x7f4f1c07d620 con 0x7f4f301543b0 2024-05-09T08:03:13.925 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 1 2024-05-09T08:03:13.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:13.910+0000 7f4f36e00640 1 -- 172.21.15.1:0/3269528404 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f4f1006be30 msgr2=0x7f4f1006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:13.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:13.910+0000 7f4f36e00640 1 --2- 172.21.15.1:0/3269528404 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f4f1006be30 0x7f4f1006e2f0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f4f200112c0 tx=0x7f4f20009190 comp rx=0 tx=0).stop 2024-05-09T08:03:13.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:13.910+0000 7f4f36e00640 1 -- 172.21.15.1:0/3269528404 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4f301543b0 msgr2=0x7f4f300ccca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:13.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:13.910+0000 7f4f36e00640 1 --2- 172.21.15.1:0/3269528404 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4f301543b0 0x7f4f300ccca0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f4f300c9240 tx=0x7f4f1c0046c0 comp rx=0 tx=0).stop 2024-05-09T08:03:13.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:13.911+0000 7f4f36e00640 1 -- 172.21.15.1:0/3269528404 shutdown_connections 2024-05-09T08:03:13.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:13.911+0000 7f4f36e00640 1 --2- 172.21.15.1:0/3269528404 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f4f1006be30 0x7f4f1006e2f0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:13.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:13.911+0000 7f4f36e00640 1 --2- 172.21.15.1:0/3269528404 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4f301543b0 0x7f4f300ccca0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:13.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:13.911+0000 7f4f36e00640 1 --2- 172.21.15.1:0/3269528404 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4f30154790 0x7f4f301f3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:13.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:13.911+0000 7f4f36e00640 1 --2- 172.21.15.1:0/3269528404 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f4f300cdf90 0x7f4f300c9860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:13.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:13.911+0000 7f4f36e00640 1 -- 172.21.15.1:0/3269528404 >> 172.21.15.1:0/3269528404 conn(0x7f4f300672f0 msgr2=0x7f4f30149460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:03:13.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:13.911+0000 7f4f36e00640 1 -- 172.21.15.1:0/3269528404 shutdown_connections 2024-05-09T08:03:13.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:13.911+0000 7f4f36e00640 1 -- 172.21.15.1:0/3269528404 wait complete. 2024-05-09T08:03:13.946 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-09T08:03:14.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:14.146+0000 7f092b200640 1 Processor -- start 2024-05-09T08:03:14.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:14.147+0000 7f092b200640 1 -- start start 2024-05-09T08:03:14.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:14.148+0000 7f092b200640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f092c0c58d0 0x7f092c1589e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:14.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:14.148+0000 7f092b200640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f092c0c6470 0x7f092c158f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:14.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:14.148+0000 7f092b200640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f092c0718e0 con 0x7f092c0c54f0 2024-05-09T08:03:14.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:14.149+0000 7f092b200640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f092c0726e0 con 0x7f092c0c58d0 2024-05-09T08:03:14.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:14.149+0000 7f092b200640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f092c070040 con 0x7f092c0c6470 2024-05-09T08:03:14.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:14.149+0000 7f0923e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f092c0c58d0 0x7f092c1589e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:14.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:14.149+0000 7f0929e00640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f092c0c54f0 0x7f092c06f930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:36884/0 (socket says 172.21.15.1:36884) 2024-05-09T08:03:14.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:14.149+0000 7f0929e00640 1 -- 172.21.15.1:0/4265042613 learned_addr learned my addr 172.21.15.1:0/4265042613 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T08:03:14.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:14.149+0000 7f092a800640 1 --2- 172.21.15.1:0/4265042613 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f092c0c6470 0x7f092c158f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:14.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:14.149+0000 7f0923e00640 1 -- 172.21.15.1:0/4265042613 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f092c0c6470 msgr2=0x7f092c158f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:03:14.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:14.149+0000 7f0923e00640 1 --2- 172.21.15.1:0/4265042613 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f092c0c6470 0x7f092c158f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:14.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:14.149+0000 7f0923e00640 1 -- 172.21.15.1:0/4265042613 >> v1:172.21.15.1:6789/0 conn(0x7f092c0c54f0 legacy=0x7f092c06f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:14.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:14.149+0000 7f0923e00640 1 -- 172.21.15.1:0/4265042613 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f092c071b60 con 0x7f092c0c58d0 2024-05-09T08:03:14.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:14.150+0000 7f0923e00640 1 --2- 172.21.15.1:0/4265042613 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f092c0c58d0 0x7f092c1589e0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f09140097a0 tx=0x7f091402a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=433f80c6ef421874 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:03:14.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:14.151+0000 7f0929400640 1 -- 172.21.15.1:0/4265042613 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0914037070 con 0x7f092c0c58d0 2024-05-09T08:03:14.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:14.151+0000 7f0929400640 1 -- 172.21.15.1:0/4265042613 <== mon.1 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0914032440 con 0x7f092c0c58d0 2024-05-09T08:03:14.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:14.151+0000 7f0929400640 1 -- 172.21.15.1:0/4265042613 <== mon.1 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0914032810 con 0x7f092c0c58d0 2024-05-09T08:03:14.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:14.151+0000 7f092b200640 1 -- 172.21.15.1:0/4265042613 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f092c0c58d0 msgr2=0x7f092c1589e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:14.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:14.151+0000 7f092b200640 1 --2- 172.21.15.1:0/4265042613 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f092c0c58d0 0x7f092c1589e0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f09140097a0 tx=0x7f091402a180 comp rx=0 tx=0).stop 2024-05-09T08:03:14.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:14.152+0000 7f092b200640 1 -- 172.21.15.1:0/4265042613 shutdown_connections 2024-05-09T08:03:14.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:14.152+0000 7f092b200640 1 --2- 172.21.15.1:0/4265042613 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f092c0c6470 0x7f092c158f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:14.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:14.152+0000 7f092b200640 1 --2- 172.21.15.1:0/4265042613 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f092c0c58d0 0x7f092c1589e0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:14.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:14.152+0000 7f092b200640 1 -- 172.21.15.1:0/4265042613 >> 172.21.15.1:0/4265042613 conn(0x7f092c06b250 msgr2=0x7f092c06b630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:03:14.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:14.152+0000 7f092b200640 1 -- 172.21.15.1:0/4265042613 shutdown_connections 2024-05-09T08:03:14.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:14.152+0000 7f092b200640 1 -- 172.21.15.1:0/4265042613 wait complete. 2024-05-09T08:03:14.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:14.154+0000 7f092b200640 1 Processor -- start 2024-05-09T08:03:14.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:14.154+0000 7f092b200640 1 -- start start 2024-05-09T08:03:14.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:14.155+0000 7f092b200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f092c0c54f0 0x7f092c14b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:14.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:14.155+0000 7f092b200640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f092c0c58d0 0x7f092c0c9b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:14.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:14.155+0000 7f092b200640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f092c0c6470 0x7f092c0caa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:14.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:14.155+0000 7f092b200640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f092c072400 con 0x7f092c0c58d0 2024-05-09T08:03:14.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:14.155+0000 7f092b200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f092c072120 con 0x7f092c0c54f0 2024-05-09T08:03:14.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:14.155+0000 7f092b200640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f092c1584c0 con 0x7f092c0c6470 2024-05-09T08:03:14.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:14.156+0000 7f0929e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f092c0c54f0 0x7f092c14b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:14.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:14.156+0000 7f0923e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f092c0c58d0 0x7f092c0c9b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:14.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:14.156+0000 7f0929e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f092c0c54f0 0x7f092c14b550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.1:43804/0 (socket says 172.21.15.1:43804) 2024-05-09T08:03:14.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:14.156+0000 7f0929e00640 1 -- 172.21.15.1:0/987311769 learned_addr learned my addr 172.21.15.1:0/987311769 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:03:14.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:14.156+0000 7f092a800640 1 --2- 172.21.15.1:0/987311769 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f092c0c6470 0x7f092c0caa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:14.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:14.156+0000 7f0929e00640 1 -- 172.21.15.1:0/987311769 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f092c0c6470 msgr2=0x7f092c0caa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:14.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:14.156+0000 7f0929e00640 1 --2- 172.21.15.1:0/987311769 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f092c0c6470 0x7f092c0caa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:14.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:14.156+0000 7f0929e00640 1 -- 172.21.15.1:0/987311769 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f092c0c58d0 msgr2=0x7f092c0c9b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:14.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:14.156+0000 7f0929e00640 1 --2- 172.21.15.1:0/987311769 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f092c0c58d0 0x7f092c0c9b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:14.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:14.156+0000 7f0929e00640 1 -- 172.21.15.1:0/987311769 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f092c155690 con 0x7f092c0c54f0 2024-05-09T08:03:14.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:14.156+0000 7f0929e00640 1 --2- 172.21.15.1:0/987311769 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f092c0c54f0 0x7f092c14b550 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f091c00c420 tx=0x7f091c00c8e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:03:14.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:14.157+0000 7f0922000640 1 -- 172.21.15.1:0/987311769 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f091c017070 con 0x7f092c0c54f0 2024-05-09T08:03:14.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:14.157+0000 7f0922000640 1 -- 172.21.15.1:0/987311769 <== mon.1 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f091c005ce0 con 0x7f092c0c54f0 2024-05-09T08:03:14.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:14.157+0000 7f0922000640 1 -- 172.21.15.1:0/987311769 <== mon.1 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f091c0072f0 con 0x7f092c0c54f0 2024-05-09T08:03:14.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:14.157+0000 7f092b200640 1 -- 172.21.15.1:0/987311769 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f092c152e10 con 0x7f092c0c54f0 2024-05-09T08:03:14.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:14.157+0000 7f092b200640 1 -- 172.21.15.1:0/987311769 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f092c1533d0 con 0x7f092c0c54f0 2024-05-09T08:03:14.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:14.158+0000 7f092b200640 1 -- 172.21.15.1:0/987311769 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f092c0c82c0 con 0x7f092c0c54f0 2024-05-09T08:03:14.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:14.159+0000 7f0922000640 1 -- 172.21.15.1:0/987311769 <== mon.1 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f091c01a070 con 0x7f092c0c54f0 2024-05-09T08:03:14.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:14.160+0000 7f0922000640 1 --2- 172.21.15.1:0/987311769 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f090806be80 0x7f090806e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:14.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:14.160+0000 7f0922000640 1 -- 172.21.15.1:0/987311769 <== mon.1 v2:172.21.15.7:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7f091c08a480 con 0x7f092c0c54f0 2024-05-09T08:03:14.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:14.160+0000 7f0923e00640 1 --2- 172.21.15.1:0/987311769 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f090806be80 0x7f090806e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:14.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:14.161+0000 7f0923e00640 1 --2- 172.21.15.1:0/987311769 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f090806be80 0x7f090806e340 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f0914034000 tx=0x7f091402e000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:03:14.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:14.165+0000 7f0922000640 1 -- 172.21.15.1:0/987311769 <== mon.1 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f092c0c82c0 con 0x7f092c0c54f0 2024-05-09T08:03:14.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:14.366+0000 7f092b200640 1 -- 172.21.15.1:0/987311769 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f092c0bf850 con 0x7f092c0c54f0 2024-05-09T08:03:14.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:14.366+0000 7f0922000640 1 -- 172.21.15.1:0/987311769 <== mon.1 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) ==== 104+0+11 (secure 0 0 0) 0x7f092c0718e0 con 0x7f092c0c54f0 2024-05-09T08:03:14.384 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 32 2024-05-09T08:03:14.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:14.369+0000 7f092b200640 1 -- 172.21.15.1:0/987311769 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f090806be80 msgr2=0x7f090806e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:14.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:14.369+0000 7f092b200640 1 --2- 172.21.15.1:0/987311769 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f090806be80 0x7f090806e340 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f0914034000 tx=0x7f091402e000 comp rx=0 tx=0).stop 2024-05-09T08:03:14.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:14.369+0000 7f092b200640 1 -- 172.21.15.1:0/987311769 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f092c0c54f0 msgr2=0x7f092c14b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:14.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:14.369+0000 7f092b200640 1 --2- 172.21.15.1:0/987311769 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f092c0c54f0 0x7f092c14b550 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f091c00c420 tx=0x7f091c00c8e0 comp rx=0 tx=0).stop 2024-05-09T08:03:14.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:14.370+0000 7f092b200640 1 -- 172.21.15.1:0/987311769 shutdown_connections 2024-05-09T08:03:14.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:14.370+0000 7f092b200640 1 --2- 172.21.15.1:0/987311769 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f090806be80 0x7f090806e340 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:14.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:14.370+0000 7f092b200640 1 --2- 172.21.15.1:0/987311769 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f092c0c6470 0x7f092c0caa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:14.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:14.370+0000 7f092b200640 1 --2- 172.21.15.1:0/987311769 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f092c0c58d0 0x7f092c0c9b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:14.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:14.370+0000 7f092b200640 1 --2- 172.21.15.1:0/987311769 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f092c0c54f0 0x7f092c14b550 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:14.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:14.370+0000 7f092b200640 1 -- 172.21.15.1:0/987311769 >> 172.21.15.1:0/987311769 conn(0x7f092c06b250 msgr2=0x7f092c14d890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:03:14.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:14.370+0000 7f092b200640 1 -- 172.21.15.1:0/987311769 shutdown_connections 2024-05-09T08:03:14.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:14.370+0000 7f092b200640 1 -- 172.21.15.1:0/987311769 wait complete. 2024-05-09T08:03:14.407 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-09T08:03:14.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:14.602+0000 7f2bc3200640 1 Processor -- start 2024-05-09T08:03:14.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:14.602+0000 7f2bc3200640 1 -- start start 2024-05-09T08:03:14.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:14.604+0000 7f2bc3200640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f2bbc0c58d0 0x7f2bbc1589e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:14.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:14.604+0000 7f2bc3200640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f2bbc0c6470 0x7f2bbc158f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:14.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:14.604+0000 7f2bc3200640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2bbc0718e0 con 0x7f2bbc0c54f0 2024-05-09T08:03:14.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:14.604+0000 7f2bc3200640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f2bbc0726e0 con 0x7f2bbc0c6470 2024-05-09T08:03:14.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:14.604+0000 7f2bc3200640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f2bbc070040 con 0x7f2bbc0c58d0 2024-05-09T08:03:14.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:14.604+0000 7f2bc2800640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f2bbc0c6470 0x7f2bbc158f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:14.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:14.604+0000 7f2bc1400640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f2bbc0c58d0 0x7f2bbc1589e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:14.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:14.604+0000 7f2bc2800640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f2bbc0c6470 0x7f2bbc158f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54590/0 (socket says 172.21.15.1:54590) 2024-05-09T08:03:14.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:14.604+0000 7f2bc2800640 1 -- 172.21.15.1:0/3035335802 learned_addr learned my addr 172.21.15.1:0/3035335802 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:03:14.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:14.604+0000 7f2bc1e00640 1 --1- 172.21.15.1:0/3035335802 >> v1:172.21.15.7:6789/0 conn(0x7f2bbc0c54f0 0x7f2bbc06f930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.1:38756/0 (socket says 172.21.15.1:38756) 2024-05-09T08:03:14.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:14.604+0000 7f2bc2800640 1 -- 172.21.15.1:0/3035335802 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f2bbc0c58d0 msgr2=0x7f2bbc1589e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:03:14.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:14.604+0000 7f2bc2800640 1 --2- 172.21.15.1:0/3035335802 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f2bbc0c58d0 0x7f2bbc1589e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:14.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:14.604+0000 7f2bc2800640 1 -- 172.21.15.1:0/3035335802 >> v1:172.21.15.7:6789/0 conn(0x7f2bbc0c54f0 legacy=0x7f2bbc06f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:14.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:14.604+0000 7f2bc2800640 1 -- 172.21.15.1:0/3035335802 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bbc071b60 con 0x7f2bbc0c6470 2024-05-09T08:03:14.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:14.605+0000 7f2bc2800640 1 --2- 172.21.15.1:0/3035335802 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2bbc0c6470 0x7f2bbc158f20 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f2bb8009e50 tx=0x7f2bb802a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=137fc1f3832e008a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:03:14.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:14.606+0000 7f2bc0a00640 1 -- 172.21.15.1:0/3035335802 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bb8036070 con 0x7f2bbc0c6470 2024-05-09T08:03:14.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:14.606+0000 7f2bc0a00640 1 -- 172.21.15.1:0/3035335802 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2bb8031600 con 0x7f2bbc0c6470 2024-05-09T08:03:14.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:14.606+0000 7f2bc0a00640 1 -- 172.21.15.1:0/3035335802 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bb80319d0 con 0x7f2bbc0c6470 2024-05-09T08:03:14.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:14.606+0000 7f2bc3200640 1 -- 172.21.15.1:0/3035335802 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2bbc0c6470 msgr2=0x7f2bbc158f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:14.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:14.606+0000 7f2bc3200640 1 --2- 172.21.15.1:0/3035335802 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2bbc0c6470 0x7f2bbc158f20 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f2bb8009e50 tx=0x7f2bb802a680 comp rx=0 tx=0).stop 2024-05-09T08:03:14.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:14.606+0000 7f2bc3200640 1 -- 172.21.15.1:0/3035335802 shutdown_connections 2024-05-09T08:03:14.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:14.607+0000 7f2bc3200640 1 --2- 172.21.15.1:0/3035335802 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2bbc0c6470 0x7f2bbc158f20 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:14.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:14.607+0000 7f2bc3200640 1 --2- 172.21.15.1:0/3035335802 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f2bbc0c58d0 0x7f2bbc1589e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:14.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:14.607+0000 7f2bc3200640 1 -- 172.21.15.1:0/3035335802 >> 172.21.15.1:0/3035335802 conn(0x7f2bbc06b250 msgr2=0x7f2bbc06b630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:03:14.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:14.607+0000 7f2bc3200640 1 -- 172.21.15.1:0/3035335802 shutdown_connections 2024-05-09T08:03:14.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:14.607+0000 7f2bc3200640 1 -- 172.21.15.1:0/3035335802 wait complete. 2024-05-09T08:03:14.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:14.608+0000 7f2bc3200640 1 Processor -- start 2024-05-09T08:03:14.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:14.609+0000 7f2bc3200640 1 -- start start 2024-05-09T08:03:14.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:14.609+0000 7f2bc3200640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2bbc0c54f0 0x7f2bbc0cbdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:14.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:14.610+0000 7f2bc3200640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f2bbc0c58d0 0x7f2bbc0c3350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:14.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:14.610+0000 7f2bc3200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2bbc0c6470 0x7f2bbc0c3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:14.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:14.610+0000 7f2bc3200640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f2bbc072400 con 0x7f2bbc0c54f0 2024-05-09T08:03:14.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:14.610+0000 7f2bc3200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2bbc072120 con 0x7f2bbc0c6470 2024-05-09T08:03:14.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:14.610+0000 7f2bc3200640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f2bbc1584c0 con 0x7f2bbc0c58d0 2024-05-09T08:03:14.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:14.610+0000 7f2bc1e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2bbc0c54f0 0x7f2bbc0cbdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:14.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:14.610+0000 7f2bc1e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2bbc0c54f0 0x7f2bbc0cbdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54602/0 (socket says 172.21.15.1:54602) 2024-05-09T08:03:14.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:14.610+0000 7f2bc1e00640 1 -- 172.21.15.1:0/2949598041 learned_addr learned my addr 172.21.15.1:0/2949598041 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:03:14.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:14.610+0000 7f2bc1400640 1 --2- 172.21.15.1:0/2949598041 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f2bbc0c58d0 0x7f2bbc0c3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:14.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:14.610+0000 7f2bc2800640 1 --2- 172.21.15.1:0/2949598041 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2bbc0c6470 0x7f2bbc0c3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:14.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:14.610+0000 7f2bc1e00640 1 -- 172.21.15.1:0/2949598041 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f2bbc0c58d0 msgr2=0x7f2bbc0c3350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:14.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:14.610+0000 7f2bc1e00640 1 --2- 172.21.15.1:0/2949598041 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f2bbc0c58d0 0x7f2bbc0c3350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:14.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:14.610+0000 7f2bc1e00640 1 -- 172.21.15.1:0/2949598041 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2bbc0c6470 msgr2=0x7f2bbc0c3890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:14.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:14.610+0000 7f2bc1e00640 1 --2- 172.21.15.1:0/2949598041 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2bbc0c6470 0x7f2bbc0c3890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:14.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:14.610+0000 7f2bc1e00640 1 -- 172.21.15.1:0/2949598041 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bbc150f20 con 0x7f2bbc0c54f0 2024-05-09T08:03:14.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:14.611+0000 7f2bc1e00640 1 --2- 172.21.15.1:0/2949598041 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2bbc0c54f0 0x7f2bbc0cbdb0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f2bb000c400 tx=0x7f2bb000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:03:14.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:14.611+0000 7f2bb6000640 1 -- 172.21.15.1:0/2949598041 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bb00074e0 con 0x7f2bbc0c54f0 2024-05-09T08:03:14.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:14.611+0000 7f2bb6000640 1 -- 172.21.15.1:0/2949598041 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2bb0007680 con 0x7f2bbc0c54f0 2024-05-09T08:03:14.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:14.611+0000 7f2bc3200640 1 -- 172.21.15.1:0/2949598041 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bbc1e30b0 con 0x7f2bbc0c54f0 2024-05-09T08:03:14.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:14.611+0000 7f2bb6000640 1 -- 172.21.15.1:0/2949598041 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bb0004a00 con 0x7f2bbc0c54f0 2024-05-09T08:03:14.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:14.611+0000 7f2bc3200640 1 -- 172.21.15.1:0/2949598041 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bbc1e3350 con 0x7f2bbc0c54f0 2024-05-09T08:03:14.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:14.612+0000 7f2bc3200640 1 -- 172.21.15.1:0/2949598041 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2bbc04f660 con 0x7f2bbc0c54f0 2024-05-09T08:03:14.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:14.613+0000 7f2bb6000640 1 -- 172.21.15.1:0/2949598041 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2bb001a020 con 0x7f2bbc0c54f0 2024-05-09T08:03:14.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:14.615+0000 7f2bb6000640 1 --2- 172.21.15.1:0/2949598041 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f2ba006be30 0x7f2ba006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:14.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:14.615+0000 7f2bb6000640 1 -- 172.21.15.1:0/2949598041 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7f2bb008aa90 con 0x7f2bbc0c54f0 2024-05-09T08:03:14.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:14.615+0000 7f2bc1400640 1 --2- 172.21.15.1:0/2949598041 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f2ba006be30 0x7f2ba006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:14.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:14.616+0000 7f2bc1400640 1 --2- 172.21.15.1:0/2949598041 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f2ba006be30 0x7f2ba006e2f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f2bbc155e70 tx=0x7f2bac009190 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:03:14.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:14.619+0000 7f2bb6000640 1 -- 172.21.15.1:0/2949598041 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f2bbc0718e0 con 0x7f2bbc0c54f0 2024-05-09T08:03:14.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:14.823+0000 7f2bc3200640 1 -- 172.21.15.1:0/2949598041 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f2bbc0bf850 con 0x7f2bbc0c54f0 2024-05-09T08:03:14.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:14.824+0000 7f2bb6000640 1 -- 172.21.15.1:0/2949598041 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) ==== 100+0+11 (secure 0 0 0) 0x7f2bb00588d0 con 0x7f2bbc0c54f0 2024-05-09T08:03:14.841 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 64 2024-05-09T08:03:14.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:14.827+0000 7f2bc3200640 1 -- 172.21.15.1:0/2949598041 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f2ba006be30 msgr2=0x7f2ba006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:14.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:14.827+0000 7f2bc3200640 1 --2- 172.21.15.1:0/2949598041 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f2ba006be30 0x7f2ba006e2f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f2bbc155e70 tx=0x7f2bac009190 comp rx=0 tx=0).stop 2024-05-09T08:03:14.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:14.827+0000 7f2bc3200640 1 -- 172.21.15.1:0/2949598041 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2bbc0c54f0 msgr2=0x7f2bbc0cbdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:14.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:14.827+0000 7f2bc3200640 1 --2- 172.21.15.1:0/2949598041 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2bbc0c54f0 0x7f2bbc0cbdb0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f2bb000c400 tx=0x7f2bb000c8c0 comp rx=0 tx=0).stop 2024-05-09T08:03:14.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:14.827+0000 7f2bc3200640 1 -- 172.21.15.1:0/2949598041 shutdown_connections 2024-05-09T08:03:14.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:14.827+0000 7f2bc3200640 1 --2- 172.21.15.1:0/2949598041 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f2ba006be30 0x7f2ba006e2f0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:14.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:14.827+0000 7f2bc3200640 1 --2- 172.21.15.1:0/2949598041 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2bbc0c6470 0x7f2bbc0c3890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:14.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:14.827+0000 7f2bc3200640 1 --2- 172.21.15.1:0/2949598041 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f2bbc0c58d0 0x7f2bbc0c3350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:14.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:14.827+0000 7f2bc3200640 1 --2- 172.21.15.1:0/2949598041 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2bbc0c54f0 0x7f2bbc0cbdb0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:14.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:14.827+0000 7f2bc3200640 1 -- 172.21.15.1:0/2949598041 >> 172.21.15.1:0/2949598041 conn(0x7f2bbc06b250 msgr2=0x7f2bbc158830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:03:14.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:14.827+0000 7f2bc3200640 1 -- 172.21.15.1:0/2949598041 shutdown_connections 2024-05-09T08:03:14.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:14.827+0000 7f2bc3200640 1 -- 172.21.15.1:0/2949598041 wait complete. 2024-05-09T08:03:14.862 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-09T08:03:15.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:15.063+0000 7fb685a00640 1 Processor -- start 2024-05-09T08:03:15.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:15.064+0000 7fb685a00640 1 -- start start 2024-05-09T08:03:15.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:15.065+0000 7fb685a00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb680154480 0x7fb68006b250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:15.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:15.065+0000 7fb685a00640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb6800c8af0 con 0x7fb6800c8680 2024-05-09T08:03:15.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:15.066+0000 7fb685a00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb6800c7300 con 0x7fb68006f4c0 2024-05-09T08:03:15.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:15.066+0000 7fb685a00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fb6800c75e0 con 0x7fb680154480 2024-05-09T08:03:15.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:15.066+0000 7fb67f400640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7fb6800c8680 0x7fb68015aed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:36888/0 (socket says 172.21.15.1:36888) 2024-05-09T08:03:15.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:15.066+0000 7fb67f400640 1 -- 172.21.15.1:0/2490570843 learned_addr learned my addr 172.21.15.1:0/2490570843 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T08:03:15.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:15.066+0000 7fb67fe00640 1 --2- 172.21.15.1:0/2490570843 >> v2:172.21.15.7:3300/0 conn(0x7fb680154480 0x7fb68006b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:15.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:15.066+0000 7fb67fe00640 1 -- 172.21.15.1:0/2490570843 >> v1:172.21.15.7:6789/0 conn(0x7fb68006f4c0 legacy=0x7fb68006f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:15.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:15.066+0000 7fb67fe00640 1 -- 172.21.15.1:0/2490570843 >> v1:172.21.15.1:6789/0 conn(0x7fb6800c8680 legacy=0x7fb68015aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:15.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:15.066+0000 7fb67fe00640 1 -- 172.21.15.1:0/2490570843 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6800c78c0 con 0x7fb680154480 2024-05-09T08:03:15.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:15.066+0000 7fb67ea00640 1 -- 172.21.15.1:0/2490570843 <== mon.1 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2916793457 0 0) 0x7fb6800c7300 con 0x7fb68006f4c0 2024-05-09T08:03:15.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:15.067+0000 7fb67fe00640 1 --2- 172.21.15.1:0/2490570843 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb680154480 0x7fb68006b250 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fb670009d90 tx=0x7fb67002a780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=36a246af5514f892 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:03:15.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:15.068+0000 7fb67ea00640 1 -- 172.21.15.1:0/2490570843 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb670037070 con 0x7fb680154480 2024-05-09T08:03:15.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:15.068+0000 7fb67ea00640 1 -- 172.21.15.1:0/2490570843 <== mon.1 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb670007740 con 0x7fb680154480 2024-05-09T08:03:15.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:15.068+0000 7fb67ea00640 1 -- 172.21.15.1:0/2490570843 <== mon.1 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb670007b10 con 0x7fb680154480 2024-05-09T08:03:15.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:15.069+0000 7fb685a00640 1 -- 172.21.15.1:0/2490570843 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb680154480 msgr2=0x7fb68006b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:15.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:15.069+0000 7fb685a00640 1 --2- 172.21.15.1:0/2490570843 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb680154480 0x7fb68006b250 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fb670009d90 tx=0x7fb67002a780 comp rx=0 tx=0).stop 2024-05-09T08:03:15.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:15.069+0000 7fb685a00640 1 -- 172.21.15.1:0/2490570843 shutdown_connections 2024-05-09T08:03:15.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:15.069+0000 7fb685a00640 1 --2- 172.21.15.1:0/2490570843 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb680154480 0x7fb68006b250 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:15.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:15.069+0000 7fb685a00640 1 -- 172.21.15.1:0/2490570843 >> 172.21.15.1:0/2490570843 conn(0x7fb6800c8e30 msgr2=0x7fb6800c9210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:03:15.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:15.069+0000 7fb685a00640 1 -- 172.21.15.1:0/2490570843 shutdown_connections 2024-05-09T08:03:15.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:15.069+0000 7fb685a00640 1 -- 172.21.15.1:0/2490570843 wait complete. 2024-05-09T08:03:15.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:15.072+0000 7fb685a00640 1 Processor -- start 2024-05-09T08:03:15.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:15.072+0000 7fb685a00640 1 -- start start 2024-05-09T08:03:15.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:15.073+0000 7fb685a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb68006f4c0 0x7fb6801f4920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:15.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:15.073+0000 7fb685a00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb6800c8680 0x7fb680156230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:15.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:15.073+0000 7fb685a00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fb680154480 0x7fb68014d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:15.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:15.073+0000 7fb685a00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fb6800c8160 con 0x7fb6800c8680 2024-05-09T08:03:15.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:15.073+0000 7fb685a00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb6800c7e80 con 0x7fb68006f4c0 2024-05-09T08:03:15.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:15.073+0000 7fb685a00640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7fb680154860 con 0x7fb680154480 2024-05-09T08:03:15.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:15.073+0000 7fb67f400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb6800c8680 0x7fb680156230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:15.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:15.073+0000 7fb67fe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb68006f4c0 0x7fb6801f4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:15.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:15.073+0000 7fb67fe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb68006f4c0 0x7fb6801f4920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.1:43814/0 (socket says 172.21.15.1:43814) 2024-05-09T08:03:15.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:15.073+0000 7fb67fe00640 1 -- 172.21.15.1:0/2640820638 learned_addr learned my addr 172.21.15.1:0/2640820638 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:03:15.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:15.074+0000 7fb685000640 1 --2- 172.21.15.1:0/2640820638 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fb680154480 0x7fb68014d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:15.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:15.074+0000 7fb67fe00640 1 -- 172.21.15.1:0/2640820638 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fb680154480 msgr2=0x7fb68014d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:15.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:15.074+0000 7fb67fe00640 1 --2- 172.21.15.1:0/2640820638 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fb680154480 0x7fb68014d1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:15.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:15.074+0000 7fb67fe00640 1 -- 172.21.15.1:0/2640820638 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb6800c8680 msgr2=0x7fb680156230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:15.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:15.074+0000 7fb67fe00640 1 --2- 172.21.15.1:0/2640820638 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb6800c8680 0x7fb680156230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:15.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:15.074+0000 7fb67fe00640 1 -- 172.21.15.1:0/2640820638 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb680156770 con 0x7fb68006f4c0 2024-05-09T08:03:15.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:15.074+0000 7fb67fe00640 1 --2- 172.21.15.1:0/2640820638 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb68006f4c0 0x7fb6801f4920 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fb670009d90 tx=0x7fb670004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:03:15.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:15.074+0000 7fb677e00640 1 -- 172.21.15.1:0/2640820638 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb670042020 con 0x7fb68006f4c0 2024-05-09T08:03:15.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:15.075+0000 7fb677e00640 1 -- 172.21.15.1:0/2640820638 <== mon.1 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb670032a70 con 0x7fb68006f4c0 2024-05-09T08:03:15.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:15.075+0000 7fb677e00640 1 -- 172.21.15.1:0/2640820638 <== mon.1 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb670037040 con 0x7fb68006f4c0 2024-05-09T08:03:15.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:15.075+0000 7fb685a00640 1 -- 172.21.15.1:0/2640820638 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb680146b20 con 0x7fb68006f4c0 2024-05-09T08:03:15.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:15.075+0000 7fb685a00640 1 -- 172.21.15.1:0/2640820638 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb680147430 con 0x7fb68006f4c0 2024-05-09T08:03:15.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:15.076+0000 7fb685a00640 1 -- 172.21.15.1:0/2640820638 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb68004f660 con 0x7fb68006f4c0 2024-05-09T08:03:15.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:15.077+0000 7fb677e00640 1 -- 172.21.15.1:0/2640820638 <== mon.1 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb670045020 con 0x7fb68006f4c0 2024-05-09T08:03:15.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:15.078+0000 7fb677e00640 1 --2- 172.21.15.1:0/2640820638 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fb65c06be60 0x7fb65c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:15.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:15.078+0000 7fb677e00640 1 -- 172.21.15.1:0/2640820638 <== mon.1 v2:172.21.15.7:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7fb6700ac490 con 0x7fb68006f4c0 2024-05-09T08:03:15.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:15.079+0000 7fb67f400640 1 --2- 172.21.15.1:0/2640820638 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fb65c06be60 0x7fb65c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:15.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:15.079+0000 7fb67f400640 1 --2- 172.21.15.1:0/2640820638 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fb65c06be60 0x7fb65c06e320 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fb680146ab0 tx=0x7fb668009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:03:15.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:15.083+0000 7fb677e00640 1 -- 172.21.15.1:0/2640820638 <== mon.1 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fb67004c030 con 0x7fb68006f4c0 2024-05-09T08:03:15.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:15.289+0000 7fb685a00640 1 -- 172.21.15.1:0/2640820638 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fb6800c6780 con 0x7fb68006f4c0 2024-05-09T08:03:15.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:15.290+0000 7fb677e00640 1 -- 172.21.15.1:0/2640820638 <== mon.1 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) ==== 103+0+11 (secure 0 0 0) 0x7fb67007a2d0 con 0x7fb68006f4c0 2024-05-09T08:03:15.307 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 32 2024-05-09T08:03:15.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:15.293+0000 7fb685a00640 1 -- 172.21.15.1:0/2640820638 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fb65c06be60 msgr2=0x7fb65c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:15.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:15.293+0000 7fb685a00640 1 --2- 172.21.15.1:0/2640820638 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fb65c06be60 0x7fb65c06e320 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fb680146ab0 tx=0x7fb668009040 comp rx=0 tx=0).stop 2024-05-09T08:03:15.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:15.293+0000 7fb685a00640 1 -- 172.21.15.1:0/2640820638 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb68006f4c0 msgr2=0x7fb6801f4920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:15.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:15.293+0000 7fb685a00640 1 --2- 172.21.15.1:0/2640820638 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb68006f4c0 0x7fb6801f4920 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fb670009d90 tx=0x7fb670004a40 comp rx=0 tx=0).stop 2024-05-09T08:03:15.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:15.293+0000 7fb685a00640 1 -- 172.21.15.1:0/2640820638 shutdown_connections 2024-05-09T08:03:15.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:15.293+0000 7fb685a00640 1 --2- 172.21.15.1:0/2640820638 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fb65c06be60 0x7fb65c06e320 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:15.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:15.293+0000 7fb685a00640 1 --2- 172.21.15.1:0/2640820638 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fb680154480 0x7fb68014d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:15.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:15.293+0000 7fb685a00640 1 --2- 172.21.15.1:0/2640820638 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb6800c8680 0x7fb680156230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:15.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:15.293+0000 7fb685a00640 1 --2- 172.21.15.1:0/2640820638 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb68006f4c0 0x7fb6801f4920 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:15.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:15.293+0000 7fb685a00640 1 -- 172.21.15.1:0/2640820638 >> 172.21.15.1:0/2640820638 conn(0x7fb6800c8e30 msgr2=0x7fb68014e690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:03:15.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:15.293+0000 7fb685a00640 1 -- 172.21.15.1:0/2640820638 shutdown_connections 2024-05-09T08:03:15.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:15.293+0000 7fb685a00640 1 -- 172.21.15.1:0/2640820638 wait complete. 2024-05-09T08:03:15.331 DEBUG:tasks.mon_thrash:Yielding 2024-05-09T08:03:15.331 INFO:teuthology.run_tasks:Running task workunit... 2024-05-09T08:03:15.345 INFO:tasks.workunit:Pulling workunits from ref be85c202969d25af6e130678e7d73a20e9c4ca11 2024-05-09T08:03:15.345 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-05-09T08:03:15.345 INFO:tasks.workunit:timeout=3h 2024-05-09T08:03:15.345 INFO:tasks.workunit:cleanup=True 2024-05-09T08:03:15.346 DEBUG:teuthology.orchestra.run.smithi001:> stat -- /home/ubuntu/cephtest/mnt.0 2024-05-09T08:03:15.346 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T08:03:15.397 INFO:teuthology.orchestra.run.smithi001.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-05-09T08:03:15.398 INFO:teuthology.orchestra.run.smithi001.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-05-09T08:03:15.398 INFO:teuthology.orchestra.run.smithi001.stdout:Device: 28h/40d Inode: 1 Links: 2 2024-05-09T08:03:15.398 INFO:teuthology.orchestra.run.smithi001.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T08:03:15.398 INFO:teuthology.orchestra.run.smithi001.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-05-09T08:03:15.398 INFO:teuthology.orchestra.run.smithi001.stdout:Access: 1970-01-01 00:00:00.000000000 +0000 2024-05-09T08:03:15.398 INFO:teuthology.orchestra.run.smithi001.stdout:Modify: 2024-05-09 08:02:20.900015500 +0000 2024-05-09T08:03:15.398 INFO:teuthology.orchestra.run.smithi001.stdout:Change: 2024-05-09 08:02:34.117820752 +0000 2024-05-09T08:03:15.398 INFO:teuthology.orchestra.run.smithi001.stdout: Birth: 2024-05-09 08:02:20.900015500 +0000 2024-05-09T08:03:15.398 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-05-09T08:03:15.398 DEBUG:teuthology.orchestra.run.smithi001:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-05-09T08:03:15.517 DEBUG:teuthology.orchestra.run.smithi001:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout be85c202969d25af6e130678e7d73a20e9c4ca11 2024-05-09T08:03:15.547 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:15.530+0000 7f2af5a00640 1 Processor -- start 2024-05-09T08:03:15.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:15.530+0000 7f2af5a00640 1 -- start start 2024-05-09T08:03:15.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:15.532+0000 7f2af5a00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f2af006e630 0x7f2af0157660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:15.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:15.532+0000 7f2af5a00640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f2af0153ce0 0x7f2af0154190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:15.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:15.532+0000 7f2af5a00640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2af00c2b20 con 0x7f2af006ea10 2024-05-09T08:03:15.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:15.532+0000 7f2af5a00640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f2af0071c80 con 0x7f2af0153ce0 2024-05-09T08:03:15.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:15.532+0000 7f2af5a00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f2af0072240 con 0x7f2af006e630 2024-05-09T08:03:15.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:15.532+0000 7f2aeea00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f2af006e630 0x7f2af0157660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:15.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:15.532+0000 7f2ae6000640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f2af006ea10 0x7f2af00c1090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:36894/0 (socket says 172.21.15.1:36894) 2024-05-09T08:03:15.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:15.532+0000 7f2ae6000640 1 -- 172.21.15.1:0/2059439922 learned_addr learned my addr 172.21.15.1:0/2059439922 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T08:03:15.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:15.532+0000 7f2aef400640 1 --2- 172.21.15.1:0/2059439922 >> v2:172.21.15.1:3300/0 conn(0x7f2af0153ce0 0x7f2af0154190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:15.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:15.533+0000 7f2aeea00640 1 -- 172.21.15.1:0/2059439922 >> v2:172.21.15.1:3300/0 conn(0x7f2af0153ce0 msgr2=0x7f2af0154190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:03:15.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:15.533+0000 7f2aeea00640 1 --2- 172.21.15.1:0/2059439922 >> v2:172.21.15.1:3300/0 conn(0x7f2af0153ce0 0x7f2af0154190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:15.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:15.533+0000 7f2aeea00640 1 -- 172.21.15.1:0/2059439922 >> v1:172.21.15.1:6789/0 conn(0x7f2af006ea10 legacy=0x7f2af00c1090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:15.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:15.533+0000 7f2aeea00640 1 -- 172.21.15.1:0/2059439922 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2af00c2e00 con 0x7f2af006e630 2024-05-09T08:03:15.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:15.533+0000 7f2aee000640 1 -- 172.21.15.1:0/2059439922 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2448923695 0 0) 0x7f2af00c2b20 con 0x7f2af006ea10 2024-05-09T08:03:15.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:15.533+0000 7f2aef400640 1 --2- 172.21.15.1:0/2059439922 >> v2:172.21.15.1:3300/0 conn(0x7f2af0153ce0 0x7f2af0154190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:03:15.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:15.534+0000 7f2aeea00640 1 --2- 172.21.15.1:0/2059439922 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2af006e630 0x7f2af0157660 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f2af00c6730 tx=0x7f2ae002a250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=15451fb2053396ec server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:03:15.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:15.535+0000 7f2aee000640 1 -- 172.21.15.1:0/2059439922 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ae0037070 con 0x7f2af006e630 2024-05-09T08:03:15.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:15.535+0000 7f2aee000640 1 -- 172.21.15.1:0/2059439922 <== mon.1 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ae0032440 con 0x7f2af006e630 2024-05-09T08:03:15.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:15.535+0000 7f2aee000640 1 -- 172.21.15.1:0/2059439922 <== mon.1 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ae0032810 con 0x7f2af006e630 2024-05-09T08:03:15.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:15.535+0000 7f2af5a00640 1 -- 172.21.15.1:0/2059439922 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2af006e630 msgr2=0x7f2af0157660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:15.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:15.535+0000 7f2af5a00640 1 --2- 172.21.15.1:0/2059439922 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2af006e630 0x7f2af0157660 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f2af00c6730 tx=0x7f2ae002a250 comp rx=0 tx=0).stop 2024-05-09T08:03:15.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:15.536+0000 7f2af5a00640 1 -- 172.21.15.1:0/2059439922 shutdown_connections 2024-05-09T08:03:15.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:15.536+0000 7f2af5a00640 1 --2- 172.21.15.1:0/2059439922 >> v2:172.21.15.1:3300/0 conn(0x7f2af0153ce0 0x7f2af0154190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:15.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:15.536+0000 7f2af5a00640 1 --2- 172.21.15.1:0/2059439922 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2af006e630 0x7f2af0157660 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:15.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:15.536+0000 7f2af5a00640 1 -- 172.21.15.1:0/2059439922 >> 172.21.15.1:0/2059439922 conn(0x7f2af00cc070 msgr2=0x7f2af00cc480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:03:15.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:15.536+0000 7f2af5a00640 1 -- 172.21.15.1:0/2059439922 shutdown_connections 2024-05-09T08:03:15.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:15.536+0000 7f2af5a00640 1 -- 172.21.15.1:0/2059439922 wait complete. 2024-05-09T08:03:15.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:15.537+0000 7f2af5a00640 1 Processor -- start 2024-05-09T08:03:15.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:15.538+0000 7f2af5a00640 1 -- start start 2024-05-09T08:03:15.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:15.539+0000 7f2af5a00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2af006e630 0x7f2af00c3f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:15.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:15.539+0000 7f2af5a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2af006ea10 0x7f2af0072940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:15.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:15.539+0000 7f2af5a00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f2af0153ce0 0x7f2af0156410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:15.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:15.539+0000 7f2af5a00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f2af00ccce0 con 0x7f2af006e630 2024-05-09T08:03:15.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:15.539+0000 7f2af5a00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2af0071ed0 con 0x7f2af006ea10 2024-05-09T08:03:15.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:15.539+0000 7f2af5a00640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f2af00cce60 con 0x7f2af0153ce0 2024-05-09T08:03:15.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:15.539+0000 7f2aeea00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2af006e630 0x7f2af00c3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:15.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:15.539+0000 7f2aeea00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2af006e630 0x7f2af00c3f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54626/0 (socket says 172.21.15.1:54626) 2024-05-09T08:03:15.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:15.539+0000 7f2aeea00640 1 -- 172.21.15.1:0/2640954914 learned_addr learned my addr 172.21.15.1:0/2640954914 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:03:15.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:15.539+0000 7f2ae6000640 1 --2- 172.21.15.1:0/2640954914 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2af006ea10 0x7f2af0072940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:15.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:15.539+0000 7f2aef400640 1 --2- 172.21.15.1:0/2640954914 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f2af0153ce0 0x7f2af0156410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:15.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:15.539+0000 7f2aeea00640 1 -- 172.21.15.1:0/2640954914 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f2af0153ce0 msgr2=0x7f2af0156410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:15.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:15.539+0000 7f2aeea00640 1 --2- 172.21.15.1:0/2640954914 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f2af0153ce0 0x7f2af0156410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:15.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:15.539+0000 7f2aeea00640 1 -- 172.21.15.1:0/2640954914 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2af006ea10 msgr2=0x7f2af0072940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:15.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:15.539+0000 7f2aeea00640 1 --2- 172.21.15.1:0/2640954914 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2af006ea10 0x7f2af0072940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:15.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:15.539+0000 7f2aeea00640 1 -- 172.21.15.1:0/2640954914 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2af00c4950 con 0x7f2af006e630 2024-05-09T08:03:15.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:15.540+0000 7f2aeea00640 1 --2- 172.21.15.1:0/2640954914 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2af006e630 0x7f2af00c3f00 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f2ae0000c00 tx=0x7f2ae00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:03:15.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:15.540+0000 7f2ae7400640 1 -- 172.21.15.1:0/2640954914 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ae0041070 con 0x7f2af006e630 2024-05-09T08:03:15.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:15.540+0000 7f2ae7400640 1 -- 172.21.15.1:0/2640954914 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ae003c890 con 0x7f2af006e630 2024-05-09T08:03:15.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:15.540+0000 7f2ae7400640 1 -- 172.21.15.1:0/2640954914 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ae0037040 con 0x7f2af006e630 2024-05-09T08:03:15.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:15.540+0000 7f2af5a00640 1 -- 172.21.15.1:0/2640954914 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2af00c4c10 con 0x7f2af006e630 2024-05-09T08:03:15.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:15.540+0000 7f2af5a00640 1 -- 172.21.15.1:0/2640954914 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2af00c51d0 con 0x7f2af006e630 2024-05-09T08:03:15.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:15.542+0000 7f2af5a00640 1 -- 172.21.15.1:0/2640954914 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2af0074d70 con 0x7f2af006e630 2024-05-09T08:03:15.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:15.543+0000 7f2ae7400640 1 -- 172.21.15.1:0/2640954914 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2ae0044070 con 0x7f2af006e630 2024-05-09T08:03:15.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:15.544+0000 7f2ae7400640 1 --2- 172.21.15.1:0/2640954914 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f2acc06be80 0x7f2acc06e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:15.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:15.544+0000 7f2ae7400640 1 -- 172.21.15.1:0/2640954914 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7f2ae00abb70 con 0x7f2af006e630 2024-05-09T08:03:15.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:15.544+0000 7f2ae6000640 1 --2- 172.21.15.1:0/2640954914 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f2acc06be80 0x7f2acc06e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:15.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:15.545+0000 7f2ae6000640 1 --2- 172.21.15.1:0/2640954914 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f2acc06be80 0x7f2acc06e340 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f2af0163550 tx=0x7f2ad8007d50 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:03:15.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:15.548+0000 7f2ae7400640 1 -- 172.21.15.1:0/2640954914 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f2ae0079a30 con 0x7f2af006e630 2024-05-09T08:03:15.567 INFO:tasks.workunit.client.0.smithi001.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-05-09T08:03:15.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:15.790+0000 7f2af5a00640 1 -- 172.21.15.1:0/2640954914 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2af00cbe50 con 0x7f2af006e630 2024-05-09T08:03:15.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:15.791+0000 7f2ae7400640 1 -- 172.21.15.1:0/2640954914 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+6807 (secure 0 0 0) 0x7f2ae007dd30 con 0x7f2af006e630 2024-05-09T08:03:15.808 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T08:03:15.808 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":10,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4220,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/1157980939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":1157980939},{"type":"v1","addr":"172.21.15.7:6837","nonce":1157980939}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4235,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6833/3269798795","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6832","nonce":3269798795},{"type":"v1","addr":"172.21.15.1:6833","nonce":3269798795}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4241,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/725115499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6835","nonce":725115499},{"type":"v1","addr":"172.21.15.1:6837","nonce":725115499}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4276,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6838/1827115274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1827115274},{"type":"v1","addr":"172.21.15.7:6838","nonce":1827115274}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/1032767352","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":1032767352},{"type":"v1","addr":"172.21.15.7:6839","nonce":1032767352}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":10,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-09T08:02:15.332696+0000","modified":"2024-05-09T08:02:49.734561+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4294},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4294":{"gid":4294,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":6,"addr":"172.21.15.1:6836/1509134635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":1509134635},{"type":"v1","addr":"172.21.15.1:6836","nonce":1509134635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4294,"qdb_cluster":[4294]},"id":1}]} 2024-05-09T08:03:15.808 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 10 2024-05-09T08:03:15.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:15.794+0000 7f2af5a00640 1 -- 172.21.15.1:0/2640954914 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f2acc06be80 msgr2=0x7f2acc06e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:15.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:15.794+0000 7f2af5a00640 1 --2- 172.21.15.1:0/2640954914 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f2acc06be80 0x7f2acc06e340 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f2af0163550 tx=0x7f2ad8007d50 comp rx=0 tx=0).stop 2024-05-09T08:03:15.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:15.794+0000 7f2af5a00640 1 -- 172.21.15.1:0/2640954914 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2af006e630 msgr2=0x7f2af00c3f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:15.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:15.794+0000 7f2af5a00640 1 --2- 172.21.15.1:0/2640954914 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2af006e630 0x7f2af00c3f00 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f2ae0000c00 tx=0x7f2ae00046c0 comp rx=0 tx=0).stop 2024-05-09T08:03:15.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:15.794+0000 7f2af5a00640 1 -- 172.21.15.1:0/2640954914 shutdown_connections 2024-05-09T08:03:15.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:15.794+0000 7f2af5a00640 1 --2- 172.21.15.1:0/2640954914 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f2acc06be80 0x7f2acc06e340 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:15.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:15.794+0000 7f2af5a00640 1 --2- 172.21.15.1:0/2640954914 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2af006ea10 0x7f2af0072940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:15.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:15.794+0000 7f2af5a00640 1 --2- 172.21.15.1:0/2640954914 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f2af0153ce0 0x7f2af0156410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:15.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:15.794+0000 7f2af5a00640 1 --2- 172.21.15.1:0/2640954914 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2af006e630 0x7f2af00c3f00 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:15.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:15.794+0000 7f2af5a00640 1 -- 172.21.15.1:0/2640954914 >> 172.21.15.1:0/2640954914 conn(0x7f2af00cc070 msgr2=0x7f2af00b0ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:03:15.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:15.794+0000 7f2af5a00640 1 -- 172.21.15.1:0/2640954914 shutdown_connections 2024-05-09T08:03:15.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:15.794+0000 7f2af5a00640 1 -- 172.21.15.1:0/2640954914 wait complete. 2024-05-09T08:03:15.830 INFO:tasks.mon_thrash.mon_thrasher:start thrashing 2024-05-09T08:03:15.831 INFO:tasks.mon_thrash.mon_thrasher:seed: 1715241792, revive delay: 20.0, thrash delay: 10.0 thrash many: False, maintain quorum: True store thrash: False, probability: 50 freeze mon: prob 10.0 duration 15.0 2024-05-09T08:03:15.831 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-05-09T08:03:15.831 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-05-09T08:03:16.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.082+0000 7f9229800640 1 Processor -- start 2024-05-09T08:03:16.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.083+0000 7f9229800640 1 -- start start 2024-05-09T08:03:16.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.084+0000 7f9229800640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f9224152500 0x7f92240c8180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:16.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.084+0000 7f9229800640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f92240cc730 0x7f922415c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:16.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.084+0000 7f9229800640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f92240c6a30 con 0x7f92240c7660 2024-05-09T08:03:16.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.084+0000 7f9229800640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f92240c5e50 con 0x7f92240cc730 2024-05-09T08:03:16.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.084+0000 7f9229800640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f92240c22d0 con 0x7f9224152500 2024-05-09T08:03:16.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.084+0000 7f921be00640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f92240cc730 0x7f922415c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:16.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.085+0000 7f921be00640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f92240cc730 0x7f922415c080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54642/0 (socket says 172.21.15.1:54642) 2024-05-09T08:03:16.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.085+0000 7f9223e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f9224152500 0x7f92240c8180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:16.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.085+0000 7f921be00640 1 -- 172.21.15.1:0/1154411637 learned_addr learned my addr 172.21.15.1:0/1154411637 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:03:16.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.085+0000 7f9228e00640 1 --1- 172.21.15.1:0/1154411637 >> v1:172.21.15.7:6789/0 conn(0x7f92240c7660 0x7f922406b250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.1:38778/0 (socket says 172.21.15.1:38778) 2024-05-09T08:03:16.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.085+0000 7f921be00640 1 -- 172.21.15.1:0/1154411637 >> v2:172.21.15.7:3300/0 conn(0x7f9224152500 msgr2=0x7f92240c8180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:03:16.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.085+0000 7f921be00640 1 --2- 172.21.15.1:0/1154411637 >> v2:172.21.15.7:3300/0 conn(0x7f9224152500 0x7f92240c8180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:16.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.085+0000 7f921be00640 1 -- 172.21.15.1:0/1154411637 >> v1:172.21.15.7:6789/0 conn(0x7f92240c7660 legacy=0x7f922406b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:16.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.085+0000 7f921be00640 1 -- 172.21.15.1:0/1154411637 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92240c6cf0 con 0x7f92240cc730 2024-05-09T08:03:16.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.086+0000 7f921be00640 1 --2- 172.21.15.1:0/1154411637 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f92240cc730 0x7f922415c080 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f922406eb20 tx=0x7f920c02a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9143aa5d1dbf366e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:03:16.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.087+0000 7f9223400640 1 -- 172.21.15.1:0/1154411637 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f920c037070 con 0x7f92240cc730 2024-05-09T08:03:16.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.087+0000 7f9223400640 1 -- 172.21.15.1:0/1154411637 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f920c02ae50 con 0x7f92240cc730 2024-05-09T08:03:16.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.087+0000 7f9223400640 1 -- 172.21.15.1:0/1154411637 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f920c032860 con 0x7f92240cc730 2024-05-09T08:03:16.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.087+0000 7f9229800640 1 -- 172.21.15.1:0/1154411637 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f92240cc730 msgr2=0x7f922415c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:16.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.087+0000 7f9229800640 1 --2- 172.21.15.1:0/1154411637 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f92240cc730 0x7f922415c080 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f922406eb20 tx=0x7f920c02a300 comp rx=0 tx=0).stop 2024-05-09T08:03:16.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.087+0000 7f9229800640 1 -- 172.21.15.1:0/1154411637 shutdown_connections 2024-05-09T08:03:16.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.087+0000 7f9229800640 1 --2- 172.21.15.1:0/1154411637 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f92240cc730 0x7f922415c080 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:16.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.087+0000 7f9229800640 1 --2- 172.21.15.1:0/1154411637 >> v2:172.21.15.7:3300/0 conn(0x7f9224152500 0x7f92240c8180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:16.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.087+0000 7f9229800640 1 -- 172.21.15.1:0/1154411637 >> 172.21.15.1:0/1154411637 conn(0x7f92240cbc10 msgr2=0x7f92240cc020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:03:16.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.087+0000 7f9229800640 1 -- 172.21.15.1:0/1154411637 shutdown_connections 2024-05-09T08:03:16.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.087+0000 7f9229800640 1 -- 172.21.15.1:0/1154411637 wait complete. 2024-05-09T08:03:16.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.089+0000 7f9229800640 1 Processor -- start 2024-05-09T08:03:16.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.089+0000 7f9229800640 1 -- start start 2024-05-09T08:03:16.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.090+0000 7f9229800640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f92240c7660 0x7f92241536e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:16.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.090+0000 7f9229800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f92240cc730 0x7f9224158330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:16.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.090+0000 7f9229800640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f9224152500 0x7f92241ec0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:16.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.090+0000 7f9229800640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f92240c72b0 con 0x7f92240c7660 2024-05-09T08:03:16.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.090+0000 7f9229800640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f92240ccb50 con 0x7f92240cc730 2024-05-09T08:03:16.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.090+0000 7f9229800640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f922415b880 con 0x7f9224152500 2024-05-09T08:03:16.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.091+0000 7f9223e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f92240c7660 0x7f92241536e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:16.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.091+0000 7f9223e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f92240c7660 0x7f92241536e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54656/0 (socket says 172.21.15.1:54656) 2024-05-09T08:03:16.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.091+0000 7f9223e00640 1 -- 172.21.15.1:0/783890629 learned_addr learned my addr 172.21.15.1:0/783890629 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:03:16.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.091+0000 7f921be00640 1 --2- 172.21.15.1:0/783890629 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f92240cc730 0x7f9224158330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:16.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.091+0000 7f9228e00640 1 --2- 172.21.15.1:0/783890629 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f9224152500 0x7f92241ec0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:16.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.091+0000 7f9223e00640 1 -- 172.21.15.1:0/783890629 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f9224152500 msgr2=0x7f92241ec0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:16.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.091+0000 7f9223e00640 1 --2- 172.21.15.1:0/783890629 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f9224152500 0x7f92241ec0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:16.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.091+0000 7f9223e00640 1 -- 172.21.15.1:0/783890629 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f92240cc730 msgr2=0x7f9224158330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:16.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.091+0000 7f9223e00640 1 --2- 172.21.15.1:0/783890629 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f92240cc730 0x7f9224158330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:16.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.091+0000 7f9223e00640 1 -- 172.21.15.1:0/783890629 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92240c0fc0 con 0x7f92240c7660 2024-05-09T08:03:16.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.091+0000 7f921be00640 1 --2- 172.21.15.1:0/783890629 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f92240cc730 0x7f9224158330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:03:16.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.092+0000 7f9223e00640 1 --2- 172.21.15.1:0/783890629 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f92240c7660 0x7f92241536e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f9214012ae0 tx=0x7f92140100c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:03:16.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.092+0000 7f9220c00640 1 -- 172.21.15.1:0/783890629 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9214010a90 con 0x7f92240c7660 2024-05-09T08:03:16.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.092+0000 7f9220c00640 1 -- 172.21.15.1:0/783890629 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f921401d070 con 0x7f92240c7660 2024-05-09T08:03:16.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.092+0000 7f9220c00640 1 -- 172.21.15.1:0/783890629 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9214010e70 con 0x7f92240c7660 2024-05-09T08:03:16.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.092+0000 7f9229800640 1 -- 172.21.15.1:0/783890629 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92240c1280 con 0x7f92240c7660 2024-05-09T08:03:16.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.092+0000 7f9229800640 1 -- 172.21.15.1:0/783890629 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9224155a10 con 0x7f92240c7660 2024-05-09T08:03:16.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.093+0000 7f9229800640 1 -- 172.21.15.1:0/783890629 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9224159d00 con 0x7f92240c7660 2024-05-09T08:03:16.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.094+0000 7f9220c00640 1 -- 172.21.15.1:0/783890629 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9214016050 con 0x7f92240c7660 2024-05-09T08:03:16.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.095+0000 7f9220c00640 1 --2- 172.21.15.1:0/783890629 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f920006be30 0x7f920006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:16.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.095+0000 7f9220c00640 1 -- 172.21.15.1:0/783890629 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7f921408ef90 con 0x7f92240c7660 2024-05-09T08:03:16.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.096+0000 7f921be00640 1 --2- 172.21.15.1:0/783890629 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f920006be30 0x7f920006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:16.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.096+0000 7f921be00640 1 --2- 172.21.15.1:0/783890629 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f920006be30 0x7f920006e2f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f920c034000 tx=0x7f920c02e000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:03:16.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.100+0000 7f9220c00640 1 -- 172.21.15.1:0/783890629 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f9224159d00 con 0x7f92240c7660 2024-05-09T08:03:16.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.380+0000 7f9229800640 1 -- 172.21.15.1:0/783890629 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f922406e8b0 con 0x7f92240c7660 2024-05-09T08:03:16.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.383+0000 7f9220c00640 1 -- 172.21.15.1:0/783890629 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1544 (secure 0 0 0) 0x7f922406e8b0 con 0x7f92240c7660 2024-05-09T08:03:16.401 INFO:teuthology.orchestra.run.smithi001.stdout:{"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":79,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"54329ec8-3270-4043-9a82-c6cd7420aa7f","modified":"2024-05-09T08:01:21.819270Z","created":"2024-05-09T08:01:21.819270Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:3300","nonce":0},{"type":"v1","addr":"172.21.15.1:6789","nonce":0}]},"addr":"172.21.15.1:6789/0","public_addr":"172.21.15.1:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:3300","nonce":0},{"type":"v1","addr":"172.21.15.7:6789","nonce":0}]},"addr":"172.21.15.7:6789/0","public_addr":"172.21.15.7:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:3301","nonce":0},{"type":"v1","addr":"172.21.15.7:6790","nonce":0}]},"addr":"172.21.15.7:6790/0","public_addr":"172.21.15.7:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-05-09T08:03:16.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.386+0000 7f9229800640 1 -- 172.21.15.1:0/783890629 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f920006be30 msgr2=0x7f920006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:16.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.386+0000 7f9229800640 1 --2- 172.21.15.1:0/783890629 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f920006be30 0x7f920006e2f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f920c034000 tx=0x7f920c02e000 comp rx=0 tx=0).stop 2024-05-09T08:03:16.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.386+0000 7f9229800640 1 -- 172.21.15.1:0/783890629 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f92240c7660 msgr2=0x7f92241536e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:16.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.386+0000 7f9229800640 1 --2- 172.21.15.1:0/783890629 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f92240c7660 0x7f92241536e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f9214012ae0 tx=0x7f92140100c0 comp rx=0 tx=0).stop 2024-05-09T08:03:16.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.387+0000 7f9229800640 1 -- 172.21.15.1:0/783890629 shutdown_connections 2024-05-09T08:03:16.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.387+0000 7f9229800640 1 --2- 172.21.15.1:0/783890629 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f920006be30 0x7f920006e2f0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:16.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.387+0000 7f9229800640 1 --2- 172.21.15.1:0/783890629 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f92240cc730 0x7f9224158330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:16.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.387+0000 7f9229800640 1 --2- 172.21.15.1:0/783890629 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f9224152500 0x7f92241ec0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:16.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.387+0000 7f9229800640 1 --2- 172.21.15.1:0/783890629 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f92240c7660 0x7f92241536e0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:16.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.387+0000 7f9229800640 1 -- 172.21.15.1:0/783890629 >> 172.21.15.1:0/783890629 conn(0x7f92240cbc10 msgr2=0x7f922406bd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:03:16.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.387+0000 7f9229800640 1 -- 172.21.15.1:0/783890629 shutdown_connections 2024-05-09T08:03:16.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.387+0000 7f9229800640 1 -- 172.21.15.1:0/783890629 wait complete. 2024-05-09T08:03:16.424 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-05-09T08:03:16.424 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-05-09T08:03:16.424 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-05-09T08:03:16.425 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-05-09T08:03:16.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.627+0000 7f8bf5e00640 1 Processor -- start 2024-05-09T08:03:16.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.627+0000 7f8bf5e00640 1 -- start start 2024-05-09T08:03:16.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.629+0000 7f8bf5e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f8bf015c360 0x7f8bf00c9570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:16.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.629+0000 7f8bf5e00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f8bf0072c20 0x7f8bf0073090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:16.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.629+0000 7f8bf5e00640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8bf0071c80 con 0x7f8bf00c5e40 2024-05-09T08:03:16.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.629+0000 7f8bf5e00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f8bf015b980 con 0x7f8bf015c360 2024-05-09T08:03:16.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.629+0000 7f8bf5e00640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f8bf0071f20 con 0x7f8bf0072c20 2024-05-09T08:03:16.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.629+0000 7f8bee000640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f8bf00c5e40 0x7f8bf00cc5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:36908/0 (socket says 172.21.15.1:36908) 2024-05-09T08:03:16.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.629+0000 7f8bee000640 1 -- 172.21.15.1:0/50143600 learned_addr learned my addr 172.21.15.1:0/50143600 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T08:03:16.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.629+0000 7f8bef400640 1 --2- 172.21.15.1:0/50143600 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f8bf0072c20 0x7f8bf0073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:16.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.629+0000 7f8beea00640 1 --2- 172.21.15.1:0/50143600 >> v2:172.21.15.7:3300/0 conn(0x7f8bf015c360 0x7f8bf00c9570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:16.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.630+0000 7f8bef400640 1 -- 172.21.15.1:0/50143600 >> v2:172.21.15.7:3300/0 conn(0x7f8bf015c360 msgr2=0x7f8bf00c9570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:03:16.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.630+0000 7f8bef400640 1 --2- 172.21.15.1:0/50143600 >> v2:172.21.15.7:3300/0 conn(0x7f8bf015c360 0x7f8bf00c9570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:16.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.630+0000 7f8bef400640 1 -- 172.21.15.1:0/50143600 >> v1:172.21.15.1:6789/0 conn(0x7f8bf00c5e40 legacy=0x7f8bf00cc5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:16.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.630+0000 7f8bef400640 1 -- 172.21.15.1:0/50143600 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bf00721e0 con 0x7f8bf0072c20 2024-05-09T08:03:16.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.630+0000 7f8bed600640 1 -- 172.21.15.1:0/50143600 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1178165774 0 0) 0x7f8bf0071c80 con 0x7f8bf00c5e40 2024-05-09T08:03:16.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.631+0000 7f8bef400640 1 --2- 172.21.15.1:0/50143600 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f8bf0072c20 0x7f8bf0073090 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f8be80097a0 tx=0x7f8be802a060 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f5e0b4cf0f7dff2e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:03:16.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.631+0000 7f8bed600640 1 -- 172.21.15.1:0/50143600 <== mon.2 v2:172.21.15.7:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8be8037070 con 0x7f8bf0072c20 2024-05-09T08:03:16.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.631+0000 7f8bed600640 1 -- 172.21.15.1:0/50143600 <== mon.2 v2:172.21.15.7:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8be802adb0 con 0x7f8bf0072c20 2024-05-09T08:03:16.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.631+0000 7f8bed600640 1 -- 172.21.15.1:0/50143600 <== mon.2 v2:172.21.15.7:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8be8032660 con 0x7f8bf0072c20 2024-05-09T08:03:16.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.632+0000 7f8bf5e00640 1 -- 172.21.15.1:0/50143600 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f8bf0072c20 msgr2=0x7f8bf0073090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:16.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.632+0000 7f8bf5e00640 1 --2- 172.21.15.1:0/50143600 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f8bf0072c20 0x7f8bf0073090 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f8be80097a0 tx=0x7f8be802a060 comp rx=0 tx=0).stop 2024-05-09T08:03:16.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.632+0000 7f8bf5e00640 1 -- 172.21.15.1:0/50143600 shutdown_connections 2024-05-09T08:03:16.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.632+0000 7f8bf5e00640 1 --2- 172.21.15.1:0/50143600 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f8bf0072c20 0x7f8bf0073090 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:16.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.632+0000 7f8bf5e00640 1 --2- 172.21.15.1:0/50143600 >> v2:172.21.15.7:3300/0 conn(0x7f8bf015c360 0x7f8bf00c9570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:16.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.632+0000 7f8bf5e00640 1 -- 172.21.15.1:0/50143600 >> 172.21.15.1:0/50143600 conn(0x7f8bf006dad0 msgr2=0x7f8bf006deb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:03:16.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.632+0000 7f8bf5e00640 1 -- 172.21.15.1:0/50143600 shutdown_connections 2024-05-09T08:03:16.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.632+0000 7f8bf5e00640 1 -- 172.21.15.1:0/50143600 wait complete. 2024-05-09T08:03:16.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.634+0000 7f8bf5e00640 1 Processor -- start 2024-05-09T08:03:16.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.634+0000 7f8bf5e00640 1 -- start start 2024-05-09T08:03:16.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.635+0000 7f8bf5e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8bf0072c20 0x7f8bf01e3930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:16.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.635+0000 7f8bf5e00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f8bf00c5e40 0x7f8bf01e3e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:16.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.635+0000 7f8bf5e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8bf015c360 0x7f8bf00c2620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:16.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.635+0000 7f8bf5e00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f8bf00cc0d0 con 0x7f8bf0072c20 2024-05-09T08:03:16.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.635+0000 7f8bf5e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f8bf00727a0 con 0x7f8bf015c360 2024-05-09T08:03:16.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.635+0000 7f8bf5e00640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f8bf00ccd00 con 0x7f8bf00c5e40 2024-05-09T08:03:16.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.635+0000 7f8beea00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8bf0072c20 0x7f8bf01e3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:16.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.635+0000 7f8beea00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8bf0072c20 0x7f8bf01e3930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54660/0 (socket says 172.21.15.1:54660) 2024-05-09T08:03:16.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.635+0000 7f8beea00640 1 -- 172.21.15.1:0/2834521148 learned_addr learned my addr 172.21.15.1:0/2834521148 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:03:16.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.635+0000 7f8bee000640 1 --2- 172.21.15.1:0/2834521148 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f8bf00c5e40 0x7f8bf01e3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:16.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.635+0000 7f8bef400640 1 --2- 172.21.15.1:0/2834521148 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8bf015c360 0x7f8bf00c2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:16.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.636+0000 7f8beea00640 1 -- 172.21.15.1:0/2834521148 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f8bf00c5e40 msgr2=0x7f8bf01e3e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:16.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.636+0000 7f8beea00640 1 --2- 172.21.15.1:0/2834521148 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f8bf00c5e40 0x7f8bf01e3e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:16.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.636+0000 7f8beea00640 1 -- 172.21.15.1:0/2834521148 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8bf015c360 msgr2=0x7f8bf00c2620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:16.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.636+0000 7f8beea00640 1 --2- 172.21.15.1:0/2834521148 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8bf015c360 0x7f8bf00c2620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:16.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.636+0000 7f8beea00640 1 -- 172.21.15.1:0/2834521148 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bf0152f20 con 0x7f8bf0072c20 2024-05-09T08:03:16.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.636+0000 7f8bef400640 1 --2- 172.21.15.1:0/2834521148 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8bf015c360 0x7f8bf00c2620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:03:16.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.636+0000 7f8beea00640 1 --2- 172.21.15.1:0/2834521148 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8bf0072c20 0x7f8bf01e3930 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f8be000e870 tx=0x7f8be000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:03:16.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.636+0000 7f8be7400640 1 -- 172.21.15.1:0/2834521148 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8be0017070 con 0x7f8bf0072c20 2024-05-09T08:03:16.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.637+0000 7f8be7400640 1 -- 172.21.15.1:0/2834521148 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8be000c940 con 0x7f8bf0072c20 2024-05-09T08:03:16.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.637+0000 7f8bf5e00640 1 -- 172.21.15.1:0/2834521148 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bf0153200 con 0x7f8bf0072c20 2024-05-09T08:03:16.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.637+0000 7f8be7400640 1 -- 172.21.15.1:0/2834521148 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8be000cd30 con 0x7f8bf0072c20 2024-05-09T08:03:16.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.637+0000 7f8bf5e00640 1 -- 172.21.15.1:0/2834521148 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bf01f12d0 con 0x7f8bf0072c20 2024-05-09T08:03:16.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.638+0000 7f8bf5e00640 1 --2- 172.21.15.1:0/2834521148 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8bf00c1690 0x7f8bf0153460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:16.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.638+0000 7f8bf5e00640 1 -- 172.21.15.1:0/2834521148 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f8bf0152c70 con 0x7f8bf00c1690 2024-05-09T08:03:16.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.639+0000 7f8bee000640 1 --2- 172.21.15.1:0/2834521148 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8bf00c1690 0x7f8bf0153460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:16.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.639+0000 7f8bee000640 1 -- 172.21.15.1:0/2834521148 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8bf00686c0 con 0x7f8bf00c1690 2024-05-09T08:03:16.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.639+0000 7f8bee000640 1 --2- 172.21.15.1:0/2834521148 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8bf00c1690 0x7f8bf0153460 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f8bf00c1380 tx=0x7f8bdc007860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:03:16.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.639+0000 7f8be7400640 1 -- 172.21.15.1:0/2834521148 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8be001a070 con 0x7f8bf0072c20 2024-05-09T08:03:16.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.640+0000 7f8be7400640 1 --2- 172.21.15.1:0/2834521148 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f8bd006be30 0x7f8bd006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:16.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.640+0000 7f8be7400640 1 -- 172.21.15.1:0/2834521148 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7f8be008b540 con 0x7f8bf0072c20 2024-05-09T08:03:16.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.641+0000 7f8bef400640 1 --2- 172.21.15.1:0/2834521148 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f8bd006be30 0x7f8bd006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:16.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.641+0000 7f8be7400640 1 -- 172.21.15.1:0/2834521148 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bdc00fa00 con 0x7f8bf00c1690 2024-05-09T08:03:16.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.641+0000 7f8be7400640 1 -- 172.21.15.1:0/2834521148 <== mon.0 v2:172.21.15.1:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f8bdc018070 con 0x7f8bf00c1690 2024-05-09T08:03:16.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.641+0000 7f8be7400640 1 -- 172.21.15.1:0/2834521148 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8bf00c1690 msgr2=0x7f8bf0153460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:16.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.641+0000 7f8be7400640 1 --2- 172.21.15.1:0/2834521148 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8bf00c1690 0x7f8bf0153460 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f8bf00c1380 tx=0x7f8bdc007860 comp rx=0 tx=0).stop 2024-05-09T08:03:16.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.641+0000 7f8be7400640 1 -- 172.21.15.1:0/2834521148 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8bf00c1690 msgr2=0x7f8bf0153460 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-05-09T08:03:16.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.641+0000 7f8be7400640 1 --2- 172.21.15.1:0/2834521148 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8bf00c1690 0x7f8bf0153460 secure :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f8bf00c1380 tx=0x7f8bdc007860 comp rx=0 tx=0).stop 2024-05-09T08:03:16.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.641+0000 7f8bef400640 1 --2- 172.21.15.1:0/2834521148 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f8bd006be30 0x7f8bd006e2f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f8be8034000 tx=0x7f8be802e000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:03:16.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.648+0000 7f8bf5e00640 1 --2- 172.21.15.1:0/2834521148 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8bf00c1e40 0x7f8bf00c2b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:16.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.648+0000 7f8bf5e00640 1 -- 172.21.15.1:0/2834521148 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f8bf0159350 con 0x7f8bf00c1e40 2024-05-09T08:03:16.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.649+0000 7f8bee000640 1 --2- 172.21.15.1:0/2834521148 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8bf00c1e40 0x7f8bf00c2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:16.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.649+0000 7f8bee000640 1 -- 172.21.15.1:0/2834521148 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f8bf00c76b0 con 0x7f8bf00c1e40 2024-05-09T08:03:16.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.649+0000 7f8bee000640 1 --2- 172.21.15.1:0/2834521148 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8bf00c1e40 0x7f8bf00c2b60 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f8bdc017000 tx=0x7f8bdc001400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:03:16.667 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.650+0000 7f8be7400640 1 -- 172.21.15.1:0/2834521148 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bdc013600 con 0x7f8bf00c1e40 2024-05-09T08:03:16.667 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.650+0000 7f8be7400640 1 -- 172.21.15.1:0/2834521148 <== mon.0 v2:172.21.15.1:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4706 (secure 0 0 0) 0x7f8bdc013600 con 0x7f8bf00c1e40 2024-05-09T08:03:16.667 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.650+0000 7f8be7400640 1 -- 172.21.15.1:0/2834521148 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8bf00c1e40 msgr2=0x7f8bf00c2b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:16.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.650+0000 7f8be7400640 1 --2- 172.21.15.1:0/2834521148 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8bf00c1e40 0x7f8bf00c2b60 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f8bdc017000 tx=0x7f8bdc001400 comp rx=0 tx=0).stop 2024-05-09T08:03:16.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.650+0000 7f8be7400640 1 -- 172.21.15.1:0/2834521148 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8bf00c1e40 msgr2=0x7f8bf00c2b60 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-05-09T08:03:16.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.650+0000 7f8be7400640 1 --2- 172.21.15.1:0/2834521148 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8bf00c1e40 0x7f8bf00c2b60 secure :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f8bdc017000 tx=0x7f8bdc001400 comp rx=0 tx=0).stop 2024-05-09T08:03:16.668 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-05-09T08:03:16.668 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "a", 2024-05-09T08:03:16.668 INFO:teuthology.orchestra.run.smithi001.stdout: "rank": 0, 2024-05-09T08:03:16.668 INFO:teuthology.orchestra.run.smithi001.stdout: "state": "leader", 2024-05-09T08:03:16.668 INFO:teuthology.orchestra.run.smithi001.stdout: "election_epoch": 4, 2024-05-09T08:03:16.668 INFO:teuthology.orchestra.run.smithi001.stdout: "uptime": 80012, 2024-05-09T08:03:16.669 INFO:teuthology.orchestra.run.smithi001.stdout: "quorum": [ 2024-05-09T08:03:16.669 INFO:teuthology.orchestra.run.smithi001.stdout: 0, 2024-05-09T08:03:16.669 INFO:teuthology.orchestra.run.smithi001.stdout: 1, 2024-05-09T08:03:16.669 INFO:teuthology.orchestra.run.smithi001.stdout: 2 2024-05-09T08:03:16.669 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:03:16.669 INFO:teuthology.orchestra.run.smithi001.stdout: "quorum_age": 79, 2024-05-09T08:03:16.669 INFO:teuthology.orchestra.run.smithi001.stdout: "features": { 2024-05-09T08:03:16.669 INFO:teuthology.orchestra.run.smithi001.stdout: "required_con": "2450521708007866372", 2024-05-09T08:03:16.670 INFO:teuthology.orchestra.run.smithi001.stdout: "required_mon": [ 2024-05-09T08:03:16.670 INFO:teuthology.orchestra.run.smithi001.stdout: "kraken", 2024-05-09T08:03:16.670 INFO:teuthology.orchestra.run.smithi001.stdout: "luminous", 2024-05-09T08:03:16.670 INFO:teuthology.orchestra.run.smithi001.stdout: "mimic", 2024-05-09T08:03:16.670 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap-prune", 2024-05-09T08:03:16.670 INFO:teuthology.orchestra.run.smithi001.stdout: "nautilus", 2024-05-09T08:03:16.670 INFO:teuthology.orchestra.run.smithi001.stdout: "octopus", 2024-05-09T08:03:16.671 INFO:teuthology.orchestra.run.smithi001.stdout: "pacific", 2024-05-09T08:03:16.671 INFO:teuthology.orchestra.run.smithi001.stdout: "elector-pinging", 2024-05-09T08:03:16.671 INFO:teuthology.orchestra.run.smithi001.stdout: "quincy", 2024-05-09T08:03:16.671 INFO:teuthology.orchestra.run.smithi001.stdout: "reef", 2024-05-09T08:03:16.671 INFO:teuthology.orchestra.run.smithi001.stdout: "squid" 2024-05-09T08:03:16.671 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:03:16.671 INFO:teuthology.orchestra.run.smithi001.stdout: "quorum_con": "4540701547738038271", 2024-05-09T08:03:16.671 INFO:teuthology.orchestra.run.smithi001.stdout: "quorum_mon": [ 2024-05-09T08:03:16.671 INFO:teuthology.orchestra.run.smithi001.stdout: "kraken", 2024-05-09T08:03:16.671 INFO:teuthology.orchestra.run.smithi001.stdout: "luminous", 2024-05-09T08:03:16.671 INFO:teuthology.orchestra.run.smithi001.stdout: "mimic", 2024-05-09T08:03:16.671 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap-prune", 2024-05-09T08:03:16.671 INFO:teuthology.orchestra.run.smithi001.stdout: "nautilus", 2024-05-09T08:03:16.671 INFO:teuthology.orchestra.run.smithi001.stdout: "octopus", 2024-05-09T08:03:16.672 INFO:teuthology.orchestra.run.smithi001.stdout: "pacific", 2024-05-09T08:03:16.672 INFO:teuthology.orchestra.run.smithi001.stdout: "elector-pinging", 2024-05-09T08:03:16.672 INFO:teuthology.orchestra.run.smithi001.stdout: "quincy", 2024-05-09T08:03:16.672 INFO:teuthology.orchestra.run.smithi001.stdout: "reef", 2024-05-09T08:03:16.672 INFO:teuthology.orchestra.run.smithi001.stdout: "squid" 2024-05-09T08:03:16.672 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:03:16.672 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:03:16.672 INFO:teuthology.orchestra.run.smithi001.stdout: "outside_quorum": [], 2024-05-09T08:03:16.672 INFO:teuthology.orchestra.run.smithi001.stdout: "extra_probe_peers": [], 2024-05-09T08:03:16.672 INFO:teuthology.orchestra.run.smithi001.stdout: "sync_provider": [], 2024-05-09T08:03:16.672 INFO:teuthology.orchestra.run.smithi001.stdout: "monmap": { 2024-05-09T08:03:16.672 INFO:teuthology.orchestra.run.smithi001.stdout: "epoch": 1, 2024-05-09T08:03:16.672 INFO:teuthology.orchestra.run.smithi001.stdout: "fsid": "54329ec8-3270-4043-9a82-c6cd7420aa7f", 2024-05-09T08:03:16.672 INFO:teuthology.orchestra.run.smithi001.stdout: "modified": "2024-05-09T08:01:21.819270Z", 2024-05-09T08:03:16.673 INFO:teuthology.orchestra.run.smithi001.stdout: "created": "2024-05-09T08:01:21.819270Z", 2024-05-09T08:03:16.673 INFO:teuthology.orchestra.run.smithi001.stdout: "min_mon_release": 19, 2024-05-09T08:03:16.673 INFO:teuthology.orchestra.run.smithi001.stdout: "min_mon_release_name": "squid", 2024-05-09T08:03:16.673 INFO:teuthology.orchestra.run.smithi001.stdout: "election_strategy": 1, 2024-05-09T08:03:16.673 INFO:teuthology.orchestra.run.smithi001.stdout: "disallowed_leaders: ": "", 2024-05-09T08:03:16.673 INFO:teuthology.orchestra.run.smithi001.stdout: "stretch_mode": false, 2024-05-09T08:03:16.673 INFO:teuthology.orchestra.run.smithi001.stdout: "tiebreaker_mon": "", 2024-05-09T08:03:16.673 INFO:teuthology.orchestra.run.smithi001.stdout: "removed_ranks: ": "", 2024-05-09T08:03:16.673 INFO:teuthology.orchestra.run.smithi001.stdout: "features": { 2024-05-09T08:03:16.673 INFO:teuthology.orchestra.run.smithi001.stdout: "persistent": [ 2024-05-09T08:03:16.673 INFO:teuthology.orchestra.run.smithi001.stdout: "kraken", 2024-05-09T08:03:16.673 INFO:teuthology.orchestra.run.smithi001.stdout: "luminous", 2024-05-09T08:03:16.674 INFO:teuthology.orchestra.run.smithi001.stdout: "mimic", 2024-05-09T08:03:16.674 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap-prune", 2024-05-09T08:03:16.674 INFO:teuthology.orchestra.run.smithi001.stdout: "nautilus", 2024-05-09T08:03:16.674 INFO:teuthology.orchestra.run.smithi001.stdout: "octopus", 2024-05-09T08:03:16.674 INFO:teuthology.orchestra.run.smithi001.stdout: "pacific", 2024-05-09T08:03:16.674 INFO:teuthology.orchestra.run.smithi001.stdout: "elector-pinging", 2024-05-09T08:03:16.674 INFO:teuthology.orchestra.run.smithi001.stdout: "quincy", 2024-05-09T08:03:16.674 INFO:teuthology.orchestra.run.smithi001.stdout: "reef", 2024-05-09T08:03:16.674 INFO:teuthology.orchestra.run.smithi001.stdout: "squid" 2024-05-09T08:03:16.674 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:03:16.674 INFO:teuthology.orchestra.run.smithi001.stdout: "optional": [] 2024-05-09T08:03:16.674 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:03:16.674 INFO:teuthology.orchestra.run.smithi001.stdout: "mons": [ 2024-05-09T08:03:16.674 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:03:16.675 INFO:teuthology.orchestra.run.smithi001.stdout: "rank": 0, 2024-05-09T08:03:16.675 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "a", 2024-05-09T08:03:16.675 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-05-09T08:03:16.675 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-05-09T08:03:16.675 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:03:16.675 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-05-09T08:03:16.675 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:3300", 2024-05-09T08:03:16.675 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:03:16.675 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:03:16.675 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:03:16.675 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-05-09T08:03:16.675 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6789", 2024-05-09T08:03:16.676 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:03:16.676 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:03:16.676 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:03:16.676 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:03:16.676 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6789/0", 2024-05-09T08:03:16.676 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.1:6789/0", 2024-05-09T08:03:16.676 INFO:teuthology.orchestra.run.smithi001.stdout: "priority": 0, 2024-05-09T08:03:16.676 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 0, 2024-05-09T08:03:16.676 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_location": "{}" 2024-05-09T08:03:16.676 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:03:16.676 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:03:16.676 INFO:teuthology.orchestra.run.smithi001.stdout: "rank": 1, 2024-05-09T08:03:16.676 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "b", 2024-05-09T08:03:16.676 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-05-09T08:03:16.677 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-05-09T08:03:16.677 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:03:16.677 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-05-09T08:03:16.677 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:3300", 2024-05-09T08:03:16.677 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:03:16.677 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:03:16.677 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:03:16.677 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-05-09T08:03:16.677 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:6789", 2024-05-09T08:03:16.677 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:03:16.677 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:03:16.677 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:03:16.677 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:03:16.677 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:6789/0", 2024-05-09T08:03:16.678 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.7:6789/0", 2024-05-09T08:03:16.678 INFO:teuthology.orchestra.run.smithi001.stdout: "priority": 0, 2024-05-09T08:03:16.678 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 0, 2024-05-09T08:03:16.678 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_location": "{}" 2024-05-09T08:03:16.678 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:03:16.678 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:03:16.678 INFO:teuthology.orchestra.run.smithi001.stdout: "rank": 2, 2024-05-09T08:03:16.678 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "c", 2024-05-09T08:03:16.678 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-05-09T08:03:16.678 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-05-09T08:03:16.678 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:03:16.678 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-05-09T08:03:16.679 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:3301", 2024-05-09T08:03:16.679 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:03:16.679 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:03:16.679 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:03:16.679 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-05-09T08:03:16.679 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:6790", 2024-05-09T08:03:16.679 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:03:16.679 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:03:16.679 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:03:16.679 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:03:16.679 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:6790/0", 2024-05-09T08:03:16.679 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.7:6790/0", 2024-05-09T08:03:16.680 INFO:teuthology.orchestra.run.smithi001.stdout: "priority": 0, 2024-05-09T08:03:16.680 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 0, 2024-05-09T08:03:16.680 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_location": "{}" 2024-05-09T08:03:16.680 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:03:16.680 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:03:16.680 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:03:16.680 INFO:teuthology.orchestra.run.smithi001.stdout: "feature_map": { 2024-05-09T08:03:16.680 INFO:teuthology.orchestra.run.smithi001.stdout: "mon": [ 2024-05-09T08:03:16.680 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:03:16.680 INFO:teuthology.orchestra.run.smithi001.stdout: "features": "0x3f03cffffffdffff", 2024-05-09T08:03:16.680 INFO:teuthology.orchestra.run.smithi001.stdout: "release": "luminous", 2024-05-09T08:03:16.680 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 1 2024-05-09T08:03:16.680 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:03:16.680 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:03:16.681 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": [ 2024-05-09T08:03:16.681 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:03:16.681 INFO:teuthology.orchestra.run.smithi001.stdout: "features": "0x3f03cffffffdffff", 2024-05-09T08:03:16.681 INFO:teuthology.orchestra.run.smithi001.stdout: "release": "luminous", 2024-05-09T08:03:16.681 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 2 2024-05-09T08:03:16.681 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:03:16.681 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:03:16.681 INFO:teuthology.orchestra.run.smithi001.stdout: "client": [ 2024-05-09T08:03:16.681 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:03:16.681 INFO:teuthology.orchestra.run.smithi001.stdout: "features": "0x3f03cffffffdffff", 2024-05-09T08:03:16.681 INFO:teuthology.orchestra.run.smithi001.stdout: "release": "luminous", 2024-05-09T08:03:16.681 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 2 2024-05-09T08:03:16.681 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:03:16.681 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:03:16.681 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:03:16.682 INFO:teuthology.orchestra.run.smithi001.stdout: "stretch_mode": false 2024-05-09T08:03:16.682 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-05-09T08:03:16.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.651+0000 7f8bf5e00640 1 -- 172.21.15.1:0/2834521148 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f8bd006be30 msgr2=0x7f8bd006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:16.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.651+0000 7f8bf5e00640 1 --2- 172.21.15.1:0/2834521148 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f8bd006be30 0x7f8bd006e2f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f8be8034000 tx=0x7f8be802e000 comp rx=0 tx=0).stop 2024-05-09T08:03:16.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.651+0000 7f8bf5e00640 1 -- 172.21.15.1:0/2834521148 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8bf0072c20 msgr2=0x7f8bf01e3930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:16.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.651+0000 7f8bf5e00640 1 --2- 172.21.15.1:0/2834521148 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8bf0072c20 0x7f8bf01e3930 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f8be000e870 tx=0x7f8be000ed30 comp rx=0 tx=0).stop 2024-05-09T08:03:16.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.651+0000 7f8bef400640 1 -- 172.21.15.1:0/2834521148 reap_dead start 2024-05-09T08:03:16.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.651+0000 7f8bef400640 1 -- 172.21.15.1:0/2834521148 reap_dead start 2024-05-09T08:03:16.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.651+0000 7f8bf5e00640 1 -- 172.21.15.1:0/2834521148 shutdown_connections 2024-05-09T08:03:16.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.651+0000 7f8bf5e00640 1 -- 172.21.15.1:0/2834521148 >> 172.21.15.1:0/2834521148 conn(0x7f8bf006dad0 msgr2=0x7f8bf00745e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:03:16.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.651+0000 7f8bf5e00640 1 -- 172.21.15.1:0/2834521148 shutdown_connections 2024-05-09T08:03:16.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.651+0000 7f8bf5e00640 1 -- 172.21.15.1:0/2834521148 wait complete. 2024-05-09T08:03:16.689 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-05-09T08:03:16.690 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-05-09T08:03:16.690 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-05-09T08:03:16.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.888+0000 7f7ccd400640 1 Processor -- start 2024-05-09T08:03:16.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.888+0000 7f7ccd400640 1 -- start start 2024-05-09T08:03:16.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.890+0000 7f7ccd400640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f7cc806f990 0x7f7cc816b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:16.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.890+0000 7f7ccd400640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7cc80c7880 con 0x7f7cc806fd70 2024-05-09T08:03:16.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.890+0000 7f7ccd400640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7cc80c7b60 con 0x7f7cc80c73f0 2024-05-09T08:03:16.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.890+0000 7f7ccd400640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f7cc816b7e0 con 0x7f7cc806f990 2024-05-09T08:03:16.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.890+0000 7f7cc6800640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f7cc806f990 0x7f7cc816b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:16.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.890+0000 7f7cc6800640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f7cc806f990 0x7f7cc816b2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54682/0 (socket says 172.21.15.1:54682) 2024-05-09T08:03:16.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.890+0000 7f7cc6800640 1 -- 172.21.15.1:0/450696674 learned_addr learned my addr 172.21.15.1:0/450696674 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:03:16.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.891+0000 7f7cc5e00640 1 -- 172.21.15.1:0/450696674 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1000511887 0 0) 0x7f7cc80c7880 con 0x7f7cc806fd70 2024-05-09T08:03:16.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.911+0000 7f7cc5e00640 1 -- 172.21.15.1:0/450696674 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7cac003500 con 0x7f7cc806fd70 2024-05-09T08:03:16.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.911+0000 7f7cc5e00640 1 -- 172.21.15.1:0/450696674 <== mon.1 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 316411504 0 0) 0x7f7cc80c7b60 con 0x7f7cc80c73f0 2024-05-09T08:03:16.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.911+0000 7f7cc5e00640 1 -- 172.21.15.1:0/450696674 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7cc80c7880 con 0x7f7cc80c73f0 2024-05-09T08:03:16.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.911+0000 7f7cc5e00640 1 -- 172.21.15.1:0/450696674 <== mon.0 v1:172.21.15.1:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3859276480 0 0) 0x7f7cac003500 con 0x7f7cc806fd70 2024-05-09T08:03:16.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.911+0000 7f7cc5e00640 1 -- 172.21.15.1:0/450696674 >> v2:172.21.15.1:3300/0 conn(0x7f7cc806f990 msgr2=0x7f7cc816b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:03:16.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.911+0000 7f7cc5e00640 1 --2- 172.21.15.1:0/450696674 >> v2:172.21.15.1:3300/0 conn(0x7f7cc806f990 0x7f7cc816b2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:16.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.911+0000 7f7cc5e00640 1 -- 172.21.15.1:0/450696674 >> v1:172.21.15.7:6789/0 conn(0x7f7cc80c73f0 legacy=0x7f7cc80c2640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:16.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.912+0000 7f7cc5e00640 1 -- 172.21.15.1:0/450696674 --> v1:172.21.15.1:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cc80c7e40 con 0x7f7cc806fd70 2024-05-09T08:03:16.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.912+0000 7f7cc5e00640 1 -- 172.21.15.1:0/450696674 <== mon.0 v1:172.21.15.1:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4179846320 0 0) 0x7f7cc00019f0 con 0x7f7cc806fd70 2024-05-09T08:03:16.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.912+0000 7f7cc5e00640 1 -- 172.21.15.1:0/450696674 >> v1:172.21.15.1:6789/0 conn(0x7f7cc806fd70 legacy=0x7f7cc81678e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:16.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.912+0000 7f7cc5e00640 1 --2- 172.21.15.1:0/450696674 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7cac004d40 0x7f7cac003890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:16.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.912+0000 7f7cc5e00640 1 --2- 172.21.15.1:0/450696674 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7cac003f00 0x7f7cac005140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:16.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.912+0000 7f7cc5e00640 1 --2- 172.21.15.1:0/450696674 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f7cac0044b0 0x7f7cac005750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:16.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.912+0000 7f7cc5e00640 1 -- 172.21.15.1:0/450696674 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f7cc816b7e0 con 0x7f7cac004d40 2024-05-09T08:03:16.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.912+0000 7f7cc5e00640 1 -- 172.21.15.1:0/450696674 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7cac005f90 con 0x7f7cac003f00 2024-05-09T08:03:16.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.912+0000 7f7cc5e00640 1 -- 172.21.15.1:0/450696674 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f7cac006190 con 0x7f7cac0044b0 2024-05-09T08:03:16.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.912+0000 7f7cc6800640 1 --2- 172.21.15.1:0/450696674 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7cac004d40 0x7f7cac003890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:16.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.912+0000 7f7cbfe00640 1 --2- 172.21.15.1:0/450696674 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7cac003f00 0x7f7cac005140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:16.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.912+0000 7f7cc7200640 1 --2- 172.21.15.1:0/450696674 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f7cac0044b0 0x7f7cac005750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:16.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.913+0000 7f7cc6800640 1 -- 172.21.15.1:0/450696674 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f7cac0044b0 msgr2=0x7f7cac005750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:03:16.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.913+0000 7f7cc6800640 1 --2- 172.21.15.1:0/450696674 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f7cac0044b0 0x7f7cac005750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:16.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.913+0000 7f7cc6800640 1 -- 172.21.15.1:0/450696674 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7cac003f00 msgr2=0x7f7cac005140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:03:16.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.913+0000 7f7cc6800640 1 --2- 172.21.15.1:0/450696674 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7cac003f00 0x7f7cac005140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:16.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.913+0000 7f7cc6800640 1 -- 172.21.15.1:0/450696674 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cac003500 con 0x7f7cac004d40 2024-05-09T08:03:16.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.913+0000 7f7cc7200640 1 -- 172.21.15.1:0/450696674 reap_dead start 2024-05-09T08:03:16.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.914+0000 7f7cc6800640 1 --2- 172.21.15.1:0/450696674 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7cac004d40 0x7f7cac003890 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f7cb8009490 tx=0x7f7cac00afc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bef4b1b069bebb4 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:03:16.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.915+0000 7f7cc5e00640 1 -- 172.21.15.1:0/450696674 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cb8037070 con 0x7f7cac004d40 2024-05-09T08:03:16.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.915+0000 7f7cc5e00640 1 -- 172.21.15.1:0/450696674 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7cb8032440 con 0x7f7cac004d40 2024-05-09T08:03:16.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.915+0000 7f7cc5e00640 1 -- 172.21.15.1:0/450696674 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cb8009060 con 0x7f7cac004d40 2024-05-09T08:03:16.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.916+0000 7f7ccd400640 1 -- 172.21.15.1:0/450696674 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7cac004d40 msgr2=0x7f7cac003890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:16.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.916+0000 7f7ccd400640 1 --2- 172.21.15.1:0/450696674 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7cac004d40 0x7f7cac003890 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f7cb8009490 tx=0x7f7cac00afc0 comp rx=0 tx=0).stop 2024-05-09T08:03:16.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.916+0000 7f7ccd400640 1 -- 172.21.15.1:0/450696674 shutdown_connections 2024-05-09T08:03:16.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.916+0000 7f7ccd400640 1 --2- 172.21.15.1:0/450696674 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7cac004d40 0x7f7cac003890 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:16.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.916+0000 7f7ccd400640 1 -- 172.21.15.1:0/450696674 >> 172.21.15.1:0/450696674 conn(0x7f7cc806dad0 msgr2=0x7f7cc806deb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:03:16.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.916+0000 7f7ccd400640 1 -- 172.21.15.1:0/450696674 shutdown_connections 2024-05-09T08:03:16.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.916+0000 7f7ccd400640 1 -- 172.21.15.1:0/450696674 wait complete. 2024-05-09T08:03:16.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.918+0000 7f7ccd400640 1 Processor -- start 2024-05-09T08:03:16.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.918+0000 7f7ccd400640 1 -- start start 2024-05-09T08:03:16.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.919+0000 7f7ccd400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7cac004d40 0x7f7cc8166260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:16.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.919+0000 7f7ccd400640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f7cc81fd6d0 0x7f7cc815d030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:16.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.919+0000 7f7ccd400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7cc80cc620 0x7f7cc815a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:16.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.919+0000 7f7ccd400640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f7cc80c87f0 con 0x7f7cac004d40 2024-05-09T08:03:16.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.919+0000 7f7ccd400640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7cc80c2120 con 0x7f7cc80cc620 2024-05-09T08:03:16.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.919+0000 7f7ccd400640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f7cc80c8970 con 0x7f7cc81fd6d0 2024-05-09T08:03:16.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.919+0000 7f7cc6800640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7cac004d40 0x7f7cc8166260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:16.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.919+0000 7f7cc6800640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7cac004d40 0x7f7cc8166260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54710/0 (socket says 172.21.15.1:54710) 2024-05-09T08:03:16.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.919+0000 7f7cc6800640 1 -- 172.21.15.1:0/3751934545 learned_addr learned my addr 172.21.15.1:0/3751934545 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:03:16.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.919+0000 7f7cc7200640 1 --2- 172.21.15.1:0/3751934545 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7cc80cc620 0x7f7cc815a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:16.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.919+0000 7f7cbfe00640 1 --2- 172.21.15.1:0/3751934545 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f7cc81fd6d0 0x7f7cc815d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:16.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.920+0000 7f7cc6800640 1 -- 172.21.15.1:0/3751934545 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f7cc81fd6d0 msgr2=0x7f7cc815d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:16.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.920+0000 7f7cc6800640 1 --2- 172.21.15.1:0/3751934545 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f7cc81fd6d0 0x7f7cc815d030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:16.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.920+0000 7f7cc6800640 1 -- 172.21.15.1:0/3751934545 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7cc80cc620 msgr2=0x7f7cc815a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:16.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.920+0000 7f7cc6800640 1 --2- 172.21.15.1:0/3751934545 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7cc80cc620 0x7f7cc815a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:16.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.920+0000 7f7cc6800640 1 -- 172.21.15.1:0/3751934545 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cc80cbe60 con 0x7f7cac004d40 2024-05-09T08:03:16.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.920+0000 7f7cc6800640 1 --2- 172.21.15.1:0/3751934545 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7cac004d40 0x7f7cc8166260 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f7cb8009580 tx=0x7f7cb8009ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:03:16.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.920+0000 7f7cbea00640 1 -- 172.21.15.1:0/3751934545 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cb8037040 con 0x7f7cac004d40 2024-05-09T08:03:16.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.921+0000 7f7cbea00640 1 -- 172.21.15.1:0/3751934545 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7cb8005990 con 0x7f7cac004d40 2024-05-09T08:03:16.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.921+0000 7f7cbea00640 1 -- 172.21.15.1:0/3751934545 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cb8005d60 con 0x7f7cac004d40 2024-05-09T08:03:16.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.921+0000 7f7ccd400640 1 -- 172.21.15.1:0/3751934545 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7cc815b600 con 0x7f7cac004d40 2024-05-09T08:03:16.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.921+0000 7f7ccd400640 1 -- 172.21.15.1:0/3751934545 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7cc815e200 con 0x7f7cac004d40 2024-05-09T08:03:16.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.922+0000 7f7ccd400640 1 --2- 172.21.15.1:0/3751934545 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7cc815e3d0 0x7f7cc80cca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:16.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.922+0000 7f7ccd400640 1 -- 172.21.15.1:0/3751934545 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7cc806f4f0 con 0x7f7cc815e3d0 2024-05-09T08:03:16.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.922+0000 7f7cbfe00640 1 --2- 172.21.15.1:0/3751934545 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7cc815e3d0 0x7f7cc80cca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:16.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.923+0000 7f7cbfe00640 1 -- 172.21.15.1:0/3751934545 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7cc8074d70 con 0x7f7cc815e3d0 2024-05-09T08:03:16.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.923+0000 7f7cbfe00640 1 --2- 172.21.15.1:0/3751934545 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7cc815e3d0 0x7f7cc80cca00 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f7cb0000c00 tx=0x7f7cb0012cf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:03:16.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.923+0000 7f7cbea00640 1 -- 172.21.15.1:0/3751934545 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7cb8042020 con 0x7f7cac004d40 2024-05-09T08:03:16.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.924+0000 7f7cbea00640 1 --2- 172.21.15.1:0/3751934545 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f7ca406be30 0x7f7ca406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:16.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.924+0000 7f7cbea00640 1 -- 172.21.15.1:0/3751934545 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7f7cb80aac20 con 0x7f7cac004d40 2024-05-09T08:03:16.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.924+0000 7f7cbea00640 1 -- 172.21.15.1:0/3751934545 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cb00052e0 con 0x7f7cc815e3d0 2024-05-09T08:03:16.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.924+0000 7f7cbea00640 1 -- 172.21.15.1:0/3751934545 <== mon.1 v2:172.21.15.7:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f7cb0005480 con 0x7f7cc815e3d0 2024-05-09T08:03:16.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.924+0000 7f7cbea00640 1 -- 172.21.15.1:0/3751934545 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7cc815e3d0 msgr2=0x7f7cc80cca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:16.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.924+0000 7f7cbea00640 1 --2- 172.21.15.1:0/3751934545 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7cc815e3d0 0x7f7cc80cca00 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f7cb0000c00 tx=0x7f7cb0012cf0 comp rx=0 tx=0).stop 2024-05-09T08:03:16.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.924+0000 7f7cbea00640 1 -- 172.21.15.1:0/3751934545 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7cc815e3d0 msgr2=0x7f7cc80cca00 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-05-09T08:03:16.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.924+0000 7f7cbea00640 1 --2- 172.21.15.1:0/3751934545 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7cc815e3d0 0x7f7cc80cca00 secure :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f7cb0000c00 tx=0x7f7cb0012cf0 comp rx=0 tx=0).stop 2024-05-09T08:03:16.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.924+0000 7f7cc7200640 1 --2- 172.21.15.1:0/3751934545 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f7ca406be30 0x7f7ca406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:16.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.925+0000 7f7cc7200640 1 --2- 172.21.15.1:0/3751934545 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f7ca406be30 0x7f7ca406e2f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f7cc80c66c0 tx=0x7f7cc0009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:03:16.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.932+0000 7f7ccd400640 1 --2- 172.21.15.1:0/3751934545 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7cc815abc0 0x7f7cc82001c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:16.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.932+0000 7f7ccd400640 1 -- 172.21.15.1:0/3751934545 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7cc80c6ee0 con 0x7f7cc815abc0 2024-05-09T08:03:16.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.932+0000 7f7cbfe00640 1 --2- 172.21.15.1:0/3751934545 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7cc815abc0 0x7f7cc82001c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:16.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.933+0000 7f7cbfe00640 1 -- 172.21.15.1:0/3751934545 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f7cc8167160 con 0x7f7cc815abc0 2024-05-09T08:03:16.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.933+0000 7f7cbfe00640 1 --2- 172.21.15.1:0/3751934545 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7cc815abc0 0x7f7cc82001c0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f7cb0001eb0 tx=0x7f7cb0007cc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:03:16.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.933+0000 7f7cbea00640 1 -- 172.21.15.1:0/3751934545 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cb0020e50 con 0x7f7cc815abc0 2024-05-09T08:03:16.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.934+0000 7f7cbea00640 1 -- 172.21.15.1:0/3751934545 <== mon.1 v2:172.21.15.7:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5017 (secure 0 0 0) 0x7f7cb0007030 con 0x7f7cc815abc0 2024-05-09T08:03:16.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.934+0000 7f7cbea00640 1 -- 172.21.15.1:0/3751934545 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7cc815abc0 msgr2=0x7f7cc82001c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:16.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.934+0000 7f7cbea00640 1 --2- 172.21.15.1:0/3751934545 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7cc815abc0 0x7f7cc82001c0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f7cb0001eb0 tx=0x7f7cb0007cc0 comp rx=0 tx=0).stop 2024-05-09T08:03:16.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.934+0000 7f7cbea00640 1 -- 172.21.15.1:0/3751934545 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7cc815abc0 msgr2=0x7f7cc82001c0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-05-09T08:03:16.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.934+0000 7f7cbea00640 1 --2- 172.21.15.1:0/3751934545 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7cc815abc0 0x7f7cc82001c0 secure :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f7cb0001eb0 tx=0x7f7cb0007cc0 comp rx=0 tx=0).stop 2024-05-09T08:03:16.952 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-05-09T08:03:16.952 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "b", 2024-05-09T08:03:16.952 INFO:teuthology.orchestra.run.smithi001.stdout: "rank": 1, 2024-05-09T08:03:16.952 INFO:teuthology.orchestra.run.smithi001.stdout: "state": "peon", 2024-05-09T08:03:16.952 INFO:teuthology.orchestra.run.smithi001.stdout: "election_epoch": 4, 2024-05-09T08:03:16.952 INFO:teuthology.orchestra.run.smithi001.stdout: "uptime": 80350, 2024-05-09T08:03:16.952 INFO:teuthology.orchestra.run.smithi001.stdout: "quorum": [ 2024-05-09T08:03:16.952 INFO:teuthology.orchestra.run.smithi001.stdout: 0, 2024-05-09T08:03:16.952 INFO:teuthology.orchestra.run.smithi001.stdout: 1, 2024-05-09T08:03:16.952 INFO:teuthology.orchestra.run.smithi001.stdout: 2 2024-05-09T08:03:16.952 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:03:16.952 INFO:teuthology.orchestra.run.smithi001.stdout: "quorum_age": 80, 2024-05-09T08:03:16.952 INFO:teuthology.orchestra.run.smithi001.stdout: "features": { 2024-05-09T08:03:16.952 INFO:teuthology.orchestra.run.smithi001.stdout: "required_con": "2450521708007866372", 2024-05-09T08:03:16.953 INFO:teuthology.orchestra.run.smithi001.stdout: "required_mon": [ 2024-05-09T08:03:16.953 INFO:teuthology.orchestra.run.smithi001.stdout: "kraken", 2024-05-09T08:03:16.953 INFO:teuthology.orchestra.run.smithi001.stdout: "luminous", 2024-05-09T08:03:16.953 INFO:teuthology.orchestra.run.smithi001.stdout: "mimic", 2024-05-09T08:03:16.953 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap-prune", 2024-05-09T08:03:16.953 INFO:teuthology.orchestra.run.smithi001.stdout: "nautilus", 2024-05-09T08:03:16.953 INFO:teuthology.orchestra.run.smithi001.stdout: "octopus", 2024-05-09T08:03:16.953 INFO:teuthology.orchestra.run.smithi001.stdout: "pacific", 2024-05-09T08:03:16.953 INFO:teuthology.orchestra.run.smithi001.stdout: "elector-pinging", 2024-05-09T08:03:16.953 INFO:teuthology.orchestra.run.smithi001.stdout: "quincy", 2024-05-09T08:03:16.953 INFO:teuthology.orchestra.run.smithi001.stdout: "reef", 2024-05-09T08:03:16.953 INFO:teuthology.orchestra.run.smithi001.stdout: "squid" 2024-05-09T08:03:16.954 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:03:16.954 INFO:teuthology.orchestra.run.smithi001.stdout: "quorum_con": "4540701547738038271", 2024-05-09T08:03:16.954 INFO:teuthology.orchestra.run.smithi001.stdout: "quorum_mon": [ 2024-05-09T08:03:16.954 INFO:teuthology.orchestra.run.smithi001.stdout: "kraken", 2024-05-09T08:03:16.954 INFO:teuthology.orchestra.run.smithi001.stdout: "luminous", 2024-05-09T08:03:16.954 INFO:teuthology.orchestra.run.smithi001.stdout: "mimic", 2024-05-09T08:03:16.954 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap-prune", 2024-05-09T08:03:16.954 INFO:teuthology.orchestra.run.smithi001.stdout: "nautilus", 2024-05-09T08:03:16.954 INFO:teuthology.orchestra.run.smithi001.stdout: "octopus", 2024-05-09T08:03:16.954 INFO:teuthology.orchestra.run.smithi001.stdout: "pacific", 2024-05-09T08:03:16.954 INFO:teuthology.orchestra.run.smithi001.stdout: "elector-pinging", 2024-05-09T08:03:16.955 INFO:teuthology.orchestra.run.smithi001.stdout: "quincy", 2024-05-09T08:03:16.955 INFO:teuthology.orchestra.run.smithi001.stdout: "reef", 2024-05-09T08:03:16.955 INFO:teuthology.orchestra.run.smithi001.stdout: "squid" 2024-05-09T08:03:16.955 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:03:16.955 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:03:16.955 INFO:teuthology.orchestra.run.smithi001.stdout: "outside_quorum": [], 2024-05-09T08:03:16.955 INFO:teuthology.orchestra.run.smithi001.stdout: "extra_probe_peers": [], 2024-05-09T08:03:16.955 INFO:teuthology.orchestra.run.smithi001.stdout: "sync_provider": [], 2024-05-09T08:03:16.955 INFO:teuthology.orchestra.run.smithi001.stdout: "monmap": { 2024-05-09T08:03:16.955 INFO:teuthology.orchestra.run.smithi001.stdout: "epoch": 1, 2024-05-09T08:03:16.955 INFO:teuthology.orchestra.run.smithi001.stdout: "fsid": "54329ec8-3270-4043-9a82-c6cd7420aa7f", 2024-05-09T08:03:16.955 INFO:teuthology.orchestra.run.smithi001.stdout: "modified": "2024-05-09T08:01:21.819270Z", 2024-05-09T08:03:16.955 INFO:teuthology.orchestra.run.smithi001.stdout: "created": "2024-05-09T08:01:21.819270Z", 2024-05-09T08:03:16.955 INFO:teuthology.orchestra.run.smithi001.stdout: "min_mon_release": 19, 2024-05-09T08:03:16.955 INFO:teuthology.orchestra.run.smithi001.stdout: "min_mon_release_name": "squid", 2024-05-09T08:03:16.956 INFO:teuthology.orchestra.run.smithi001.stdout: "election_strategy": 1, 2024-05-09T08:03:16.956 INFO:teuthology.orchestra.run.smithi001.stdout: "disallowed_leaders: ": "", 2024-05-09T08:03:16.956 INFO:teuthology.orchestra.run.smithi001.stdout: "stretch_mode": false, 2024-05-09T08:03:16.956 INFO:teuthology.orchestra.run.smithi001.stdout: "tiebreaker_mon": "", 2024-05-09T08:03:16.956 INFO:teuthology.orchestra.run.smithi001.stdout: "removed_ranks: ": "", 2024-05-09T08:03:16.956 INFO:teuthology.orchestra.run.smithi001.stdout: "features": { 2024-05-09T08:03:16.956 INFO:teuthology.orchestra.run.smithi001.stdout: "persistent": [ 2024-05-09T08:03:16.956 INFO:teuthology.orchestra.run.smithi001.stdout: "kraken", 2024-05-09T08:03:16.956 INFO:teuthology.orchestra.run.smithi001.stdout: "luminous", 2024-05-09T08:03:16.956 INFO:teuthology.orchestra.run.smithi001.stdout: "mimic", 2024-05-09T08:03:16.956 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap-prune", 2024-05-09T08:03:16.956 INFO:teuthology.orchestra.run.smithi001.stdout: "nautilus", 2024-05-09T08:03:16.956 INFO:teuthology.orchestra.run.smithi001.stdout: "octopus", 2024-05-09T08:03:16.956 INFO:teuthology.orchestra.run.smithi001.stdout: "pacific", 2024-05-09T08:03:16.957 INFO:teuthology.orchestra.run.smithi001.stdout: "elector-pinging", 2024-05-09T08:03:16.957 INFO:teuthology.orchestra.run.smithi001.stdout: "quincy", 2024-05-09T08:03:16.957 INFO:teuthology.orchestra.run.smithi001.stdout: "reef", 2024-05-09T08:03:16.957 INFO:teuthology.orchestra.run.smithi001.stdout: "squid" 2024-05-09T08:03:16.957 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:03:16.957 INFO:teuthology.orchestra.run.smithi001.stdout: "optional": [] 2024-05-09T08:03:16.957 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:03:16.957 INFO:teuthology.orchestra.run.smithi001.stdout: "mons": [ 2024-05-09T08:03:16.957 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:03:16.957 INFO:teuthology.orchestra.run.smithi001.stdout: "rank": 0, 2024-05-09T08:03:16.957 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "a", 2024-05-09T08:03:16.957 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-05-09T08:03:16.957 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-05-09T08:03:16.957 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:03:16.957 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-05-09T08:03:16.958 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:3300", 2024-05-09T08:03:16.958 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:03:16.958 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:03:16.958 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:03:16.958 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-05-09T08:03:16.958 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6789", 2024-05-09T08:03:16.958 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:03:16.958 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:03:16.958 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:03:16.958 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:03:16.958 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6789/0", 2024-05-09T08:03:16.959 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.1:6789/0", 2024-05-09T08:03:16.959 INFO:teuthology.orchestra.run.smithi001.stdout: "priority": 0, 2024-05-09T08:03:16.959 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 0, 2024-05-09T08:03:16.959 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_location": "{}" 2024-05-09T08:03:16.959 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:03:16.959 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:03:16.959 INFO:teuthology.orchestra.run.smithi001.stdout: "rank": 1, 2024-05-09T08:03:16.959 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "b", 2024-05-09T08:03:16.959 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-05-09T08:03:16.959 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-05-09T08:03:16.959 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:03:16.959 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-05-09T08:03:16.959 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:3300", 2024-05-09T08:03:16.959 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:03:16.959 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:03:16.960 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:03:16.960 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-05-09T08:03:16.960 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:6789", 2024-05-09T08:03:16.960 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:03:16.960 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:03:16.960 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:03:16.960 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:03:16.960 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:6789/0", 2024-05-09T08:03:16.960 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.7:6789/0", 2024-05-09T08:03:16.960 INFO:teuthology.orchestra.run.smithi001.stdout: "priority": 0, 2024-05-09T08:03:16.960 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 0, 2024-05-09T08:03:16.960 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_location": "{}" 2024-05-09T08:03:16.960 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:03:16.960 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:03:16.961 INFO:teuthology.orchestra.run.smithi001.stdout: "rank": 2, 2024-05-09T08:03:16.961 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "c", 2024-05-09T08:03:16.961 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-05-09T08:03:16.961 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-05-09T08:03:16.961 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:03:16.961 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-05-09T08:03:16.961 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:3301", 2024-05-09T08:03:16.961 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:03:16.961 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:03:16.961 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:03:16.961 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-05-09T08:03:16.961 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:6790", 2024-05-09T08:03:16.962 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:03:16.962 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:03:16.962 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:03:16.962 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:03:16.962 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:6790/0", 2024-05-09T08:03:16.962 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.7:6790/0", 2024-05-09T08:03:16.962 INFO:teuthology.orchestra.run.smithi001.stdout: "priority": 0, 2024-05-09T08:03:16.962 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 0, 2024-05-09T08:03:16.962 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_location": "{}" 2024-05-09T08:03:16.962 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:03:16.962 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:03:16.962 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:03:16.962 INFO:teuthology.orchestra.run.smithi001.stdout: "feature_map": { 2024-05-09T08:03:16.963 INFO:teuthology.orchestra.run.smithi001.stdout: "mon": [ 2024-05-09T08:03:16.963 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:03:16.963 INFO:teuthology.orchestra.run.smithi001.stdout: "features": "0x3f03cffffffdffff", 2024-05-09T08:03:16.963 INFO:teuthology.orchestra.run.smithi001.stdout: "release": "luminous", 2024-05-09T08:03:16.963 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 1 2024-05-09T08:03:16.963 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:03:16.963 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:03:16.963 INFO:teuthology.orchestra.run.smithi001.stdout: "mds": [ 2024-05-09T08:03:16.963 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:03:16.963 INFO:teuthology.orchestra.run.smithi001.stdout: "features": "0x3f03cffffffdffff", 2024-05-09T08:03:16.963 INFO:teuthology.orchestra.run.smithi001.stdout: "release": "luminous", 2024-05-09T08:03:16.963 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 3 2024-05-09T08:03:16.963 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:03:16.963 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:03:16.963 INFO:teuthology.orchestra.run.smithi001.stdout: "client": [ 2024-05-09T08:03:16.963 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:03:16.964 INFO:teuthology.orchestra.run.smithi001.stdout: "features": "0x2f018fb87aa4aafe", 2024-05-09T08:03:16.964 INFO:teuthology.orchestra.run.smithi001.stdout: "release": "luminous", 2024-05-09T08:03:16.964 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 1 2024-05-09T08:03:16.964 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:03:16.964 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:03:16.964 INFO:teuthology.orchestra.run.smithi001.stdout: "features": "0x3f03cffffffdffff", 2024-05-09T08:03:16.964 INFO:teuthology.orchestra.run.smithi001.stdout: "release": "luminous", 2024-05-09T08:03:16.964 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 2 2024-05-09T08:03:16.964 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:03:16.964 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:03:16.964 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr": [ 2024-05-09T08:03:16.964 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:03:16.964 INFO:teuthology.orchestra.run.smithi001.stdout: "features": "0x3f03cffffffdffff", 2024-05-09T08:03:16.964 INFO:teuthology.orchestra.run.smithi001.stdout: "release": "luminous", 2024-05-09T08:03:16.964 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 2 2024-05-09T08:03:16.964 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:03:16.965 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:03:16.965 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:03:16.965 INFO:teuthology.orchestra.run.smithi001.stdout: "stretch_mode": false 2024-05-09T08:03:16.965 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-05-09T08:03:16.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.934+0000 7f7ccd400640 1 -- 172.21.15.1:0/3751934545 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f7ca406be30 msgr2=0x7f7ca406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:16.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.934+0000 7f7ccd400640 1 --2- 172.21.15.1:0/3751934545 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f7ca406be30 0x7f7ca406e2f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f7cc80c66c0 tx=0x7f7cc0009040 comp rx=0 tx=0).stop 2024-05-09T08:03:16.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.934+0000 7f7ccd400640 1 -- 172.21.15.1:0/3751934545 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7cac004d40 msgr2=0x7f7cc8166260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:16.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.934+0000 7f7ccd400640 1 --2- 172.21.15.1:0/3751934545 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7cac004d40 0x7f7cc8166260 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f7cb8009580 tx=0x7f7cb8009ad0 comp rx=0 tx=0).stop 2024-05-09T08:03:16.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.934+0000 7f7cc7200640 1 -- 172.21.15.1:0/3751934545 reap_dead start 2024-05-09T08:03:16.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.934+0000 7f7cc7200640 1 -- 172.21.15.1:0/3751934545 reap_dead start 2024-05-09T08:03:16.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.935+0000 7f7ccd400640 1 -- 172.21.15.1:0/3751934545 shutdown_connections 2024-05-09T08:03:16.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.935+0000 7f7ccd400640 1 -- 172.21.15.1:0/3751934545 >> 172.21.15.1:0/3751934545 conn(0x7f7cc806dad0 msgr2=0x7f7cc8071c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:03:16.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.935+0000 7f7ccd400640 1 -- 172.21.15.1:0/3751934545 shutdown_connections 2024-05-09T08:03:16.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:16.935+0000 7f7ccd400640 1 -- 172.21.15.1:0/3751934545 wait complete. 2024-05-09T08:03:16.970 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-05-09T08:03:16.970 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-05-09T08:03:16.970 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-05-09T08:03:17.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:17.168+0000 7fedf7000640 1 Processor -- start 2024-05-09T08:03:17.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:17.168+0000 7fedf7000640 1 -- start start 2024-05-09T08:03:17.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:17.170+0000 7fedf7000640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fedf015c360 0x7fedf00c9570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:17.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:17.170+0000 7fedf7000640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fedf00c5e40 0x7fedf00cc5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:17.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:17.171+0000 7fedf7000640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fedf0071c80 con 0x7fedf00ccb30 2024-05-09T08:03:17.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:17.171+0000 7fedf7000640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fedf015b980 con 0x7fedf00c5e40 2024-05-09T08:03:17.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:17.171+0000 7fedf7000640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7fedf0071f20 con 0x7fedf015c360 2024-05-09T08:03:17.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:17.171+0000 7fedefe00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fedf015c360 0x7fedf00c9570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:17.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:17.171+0000 7fede7e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fedf00c5e40 0x7fedf00cc5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:17.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:17.171+0000 7fedefe00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fedf015c360 0x7fedf00c9570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3301/0 says I am v2:172.21.15.1:60460/0 (socket says 172.21.15.1:60460) 2024-05-09T08:03:17.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:17.171+0000 7fedefe00640 1 -- 172.21.15.1:0/4174114632 learned_addr learned my addr 172.21.15.1:0/4174114632 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:03:17.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:17.171+0000 7fedefe00640 1 -- 172.21.15.1:0/4174114632 >> v2:172.21.15.7:3300/0 conn(0x7fedf00c5e40 msgr2=0x7fedf00cc5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:03:17.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:17.171+0000 7fedefe00640 1 --2- 172.21.15.1:0/4174114632 >> v2:172.21.15.7:3300/0 conn(0x7fedf00c5e40 0x7fedf00cc5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:17.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:17.171+0000 7fedefe00640 1 -- 172.21.15.1:0/4174114632 >> v1:172.21.15.1:6789/0 conn(0x7fedf00ccb30 legacy=0x7fedf0072c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:17.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:17.171+0000 7fedefe00640 1 -- 172.21.15.1:0/4174114632 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedf00721e0 con 0x7fedf015c360 2024-05-09T08:03:17.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:17.172+0000 7fedefe00640 1 --2- 172.21.15.1:0/4174114632 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fedf015c360 0x7fedf00c9570 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fede0009870 tx=0x7fede002a130 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2d899d79955d63a1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:03:17.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:17.173+0000 7fedef400640 1 -- 172.21.15.1:0/4174114632 <== mon.2 v2:172.21.15.7:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fede0037070 con 0x7fedf015c360 2024-05-09T08:03:17.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:17.173+0000 7fedef400640 1 -- 172.21.15.1:0/4174114632 <== mon.2 v2:172.21.15.7:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fede0032440 con 0x7fedf015c360 2024-05-09T08:03:17.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:17.173+0000 7fedef400640 1 -- 172.21.15.1:0/4174114632 <== mon.2 v2:172.21.15.7:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fede0032810 con 0x7fedf015c360 2024-05-09T08:03:17.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:17.173+0000 7fedf7000640 1 -- 172.21.15.1:0/4174114632 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fedf015c360 msgr2=0x7fedf00c9570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:17.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:17.173+0000 7fedf7000640 1 --2- 172.21.15.1:0/4174114632 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fedf015c360 0x7fedf00c9570 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fede0009870 tx=0x7fede002a130 comp rx=0 tx=0).stop 2024-05-09T08:03:17.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:17.173+0000 7fedf7000640 1 -- 172.21.15.1:0/4174114632 shutdown_connections 2024-05-09T08:03:17.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:17.173+0000 7fedf7000640 1 --2- 172.21.15.1:0/4174114632 >> v2:172.21.15.7:3300/0 conn(0x7fedf00c5e40 0x7fedf00cc5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:17.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:17.173+0000 7fedf7000640 1 --2- 172.21.15.1:0/4174114632 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fedf015c360 0x7fedf00c9570 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:17.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:17.173+0000 7fedf7000640 1 -- 172.21.15.1:0/4174114632 >> 172.21.15.1:0/4174114632 conn(0x7fedf006dad0 msgr2=0x7fedf006deb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:03:17.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:17.173+0000 7fedf7000640 1 -- 172.21.15.1:0/4174114632 shutdown_connections 2024-05-09T08:03:17.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:17.173+0000 7fedf7000640 1 -- 172.21.15.1:0/4174114632 wait complete. 2024-05-09T08:03:17.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:17.175+0000 7fedf7000640 1 Processor -- start 2024-05-09T08:03:17.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:17.175+0000 7fedf7000640 1 -- start start 2024-05-09T08:03:17.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:17.176+0000 7fedf7000640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fedf00c5e40 0x7fedf0140bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:17.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:17.176+0000 7fedf7000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fedf00ccb30 0x7fedf00c1820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:17.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:17.176+0000 7fedf7000640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fedf015c360 0x7fedf0146090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:17.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:17.176+0000 7fedf7000640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fedf00cc0d0 con 0x7fedf00c5e40 2024-05-09T08:03:17.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:17.176+0000 7fedf7000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fedf00727a0 con 0x7fedf00ccb30 2024-05-09T08:03:17.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:17.176+0000 7fedf7000640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7fedf015c740 con 0x7fedf015c360 2024-05-09T08:03:17.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:17.177+0000 7fedefe00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fedf00c5e40 0x7fedf0140bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:17.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:17.177+0000 7fedefe00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fedf00c5e40 0x7fedf0140bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54712/0 (socket says 172.21.15.1:54712) 2024-05-09T08:03:17.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:17.177+0000 7fedefe00640 1 -- 172.21.15.1:0/2318646701 learned_addr learned my addr 172.21.15.1:0/2318646701 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:03:17.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:17.177+0000 7fede7e00640 1 --2- 172.21.15.1:0/2318646701 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fedf00ccb30 0x7fedf00c1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:17.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:17.177+0000 7fedf5000640 1 --2- 172.21.15.1:0/2318646701 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fedf015c360 0x7fedf0146090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:17.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:17.177+0000 7fedefe00640 1 -- 172.21.15.1:0/2318646701 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fedf015c360 msgr2=0x7fedf0146090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:17.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:17.177+0000 7fedefe00640 1 --2- 172.21.15.1:0/2318646701 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fedf015c360 0x7fedf0146090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:17.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:17.177+0000 7fedefe00640 1 -- 172.21.15.1:0/2318646701 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fedf00ccb30 msgr2=0x7fedf00c1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:17.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:17.177+0000 7fedefe00640 1 --2- 172.21.15.1:0/2318646701 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fedf00ccb30 0x7fedf00c1820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:17.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:17.177+0000 7fedefe00640 1 -- 172.21.15.1:0/2318646701 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedf00c1340 con 0x7fedf00c5e40 2024-05-09T08:03:17.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:17.177+0000 7fedefe00640 1 --2- 172.21.15.1:0/2318646701 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fedf00c5e40 0x7fedf0140bb0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fede002a170 tx=0x7fede00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:03:17.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:17.178+0000 7fedecc00640 1 -- 172.21.15.1:0/2318646701 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fede0041030 con 0x7fedf00c5e40 2024-05-09T08:03:17.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:17.178+0000 7fedecc00640 1 -- 172.21.15.1:0/2318646701 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fede0030440 con 0x7fedf00c5e40 2024-05-09T08:03:17.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:17.178+0000 7fedecc00640 1 -- 172.21.15.1:0/2318646701 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fede0037040 con 0x7fedf00c5e40 2024-05-09T08:03:17.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:17.178+0000 7fedf7000640 1 -- 172.21.15.1:0/2318646701 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fedf00ca030 con 0x7fedf00c5e40 2024-05-09T08:03:17.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:17.178+0000 7fedf7000640 1 -- 172.21.15.1:0/2318646701 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fedf00c76a0 con 0x7fedf00c5e40 2024-05-09T08:03:17.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:17.179+0000 7fedf7000640 1 --2- 172.21.15.1:0/2318646701 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fedf00cba00 0x7fedf01465d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:17.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:17.179+0000 7fedf7000640 1 -- 172.21.15.1:0/2318646701 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7fedf00c9e00 con 0x7fedf00cba00 2024-05-09T08:03:17.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:17.179+0000 7fede7e00640 1 --2- 172.21.15.1:0/2318646701 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fedf00cba00 0x7fedf01465d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:17.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:17.179+0000 7fede7e00640 1 -- 172.21.15.1:0/2318646701 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fedf00c1510 con 0x7fedf00cba00 2024-05-09T08:03:17.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:17.180+0000 7fede7e00640 1 --2- 172.21.15.1:0/2318646701 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fedf00cba00 0x7fedf01465d0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fedf00b1b10 tx=0x7fedd8011cf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:03:17.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:17.180+0000 7fedecc00640 1 -- 172.21.15.1:0/2318646701 <== mon.2 v2:172.21.15.7:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedd8019070 con 0x7fedf00cba00 2024-05-09T08:03:17.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:17.180+0000 7fedecc00640 1 -- 172.21.15.1:0/2318646701 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fede0044070 con 0x7fedf00c5e40 2024-05-09T08:03:17.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:17.181+0000 7fedecc00640 1 --2- 172.21.15.1:0/2318646701 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fedcc06bd60 0x7fedcc06e220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:17.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:17.181+0000 7fedecc00640 1 -- 172.21.15.1:0/2318646701 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7fede00ab4c0 con 0x7fedf00c5e40 2024-05-09T08:03:17.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:17.181+0000 7fedecc00640 1 -- 172.21.15.1:0/2318646701 <== mon.2 v2:172.21.15.7:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fedd8004240 con 0x7fedf00cba00 2024-05-09T08:03:17.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:17.181+0000 7fedecc00640 1 -- 172.21.15.1:0/2318646701 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fedf00cba00 msgr2=0x7fedf01465d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:17.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:17.181+0000 7fedecc00640 1 --2- 172.21.15.1:0/2318646701 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fedf00cba00 0x7fedf01465d0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fedf00b1b10 tx=0x7fedd8011cf0 comp rx=0 tx=0).stop 2024-05-09T08:03:17.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:17.181+0000 7fedecc00640 1 -- 172.21.15.1:0/2318646701 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fedf00cba00 msgr2=0x7fedf01465d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-05-09T08:03:17.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:17.181+0000 7fedf5000640 1 --2- 172.21.15.1:0/2318646701 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fedcc06bd60 0x7fedcc06e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:17.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:17.181+0000 7fedecc00640 1 --2- 172.21.15.1:0/2318646701 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fedf00cba00 0x7fedf01465d0 secure :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fedf00b1b10 tx=0x7fedd8011cf0 comp rx=0 tx=0).stop 2024-05-09T08:03:17.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:17.182+0000 7fedf5000640 1 --2- 172.21.15.1:0/2318646701 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fedcc06bd60 0x7fedcc06e220 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fede80012e0 tx=0x7fede8009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:03:17.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:17.189+0000 7fedf7000640 1 --2- 172.21.15.1:0/2318646701 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fedf00c78e0 0x7fedf01f3420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:17.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:17.189+0000 7fedf7000640 1 -- 172.21.15.1:0/2318646701 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7fedf00c7cf0 con 0x7fedf00c78e0 2024-05-09T08:03:17.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:17.189+0000 7fede7e00640 1 --2- 172.21.15.1:0/2318646701 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fedf00c78e0 0x7fedf01f3420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:17.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:17.189+0000 7fede7e00640 1 -- 172.21.15.1:0/2318646701 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fedf00684b0 con 0x7fedf00c78e0 2024-05-09T08:03:17.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:17.190+0000 7fede7e00640 1 --2- 172.21.15.1:0/2318646701 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fedf00c78e0 0x7fedf01f3420 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fedf00c2600 tx=0x7fedd80046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:03:17.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:17.190+0000 7fedecc00640 1 -- 172.21.15.1:0/2318646701 <== mon.2 v2:172.21.15.7:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedd8003d10 con 0x7fedf00c78e0 2024-05-09T08:03:17.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:17.190+0000 7fedecc00640 1 -- 172.21.15.1:0/2318646701 <== mon.2 v2:172.21.15.7:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4874 (secure 0 0 0) 0x7fedd8003cf0 con 0x7fedf00c78e0 2024-05-09T08:03:17.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:17.190+0000 7fedecc00640 1 -- 172.21.15.1:0/2318646701 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fedf00c78e0 msgr2=0x7fedf01f3420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:17.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:17.190+0000 7fedecc00640 1 --2- 172.21.15.1:0/2318646701 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fedf00c78e0 0x7fedf01f3420 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fedf00c2600 tx=0x7fedd80046c0 comp rx=0 tx=0).stop 2024-05-09T08:03:17.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:17.190+0000 7fedecc00640 1 -- 172.21.15.1:0/2318646701 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fedf00c78e0 msgr2=0x7fedf01f3420 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-05-09T08:03:17.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:17.190+0000 7fedecc00640 1 --2- 172.21.15.1:0/2318646701 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fedf00c78e0 0x7fedf01f3420 secure :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fedf00c2600 tx=0x7fedd80046c0 comp rx=0 tx=0).stop 2024-05-09T08:03:17.208 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-05-09T08:03:17.208 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "c", 2024-05-09T08:03:17.208 INFO:teuthology.orchestra.run.smithi001.stdout: "rank": 2, 2024-05-09T08:03:17.208 INFO:teuthology.orchestra.run.smithi001.stdout: "state": "peon", 2024-05-09T08:03:17.209 INFO:teuthology.orchestra.run.smithi001.stdout: "election_epoch": 4, 2024-05-09T08:03:17.209 INFO:teuthology.orchestra.run.smithi001.stdout: "uptime": 80606, 2024-05-09T08:03:17.209 INFO:teuthology.orchestra.run.smithi001.stdout: "quorum": [ 2024-05-09T08:03:17.209 INFO:teuthology.orchestra.run.smithi001.stdout: 0, 2024-05-09T08:03:17.209 INFO:teuthology.orchestra.run.smithi001.stdout: 1, 2024-05-09T08:03:17.209 INFO:teuthology.orchestra.run.smithi001.stdout: 2 2024-05-09T08:03:17.209 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:03:17.209 INFO:teuthology.orchestra.run.smithi001.stdout: "quorum_age": 80, 2024-05-09T08:03:17.209 INFO:teuthology.orchestra.run.smithi001.stdout: "features": { 2024-05-09T08:03:17.209 INFO:teuthology.orchestra.run.smithi001.stdout: "required_con": "2450521708007866372", 2024-05-09T08:03:17.209 INFO:teuthology.orchestra.run.smithi001.stdout: "required_mon": [ 2024-05-09T08:03:17.209 INFO:teuthology.orchestra.run.smithi001.stdout: "kraken", 2024-05-09T08:03:17.209 INFO:teuthology.orchestra.run.smithi001.stdout: "luminous", 2024-05-09T08:03:17.209 INFO:teuthology.orchestra.run.smithi001.stdout: "mimic", 2024-05-09T08:03:17.209 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap-prune", 2024-05-09T08:03:17.210 INFO:teuthology.orchestra.run.smithi001.stdout: "nautilus", 2024-05-09T08:03:17.210 INFO:teuthology.orchestra.run.smithi001.stdout: "octopus", 2024-05-09T08:03:17.210 INFO:teuthology.orchestra.run.smithi001.stdout: "pacific", 2024-05-09T08:03:17.210 INFO:teuthology.orchestra.run.smithi001.stdout: "elector-pinging", 2024-05-09T08:03:17.210 INFO:teuthology.orchestra.run.smithi001.stdout: "quincy", 2024-05-09T08:03:17.210 INFO:teuthology.orchestra.run.smithi001.stdout: "reef", 2024-05-09T08:03:17.210 INFO:teuthology.orchestra.run.smithi001.stdout: "squid" 2024-05-09T08:03:17.210 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:03:17.210 INFO:teuthology.orchestra.run.smithi001.stdout: "quorum_con": "4540701547738038271", 2024-05-09T08:03:17.210 INFO:teuthology.orchestra.run.smithi001.stdout: "quorum_mon": [ 2024-05-09T08:03:17.210 INFO:teuthology.orchestra.run.smithi001.stdout: "kraken", 2024-05-09T08:03:17.210 INFO:teuthology.orchestra.run.smithi001.stdout: "luminous", 2024-05-09T08:03:17.210 INFO:teuthology.orchestra.run.smithi001.stdout: "mimic", 2024-05-09T08:03:17.210 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap-prune", 2024-05-09T08:03:17.210 INFO:teuthology.orchestra.run.smithi001.stdout: "nautilus", 2024-05-09T08:03:17.211 INFO:teuthology.orchestra.run.smithi001.stdout: "octopus", 2024-05-09T08:03:17.211 INFO:teuthology.orchestra.run.smithi001.stdout: "pacific", 2024-05-09T08:03:17.211 INFO:teuthology.orchestra.run.smithi001.stdout: "elector-pinging", 2024-05-09T08:03:17.211 INFO:teuthology.orchestra.run.smithi001.stdout: "quincy", 2024-05-09T08:03:17.211 INFO:teuthology.orchestra.run.smithi001.stdout: "reef", 2024-05-09T08:03:17.211 INFO:teuthology.orchestra.run.smithi001.stdout: "squid" 2024-05-09T08:03:17.211 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:03:17.211 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:03:17.211 INFO:teuthology.orchestra.run.smithi001.stdout: "outside_quorum": [], 2024-05-09T08:03:17.211 INFO:teuthology.orchestra.run.smithi001.stdout: "extra_probe_peers": [], 2024-05-09T08:03:17.211 INFO:teuthology.orchestra.run.smithi001.stdout: "sync_provider": [], 2024-05-09T08:03:17.211 INFO:teuthology.orchestra.run.smithi001.stdout: "monmap": { 2024-05-09T08:03:17.211 INFO:teuthology.orchestra.run.smithi001.stdout: "epoch": 1, 2024-05-09T08:03:17.211 INFO:teuthology.orchestra.run.smithi001.stdout: "fsid": "54329ec8-3270-4043-9a82-c6cd7420aa7f", 2024-05-09T08:03:17.211 INFO:teuthology.orchestra.run.smithi001.stdout: "modified": "2024-05-09T08:01:21.819270Z", 2024-05-09T08:03:17.211 INFO:teuthology.orchestra.run.smithi001.stdout: "created": "2024-05-09T08:01:21.819270Z", 2024-05-09T08:03:17.212 INFO:teuthology.orchestra.run.smithi001.stdout: "min_mon_release": 19, 2024-05-09T08:03:17.212 INFO:teuthology.orchestra.run.smithi001.stdout: "min_mon_release_name": "squid", 2024-05-09T08:03:17.212 INFO:teuthology.orchestra.run.smithi001.stdout: "election_strategy": 1, 2024-05-09T08:03:17.212 INFO:teuthology.orchestra.run.smithi001.stdout: "disallowed_leaders: ": "", 2024-05-09T08:03:17.212 INFO:teuthology.orchestra.run.smithi001.stdout: "stretch_mode": false, 2024-05-09T08:03:17.212 INFO:teuthology.orchestra.run.smithi001.stdout: "tiebreaker_mon": "", 2024-05-09T08:03:17.212 INFO:teuthology.orchestra.run.smithi001.stdout: "removed_ranks: ": "", 2024-05-09T08:03:17.212 INFO:teuthology.orchestra.run.smithi001.stdout: "features": { 2024-05-09T08:03:17.212 INFO:teuthology.orchestra.run.smithi001.stdout: "persistent": [ 2024-05-09T08:03:17.212 INFO:teuthology.orchestra.run.smithi001.stdout: "kraken", 2024-05-09T08:03:17.212 INFO:teuthology.orchestra.run.smithi001.stdout: "luminous", 2024-05-09T08:03:17.212 INFO:teuthology.orchestra.run.smithi001.stdout: "mimic", 2024-05-09T08:03:17.212 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap-prune", 2024-05-09T08:03:17.212 INFO:teuthology.orchestra.run.smithi001.stdout: "nautilus", 2024-05-09T08:03:17.213 INFO:teuthology.orchestra.run.smithi001.stdout: "octopus", 2024-05-09T08:03:17.213 INFO:teuthology.orchestra.run.smithi001.stdout: "pacific", 2024-05-09T08:03:17.213 INFO:teuthology.orchestra.run.smithi001.stdout: "elector-pinging", 2024-05-09T08:03:17.213 INFO:teuthology.orchestra.run.smithi001.stdout: "quincy", 2024-05-09T08:03:17.213 INFO:teuthology.orchestra.run.smithi001.stdout: "reef", 2024-05-09T08:03:17.213 INFO:teuthology.orchestra.run.smithi001.stdout: "squid" 2024-05-09T08:03:17.213 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:03:17.213 INFO:teuthology.orchestra.run.smithi001.stdout: "optional": [] 2024-05-09T08:03:17.213 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:03:17.213 INFO:teuthology.orchestra.run.smithi001.stdout: "mons": [ 2024-05-09T08:03:17.213 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:03:17.213 INFO:teuthology.orchestra.run.smithi001.stdout: "rank": 0, 2024-05-09T08:03:17.213 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "a", 2024-05-09T08:03:17.213 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-05-09T08:03:17.214 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-05-09T08:03:17.214 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:03:17.214 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-05-09T08:03:17.214 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:3300", 2024-05-09T08:03:17.214 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:03:17.214 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:03:17.214 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:03:17.214 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-05-09T08:03:17.214 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6789", 2024-05-09T08:03:17.214 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:03:17.214 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:03:17.214 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:03:17.214 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:03:17.215 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6789/0", 2024-05-09T08:03:17.215 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.1:6789/0", 2024-05-09T08:03:17.215 INFO:teuthology.orchestra.run.smithi001.stdout: "priority": 0, 2024-05-09T08:03:17.215 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 0, 2024-05-09T08:03:17.215 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_location": "{}" 2024-05-09T08:03:17.215 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:03:17.215 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:03:17.215 INFO:teuthology.orchestra.run.smithi001.stdout: "rank": 1, 2024-05-09T08:03:17.215 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "b", 2024-05-09T08:03:17.215 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-05-09T08:03:17.215 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-05-09T08:03:17.215 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:03:17.215 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-05-09T08:03:17.215 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:3300", 2024-05-09T08:03:17.216 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:03:17.216 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:03:17.216 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:03:17.216 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-05-09T08:03:17.216 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:6789", 2024-05-09T08:03:17.216 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:03:17.216 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:03:17.216 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:03:17.216 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:03:17.216 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:6789/0", 2024-05-09T08:03:17.216 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.7:6789/0", 2024-05-09T08:03:17.216 INFO:teuthology.orchestra.run.smithi001.stdout: "priority": 0, 2024-05-09T08:03:17.216 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 0, 2024-05-09T08:03:17.216 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_location": "{}" 2024-05-09T08:03:17.216 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:03:17.217 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:03:17.217 INFO:teuthology.orchestra.run.smithi001.stdout: "rank": 2, 2024-05-09T08:03:17.217 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "c", 2024-05-09T08:03:17.217 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-05-09T08:03:17.217 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-05-09T08:03:17.217 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:03:17.217 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-05-09T08:03:17.217 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:3301", 2024-05-09T08:03:17.217 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:03:17.217 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:03:17.218 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:03:17.218 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-05-09T08:03:17.218 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:6790", 2024-05-09T08:03:17.218 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:03:17.218 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:03:17.218 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:03:17.218 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:03:17.218 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:6790/0", 2024-05-09T08:03:17.218 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.7:6790/0", 2024-05-09T08:03:17.218 INFO:teuthology.orchestra.run.smithi001.stdout: "priority": 0, 2024-05-09T08:03:17.218 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 0, 2024-05-09T08:03:17.218 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_location": "{}" 2024-05-09T08:03:17.219 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:03:17.219 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:03:17.219 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:03:17.219 INFO:teuthology.orchestra.run.smithi001.stdout: "feature_map": { 2024-05-09T08:03:17.219 INFO:teuthology.orchestra.run.smithi001.stdout: "mon": [ 2024-05-09T08:03:17.219 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:03:17.219 INFO:teuthology.orchestra.run.smithi001.stdout: "features": "0x3f03cffffffdffff", 2024-05-09T08:03:17.219 INFO:teuthology.orchestra.run.smithi001.stdout: "release": "luminous", 2024-05-09T08:03:17.219 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 1 2024-05-09T08:03:17.219 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:03:17.219 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:03:17.219 INFO:teuthology.orchestra.run.smithi001.stdout: "mds": [ 2024-05-09T08:03:17.219 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:03:17.219 INFO:teuthology.orchestra.run.smithi001.stdout: "features": "0x3f03cffffffdffff", 2024-05-09T08:03:17.220 INFO:teuthology.orchestra.run.smithi001.stdout: "release": "luminous", 2024-05-09T08:03:17.220 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 3 2024-05-09T08:03:17.220 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:03:17.220 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:03:17.220 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": [ 2024-05-09T08:03:17.220 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:03:17.220 INFO:teuthology.orchestra.run.smithi001.stdout: "features": "0x3f03cffffffdffff", 2024-05-09T08:03:17.220 INFO:teuthology.orchestra.run.smithi001.stdout: "release": "luminous", 2024-05-09T08:03:17.220 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 6 2024-05-09T08:03:17.220 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:03:17.220 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:03:17.220 INFO:teuthology.orchestra.run.smithi001.stdout: "client": [ 2024-05-09T08:03:17.220 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:03:17.221 INFO:teuthology.orchestra.run.smithi001.stdout: "features": "0x3f03cffffffdffff", 2024-05-09T08:03:17.221 INFO:teuthology.orchestra.run.smithi001.stdout: "release": "luminous", 2024-05-09T08:03:17.221 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 4 2024-05-09T08:03:17.221 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:03:17.221 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:03:17.221 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:03:17.221 INFO:teuthology.orchestra.run.smithi001.stdout: "stretch_mode": false 2024-05-09T08:03:17.221 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-05-09T08:03:17.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:17.191+0000 7fedf7000640 1 -- 172.21.15.1:0/2318646701 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fedcc06bd60 msgr2=0x7fedcc06e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:17.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:17.191+0000 7fedf7000640 1 --2- 172.21.15.1:0/2318646701 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fedcc06bd60 0x7fedcc06e220 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fede80012e0 tx=0x7fede8009040 comp rx=0 tx=0).stop 2024-05-09T08:03:17.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:17.191+0000 7fedf7000640 1 -- 172.21.15.1:0/2318646701 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fedf00c5e40 msgr2=0x7fedf0140bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:17.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:17.191+0000 7fedf7000640 1 --2- 172.21.15.1:0/2318646701 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fedf00c5e40 0x7fedf0140bb0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fede002a170 tx=0x7fede00046c0 comp rx=0 tx=0).stop 2024-05-09T08:03:17.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:17.191+0000 7fedf5000640 1 -- 172.21.15.1:0/2318646701 reap_dead start 2024-05-09T08:03:17.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:17.191+0000 7fedf5000640 1 -- 172.21.15.1:0/2318646701 reap_dead start 2024-05-09T08:03:17.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:17.191+0000 7fedf7000640 1 -- 172.21.15.1:0/2318646701 shutdown_connections 2024-05-09T08:03:17.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:17.191+0000 7fedf7000640 1 -- 172.21.15.1:0/2318646701 >> 172.21.15.1:0/2318646701 conn(0x7fedf006dad0 msgr2=0x7fedf006deb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:03:17.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:17.191+0000 7fedf7000640 1 -- 172.21.15.1:0/2318646701 shutdown_connections 2024-05-09T08:03:17.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:17.191+0000 7fedf7000640 1 -- 172.21.15.1:0/2318646701 wait complete. 2024-05-09T08:03:17.228 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-05-09T08:03:17.228 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['b'] 2024-05-09T08:03:17.228 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2024-05-09T08:03:17.228 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.b 2024-05-09T08:03:17.229 INFO:tasks.mon_thrash.mon_thrasher:killing mon.b 2024-05-09T08:03:17.229 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-05-09T08:03:17.229 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T08:03:17.245 INFO:tasks.ceph.mon.b:Stopped 2024-05-09T08:03:17.246 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2024-05-09T08:03:17.246 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-05-09T08:03:17.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:17.442+0000 7f68aca00640 1 Processor -- start 2024-05-09T08:03:17.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:17.443+0000 7f68aca00640 1 -- start start 2024-05-09T08:03:17.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:17.444+0000 7f68aca00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f68a8158cf0 0x7f68a8162850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:17.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:17.444+0000 7f68aca00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f68a815f1f0 0x7f68a806b250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:17.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:17.444+0000 7f68aca00640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f68a815f610 0x7f68a806b790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:17.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:17.444+0000 7f68aca00640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f68a815ed60 con 0x7f68a815f610 2024-05-09T08:03:17.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:17.444+0000 7f68aca00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f68a80c8210 con 0x7f68a8158cf0 2024-05-09T08:03:17.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:17.444+0000 7f68aca00640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f68a806bfd0 con 0x7f68a815f1f0 2024-05-09T08:03:17.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:17.444+0000 7f68a6800640 1 -- >> v2:172.21.15.7:3300/0 conn(0x7f68a8158cf0 msgr2=0x7f68a8162850 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.7:3300/0 2024-05-09T08:03:17.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:17.445+0000 7f68a6800640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f68a8158cf0 0x7f68a8162850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T08:03:17.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:17.445+0000 7f68a7200640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f68a815f610 0x7f68a806b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:17.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:17.445+0000 7f68a7200640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f68a815f610 0x7f68a806b790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54738/0 (socket says 172.21.15.1:54738) 2024-05-09T08:03:17.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:17.445+0000 7f68a7200640 1 -- 172.21.15.1:0/330568985 learned_addr learned my addr 172.21.15.1:0/330568985 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:03:17.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:17.445+0000 7f68a5e00640 1 --2- 172.21.15.1:0/330568985 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f68a815f1f0 0x7f68a806b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:17.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:17.445+0000 7f68a7200640 1 -- 172.21.15.1:0/330568985 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f68a815f1f0 msgr2=0x7f68a806b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:03:17.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:17.445+0000 7f68a7200640 1 --2- 172.21.15.1:0/330568985 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f68a815f1f0 0x7f68a806b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:17.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:17.445+0000 7f68a7200640 1 -- 172.21.15.1:0/330568985 >> v2:172.21.15.7:3300/0 conn(0x7f68a8158cf0 msgr2=0x7f68a8162850 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T08:03:17.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:17.445+0000 7f68a7200640 1 --2- 172.21.15.1:0/330568985 >> v2:172.21.15.7:3300/0 conn(0x7f68a8158cf0 0x7f68a8162850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:17.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:17.445+0000 7f68a7200640 1 -- 172.21.15.1:0/330568985 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68a8162d90 con 0x7f68a815f610 2024-05-09T08:03:17.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:17.447+0000 7f68a7200640 1 --2- 172.21.15.1:0/330568985 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f68a815f610 0x7f68a806b790 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f68a8071230 tx=0x7f68a002a290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a89f9ad46a7581f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:03:17.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:17.447+0000 7f68a5400640 1 -- 172.21.15.1:0/330568985 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68a0037070 con 0x7f68a815f610 2024-05-09T08:03:17.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:17.447+0000 7f68a5400640 1 -- 172.21.15.1:0/330568985 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68a0032440 con 0x7f68a815f610 2024-05-09T08:03:17.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:17.447+0000 7f68a5400640 1 -- 172.21.15.1:0/330568985 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68a00327f0 con 0x7f68a815f610 2024-05-09T08:03:17.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:17.447+0000 7f68aca00640 1 -- 172.21.15.1:0/330568985 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f68a815f610 msgr2=0x7f68a806b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:17.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:17.447+0000 7f68aca00640 1 --2- 172.21.15.1:0/330568985 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f68a815f610 0x7f68a806b790 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f68a8071230 tx=0x7f68a002a290 comp rx=0 tx=0).stop 2024-05-09T08:03:17.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:17.448+0000 7f68aca00640 1 -- 172.21.15.1:0/330568985 shutdown_connections 2024-05-09T08:03:17.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:17.448+0000 7f68aca00640 1 --2- 172.21.15.1:0/330568985 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f68a815f1f0 0x7f68a806b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:17.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:17.448+0000 7f68aca00640 1 --2- 172.21.15.1:0/330568985 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f68a815f610 0x7f68a806b790 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:17.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:17.448+0000 7f68aca00640 1 --2- 172.21.15.1:0/330568985 >> v2:172.21.15.7:3300/0 conn(0x7f68a8158cf0 0x7f68a8162850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:17.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:17.448+0000 7f68aca00640 1 -- 172.21.15.1:0/330568985 >> 172.21.15.1:0/330568985 conn(0x7f68a80cbc10 msgr2=0x7f68a80cc020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:03:17.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:17.448+0000 7f68aca00640 1 -- 172.21.15.1:0/330568985 shutdown_connections 2024-05-09T08:03:17.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:17.448+0000 7f68aca00640 1 -- 172.21.15.1:0/330568985 wait complete. 2024-05-09T08:03:17.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:17.450+0000 7f68aca00640 1 Processor -- start 2024-05-09T08:03:17.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:17.450+0000 7f68aca00640 1 -- start start 2024-05-09T08:03:17.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:17.451+0000 7f68aca00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f68a8158cf0 0x7f68a81f89c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:17.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:17.451+0000 7f68aca00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f68a815f1f0 0x7f68a814b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:17.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:17.451+0000 7f68aca00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68a815f610 0x7f68a81f5110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:17.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:17.451+0000 7f68aca00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f68a80c7c50 con 0x7f68a8158cf0 2024-05-09T08:03:17.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:17.451+0000 7f68aca00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f68a80c7970 con 0x7f68a815f610 2024-05-09T08:03:17.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:17.451+0000 7f68aca00640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f68a80c7f30 con 0x7f68a815f1f0 2024-05-09T08:03:17.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:17.451+0000 7f68a6800640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f68a8158cf0 0x7f68a81f89c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:17.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:17.451+0000 7f68a7200640 1 -- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68a815f610 msgr2=0x7f68a81f5110 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.7:3300/0 2024-05-09T08:03:17.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:17.451+0000 7f68a6800640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f68a8158cf0 0x7f68a81f89c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54754/0 (socket says 172.21.15.1:54754) 2024-05-09T08:03:17.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:17.451+0000 7f68a6800640 1 -- 172.21.15.1:0/4161067381 learned_addr learned my addr 172.21.15.1:0/4161067381 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:03:17.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:17.451+0000 7f68a7200640 1 --2- 172.21.15.1:0/4161067381 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68a815f610 0x7f68a81f5110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T08:03:17.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:17.451+0000 7f68a5e00640 1 --2- 172.21.15.1:0/4161067381 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f68a815f1f0 0x7f68a814b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:17.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:17.451+0000 7f68a6800640 1 -- 172.21.15.1:0/4161067381 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f68a815f1f0 msgr2=0x7f68a814b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:17.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:17.451+0000 7f68a6800640 1 --2- 172.21.15.1:0/4161067381 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f68a815f1f0 0x7f68a814b1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:17.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:17.451+0000 7f68a6800640 1 -- 172.21.15.1:0/4161067381 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68a815f610 msgr2=0x7f68a81f5110 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T08:03:17.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:17.451+0000 7f68a6800640 1 --2- 172.21.15.1:0/4161067381 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68a815f610 0x7f68a81f5110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:17.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:17.451+0000 7f68a6800640 1 -- 172.21.15.1:0/4161067381 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68a81f4c30 con 0x7f68a8158cf0 2024-05-09T08:03:17.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:17.452+0000 7f68a6800640 1 --2- 172.21.15.1:0/4161067381 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f68a8158cf0 0x7f68a81f89c0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f689800b9e0 tx=0x7f689800bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:03:17.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:17.452+0000 7f689ea00640 1 -- 172.21.15.1:0/4161067381 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f689800cba0 con 0x7f68a8158cf0 2024-05-09T08:03:17.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:17.452+0000 7f689ea00640 1 -- 172.21.15.1:0/4161067381 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6898010070 con 0x7f68a8158cf0 2024-05-09T08:03:17.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:17.452+0000 7f689ea00640 1 -- 172.21.15.1:0/4161067381 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6898015470 con 0x7f68a8158cf0 2024-05-09T08:03:17.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:17.452+0000 7f68aca00640 1 -- 172.21.15.1:0/4161067381 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68a8150160 con 0x7f68a8158cf0 2024-05-09T08:03:17.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:17.452+0000 7f68aca00640 1 -- 172.21.15.1:0/4161067381 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68a8150a70 con 0x7f68a8158cf0 2024-05-09T08:03:17.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:17.453+0000 7f68aca00640 1 -- 172.21.15.1:0/4161067381 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68a806e8b0 con 0x7f68a8158cf0 2024-05-09T08:03:17.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:17.454+0000 7f689ea00640 1 -- 172.21.15.1:0/4161067381 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f689801e3f0 con 0x7f68a8158cf0 2024-05-09T08:03:17.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:17.455+0000 7f689ea00640 1 --2- 172.21.15.1:0/4161067381 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f688806bde0 0x7f688806e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:17.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:17.456+0000 7f689ea00640 1 -- 172.21.15.1:0/4161067381 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7f689808b470 con 0x7f68a8158cf0 2024-05-09T08:03:17.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:17.456+0000 7f68a5e00640 1 --2- 172.21.15.1:0/4161067381 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f688806bde0 0x7f688806e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:17.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:17.456+0000 7f68a5e00640 1 --2- 172.21.15.1:0/4161067381 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f688806bde0 0x7f688806e2a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f68a8162830 tx=0x7f6894009190 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:03:17.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:17.460+0000 7f689ea00640 1 -- 172.21.15.1:0/4161067381 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f68a806e8b0 con 0x7f68a8158cf0 2024-05-09T08:03:17.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:17.744+0000 7f68aca00640 1 -- 172.21.15.1:0/4161067381 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f68a8072000 con 0x7f68a8158cf0 2024-05-09T08:03:17.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:17.744+0000 7f689ea00640 1 -- 172.21.15.1:0/4161067381 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1544 (secure 0 0 0) 0x7f68a8072000 con 0x7f68a8158cf0 2024-05-09T08:03:17.762 INFO:teuthology.orchestra.run.smithi001.stdout:{"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":81,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"54329ec8-3270-4043-9a82-c6cd7420aa7f","modified":"2024-05-09T08:01:21.819270Z","created":"2024-05-09T08:01:21.819270Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:3300","nonce":0},{"type":"v1","addr":"172.21.15.1:6789","nonce":0}]},"addr":"172.21.15.1:6789/0","public_addr":"172.21.15.1:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:3300","nonce":0},{"type":"v1","addr":"172.21.15.7:6789","nonce":0}]},"addr":"172.21.15.7:6789/0","public_addr":"172.21.15.7:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:3301","nonce":0},{"type":"v1","addr":"172.21.15.7:6790","nonce":0}]},"addr":"172.21.15.7:6790/0","public_addr":"172.21.15.7:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-05-09T08:03:17.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:17.747+0000 7f68aca00640 1 -- 172.21.15.1:0/4161067381 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f688806bde0 msgr2=0x7f688806e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:17.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:17.747+0000 7f68aca00640 1 --2- 172.21.15.1:0/4161067381 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f688806bde0 0x7f688806e2a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f68a8162830 tx=0x7f6894009190 comp rx=0 tx=0).stop 2024-05-09T08:03:17.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:17.747+0000 7f68aca00640 1 -- 172.21.15.1:0/4161067381 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f68a8158cf0 msgr2=0x7f68a81f89c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:17.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:17.747+0000 7f68aca00640 1 --2- 172.21.15.1:0/4161067381 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f68a8158cf0 0x7f68a81f89c0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f689800b9e0 tx=0x7f689800bea0 comp rx=0 tx=0).stop 2024-05-09T08:03:17.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:17.747+0000 7f68aca00640 1 -- 172.21.15.1:0/4161067381 shutdown_connections 2024-05-09T08:03:17.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:17.747+0000 7f68aca00640 1 --2- 172.21.15.1:0/4161067381 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f688806bde0 0x7f688806e2a0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:17.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:17.747+0000 7f68aca00640 1 --2- 172.21.15.1:0/4161067381 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68a815f610 0x7f68a81f5110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:17.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:17.747+0000 7f68aca00640 1 --2- 172.21.15.1:0/4161067381 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f68a815f1f0 0x7f68a814b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:17.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:17.747+0000 7f68aca00640 1 --2- 172.21.15.1:0/4161067381 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f68a8158cf0 0x7f68a81f89c0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:17.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:17.747+0000 7f68aca00640 1 -- 172.21.15.1:0/4161067381 >> 172.21.15.1:0/4161067381 conn(0x7f68a80cbc10 msgr2=0x7f68a80c4640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:03:17.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:17.748+0000 7f68aca00640 1 -- 172.21.15.1:0/4161067381 shutdown_connections 2024-05-09T08:03:17.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:17.748+0000 7f68aca00640 1 -- 172.21.15.1:0/4161067381 wait complete. 2024-05-09T08:03:19.932 DEBUG:teuthology.orchestra.run.smithi001:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T08:03:19.940 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T08:03:20.785 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-05-09T08:03:21.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:20.989+0000 7fd7d0000640 1 Processor -- start 2024-05-09T08:03:21.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:20.990+0000 7fd7d0000640 1 -- start start 2024-05-09T08:03:21.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:20.991+0000 7fd7d0000640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd7c806dca0 0x7fd7c8072900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:21.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:20.991+0000 7fd7d0000640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fd7c80c6ea0 0x7fd7c8159010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:21.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:20.991+0000 7fd7d0000640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd7c8073200 con 0x7fd7c806ece0 2024-05-09T08:03:21.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:20.991+0000 7fd7d0000640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7fd7c8072e40 con 0x7fd7c80c6ea0 2024-05-09T08:03:21.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:20.991+0000 7fd7d0000640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fd7c80737c0 con 0x7fd7c806dca0 2024-05-09T08:03:21.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:20.992+0000 7fd7ccc00640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fd7c80c6ea0 0x7fd7c8159010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:21.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:20.992+0000 7fd7cd600640 1 -- >> v2:172.21.15.7:3300/0 conn(0x7fd7c806dca0 msgr2=0x7fd7c8072900 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.7:3300/0 2024-05-09T08:03:21.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:20.992+0000 7fd7ce000640 1 -- >> v1:172.21.15.7:6789/0 conn(0x7fd7c806ece0 legacy=0x7fd7c806e080 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.7:6789/0 2024-05-09T08:03:21.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:20.992+0000 7fd7cd600640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd7c806dca0 0x7fd7c8072900 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T08:03:21.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:20.992+0000 7fd7ccc00640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fd7c80c6ea0 0x7fd7c8159010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54756/0 (socket says 172.21.15.1:54756) 2024-05-09T08:03:21.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:20.992+0000 7fd7ccc00640 1 -- 172.21.15.1:0/3883918847 learned_addr learned my addr 172.21.15.1:0/3883918847 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:03:21.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:20.992+0000 7fd7ccc00640 1 -- 172.21.15.1:0/3883918847 >> v2:172.21.15.7:3300/0 conn(0x7fd7c806dca0 msgr2=0x7fd7c8072900 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T08:03:21.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:20.992+0000 7fd7ccc00640 1 --2- 172.21.15.1:0/3883918847 >> v2:172.21.15.7:3300/0 conn(0x7fd7c806dca0 0x7fd7c8072900 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:21.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:20.992+0000 7fd7ccc00640 1 -- 172.21.15.1:0/3883918847 >> v1:172.21.15.7:6789/0 conn(0x7fd7c806ece0 legacy=0x7fd7c806e080 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T08:03:21.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:20.992+0000 7fd7ccc00640 1 -- 172.21.15.1:0/3883918847 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7c80734e0 con 0x7fd7c80c6ea0 2024-05-09T08:03:21.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:20.993+0000 7fd7ccc00640 1 --2- 172.21.15.1:0/3883918847 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd7c80c6ea0 0x7fd7c8159010 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fd7c80cccb0 tx=0x7fd7b802a2e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22dfb92190684f41 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:03:21.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:20.994+0000 7fd7c7e00640 1 -- 172.21.15.1:0/3883918847 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7b8037070 con 0x7fd7c80c6ea0 2024-05-09T08:03:21.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:20.994+0000 7fd7c7e00640 1 -- 172.21.15.1:0/3883918847 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7b8032440 con 0x7fd7c80c6ea0 2024-05-09T08:03:21.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:20.994+0000 7fd7c7e00640 1 -- 172.21.15.1:0/3883918847 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7b8032810 con 0x7fd7c80c6ea0 2024-05-09T08:03:21.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:20.995+0000 7fd7d0000640 1 -- 172.21.15.1:0/3883918847 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd7c80c6ea0 msgr2=0x7fd7c8159010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:21.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:20.995+0000 7fd7d0000640 1 --2- 172.21.15.1:0/3883918847 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd7c80c6ea0 0x7fd7c8159010 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fd7c80cccb0 tx=0x7fd7b802a2e0 comp rx=0 tx=0).stop 2024-05-09T08:03:21.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:20.995+0000 7fd7d0000640 1 -- 172.21.15.1:0/3883918847 shutdown_connections 2024-05-09T08:03:21.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:20.995+0000 7fd7d0000640 1 --2- 172.21.15.1:0/3883918847 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd7c80c6ea0 0x7fd7c8159010 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:21.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:20.995+0000 7fd7d0000640 1 --2- 172.21.15.1:0/3883918847 >> v2:172.21.15.7:3300/0 conn(0x7fd7c806dca0 0x7fd7c8072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:21.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:20.995+0000 7fd7d0000640 1 -- 172.21.15.1:0/3883918847 >> 172.21.15.1:0/3883918847 conn(0x7fd7c80cc070 msgr2=0x7fd7c80cc480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:03:21.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:20.995+0000 7fd7d0000640 1 -- 172.21.15.1:0/3883918847 shutdown_connections 2024-05-09T08:03:21.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:20.995+0000 7fd7d0000640 1 -- 172.21.15.1:0/3883918847 wait complete. 2024-05-09T08:03:21.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:20.997+0000 7fd7d0000640 1 Processor -- start 2024-05-09T08:03:21.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:20.997+0000 7fd7d0000640 1 -- start start 2024-05-09T08:03:21.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:20.998+0000 7fd7d0000640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fd7c806dca0 0x7fd7c80c5a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:21.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:20.998+0000 7fd7d0000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd7c806ece0 0x7fd7c8146050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:21.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:20.998+0000 7fd7d0000640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd7c80c6ea0 0x7fd7c8146590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:21.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:20.998+0000 7fd7d0000640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fd7c8072470 con 0x7fd7c80c6ea0 2024-05-09T08:03:21.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:20.998+0000 7fd7d0000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd7c8159550 con 0x7fd7c806ece0 2024-05-09T08:03:21.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:20.998+0000 7fd7d0000640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7fd7c8072100 con 0x7fd7c806dca0 2024-05-09T08:03:21.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:20.998+0000 7fd7ccc00640 1 -- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd7c806ece0 msgr2=0x7fd7c8146050 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.7:3300/0 2024-05-09T08:03:21.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:20.998+0000 7fd7ccc00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd7c806ece0 0x7fd7c8146050 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T08:03:21.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:20.998+0000 7fd7ce000640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd7c80c6ea0 0x7fd7c8146590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:21.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:20.998+0000 7fd7cd600640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fd7c806dca0 0x7fd7c80c5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:21.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:20.998+0000 7fd7cd600640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fd7c806dca0 0x7fd7c80c5a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3301/0 says I am v2:172.21.15.1:60530/0 (socket says 172.21.15.1:60530) 2024-05-09T08:03:21.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:20.998+0000 7fd7cd600640 1 -- 172.21.15.1:0/3649719540 learned_addr learned my addr 172.21.15.1:0/3649719540 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:03:21.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:20.998+0000 7fd7ce000640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd7c80c6ea0 0x7fd7c8146590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54758/0 (socket says 172.21.15.1:54758) 2024-05-09T08:03:21.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:20.999+0000 7fd7cd600640 1 -- 172.21.15.1:0/3649719540 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd7c806ece0 msgr2=0x7fd7c8146050 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T08:03:21.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:20.999+0000 7fd7cd600640 1 --2- 172.21.15.1:0/3649719540 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd7c806ece0 0x7fd7c8146050 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:21.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:20.999+0000 7fd7cd600640 1 -- 172.21.15.1:0/3649719540 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd7c80c6ea0 msgr2=0x7fd7c8146590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:21.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:20.999+0000 7fd7cd600640 1 --2- 172.21.15.1:0/3649719540 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd7c80c6ea0 0x7fd7c8146590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:21.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:20.999+0000 7fd7cd600640 1 -- 172.21.15.1:0/3649719540 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7c8152e70 con 0x7fd7c806dca0 2024-05-09T08:03:21.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:20.999+0000 7fd7cd600640 1 --2- 172.21.15.1:0/3649719540 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fd7c806dca0 0x7fd7c80c5a00 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fd7bc00b580 tx=0x7fd7bc00ba40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:03:21.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:20.999+0000 7fd7c5600640 1 -- 172.21.15.1:0/3649719540 <== mon.2 v2:172.21.15.7:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7bc013020 con 0x7fd7c806dca0 2024-05-09T08:03:21.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:20.999+0000 7fd7c5600640 1 -- 172.21.15.1:0/3649719540 <== mon.2 v2:172.21.15.7:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7bc003de0 con 0x7fd7c806dca0 2024-05-09T08:03:21.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:20.999+0000 7fd7c5600640 1 -- 172.21.15.1:0/3649719540 <== mon.2 v2:172.21.15.7:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7bc0052b0 con 0x7fd7c806dca0 2024-05-09T08:03:21.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:20.999+0000 7fd7d0000640 1 -- 172.21.15.1:0/3649719540 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7c81561f0 con 0x7fd7c806dca0 2024-05-09T08:03:21.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:21.000+0000 7fd7d0000640 1 -- 172.21.15.1:0/3649719540 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7c8153410 con 0x7fd7c806dca0 2024-05-09T08:03:21.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:21.001+0000 7fd7d0000640 1 -- 172.21.15.1:0/3649719540 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7c8074d70 con 0x7fd7c806dca0 2024-05-09T08:03:21.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:21.002+0000 7fd7c5600640 1 -- 172.21.15.1:0/3649719540 <== mon.2 v2:172.21.15.7:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd7bc016020 con 0x7fd7c806dca0 2024-05-09T08:03:21.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:21.003+0000 7fd7c5600640 1 --2- 172.21.15.1:0/3649719540 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fd7ac06bde0 0x7fd7ac06e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:21.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:21.003+0000 7fd7c5600640 1 -- 172.21.15.1:0/3649719540 <== mon.2 v2:172.21.15.7:3301/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7fd7bc08afe0 con 0x7fd7c806dca0 2024-05-09T08:03:21.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:21.003+0000 7fd7ccc00640 1 --2- 172.21.15.1:0/3649719540 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fd7ac06bde0 0x7fd7ac06e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:21.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:21.004+0000 7fd7ccc00640 1 --2- 172.21.15.1:0/3649719540 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fd7ac06bde0 0x7fd7ac06e2a0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fd7b8034000 tx=0x7fd7b802e000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:03:21.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:21.008+0000 7fd7c5600640 1 -- 172.21.15.1:0/3649719540 <== mon.2 v2:172.21.15.7:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fd7bc01d020 con 0x7fd7c806dca0 2024-05-09T08:03:21.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:21.291+0000 7fd7d0000640 1 -- 172.21.15.1:0/3649719540 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fd7c80ccd20 con 0x7fd7c806dca0 2024-05-09T08:03:21.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:21.292+0000 7fd7c5600640 1 -- 172.21.15.1:0/3649719540 <== mon.2 v2:172.21.15.7:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1544 (secure 0 0 0) 0x7fd7bc059520 con 0x7fd7c806dca0 2024-05-09T08:03:21.309 INFO:teuthology.orchestra.run.smithi001.stdout:{"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":84,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"54329ec8-3270-4043-9a82-c6cd7420aa7f","modified":"2024-05-09T08:01:21.819270Z","created":"2024-05-09T08:01:21.819270Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:3300","nonce":0},{"type":"v1","addr":"172.21.15.1:6789","nonce":0}]},"addr":"172.21.15.1:6789/0","public_addr":"172.21.15.1:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:3300","nonce":0},{"type":"v1","addr":"172.21.15.7:6789","nonce":0}]},"addr":"172.21.15.7:6789/0","public_addr":"172.21.15.7:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:3301","nonce":0},{"type":"v1","addr":"172.21.15.7:6790","nonce":0}]},"addr":"172.21.15.7:6790/0","public_addr":"172.21.15.7:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-05-09T08:03:21.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:21.295+0000 7fd7d0000640 1 -- 172.21.15.1:0/3649719540 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fd7ac06bde0 msgr2=0x7fd7ac06e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:21.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:21.295+0000 7fd7d0000640 1 --2- 172.21.15.1:0/3649719540 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fd7ac06bde0 0x7fd7ac06e2a0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fd7b8034000 tx=0x7fd7b802e000 comp rx=0 tx=0).stop 2024-05-09T08:03:21.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:21.295+0000 7fd7d0000640 1 -- 172.21.15.1:0/3649719540 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fd7c806dca0 msgr2=0x7fd7c80c5a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:21.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:21.295+0000 7fd7d0000640 1 --2- 172.21.15.1:0/3649719540 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fd7c806dca0 0x7fd7c80c5a00 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fd7bc00b580 tx=0x7fd7bc00ba40 comp rx=0 tx=0).stop 2024-05-09T08:03:21.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:21.295+0000 7fd7d0000640 1 -- 172.21.15.1:0/3649719540 shutdown_connections 2024-05-09T08:03:21.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:21.295+0000 7fd7d0000640 1 --2- 172.21.15.1:0/3649719540 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fd7ac06bde0 0x7fd7ac06e2a0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:21.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:21.295+0000 7fd7d0000640 1 --2- 172.21.15.1:0/3649719540 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd7c806ece0 0x7fd7c8146050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:21.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:21.295+0000 7fd7d0000640 1 --2- 172.21.15.1:0/3649719540 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd7c80c6ea0 0x7fd7c8146590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:21.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:21.295+0000 7fd7d0000640 1 --2- 172.21.15.1:0/3649719540 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fd7c806dca0 0x7fd7c80c5a00 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:21.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:21.295+0000 7fd7d0000640 1 -- 172.21.15.1:0/3649719540 >> 172.21.15.1:0/3649719540 conn(0x7fd7c80cc070 msgr2=0x7fd7c8152050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:03:21.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:21.295+0000 7fd7d0000640 1 -- 172.21.15.1:0/3649719540 shutdown_connections 2024-05-09T08:03:21.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:21.295+0000 7fd7d0000640 1 -- 172.21.15.1:0/3649719540 wait complete. 2024-05-09T08:03:24.333 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-05-09T08:03:24.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:24.535+0000 7f0e8fc00640 1 Processor -- start 2024-05-09T08:03:24.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:24.536+0000 7f0e8fc00640 1 -- start start 2024-05-09T08:03:24.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:24.537+0000 7f0e8fc00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f0e880724e0 0x7f0e880c8a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:24.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:24.538+0000 7f0e8fc00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f0e880c7650 0x7f0e88164f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:24.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:24.538+0000 7f0e8fc00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e8815b0f0 con 0x7f0e88066ed0 2024-05-09T08:03:24.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:24.538+0000 7f0e8fc00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f0e880c7d30 con 0x7f0e880c7650 2024-05-09T08:03:24.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:24.538+0000 7f0e8fc00640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f0e8815b3d0 con 0x7f0e880724e0 2024-05-09T08:03:24.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:24.538+0000 7f0e7fe00640 1 -- >> v1:172.21.15.7:6789/0 conn(0x7f0e88066ed0 legacy=0x7f0e880728c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.7:6789/0 2024-05-09T08:03:24.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:24.538+0000 7f0e8dc00640 1 -- >> v2:172.21.15.7:3300/0 conn(0x7f0e880c7650 msgr2=0x7f0e88164f50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.7:3300/0 2024-05-09T08:03:24.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:24.538+0000 7f0e8dc00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f0e880c7650 0x7f0e88164f50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T08:03:24.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:24.538+0000 7f0e8d200640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f0e880724e0 0x7f0e880c8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:24.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:24.538+0000 7f0e8d200640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f0e880724e0 0x7f0e880c8a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3301/0 says I am v2:172.21.15.1:33326/0 (socket says 172.21.15.1:33326) 2024-05-09T08:03:24.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:24.538+0000 7f0e8d200640 1 -- 172.21.15.1:0/3197099780 learned_addr learned my addr 172.21.15.1:0/3197099780 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:03:24.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:24.538+0000 7f0e8d200640 1 -- 172.21.15.1:0/3197099780 >> v2:172.21.15.7:3300/0 conn(0x7f0e880c7650 msgr2=0x7f0e88164f50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T08:03:24.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:24.538+0000 7f0e8d200640 1 --2- 172.21.15.1:0/3197099780 >> v2:172.21.15.7:3300/0 conn(0x7f0e880c7650 0x7f0e88164f50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:24.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:24.538+0000 7f0e8d200640 1 -- 172.21.15.1:0/3197099780 >> v1:172.21.15.7:6789/0 conn(0x7f0e88066ed0 legacy=0x7f0e880728c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T08:03:24.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:24.538+0000 7f0e8d200640 1 -- 172.21.15.1:0/3197099780 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e880c7f80 con 0x7f0e880724e0 2024-05-09T08:03:24.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:24.540+0000 7f0e8d200640 1 --2- 172.21.15.1:0/3197099780 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f0e880724e0 0x7f0e880c8a60 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f0e78009e50 tx=0x7f0e7802a5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=905696d6083cd0f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:03:24.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:24.540+0000 7f0e87e00640 1 -- 172.21.15.1:0/3197099780 <== mon.2 v2:172.21.15.7:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e78036070 con 0x7f0e880724e0 2024-05-09T08:03:24.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:24.540+0000 7f0e87e00640 1 -- 172.21.15.1:0/3197099780 <== mon.2 v2:172.21.15.7:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e78031600 con 0x7f0e880724e0 2024-05-09T08:03:24.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:24.540+0000 7f0e87e00640 1 -- 172.21.15.1:0/3197099780 <== mon.2 v2:172.21.15.7:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e780319d0 con 0x7f0e880724e0 2024-05-09T08:03:24.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:24.541+0000 7f0e8fc00640 1 -- 172.21.15.1:0/3197099780 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f0e880724e0 msgr2=0x7f0e880c8a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:24.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:24.541+0000 7f0e8fc00640 1 --2- 172.21.15.1:0/3197099780 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f0e880724e0 0x7f0e880c8a60 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f0e78009e50 tx=0x7f0e7802a5d0 comp rx=0 tx=0).stop 2024-05-09T08:03:24.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:24.541+0000 7f0e8fc00640 1 -- 172.21.15.1:0/3197099780 shutdown_connections 2024-05-09T08:03:24.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:24.541+0000 7f0e8fc00640 1 --2- 172.21.15.1:0/3197099780 >> v2:172.21.15.7:3300/0 conn(0x7f0e880c7650 0x7f0e88164f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:24.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:24.541+0000 7f0e8fc00640 1 --2- 172.21.15.1:0/3197099780 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f0e880724e0 0x7f0e880c8a60 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:24.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:24.541+0000 7f0e8fc00640 1 -- 172.21.15.1:0/3197099780 >> 172.21.15.1:0/3197099780 conn(0x7f0e880cc5f0 msgr2=0x7f0e880cca00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:03:24.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:24.541+0000 7f0e8fc00640 1 -- 172.21.15.1:0/3197099780 shutdown_connections 2024-05-09T08:03:24.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:24.541+0000 7f0e8fc00640 1 -- 172.21.15.1:0/3197099780 wait complete. 2024-05-09T08:03:24.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:24.543+0000 7f0e8fc00640 1 Processor -- start 2024-05-09T08:03:24.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:24.543+0000 7f0e8fc00640 1 -- start start 2024-05-09T08:03:24.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:24.544+0000 7f0e8fc00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0e88066ed0 0x7f0e8814d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:24.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:24.544+0000 7f0e8fc00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e880724e0 0x7f0e88158540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:24.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:24.544+0000 7f0e8fc00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f0e880c7650 0x7f0e88158a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:24.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:24.544+0000 7f0e8fc00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f0e881516c0 con 0x7f0e88066ed0 2024-05-09T08:03:24.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:24.544+0000 7f0e8fc00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0e880c8540 con 0x7f0e880724e0 2024-05-09T08:03:24.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:24.544+0000 7f0e8fc00640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f0e88067400 con 0x7f0e880c7650 2024-05-09T08:03:24.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:24.544+0000 7f0e7fe00640 1 -- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e880724e0 msgr2=0x7f0e88158540 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.7:3300/0 2024-05-09T08:03:24.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:24.544+0000 7f0e7fe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e880724e0 0x7f0e88158540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T08:03:24.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:24.544+0000 7f0e8d200640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0e88066ed0 0x7f0e8814d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:24.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:24.544+0000 7f0e8d200640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0e88066ed0 0x7f0e8814d0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:37078/0 (socket says 172.21.15.1:37078) 2024-05-09T08:03:24.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:24.544+0000 7f0e8d200640 1 -- 172.21.15.1:0/3126992839 learned_addr learned my addr 172.21.15.1:0/3126992839 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:03:24.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:24.544+0000 7f0e8dc00640 1 --2- 172.21.15.1:0/3126992839 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f0e880c7650 0x7f0e88158a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:24.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:24.545+0000 7f0e8d200640 1 -- 172.21.15.1:0/3126992839 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f0e880c7650 msgr2=0x7f0e88158a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:24.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:24.545+0000 7f0e8d200640 1 --2- 172.21.15.1:0/3126992839 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f0e880c7650 0x7f0e88158a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:24.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:24.545+0000 7f0e8d200640 1 -- 172.21.15.1:0/3126992839 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e880724e0 msgr2=0x7f0e88158540 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T08:03:24.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:24.545+0000 7f0e8d200640 1 --2- 172.21.15.1:0/3126992839 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e880724e0 0x7f0e88158540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:24.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:24.545+0000 7f0e8d200640 1 -- 172.21.15.1:0/3126992839 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e881f42c0 con 0x7f0e88066ed0 2024-05-09T08:03:24.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:24.545+0000 7f0e8d200640 1 --2- 172.21.15.1:0/3126992839 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0e88066ed0 0x7f0e8814d0d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f0e78034040 tx=0x7f0e78004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:03:24.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:24.545+0000 7f0e85600640 1 -- 172.21.15.1:0/3126992839 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e78041020 con 0x7f0e88066ed0 2024-05-09T08:03:24.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:24.545+0000 7f0e85600640 1 -- 172.21.15.1:0/3126992839 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e78007820 con 0x7f0e88066ed0 2024-05-09T08:03:24.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:24.545+0000 7f0e85600640 1 -- 172.21.15.1:0/3126992839 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e78036040 con 0x7f0e88066ed0 2024-05-09T08:03:24.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:24.545+0000 7f0e8fc00640 1 -- 172.21.15.1:0/3126992839 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e881f45a0 con 0x7f0e88066ed0 2024-05-09T08:03:24.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:24.546+0000 7f0e8fc00640 1 -- 172.21.15.1:0/3126992839 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e881f4b60 con 0x7f0e88066ed0 2024-05-09T08:03:24.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:24.547+0000 7f0e8fc00640 1 -- 172.21.15.1:0/3126992839 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e8815ae70 con 0x7f0e88066ed0 2024-05-09T08:03:24.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:24.548+0000 7f0e85600640 1 -- 172.21.15.1:0/3126992839 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0e78044020 con 0x7f0e88066ed0 2024-05-09T08:03:24.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:24.549+0000 7f0e85600640 1 --2- 172.21.15.1:0/3126992839 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f0e6406bde0 0x7f0e6406e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:24.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:24.549+0000 7f0e85600640 1 -- 172.21.15.1:0/3126992839 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7f0e780ac120 con 0x7f0e88066ed0 2024-05-09T08:03:24.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:24.549+0000 7f0e7fe00640 1 --2- 172.21.15.1:0/3126992839 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f0e6406bde0 0x7f0e6406e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:24.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:24.550+0000 7f0e7fe00640 1 --2- 172.21.15.1:0/3126992839 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f0e6406bde0 0x7f0e6406e2a0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f0e70006fd0 tx=0x7f0e70006d80 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:03:24.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:24.554+0000 7f0e85600640 1 -- 172.21.15.1:0/3126992839 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f0e8815ae70 con 0x7f0e88066ed0 2024-05-09T08:03:24.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:24.830+0000 7f0e8fc00640 1 -- 172.21.15.1:0/3126992839 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f0e8814c2c0 con 0x7f0e88066ed0 2024-05-09T08:03:24.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:24.830+0000 7f0e85600640 1 -- 172.21.15.1:0/3126992839 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1544 (secure 0 0 0) 0x7f0e8814c2c0 con 0x7f0e88066ed0 2024-05-09T08:03:24.847 INFO:teuthology.orchestra.run.smithi001.stdout:{"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":88,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"54329ec8-3270-4043-9a82-c6cd7420aa7f","modified":"2024-05-09T08:01:21.819270Z","created":"2024-05-09T08:01:21.819270Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:3300","nonce":0},{"type":"v1","addr":"172.21.15.1:6789","nonce":0}]},"addr":"172.21.15.1:6789/0","public_addr":"172.21.15.1:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:3300","nonce":0},{"type":"v1","addr":"172.21.15.7:6789","nonce":0}]},"addr":"172.21.15.7:6789/0","public_addr":"172.21.15.7:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:3301","nonce":0},{"type":"v1","addr":"172.21.15.7:6790","nonce":0}]},"addr":"172.21.15.7:6790/0","public_addr":"172.21.15.7:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-05-09T08:03:24.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:24.833+0000 7f0e8fc00640 1 -- 172.21.15.1:0/3126992839 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f0e6406bde0 msgr2=0x7f0e6406e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:24.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:24.833+0000 7f0e8fc00640 1 --2- 172.21.15.1:0/3126992839 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f0e6406bde0 0x7f0e6406e2a0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f0e70006fd0 tx=0x7f0e70006d80 comp rx=0 tx=0).stop 2024-05-09T08:03:24.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:24.833+0000 7f0e8fc00640 1 -- 172.21.15.1:0/3126992839 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0e88066ed0 msgr2=0x7f0e8814d0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:24.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:24.833+0000 7f0e8fc00640 1 --2- 172.21.15.1:0/3126992839 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0e88066ed0 0x7f0e8814d0d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f0e78034040 tx=0x7f0e78004a40 comp rx=0 tx=0).stop 2024-05-09T08:03:24.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:24.833+0000 7f0e8fc00640 1 -- 172.21.15.1:0/3126992839 shutdown_connections 2024-05-09T08:03:24.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:24.833+0000 7f0e8fc00640 1 --2- 172.21.15.1:0/3126992839 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f0e6406bde0 0x7f0e6406e2a0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:24.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:24.833+0000 7f0e8fc00640 1 --2- 172.21.15.1:0/3126992839 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e880724e0 0x7f0e88158540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:24.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:24.833+0000 7f0e8fc00640 1 --2- 172.21.15.1:0/3126992839 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f0e880c7650 0x7f0e88158a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:24.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:24.833+0000 7f0e8fc00640 1 --2- 172.21.15.1:0/3126992839 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0e88066ed0 0x7f0e8814d0d0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:24.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:24.833+0000 7f0e8fc00640 1 -- 172.21.15.1:0/3126992839 >> 172.21.15.1:0/3126992839 conn(0x7f0e880cc5f0 msgr2=0x7f0e8814b2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:03:24.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:24.833+0000 7f0e8fc00640 1 -- 172.21.15.1:0/3126992839 shutdown_connections 2024-05-09T08:03:24.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:24.833+0000 7f0e8fc00640 1 -- 172.21.15.1:0/3126992839 wait complete. 2024-05-09T08:03:27.870 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-05-09T08:03:28.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:28.070+0000 7ff634e00640 1 Processor -- start 2024-05-09T08:03:28.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:28.071+0000 7ff634e00640 1 -- start start 2024-05-09T08:03:28.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:28.072+0000 7ff634e00640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7ff630152500 0x7ff6300c8180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:28.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:28.072+0000 7ff634e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff6300cc730 0x7ff63015c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:28.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:28.072+0000 7ff634e00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ff6300c7660 0x7ff6300c7ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:28.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:28.072+0000 7ff634e00640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7ff6300c5e50 con 0x7ff630152500 2024-05-09T08:03:28.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:28.072+0000 7ff634e00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7ff6300c22d0 con 0x7ff6300cc730 2024-05-09T08:03:28.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:28.072+0000 7ff634e00640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7ff6300ccb50 con 0x7ff6300c7660 2024-05-09T08:03:28.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:28.073+0000 7ff627e00640 1 -- >> v2:172.21.15.7:3300/0 conn(0x7ff6300cc730 msgr2=0x7ff63015c080 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.7:3300/0 2024-05-09T08:03:28.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:28.073+0000 7ff627e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff6300cc730 0x7ff63015c080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T08:03:28.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:28.073+0000 7ff62f400640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7ff630152500 0x7ff6300c8180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:28.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:28.073+0000 7ff62f400640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7ff630152500 0x7ff6300c8180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:37092/0 (socket says 172.21.15.1:37092) 2024-05-09T08:03:28.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:28.073+0000 7ff62f400640 1 -- 172.21.15.1:0/1055609593 learned_addr learned my addr 172.21.15.1:0/1055609593 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:03:28.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:28.073+0000 7ff62fe00640 1 --2- 172.21.15.1:0/1055609593 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ff6300c7660 0x7ff6300c7ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:28.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:28.073+0000 7ff62f400640 1 -- 172.21.15.1:0/1055609593 >> v2:172.21.15.1:3300/0 conn(0x7ff630152500 msgr2=0x7ff6300c8180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-09T08:03:28.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:28.073+0000 7ff62f400640 1 -- 172.21.15.1:0/1055609593 >> v2:172.21.15.1:3300/0 conn(0x7ff630152500 msgr2=0x7ff6300c8180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-09T08:03:28.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:28.073+0000 7ff62f400640 1 --2- 172.21.15.1:0/1055609593 >> v2:172.21.15.1:3300/0 conn(0x7ff630152500 0x7ff6300c8180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-09T08:03:28.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:28.073+0000 7ff62fe00640 1 -- 172.21.15.1:0/1055609593 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ff6300c7660 msgr2=0x7ff6300c7ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-05-09T08:03:28.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:28.073+0000 7ff62fe00640 1 -- 172.21.15.1:0/1055609593 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ff6300c7660 msgr2=0x7ff6300c7ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-09T08:03:28.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:28.073+0000 7ff62f400640 1 --2- 172.21.15.1:0/1055609593 >> v2:172.21.15.1:3300/0 conn(0x7ff630152500 0x7ff6300c8180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T08:03:28.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:28.073+0000 7ff62fe00640 1 --2- 172.21.15.1:0/1055609593 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ff6300c7660 0x7ff6300c7ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-09T08:03:28.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:28.073+0000 7ff62fe00640 1 --2- 172.21.15.1:0/1055609593 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ff6300c7660 0x7ff6300c7ab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T08:03:28.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:28.274+0000 7ff627e00640 1 -- 172.21.15.1:0/1055609593 >> v2:172.21.15.7:3300/0 conn(0x7ff6300cc730 msgr2=0x7ff63015c080 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.7:3300/0 2024-05-09T08:03:28.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:28.274+0000 7ff627e00640 1 --2- 172.21.15.1:0/1055609593 >> v2:172.21.15.7:3300/0 conn(0x7ff6300cc730 0x7ff63015c080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-09T08:03:28.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:28.274+0000 7ff62fe00640 1 --2- 172.21.15.1:0/1055609593 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ff6300c7660 0x7ff6300c7ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:28.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:28.274+0000 7ff62f400640 1 --2- 172.21.15.1:0/1055609593 >> v2:172.21.15.1:3300/0 conn(0x7ff630152500 0x7ff6300c8180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:28.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:28.274+0000 7ff62fe00640 1 -- 172.21.15.1:0/1055609593 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ff6300c7660 msgr2=0x7ff6300c7ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-05-09T08:03:28.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:28.274+0000 7ff62fe00640 1 -- 172.21.15.1:0/1055609593 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ff6300c7660 msgr2=0x7ff6300c7ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-09T08:03:28.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:28.274+0000 7ff62fe00640 1 --2- 172.21.15.1:0/1055609593 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ff6300c7660 0x7ff6300c7ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-09T08:03:28.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:28.275+0000 7ff62fe00640 1 --2- 172.21.15.1:0/1055609593 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ff6300c7660 0x7ff6300c7ab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-09T08:03:28.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:28.275+0000 7ff62f400640 1 -- 172.21.15.1:0/1055609593 >> v2:172.21.15.1:3300/0 conn(0x7ff630152500 msgr2=0x7ff6300c8180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-09T08:03:28.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:28.275+0000 7ff62f400640 1 -- 172.21.15.1:0/1055609593 >> v2:172.21.15.1:3300/0 conn(0x7ff630152500 msgr2=0x7ff6300c8180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-09T08:03:28.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:28.275+0000 7ff62f400640 1 --2- 172.21.15.1:0/1055609593 >> v2:172.21.15.1:3300/0 conn(0x7ff630152500 0x7ff6300c8180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-09T08:03:28.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:28.275+0000 7ff62f400640 1 --2- 172.21.15.1:0/1055609593 >> v2:172.21.15.1:3300/0 conn(0x7ff630152500 0x7ff6300c8180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-09T08:03:28.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:28.675+0000 7ff627e00640 1 -- 172.21.15.1:0/1055609593 >> v2:172.21.15.7:3300/0 conn(0x7ff6300cc730 msgr2=0x7ff63015c080 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.7:3300/0 2024-05-09T08:03:28.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:28.675+0000 7ff627e00640 1 --2- 172.21.15.1:0/1055609593 >> v2:172.21.15.7:3300/0 conn(0x7ff6300cc730 0x7ff63015c080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-09T08:03:28.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:28.676+0000 7ff62fe00640 1 --2- 172.21.15.1:0/1055609593 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ff6300c7660 0x7ff6300c7ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:28.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:28.676+0000 7ff62f400640 1 --2- 172.21.15.1:0/1055609593 >> v2:172.21.15.1:3300/0 conn(0x7ff630152500 0x7ff6300c8180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:28.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:28.677+0000 7ff62fe00640 1 -- 172.21.15.1:0/1055609593 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ff6300c7660 msgr2=0x7ff6300c7ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-09T08:03:28.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:28.677+0000 7ff62fe00640 1 -- 172.21.15.1:0/1055609593 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ff6300c7660 msgr2=0x7ff6300c7ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-09T08:03:28.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:28.677+0000 7ff62fe00640 1 --2- 172.21.15.1:0/1055609593 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ff6300c7660 0x7ff6300c7ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-09T08:03:28.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:28.677+0000 7ff62fe00640 1 --2- 172.21.15.1:0/1055609593 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ff6300c7660 0x7ff6300c7ab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-09T08:03:28.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:28.677+0000 7ff62f400640 1 -- 172.21.15.1:0/1055609593 >> v2:172.21.15.1:3300/0 conn(0x7ff630152500 msgr2=0x7ff6300c8180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-05-09T08:03:28.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:28.677+0000 7ff62f400640 1 -- 172.21.15.1:0/1055609593 >> v2:172.21.15.1:3300/0 conn(0x7ff630152500 msgr2=0x7ff6300c8180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-09T08:03:28.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:28.677+0000 7ff62f400640 1 --2- 172.21.15.1:0/1055609593 >> v2:172.21.15.1:3300/0 conn(0x7ff630152500 0x7ff6300c8180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-09T08:03:28.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:28.677+0000 7ff62f400640 1 --2- 172.21.15.1:0/1055609593 >> v2:172.21.15.1:3300/0 conn(0x7ff630152500 0x7ff6300c8180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-09T08:03:29.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:29.476+0000 7ff627e00640 1 -- 172.21.15.1:0/1055609593 >> v2:172.21.15.7:3300/0 conn(0x7ff6300cc730 msgr2=0x7ff63015c080 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.7:3300/0 2024-05-09T08:03:29.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:29.476+0000 7ff627e00640 1 --2- 172.21.15.1:0/1055609593 >> v2:172.21.15.7:3300/0 conn(0x7ff6300cc730 0x7ff63015c080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-09T08:03:29.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:29.478+0000 7ff62fe00640 1 --2- 172.21.15.1:0/1055609593 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ff6300c7660 0x7ff6300c7ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:29.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:29.478+0000 7ff62f400640 1 --2- 172.21.15.1:0/1055609593 >> v2:172.21.15.1:3300/0 conn(0x7ff630152500 0x7ff6300c8180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:29.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:29.479+0000 7ff62fe00640 1 -- 172.21.15.1:0/1055609593 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ff6300c7660 msgr2=0x7ff6300c7ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-05-09T08:03:29.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:29.479+0000 7ff62fe00640 1 -- 172.21.15.1:0/1055609593 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ff6300c7660 msgr2=0x7ff6300c7ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-09T08:03:29.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:29.479+0000 7ff62fe00640 1 --2- 172.21.15.1:0/1055609593 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ff6300c7660 0x7ff6300c7ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-09T08:03:29.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:29.479+0000 7ff62fe00640 1 --2- 172.21.15.1:0/1055609593 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ff6300c7660 0x7ff6300c7ab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-09T08:03:29.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:29.479+0000 7ff62f400640 1 -- 172.21.15.1:0/1055609593 >> v2:172.21.15.1:3300/0 conn(0x7ff630152500 msgr2=0x7ff6300c8180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-09T08:03:29.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:29.479+0000 7ff62f400640 1 -- 172.21.15.1:0/1055609593 >> v2:172.21.15.1:3300/0 conn(0x7ff630152500 msgr2=0x7ff6300c8180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-09T08:03:29.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:29.479+0000 7ff62f400640 1 --2- 172.21.15.1:0/1055609593 >> v2:172.21.15.1:3300/0 conn(0x7ff630152500 0x7ff6300c8180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-09T08:03:29.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:29.479+0000 7ff62f400640 1 --2- 172.21.15.1:0/1055609593 >> v2:172.21.15.1:3300/0 conn(0x7ff630152500 0x7ff6300c8180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-09T08:03:31.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.071+0000 7ff62d600640 1 -- 172.21.15.1:0/1055609593 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ff6300c7660 msgr2=0x7ff6300c7ab0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T08:03:31.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.071+0000 7ff62d600640 1 --2- 172.21.15.1:0/1055609593 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ff6300c7660 0x7ff6300c7ab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:31.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.072+0000 7ff62d600640 1 -- 172.21.15.1:0/1055609593 >> v2:172.21.15.7:3300/0 conn(0x7ff6300cc730 msgr2=0x7ff63015c080 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T08:03:31.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.072+0000 7ff62d600640 1 --2- 172.21.15.1:0/1055609593 >> v2:172.21.15.7:3300/0 conn(0x7ff6300cc730 0x7ff63015c080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:31.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.072+0000 7ff62d600640 1 -- 172.21.15.1:0/1055609593 >> v2:172.21.15.1:3300/0 conn(0x7ff630152500 msgr2=0x7ff6300c8180 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T08:03:31.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.072+0000 7ff62d600640 1 --2- 172.21.15.1:0/1055609593 >> v2:172.21.15.1:3300/0 conn(0x7ff630152500 0x7ff6300c8180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:31.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.072+0000 7ff62d600640 1 -- 172.21.15.1:0/1055609593 --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff6300c6a30 con 0x7ff60c0073b0 2024-05-09T08:03:31.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.072+0000 7ff62d600640 1 -- 172.21.15.1:0/1055609593 --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff60c00c120 con 0x7ff60c003690 2024-05-09T08:03:31.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.072+0000 7ff62f400640 1 -- 172.21.15.1:0/1055609593 >> v1:172.21.15.7:6789/0 conn(0x7ff60c003690 legacy=0x7ff60c005af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.7:6789/0 2024-05-09T08:03:31.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.273+0000 7ff62f400640 1 -- 172.21.15.1:0/1055609593 >> v1:172.21.15.7:6789/0 conn(0x7ff60c003690 legacy=0x7ff60c005af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.7:6789/0 2024-05-09T08:03:31.420 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.402+0000 7ff62ea00640 1 -- 172.21.15.1:0/1055609593 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2045875075 0 0) 0x7ff6300c6a30 con 0x7ff60c0073b0 2024-05-09T08:03:31.420 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.402+0000 7ff62ea00640 1 -- 172.21.15.1:0/1055609593 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff614003520 con 0x7ff60c0073b0 2024-05-09T08:03:31.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.408+0000 7ff62ea00640 1 -- 172.21.15.1:0/1055609593 <== mon.0 v1:172.21.15.1:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2224404092 0 0) 0x7ff614003520 con 0x7ff60c0073b0 2024-05-09T08:03:31.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.408+0000 7ff62ea00640 1 -- 172.21.15.1:0/1055609593 >> v1:172.21.15.7:6789/0 conn(0x7ff60c003690 legacy=0x7ff60c005af0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T08:03:31.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.408+0000 7ff62ea00640 1 -- 172.21.15.1:0/1055609593 --> v1:172.21.15.1:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff60c00d3c0 con 0x7ff60c0073b0 2024-05-09T08:03:31.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.408+0000 7ff62ea00640 1 -- 172.21.15.1:0/1055609593 <== mon.0 v1:172.21.15.1:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4179846320 0 0) 0x7ff618004240 con 0x7ff60c0073b0 2024-05-09T08:03:31.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.408+0000 7ff62ea00640 1 -- 172.21.15.1:0/1055609593 >> v1:172.21.15.1:6789/0 conn(0x7ff60c0073b0 legacy=0x7ff60c009870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:31.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.408+0000 7ff62fe00640 1 -- 172.21.15.1:0/1055609593 reap_dead start 2024-05-09T08:03:31.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.408+0000 7ff62ea00640 1 --2- 172.21.15.1:0/1055609593 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ff6300c7660 0x7ff6140039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:31.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.408+0000 7ff62ea00640 1 --2- 172.21.15.1:0/1055609593 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff6140040b0 0x7ff614004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:31.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.409+0000 7ff62ea00640 1 --2- 172.21.15.1:0/1055609593 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff614004d90 0x7ff614005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:31.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.409+0000 7ff62ea00640 1 -- 172.21.15.1:0/1055609593 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ff61400bc70 con 0x7ff614004d90 2024-05-09T08:03:31.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.409+0000 7ff62ea00640 1 -- 172.21.15.1:0/1055609593 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff61400bdf0 con 0x7ff6140040b0 2024-05-09T08:03:31.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.409+0000 7ff62ea00640 1 -- 172.21.15.1:0/1055609593 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7ff61400bff0 con 0x7ff6300c7660 2024-05-09T08:03:31.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.409+0000 7ff62f400640 1 --2- 172.21.15.1:0/1055609593 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ff6300c7660 0x7ff6140039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:31.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.409+0000 7ff627e00640 1 -- 172.21.15.1:0/1055609593 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff6140040b0 msgr2=0x7ff614004800 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.7:3300/0 2024-05-09T08:03:31.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.409+0000 7ff627e00640 1 --2- 172.21.15.1:0/1055609593 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff6140040b0 0x7ff614004800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T08:03:31.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.410+0000 7ff62f400640 1 -- 172.21.15.1:0/1055609593 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff6140040b0 msgr2=0x7ff614004800 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T08:03:31.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.410+0000 7ff62f400640 1 --2- 172.21.15.1:0/1055609593 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff6140040b0 0x7ff614004800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:31.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.410+0000 7ff62f400640 1 -- 172.21.15.1:0/1055609593 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff614004d90 msgr2=0x7ff614005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:03:31.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.410+0000 7ff62f400640 1 --2- 172.21.15.1:0/1055609593 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff614004d90 0x7ff614005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:31.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.410+0000 7ff62f400640 1 -- 172.21.15.1:0/1055609593 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff614003520 con 0x7ff6300c7660 2024-05-09T08:03:31.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.412+0000 7ff62f400640 1 --2- 172.21.15.1:0/1055609593 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ff6300c7660 0x7ff6140039e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ff620009100 tx=0x7ff614005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5a1fd0c55508cca2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:03:31.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.414+0000 7ff62ea00640 1 -- 172.21.15.1:0/1055609593 <== mon.2 v2:172.21.15.7:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff620031070 con 0x7ff6300c7660 2024-05-09T08:03:31.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.414+0000 7ff62ea00640 1 -- 172.21.15.1:0/1055609593 <== mon.2 v2:172.21.15.7:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff62002b770 con 0x7ff6300c7660 2024-05-09T08:03:31.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.414+0000 7ff62ea00640 1 -- 172.21.15.1:0/1055609593 <== mon.2 v2:172.21.15.7:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff62000f040 con 0x7ff6300c7660 2024-05-09T08:03:31.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.415+0000 7ff634e00640 1 -- 172.21.15.1:0/1055609593 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ff6300c7660 msgr2=0x7ff6140039e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:31.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.415+0000 7ff634e00640 1 --2- 172.21.15.1:0/1055609593 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ff6300c7660 0x7ff6140039e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ff620009100 tx=0x7ff614005830 comp rx=0 tx=0).stop 2024-05-09T08:03:31.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.415+0000 7ff634e00640 1 -- 172.21.15.1:0/1055609593 shutdown_connections 2024-05-09T08:03:31.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.416+0000 7ff634e00640 1 --2- 172.21.15.1:0/1055609593 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff6140040b0 0x7ff614004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:31.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.416+0000 7ff634e00640 1 --2- 172.21.15.1:0/1055609593 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff614004d90 0x7ff614005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:31.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.416+0000 7ff634e00640 1 --2- 172.21.15.1:0/1055609593 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ff6300c7660 0x7ff6140039e0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:31.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.416+0000 7ff634e00640 1 -- 172.21.15.1:0/1055609593 >> 172.21.15.1:0/1055609593 conn(0x7ff6300cbc10 msgr2=0x7ff6300cc020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:03:31.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.416+0000 7ff634e00640 1 -- 172.21.15.1:0/1055609593 shutdown_connections 2024-05-09T08:03:31.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.416+0000 7ff634e00640 1 -- 172.21.15.1:0/1055609593 wait complete. 2024-05-09T08:03:31.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.419+0000 7ff634e00640 1 Processor -- start 2024-05-09T08:03:31.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.419+0000 7ff634e00640 1 -- start start 2024-05-09T08:03:31.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.420+0000 7ff634e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff6140040b0 0x7ff6301eed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:31.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.420+0000 7ff634e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff614004d90 0x7ff63014b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:31.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.420+0000 7ff634e00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ff6300c7660 0x7ff630159cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:31.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.420+0000 7ff634e00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ff6300c72b0 con 0x7ff614004d90 2024-05-09T08:03:31.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.420+0000 7ff634e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff6300c6fd0 con 0x7ff6140040b0 2024-05-09T08:03:31.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.420+0000 7ff634e00640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7ff63015b880 con 0x7ff6300c7660 2024-05-09T08:03:31.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.420+0000 7ff62f400640 1 -- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff6140040b0 msgr2=0x7ff6301eed60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.7:3300/0 2024-05-09T08:03:31.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.420+0000 7ff62f400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff6140040b0 0x7ff6301eed60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T08:03:31.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.421+0000 7ff62fe00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ff6300c7660 0x7ff630159cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:31.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.421+0000 7ff627e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff614004d90 0x7ff63014b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:31.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.421+0000 7ff62fe00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ff6300c7660 0x7ff630159cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3301/0 says I am v2:172.21.15.1:33376/0 (socket says 172.21.15.1:33376) 2024-05-09T08:03:31.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.421+0000 7ff62fe00640 1 -- 172.21.15.1:0/2260137164 learned_addr learned my addr 172.21.15.1:0/2260137164 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:03:31.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.421+0000 7ff627e00640 1 -- 172.21.15.1:0/2260137164 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ff6300c7660 msgr2=0x7ff630159cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:31.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.421+0000 7ff627e00640 1 --2- 172.21.15.1:0/2260137164 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ff6300c7660 0x7ff630159cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:31.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.421+0000 7ff627e00640 1 -- 172.21.15.1:0/2260137164 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff6140040b0 msgr2=0x7ff6301eed60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T08:03:31.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.421+0000 7ff627e00640 1 --2- 172.21.15.1:0/2260137164 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff6140040b0 0x7ff6301eed60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:31.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.421+0000 7ff627e00640 1 -- 172.21.15.1:0/2260137164 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6301ef2a0 con 0x7ff614004d90 2024-05-09T08:03:31.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.422+0000 7ff627e00640 1 --2- 172.21.15.1:0/2260137164 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff614004d90 0x7ff63014b810 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ff618003bd0 tx=0x7ff61800cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:03:31.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.422+0000 7ff627400640 1 -- 172.21.15.1:0/2260137164 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff618017070 con 0x7ff614004d90 2024-05-09T08:03:31.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.422+0000 7ff627400640 1 -- 172.21.15.1:0/2260137164 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff61800aa20 con 0x7ff614004d90 2024-05-09T08:03:31.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.422+0000 7ff627400640 1 -- 172.21.15.1:0/2260137164 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff618007e60 con 0x7ff614004d90 2024-05-09T08:03:31.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.422+0000 7ff634e00640 1 -- 172.21.15.1:0/2260137164 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6301ebd70 con 0x7ff614004d90 2024-05-09T08:03:31.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.422+0000 7ff634e00640 1 -- 172.21.15.1:0/2260137164 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6301eafb0 con 0x7ff614004d90 2024-05-09T08:03:31.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.424+0000 7ff634e00640 1 -- 172.21.15.1:0/2260137164 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff63004f660 con 0x7ff614004d90 2024-05-09T08:03:31.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.425+0000 7ff627400640 1 -- 172.21.15.1:0/2260137164 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff61801a070 con 0x7ff614004d90 2024-05-09T08:03:31.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.426+0000 7ff627400640 1 --2- 172.21.15.1:0/2260137164 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7ff60c06c600 0x7ff60c06eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:31.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.426+0000 7ff627400640 1 -- 172.21.15.1:0/2260137164 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7ff61808a900 con 0x7ff614004d90 2024-05-09T08:03:31.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.426+0000 7ff62f400640 1 --2- 172.21.15.1:0/2260137164 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7ff60c06c600 0x7ff60c06eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:31.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.427+0000 7ff62f400640 1 --2- 172.21.15.1:0/2260137164 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7ff60c06c600 0x7ff60c06eac0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7ff620036000 tx=0x7ff620010040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:03:31.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.431+0000 7ff627400640 1 -- 172.21.15.1:0/2260137164 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ff6180587d0 con 0x7ff614004d90 2024-05-09T08:03:31.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.709+0000 7ff634e00640 1 -- 172.21.15.1:0/2260137164 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7ff6300734f0 con 0x7ff614004d90 2024-05-09T08:03:31.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.710+0000 7ff627400640 1 -- 172.21.15.1:0/2260137164 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1537 (secure 0 0 0) 0x7ff61805cad0 con 0x7ff614004d90 2024-05-09T08:03:31.727 INFO:teuthology.orchestra.run.smithi001.stdout:{"election_epoch":8,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"54329ec8-3270-4043-9a82-c6cd7420aa7f","modified":"2024-05-09T08:01:21.819270Z","created":"2024-05-09T08:01:21.819270Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:3300","nonce":0},{"type":"v1","addr":"172.21.15.1:6789","nonce":0}]},"addr":"172.21.15.1:6789/0","public_addr":"172.21.15.1:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:3300","nonce":0},{"type":"v1","addr":"172.21.15.7:6789","nonce":0}]},"addr":"172.21.15.7:6789/0","public_addr":"172.21.15.7:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:3301","nonce":0},{"type":"v1","addr":"172.21.15.7:6790","nonce":0}]},"addr":"172.21.15.7:6790/0","public_addr":"172.21.15.7:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-05-09T08:03:31.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.713+0000 7ff634e00640 1 -- 172.21.15.1:0/2260137164 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7ff60c06c600 msgr2=0x7ff60c06eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:31.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.713+0000 7ff634e00640 1 --2- 172.21.15.1:0/2260137164 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7ff60c06c600 0x7ff60c06eac0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7ff620036000 tx=0x7ff620010040 comp rx=0 tx=0).stop 2024-05-09T08:03:31.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.713+0000 7ff634e00640 1 -- 172.21.15.1:0/2260137164 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff614004d90 msgr2=0x7ff63014b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:31.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.713+0000 7ff634e00640 1 --2- 172.21.15.1:0/2260137164 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff614004d90 0x7ff63014b810 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ff618003bd0 tx=0x7ff61800cc50 comp rx=0 tx=0).stop 2024-05-09T08:03:31.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.713+0000 7ff634e00640 1 -- 172.21.15.1:0/2260137164 shutdown_connections 2024-05-09T08:03:31.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.713+0000 7ff634e00640 1 --2- 172.21.15.1:0/2260137164 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7ff60c06c600 0x7ff60c06eac0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:31.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.713+0000 7ff634e00640 1 --2- 172.21.15.1:0/2260137164 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ff6300c7660 0x7ff630159cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:31.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.713+0000 7ff634e00640 1 --2- 172.21.15.1:0/2260137164 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff614004d90 0x7ff63014b810 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:31.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.713+0000 7ff634e00640 1 --2- 172.21.15.1:0/2260137164 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff6140040b0 0x7ff6301eed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:31.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.713+0000 7ff634e00640 1 -- 172.21.15.1:0/2260137164 >> 172.21.15.1:0/2260137164 conn(0x7ff6300cbc10 msgr2=0x7ff6300cb360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:03:31.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.714+0000 7ff634e00640 1 -- 172.21.15.1:0/2260137164 shutdown_connections 2024-05-09T08:03:31.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.714+0000 7ff634e00640 1 -- 172.21.15.1:0/2260137164 wait complete. 2024-05-09T08:03:31.751 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2024-05-09T08:03:31.751 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon a 2024-05-09T08:03:31.751 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-05-09T08:03:31.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.951+0000 7f1528c00640 1 Processor -- start 2024-05-09T08:03:31.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.951+0000 7f1528c00640 1 -- start start 2024-05-09T08:03:31.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.953+0000 7f1528c00640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f15240c58d0 0x7f15241589e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:31.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.953+0000 7f1528c00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f15240c6470 0x7f1524158f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:31.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.953+0000 7f1528c00640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f15240718e0 con 0x7f15240c54f0 2024-05-09T08:03:31.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.953+0000 7f1528c00640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f15240726e0 con 0x7f15240c58d0 2024-05-09T08:03:31.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.953+0000 7f1528c00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f1524070040 con 0x7f15240c6470 2024-05-09T08:03:31.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.953+0000 7f1523400640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f15240c54f0 0x7f152406f930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:52590/0 (socket says 172.21.15.1:52590) 2024-05-09T08:03:31.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.953+0000 7f1523400640 1 -- 172.21.15.1:0/1224896085 learned_addr learned my addr 172.21.15.1:0/1224896085 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T08:03:31.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.953+0000 7f1523e00640 1 -- 172.21.15.1:0/1224896085 >> v2:172.21.15.7:3300/0 conn(0x7f15240c6470 msgr2=0x7f1524158f20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.7:3300/0 2024-05-09T08:03:31.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.953+0000 7f151be00640 1 --2- 172.21.15.1:0/1224896085 >> v2:172.21.15.1:3300/0 conn(0x7f15240c58d0 0x7f15241589e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:31.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.953+0000 7f1523e00640 1 --2- 172.21.15.1:0/1224896085 >> v2:172.21.15.7:3300/0 conn(0x7f15240c6470 0x7f1524158f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T08:03:31.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.953+0000 7f1522a00640 1 -- 172.21.15.1:0/1224896085 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3383622457 0 0) 0x7f1514001110 con 0x7f15240c54f0 2024-05-09T08:03:31.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.953+0000 7f1522a00640 1 -- 172.21.15.1:0/1224896085 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1508003520 con 0x7f15240c54f0 2024-05-09T08:03:31.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.953+0000 7f151be00640 1 -- 172.21.15.1:0/1224896085 >> v2:172.21.15.7:3300/0 conn(0x7f15240c6470 msgr2=0x7f1524158f20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T08:03:31.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.953+0000 7f151be00640 1 --2- 172.21.15.1:0/1224896085 >> v2:172.21.15.7:3300/0 conn(0x7f15240c6470 0x7f1524158f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:31.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.953+0000 7f151be00640 1 -- 172.21.15.1:0/1224896085 >> v1:172.21.15.1:6789/0 conn(0x7f15240c54f0 legacy=0x7f152406f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:31.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.954+0000 7f151be00640 1 -- 172.21.15.1:0/1224896085 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1524071b60 con 0x7f15240c58d0 2024-05-09T08:03:31.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.955+0000 7f151be00640 1 --2- 172.21.15.1:0/1224896085 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f15240c58d0 0x7f15241589e0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f1524073080 tx=0x7f150c02a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac9e79227dce69ba server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:03:31.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.955+0000 7f1522a00640 1 -- 172.21.15.1:0/1224896085 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f150c037070 con 0x7f15240c58d0 2024-05-09T08:03:31.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.955+0000 7f1522a00640 1 -- 172.21.15.1:0/1224896085 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f150c032440 con 0x7f15240c58d0 2024-05-09T08:03:31.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.955+0000 7f1522a00640 1 -- 172.21.15.1:0/1224896085 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f150c032810 con 0x7f15240c58d0 2024-05-09T08:03:31.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.956+0000 7f1528c00640 1 -- 172.21.15.1:0/1224896085 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f15240c58d0 msgr2=0x7f15241589e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:31.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.956+0000 7f1528c00640 1 --2- 172.21.15.1:0/1224896085 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f15240c58d0 0x7f15241589e0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f1524073080 tx=0x7f150c02a1c0 comp rx=0 tx=0).stop 2024-05-09T08:03:31.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.956+0000 7f1528c00640 1 -- 172.21.15.1:0/1224896085 shutdown_connections 2024-05-09T08:03:31.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.956+0000 7f1528c00640 1 --2- 172.21.15.1:0/1224896085 >> v2:172.21.15.7:3300/0 conn(0x7f15240c6470 0x7f1524158f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:31.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.956+0000 7f1528c00640 1 --2- 172.21.15.1:0/1224896085 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f15240c58d0 0x7f15241589e0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:31.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.956+0000 7f1528c00640 1 -- 172.21.15.1:0/1224896085 >> 172.21.15.1:0/1224896085 conn(0x7f152406b250 msgr2=0x7f152406b630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:03:31.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.956+0000 7f1528c00640 1 -- 172.21.15.1:0/1224896085 shutdown_connections 2024-05-09T08:03:31.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.956+0000 7f1528c00640 1 -- 172.21.15.1:0/1224896085 wait complete. 2024-05-09T08:03:31.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.958+0000 7f1528c00640 1 Processor -- start 2024-05-09T08:03:31.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.959+0000 7f1528c00640 1 -- start start 2024-05-09T08:03:31.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.960+0000 7f1528c00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f15240c54f0 0x7f15240c9b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:31.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.960+0000 7f1528c00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f15240c58d0 0x7f15240c4180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:31.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.960+0000 7f1528c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15240c6470 0x7f15241f7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:31.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.960+0000 7f1528c00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f1524072400 con 0x7f15240c58d0 2024-05-09T08:03:31.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.960+0000 7f1528c00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1524072120 con 0x7f15240c6470 2024-05-09T08:03:31.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.960+0000 7f1528c00640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f15241584c0 con 0x7f15240c54f0 2024-05-09T08:03:31.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.960+0000 7f1523e00640 1 -- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15240c6470 msgr2=0x7f15241f7cb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.7:3300/0 2024-05-09T08:03:31.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.960+0000 7f1523e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15240c6470 0x7f15241f7cb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T08:03:31.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.960+0000 7f151be00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f15240c58d0 0x7f15240c4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:31.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.961+0000 7f1523400640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f15240c54f0 0x7f15240c9b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:31.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.961+0000 7f151be00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f15240c58d0 0x7f15240c4180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:37132/0 (socket says 172.21.15.1:37132) 2024-05-09T08:03:31.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.961+0000 7f151be00640 1 -- 172.21.15.1:0/2542262435 learned_addr learned my addr 172.21.15.1:0/2542262435 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:03:31.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.961+0000 7f151be00640 1 -- 172.21.15.1:0/2542262435 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f15240c54f0 msgr2=0x7f15240c9b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:31.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.961+0000 7f151be00640 1 --2- 172.21.15.1:0/2542262435 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f15240c54f0 0x7f15240c9b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:31.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.961+0000 7f151be00640 1 -- 172.21.15.1:0/2542262435 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15240c6470 msgr2=0x7f15241f7cb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T08:03:31.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.961+0000 7f151be00640 1 --2- 172.21.15.1:0/2542262435 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15240c6470 0x7f15241f7cb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:31.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.961+0000 7f151be00640 1 -- 172.21.15.1:0/2542262435 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15241f88e0 con 0x7f15240c58d0 2024-05-09T08:03:31.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.961+0000 7f1523400640 1 --2- 172.21.15.1:0/2542262435 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f15240c54f0 0x7f15240c9b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:03:31.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.962+0000 7f151be00640 1 --2- 172.21.15.1:0/2542262435 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f15240c58d0 0x7f15240c4180 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f150c009890 tx=0x7f150c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:03:31.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.962+0000 7f151b400640 1 -- 172.21.15.1:0/2542262435 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f150c041070 con 0x7f15240c58d0 2024-05-09T08:03:31.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.962+0000 7f151b400640 1 -- 172.21.15.1:0/2542262435 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f150c03c890 con 0x7f15240c58d0 2024-05-09T08:03:31.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.962+0000 7f151b400640 1 -- 172.21.15.1:0/2542262435 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f150c037040 con 0x7f15240c58d0 2024-05-09T08:03:31.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.962+0000 7f1528c00640 1 -- 172.21.15.1:0/2542262435 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1524144760 con 0x7f15240c58d0 2024-05-09T08:03:31.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.962+0000 7f1528c00640 1 -- 172.21.15.1:0/2542262435 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15240c8f50 con 0x7f15240c58d0 2024-05-09T08:03:31.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.963+0000 7f1528c00640 1 --2- 172.21.15.1:0/2542262435 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f152414ffb0 0x7f15241f4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:31.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.963+0000 7f1528c00640 1 -- 172.21.15.1:0/2542262435 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f1524144a40 con 0x7f152414ffb0 2024-05-09T08:03:31.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.964+0000 7f1523400640 1 --2- 172.21.15.1:0/2542262435 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f152414ffb0 0x7f15241f4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:31.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.964+0000 7f1523400640 1 -- 172.21.15.1:0/2542262435 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f152404f660 con 0x7f152414ffb0 2024-05-09T08:03:31.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.964+0000 7f1523400640 1 --2- 172.21.15.1:0/2542262435 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f152414ffb0 0x7f15241f4720 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f1514000f60 tx=0x7f151400ff70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:03:31.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.964+0000 7f151b400640 1 -- 172.21.15.1:0/2542262435 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f150c044070 con 0x7f15240c58d0 2024-05-09T08:03:31.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.965+0000 7f151b400640 1 --2- 172.21.15.1:0/2542262435 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f150006bde0 0x7f150006e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:31.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.965+0000 7f151b400640 1 -- 172.21.15.1:0/2542262435 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7f150c0abb50 con 0x7f15240c58d0 2024-05-09T08:03:31.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.966+0000 7f151b400640 1 -- 172.21.15.1:0/2542262435 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1514005ac0 con 0x7f152414ffb0 2024-05-09T08:03:31.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.966+0000 7f151b400640 1 -- 172.21.15.1:0/2542262435 <== mon.0 v2:172.21.15.1:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f1514018070 con 0x7f152414ffb0 2024-05-09T08:03:31.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.966+0000 7f1523e00640 1 --2- 172.21.15.1:0/2542262435 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f150006bde0 0x7f150006e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:31.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.966+0000 7f151b400640 1 -- 172.21.15.1:0/2542262435 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f152414ffb0 msgr2=0x7f15241f4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:31.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.966+0000 7f151b400640 1 --2- 172.21.15.1:0/2542262435 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f152414ffb0 0x7f15241f4720 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f1514000f60 tx=0x7f151400ff70 comp rx=0 tx=0).stop 2024-05-09T08:03:31.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.966+0000 7f151b400640 1 -- 172.21.15.1:0/2542262435 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f152414ffb0 msgr2=0x7f15241f4720 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-05-09T08:03:31.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.966+0000 7f151b400640 1 --2- 172.21.15.1:0/2542262435 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f152414ffb0 0x7f15241f4720 secure :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f1514000f60 tx=0x7f151400ff70 comp rx=0 tx=0).stop 2024-05-09T08:03:31.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.966+0000 7f1523e00640 1 --2- 172.21.15.1:0/2542262435 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f150006bde0 0x7f150006e2a0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f151c003cb0 tx=0x7f151c0091f0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:03:31.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.973+0000 7f1528c00640 1 --2- 172.21.15.1:0/2542262435 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f15240b17e0 0x7f15241f4c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:31.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.973+0000 7f1528c00640 1 -- 172.21.15.1:0/2542262435 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f1524151d80 con 0x7f15240b17e0 2024-05-09T08:03:31.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.973+0000 7f1523400640 1 --2- 172.21.15.1:0/2542262435 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f15240b17e0 0x7f15241f4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:31.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.974+0000 7f1523400640 1 -- 172.21.15.1:0/2542262435 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f1524152860 con 0x7f15240b17e0 2024-05-09T08:03:31.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.974+0000 7f1523400640 1 --2- 172.21.15.1:0/2542262435 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f15240b17e0 0x7f15241f4c60 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f1514017000 tx=0x7f1514000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:03:31.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.974+0000 7f151b400640 1 -- 172.21.15.1:0/2542262435 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1514007730 con 0x7f15240b17e0 2024-05-09T08:03:31.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.975+0000 7f151b400640 1 -- 172.21.15.1:0/2542262435 <== mon.0 v2:172.21.15.1:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4694 (secure 0 0 0) 0x7f151400da20 con 0x7f15240b17e0 2024-05-09T08:03:31.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.975+0000 7f151b400640 1 -- 172.21.15.1:0/2542262435 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f15240b17e0 msgr2=0x7f15241f4c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:31.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.975+0000 7f151b400640 1 --2- 172.21.15.1:0/2542262435 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f15240b17e0 0x7f15241f4c60 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f1514017000 tx=0x7f1514000ed0 comp rx=0 tx=0).stop 2024-05-09T08:03:31.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.975+0000 7f151b400640 1 -- 172.21.15.1:0/2542262435 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f15240b17e0 msgr2=0x7f15241f4c60 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-05-09T08:03:31.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.975+0000 7f151b400640 1 --2- 172.21.15.1:0/2542262435 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f15240b17e0 0x7f15241f4c60 secure :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f1514017000 tx=0x7f1514000ed0 comp rx=0 tx=0).stop 2024-05-09T08:03:31.993 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-05-09T08:03:31.993 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "a", 2024-05-09T08:03:31.993 INFO:teuthology.orchestra.run.smithi001.stdout: "rank": 0, 2024-05-09T08:03:31.993 INFO:teuthology.orchestra.run.smithi001.stdout: "state": "leader", 2024-05-09T08:03:31.994 INFO:teuthology.orchestra.run.smithi001.stdout: "election_epoch": 8, 2024-05-09T08:03:31.994 INFO:teuthology.orchestra.run.smithi001.stdout: "uptime": 95337, 2024-05-09T08:03:31.994 INFO:teuthology.orchestra.run.smithi001.stdout: "quorum": [ 2024-05-09T08:03:31.994 INFO:teuthology.orchestra.run.smithi001.stdout: 0, 2024-05-09T08:03:31.994 INFO:teuthology.orchestra.run.smithi001.stdout: 2 2024-05-09T08:03:31.994 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:03:31.994 INFO:teuthology.orchestra.run.smithi001.stdout: "quorum_age": 0, 2024-05-09T08:03:31.994 INFO:teuthology.orchestra.run.smithi001.stdout: "features": { 2024-05-09T08:03:31.994 INFO:teuthology.orchestra.run.smithi001.stdout: "required_con": "2450521708007866372", 2024-05-09T08:03:31.994 INFO:teuthology.orchestra.run.smithi001.stdout: "required_mon": [ 2024-05-09T08:03:31.994 INFO:teuthology.orchestra.run.smithi001.stdout: "kraken", 2024-05-09T08:03:31.994 INFO:teuthology.orchestra.run.smithi001.stdout: "luminous", 2024-05-09T08:03:31.995 INFO:teuthology.orchestra.run.smithi001.stdout: "mimic", 2024-05-09T08:03:31.995 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap-prune", 2024-05-09T08:03:31.995 INFO:teuthology.orchestra.run.smithi001.stdout: "nautilus", 2024-05-09T08:03:31.995 INFO:teuthology.orchestra.run.smithi001.stdout: "octopus", 2024-05-09T08:03:31.995 INFO:teuthology.orchestra.run.smithi001.stdout: "pacific", 2024-05-09T08:03:31.995 INFO:teuthology.orchestra.run.smithi001.stdout: "elector-pinging", 2024-05-09T08:03:31.995 INFO:teuthology.orchestra.run.smithi001.stdout: "quincy", 2024-05-09T08:03:31.995 INFO:teuthology.orchestra.run.smithi001.stdout: "reef", 2024-05-09T08:03:31.995 INFO:teuthology.orchestra.run.smithi001.stdout: "squid" 2024-05-09T08:03:31.995 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:03:31.995 INFO:teuthology.orchestra.run.smithi001.stdout: "quorum_con": "4540701547738038271", 2024-05-09T08:03:31.995 INFO:teuthology.orchestra.run.smithi001.stdout: "quorum_mon": [ 2024-05-09T08:03:31.995 INFO:teuthology.orchestra.run.smithi001.stdout: "kraken", 2024-05-09T08:03:31.995 INFO:teuthology.orchestra.run.smithi001.stdout: "luminous", 2024-05-09T08:03:31.996 INFO:teuthology.orchestra.run.smithi001.stdout: "mimic", 2024-05-09T08:03:31.996 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap-prune", 2024-05-09T08:03:31.996 INFO:teuthology.orchestra.run.smithi001.stdout: "nautilus", 2024-05-09T08:03:31.996 INFO:teuthology.orchestra.run.smithi001.stdout: "octopus", 2024-05-09T08:03:31.996 INFO:teuthology.orchestra.run.smithi001.stdout: "pacific", 2024-05-09T08:03:31.996 INFO:teuthology.orchestra.run.smithi001.stdout: "elector-pinging", 2024-05-09T08:03:31.996 INFO:teuthology.orchestra.run.smithi001.stdout: "quincy", 2024-05-09T08:03:31.996 INFO:teuthology.orchestra.run.smithi001.stdout: "reef", 2024-05-09T08:03:31.996 INFO:teuthology.orchestra.run.smithi001.stdout: "squid" 2024-05-09T08:03:31.996 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:03:31.996 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:03:31.996 INFO:teuthology.orchestra.run.smithi001.stdout: "outside_quorum": [], 2024-05-09T08:03:31.996 INFO:teuthology.orchestra.run.smithi001.stdout: "extra_probe_peers": [], 2024-05-09T08:03:31.997 INFO:teuthology.orchestra.run.smithi001.stdout: "sync_provider": [], 2024-05-09T08:03:31.997 INFO:teuthology.orchestra.run.smithi001.stdout: "monmap": { 2024-05-09T08:03:31.997 INFO:teuthology.orchestra.run.smithi001.stdout: "epoch": 1, 2024-05-09T08:03:31.997 INFO:teuthology.orchestra.run.smithi001.stdout: "fsid": "54329ec8-3270-4043-9a82-c6cd7420aa7f", 2024-05-09T08:03:31.997 INFO:teuthology.orchestra.run.smithi001.stdout: "modified": "2024-05-09T08:01:21.819270Z", 2024-05-09T08:03:31.997 INFO:teuthology.orchestra.run.smithi001.stdout: "created": "2024-05-09T08:01:21.819270Z", 2024-05-09T08:03:31.997 INFO:teuthology.orchestra.run.smithi001.stdout: "min_mon_release": 19, 2024-05-09T08:03:31.997 INFO:teuthology.orchestra.run.smithi001.stdout: "min_mon_release_name": "squid", 2024-05-09T08:03:31.997 INFO:teuthology.orchestra.run.smithi001.stdout: "election_strategy": 1, 2024-05-09T08:03:31.997 INFO:teuthology.orchestra.run.smithi001.stdout: "disallowed_leaders: ": "", 2024-05-09T08:03:31.997 INFO:teuthology.orchestra.run.smithi001.stdout: "stretch_mode": false, 2024-05-09T08:03:31.997 INFO:teuthology.orchestra.run.smithi001.stdout: "tiebreaker_mon": "", 2024-05-09T08:03:31.997 INFO:teuthology.orchestra.run.smithi001.stdout: "removed_ranks: ": "", 2024-05-09T08:03:31.997 INFO:teuthology.orchestra.run.smithi001.stdout: "features": { 2024-05-09T08:03:31.998 INFO:teuthology.orchestra.run.smithi001.stdout: "persistent": [ 2024-05-09T08:03:31.998 INFO:teuthology.orchestra.run.smithi001.stdout: "kraken", 2024-05-09T08:03:31.998 INFO:teuthology.orchestra.run.smithi001.stdout: "luminous", 2024-05-09T08:03:31.998 INFO:teuthology.orchestra.run.smithi001.stdout: "mimic", 2024-05-09T08:03:31.998 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap-prune", 2024-05-09T08:03:31.998 INFO:teuthology.orchestra.run.smithi001.stdout: "nautilus", 2024-05-09T08:03:31.998 INFO:teuthology.orchestra.run.smithi001.stdout: "octopus", 2024-05-09T08:03:31.998 INFO:teuthology.orchestra.run.smithi001.stdout: "pacific", 2024-05-09T08:03:31.998 INFO:teuthology.orchestra.run.smithi001.stdout: "elector-pinging", 2024-05-09T08:03:31.998 INFO:teuthology.orchestra.run.smithi001.stdout: "quincy", 2024-05-09T08:03:31.998 INFO:teuthology.orchestra.run.smithi001.stdout: "reef", 2024-05-09T08:03:31.998 INFO:teuthology.orchestra.run.smithi001.stdout: "squid" 2024-05-09T08:03:31.998 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:03:31.999 INFO:teuthology.orchestra.run.smithi001.stdout: "optional": [] 2024-05-09T08:03:31.999 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:03:31.999 INFO:teuthology.orchestra.run.smithi001.stdout: "mons": [ 2024-05-09T08:03:31.999 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:03:31.999 INFO:teuthology.orchestra.run.smithi001.stdout: "rank": 0, 2024-05-09T08:03:31.999 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "a", 2024-05-09T08:03:31.999 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-05-09T08:03:31.999 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-05-09T08:03:31.999 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:03:31.999 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-05-09T08:03:31.999 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:3300", 2024-05-09T08:03:31.999 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:03:32.000 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:03:32.000 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:03:32.000 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-05-09T08:03:32.000 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6789", 2024-05-09T08:03:32.000 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:03:32.000 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:03:32.000 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:03:32.000 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:03:32.000 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6789/0", 2024-05-09T08:03:32.000 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.1:6789/0", 2024-05-09T08:03:32.000 INFO:teuthology.orchestra.run.smithi001.stdout: "priority": 0, 2024-05-09T08:03:32.000 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 0, 2024-05-09T08:03:32.000 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_location": "{}" 2024-05-09T08:03:32.000 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:03:32.001 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:03:32.001 INFO:teuthology.orchestra.run.smithi001.stdout: "rank": 1, 2024-05-09T08:03:32.001 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "b", 2024-05-09T08:03:32.001 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-05-09T08:03:32.001 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-05-09T08:03:32.001 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:03:32.001 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-05-09T08:03:32.001 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:3300", 2024-05-09T08:03:32.001 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:03:32.001 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:03:32.001 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:03:32.002 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-05-09T08:03:32.002 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:6789", 2024-05-09T08:03:32.002 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:03:32.002 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:03:32.002 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:03:32.002 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:03:32.002 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:6789/0", 2024-05-09T08:03:32.002 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.7:6789/0", 2024-05-09T08:03:32.002 INFO:teuthology.orchestra.run.smithi001.stdout: "priority": 0, 2024-05-09T08:03:32.002 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 0, 2024-05-09T08:03:32.002 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_location": "{}" 2024-05-09T08:03:32.002 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:03:32.003 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:03:32.003 INFO:teuthology.orchestra.run.smithi001.stdout: "rank": 2, 2024-05-09T08:03:32.003 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "c", 2024-05-09T08:03:32.003 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-05-09T08:03:32.003 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-05-09T08:03:32.003 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:03:32.003 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-05-09T08:03:32.003 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:3301", 2024-05-09T08:03:32.003 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:03:32.003 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:03:32.003 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:03:32.004 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-05-09T08:03:32.004 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:6790", 2024-05-09T08:03:32.004 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:03:32.004 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:03:32.004 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:03:32.004 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:03:32.004 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:6790/0", 2024-05-09T08:03:32.004 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.7:6790/0", 2024-05-09T08:03:32.004 INFO:teuthology.orchestra.run.smithi001.stdout: "priority": 0, 2024-05-09T08:03:32.004 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 0, 2024-05-09T08:03:32.004 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_location": "{}" 2024-05-09T08:03:32.004 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:03:32.004 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:03:32.004 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:03:32.005 INFO:teuthology.orchestra.run.smithi001.stdout: "feature_map": { 2024-05-09T08:03:32.005 INFO:teuthology.orchestra.run.smithi001.stdout: "mon": [ 2024-05-09T08:03:32.005 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:03:32.005 INFO:teuthology.orchestra.run.smithi001.stdout: "features": "0x3f03cffffffdffff", 2024-05-09T08:03:32.005 INFO:teuthology.orchestra.run.smithi001.stdout: "release": "luminous", 2024-05-09T08:03:32.005 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 1 2024-05-09T08:03:32.005 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:03:32.005 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:03:32.005 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": [ 2024-05-09T08:03:32.005 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:03:32.005 INFO:teuthology.orchestra.run.smithi001.stdout: "features": "0x3f03cffffffdffff", 2024-05-09T08:03:32.005 INFO:teuthology.orchestra.run.smithi001.stdout: "release": "luminous", 2024-05-09T08:03:32.005 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 2 2024-05-09T08:03:32.006 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:03:32.006 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:03:32.006 INFO:teuthology.orchestra.run.smithi001.stdout: "client": [ 2024-05-09T08:03:32.006 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:03:32.006 INFO:teuthology.orchestra.run.smithi001.stdout: "features": "0x3f03cffffffdffff", 2024-05-09T08:03:32.006 INFO:teuthology.orchestra.run.smithi001.stdout: "release": "luminous", 2024-05-09T08:03:32.006 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 2 2024-05-09T08:03:32.006 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:03:32.006 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:03:32.006 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:03:32.006 INFO:teuthology.orchestra.run.smithi001.stdout: "stretch_mode": false 2024-05-09T08:03:32.006 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-05-09T08:03:32.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.975+0000 7f1528c00640 1 -- 172.21.15.1:0/2542262435 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f150006bde0 msgr2=0x7f150006e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:32.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.975+0000 7f1528c00640 1 --2- 172.21.15.1:0/2542262435 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f150006bde0 0x7f150006e2a0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f151c003cb0 tx=0x7f151c0091f0 comp rx=0 tx=0).stop 2024-05-09T08:03:32.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.975+0000 7f1528c00640 1 -- 172.21.15.1:0/2542262435 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f15240c58d0 msgr2=0x7f15240c4180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:32.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.975+0000 7f1528c00640 1 --2- 172.21.15.1:0/2542262435 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f15240c58d0 0x7f15240c4180 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f150c009890 tx=0x7f150c0046c0 comp rx=0 tx=0).stop 2024-05-09T08:03:32.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.975+0000 7f1523e00640 1 -- 172.21.15.1:0/2542262435 reap_dead start 2024-05-09T08:03:32.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.975+0000 7f1523e00640 1 -- 172.21.15.1:0/2542262435 reap_dead start 2024-05-09T08:03:32.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.975+0000 7f1528c00640 1 -- 172.21.15.1:0/2542262435 shutdown_connections 2024-05-09T08:03:32.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.975+0000 7f1528c00640 1 -- 172.21.15.1:0/2542262435 >> 172.21.15.1:0/2542262435 conn(0x7f152406b250 msgr2=0x7f1524143f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:03:32.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.975+0000 7f1528c00640 1 -- 172.21.15.1:0/2542262435 shutdown_connections 2024-05-09T08:03:32.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:31.975+0000 7f1528c00640 1 -- 172.21.15.1:0/2542262435 wait complete. 2024-05-09T08:03:32.009 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-05-09T08:03:32.009 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon c 2024-05-09T08:03:32.009 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-05-09T08:03:32.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:32.194+0000 7f028b200640 1 Processor -- start 2024-05-09T08:03:32.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:32.194+0000 7f028b200640 1 -- start start 2024-05-09T08:03:32.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:32.196+0000 7f028b200640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f028c0c54f0 0x7f028c06f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:32.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:32.196+0000 7f028b200640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f028c0c6470 0x7f028c151d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:32.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:32.196+0000 7f028b200640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f028c0718e0 con 0x7f028c0c58d0 2024-05-09T08:03:32.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:32.196+0000 7f028b200640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f028c0726e0 con 0x7f028c0c6470 2024-05-09T08:03:32.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:32.196+0000 7f028b200640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f028c071b60 con 0x7f028c0c54f0 2024-05-09T08:03:32.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:32.196+0000 7f0289e00640 1 -- >> v2:172.21.15.7:3300/0 conn(0x7f028c0c54f0 msgr2=0x7f028c06f930 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.7:3300/0 2024-05-09T08:03:32.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:32.196+0000 7f028a800640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f028c0c6470 0x7f028c151d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:32.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:32.196+0000 7f0289e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f028c0c54f0 0x7f028c06f930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T08:03:32.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:32.196+0000 7f0289400640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f028c0c58d0 0x7f028c1589e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:52600/0 (socket says 172.21.15.1:52600) 2024-05-09T08:03:32.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:32.196+0000 7f028a800640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f028c0c6470 0x7f028c151d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:37152/0 (socket says 172.21.15.1:37152) 2024-05-09T08:03:32.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:32.196+0000 7f0289400640 1 -- 172.21.15.1:0/138345024 learned_addr learned my addr 172.21.15.1:0/138345024 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T08:03:32.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:32.197+0000 7f028a800640 1 -- 172.21.15.1:0/138345024 >> v2:172.21.15.7:3300/0 conn(0x7f028c0c54f0 msgr2=0x7f028c06f930 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T08:03:32.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:32.197+0000 7f028a800640 1 --2- 172.21.15.1:0/138345024 >> v2:172.21.15.7:3300/0 conn(0x7f028c0c54f0 0x7f028c06f930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:32.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:32.197+0000 7f028a800640 1 -- 172.21.15.1:0/138345024 >> v1:172.21.15.1:6789/0 conn(0x7f028c0c58d0 legacy=0x7f028c1589e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:32.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:32.197+0000 7f028a800640 1 -- 172.21.15.1:0/138345024 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f028c071e40 con 0x7f028c0c6470 2024-05-09T08:03:32.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:32.197+0000 7f0288a00640 1 -- 172.21.15.1:0/138345024 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1968068623 0 0) 0x7f0278001110 con 0x7f028c0c58d0 2024-05-09T08:03:32.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:32.198+0000 7f028a800640 1 --2- 172.21.15.1:0/138345024 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f028c0c6470 0x7f028c151d60 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f028c0c96b0 tx=0x7f028402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8030590d4b8db3ca server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:03:32.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:32.198+0000 7f0288a00640 1 -- 172.21.15.1:0/138345024 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0284007740 con 0x7f028c0c6470 2024-05-09T08:03:32.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:32.198+0000 7f0288a00640 1 -- 172.21.15.1:0/138345024 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02840078e0 con 0x7f028c0c6470 2024-05-09T08:03:32.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:32.198+0000 7f0288a00640 1 -- 172.21.15.1:0/138345024 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0284007cb0 con 0x7f028c0c6470 2024-05-09T08:03:32.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:32.199+0000 7f028b200640 1 -- 172.21.15.1:0/138345024 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f028c0c6470 msgr2=0x7f028c151d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:32.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:32.199+0000 7f028b200640 1 --2- 172.21.15.1:0/138345024 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f028c0c6470 0x7f028c151d60 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f028c0c96b0 tx=0x7f028402a6a0 comp rx=0 tx=0).stop 2024-05-09T08:03:32.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:32.199+0000 7f028b200640 1 -- 172.21.15.1:0/138345024 shutdown_connections 2024-05-09T08:03:32.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:32.199+0000 7f028b200640 1 --2- 172.21.15.1:0/138345024 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f028c0c6470 0x7f028c151d60 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:32.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:32.199+0000 7f028b200640 1 --2- 172.21.15.1:0/138345024 >> v2:172.21.15.7:3300/0 conn(0x7f028c0c54f0 0x7f028c06f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:32.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:32.199+0000 7f028b200640 1 -- 172.21.15.1:0/138345024 >> 172.21.15.1:0/138345024 conn(0x7f028c06b250 msgr2=0x7f028c06b630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:03:32.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:32.199+0000 7f028b200640 1 -- 172.21.15.1:0/138345024 shutdown_connections 2024-05-09T08:03:32.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:32.199+0000 7f028b200640 1 -- 172.21.15.1:0/138345024 wait complete. 2024-05-09T08:03:32.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:32.201+0000 7f028b200640 1 Processor -- start 2024-05-09T08:03:32.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:32.201+0000 7f028b200640 1 -- start start 2024-05-09T08:03:32.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:32.202+0000 7f028b200640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f028c0c54f0 0x7f028c0cc680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:32.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:32.202+0000 7f028b200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f028c0c58d0 0x7f028c0cad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:32.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:32.202+0000 7f028b200640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f028c0c6470 0x7f028c1e4b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:32.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:32.202+0000 7f028b200640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f028c1584c0 con 0x7f028c0c54f0 2024-05-09T08:03:32.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:32.202+0000 7f028b200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f028c072400 con 0x7f028c0c58d0 2024-05-09T08:03:32.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:32.202+0000 7f028b200640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f028c1590f0 con 0x7f028c0c6470 2024-05-09T08:03:32.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:32.202+0000 7f0289400640 1 -- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f028c0c58d0 msgr2=0x7f028c0cad00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.7:3300/0 2024-05-09T08:03:32.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:32.202+0000 7f0289400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f028c0c58d0 0x7f028c0cad00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T08:03:32.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:32.202+0000 7f0289e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f028c0c54f0 0x7f028c0cc680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:32.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:32.202+0000 7f0289e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f028c0c54f0 0x7f028c0cc680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:37160/0 (socket says 172.21.15.1:37160) 2024-05-09T08:03:32.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:32.202+0000 7f0289e00640 1 -- 172.21.15.1:0/3603465284 learned_addr learned my addr 172.21.15.1:0/3603465284 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:03:32.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:32.202+0000 7f028a800640 1 --2- 172.21.15.1:0/3603465284 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f028c0c6470 0x7f028c1e4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:32.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:32.203+0000 7f0289e00640 1 -- 172.21.15.1:0/3603465284 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f028c0c6470 msgr2=0x7f028c1e4b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:32.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:32.203+0000 7f0289e00640 1 --2- 172.21.15.1:0/3603465284 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f028c0c6470 0x7f028c1e4b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:32.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:32.203+0000 7f0289e00640 1 -- 172.21.15.1:0/3603465284 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f028c0c58d0 msgr2=0x7f028c0cad00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T08:03:32.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:32.203+0000 7f0289e00640 1 --2- 172.21.15.1:0/3603465284 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f028c0c58d0 0x7f028c0cad00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:32.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:32.203+0000 7f0289e00640 1 -- 172.21.15.1:0/3603465284 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f028c0c9e40 con 0x7f028c0c54f0 2024-05-09T08:03:32.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:32.203+0000 7f0289e00640 1 --2- 172.21.15.1:0/3603465284 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f028c0c54f0 0x7f028c0cc680 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f027c00b580 tx=0x7f027c00ba40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:03:32.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:32.203+0000 7f0282000640 1 -- 172.21.15.1:0/3603465284 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f027c013020 con 0x7f028c0c54f0 2024-05-09T08:03:32.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:32.203+0000 7f0282000640 1 -- 172.21.15.1:0/3603465284 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f027c003de0 con 0x7f028c0c54f0 2024-05-09T08:03:32.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:32.203+0000 7f0282000640 1 -- 172.21.15.1:0/3603465284 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f027c0052b0 con 0x7f028c0c54f0 2024-05-09T08:03:32.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:32.203+0000 7f028b200640 1 -- 172.21.15.1:0/3603465284 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f028c155150 con 0x7f028c0c54f0 2024-05-09T08:03:32.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:32.203+0000 7f028b200640 1 -- 172.21.15.1:0/3603465284 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f028c153f50 con 0x7f028c0c54f0 2024-05-09T08:03:32.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:32.204+0000 7f028b200640 1 --2- 172.21.15.1:0/3603465284 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f028c0cbca0 0x7f028c154470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:32.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:32.204+0000 7f028b200640 1 -- 172.21.15.1:0/3603465284 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f028c14f5a0 con 0x7f028c0cbca0 2024-05-09T08:03:32.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:32.205+0000 7f0289400640 1 --2- 172.21.15.1:0/3603465284 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f028c0cbca0 0x7f028c154470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:32.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:32.205+0000 7f0289400640 1 -- 172.21.15.1:0/3603465284 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f028c154150 con 0x7f028c0cbca0 2024-05-09T08:03:32.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:32.206+0000 7f0289400640 1 --2- 172.21.15.1:0/3603465284 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f028c0cbca0 0x7f028c154470 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f028c0c5ce0 tx=0x7f027800cb10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:03:32.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:32.206+0000 7f0282000640 1 -- 172.21.15.1:0/3603465284 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f027c016020 con 0x7f028c0c54f0 2024-05-09T08:03:32.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:32.207+0000 7f0282000640 1 --2- 172.21.15.1:0/3603465284 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f026c06bde0 0x7f026c06e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:32.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:32.207+0000 7f0282000640 1 -- 172.21.15.1:0/3603465284 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7f027c08aed0 con 0x7f028c0c54f0 2024-05-09T08:03:32.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:32.207+0000 7f0282000640 1 -- 172.21.15.1:0/3603465284 <== mon.2 v2:172.21.15.7:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0278017070 con 0x7f028c0cbca0 2024-05-09T08:03:32.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:32.207+0000 7f0282000640 1 -- 172.21.15.1:0/3603465284 <== mon.2 v2:172.21.15.7:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f027800ce50 con 0x7f028c0cbca0 2024-05-09T08:03:32.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:32.207+0000 7f0282000640 1 -- 172.21.15.1:0/3603465284 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f028c0cbca0 msgr2=0x7f028c154470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:32.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:32.207+0000 7f0282000640 1 --2- 172.21.15.1:0/3603465284 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f028c0cbca0 0x7f028c154470 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f028c0c5ce0 tx=0x7f027800cb10 comp rx=0 tx=0).stop 2024-05-09T08:03:32.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:32.207+0000 7f0282000640 1 -- 172.21.15.1:0/3603465284 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f028c0cbca0 msgr2=0x7f028c154470 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-05-09T08:03:32.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:32.207+0000 7f0282000640 1 --2- 172.21.15.1:0/3603465284 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f028c0cbca0 0x7f028c154470 secure :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f028c0c5ce0 tx=0x7f027800cb10 comp rx=0 tx=0).stop 2024-05-09T08:03:32.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:32.207+0000 7f028a800640 1 --2- 172.21.15.1:0/3603465284 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f026c06bde0 0x7f026c06e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:32.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:32.208+0000 7f028a800640 1 --2- 172.21.15.1:0/3603465284 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f026c06bde0 0x7f026c06e2a0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f0284034000 tx=0x7f028402f000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:03:32.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:32.215+0000 7f028b200640 1 --2- 172.21.15.1:0/3603465284 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f028c1549b0 0x7f028c1f3780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:32.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:32.215+0000 7f028b200640 1 -- 172.21.15.1:0/3603465284 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f028c154dc0 con 0x7f028c1549b0 2024-05-09T08:03:32.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:32.215+0000 7f0289400640 1 --2- 172.21.15.1:0/3603465284 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f028c1549b0 0x7f028c1f3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:32.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:32.216+0000 7f0289400640 1 -- 172.21.15.1:0/3603465284 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f028c04f660 con 0x7f028c1549b0 2024-05-09T08:03:32.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:32.216+0000 7f0289400640 1 --2- 172.21.15.1:0/3603465284 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f028c1549b0 0x7f028c1f3780 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f028c0c5ce0 tx=0x7f027800f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:03:32.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:32.216+0000 7f0282000640 1 -- 172.21.15.1:0/3603465284 <== mon.2 v2:172.21.15.7:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0278004c20 con 0x7f028c1549b0 2024-05-09T08:03:32.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:32.216+0000 7f0282000640 1 -- 172.21.15.1:0/3603465284 <== mon.2 v2:172.21.15.7:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5175 (secure 0 0 0) 0x7f027801a6c0 con 0x7f028c1549b0 2024-05-09T08:03:32.234 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-05-09T08:03:32.234 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "c", 2024-05-09T08:03:32.234 INFO:teuthology.orchestra.run.smithi001.stdout: "rank": 2, 2024-05-09T08:03:32.234 INFO:teuthology.orchestra.run.smithi001.stdout: "state": "peon", 2024-05-09T08:03:32.234 INFO:teuthology.orchestra.run.smithi001.stdout: "election_epoch": 8, 2024-05-09T08:03:32.234 INFO:teuthology.orchestra.run.smithi001.stdout: "uptime": 95632, 2024-05-09T08:03:32.234 INFO:teuthology.orchestra.run.smithi001.stdout: "quorum": [ 2024-05-09T08:03:32.235 INFO:teuthology.orchestra.run.smithi001.stdout: 0, 2024-05-09T08:03:32.235 INFO:teuthology.orchestra.run.smithi001.stdout: 2 2024-05-09T08:03:32.235 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:03:32.235 INFO:teuthology.orchestra.run.smithi001.stdout: "quorum_age": 0, 2024-05-09T08:03:32.235 INFO:teuthology.orchestra.run.smithi001.stdout: "features": { 2024-05-09T08:03:32.235 INFO:teuthology.orchestra.run.smithi001.stdout: "required_con": "2450521708007866372", 2024-05-09T08:03:32.235 INFO:teuthology.orchestra.run.smithi001.stdout: "required_mon": [ 2024-05-09T08:03:32.235 INFO:teuthology.orchestra.run.smithi001.stdout: "kraken", 2024-05-09T08:03:32.235 INFO:teuthology.orchestra.run.smithi001.stdout: "luminous", 2024-05-09T08:03:32.235 INFO:teuthology.orchestra.run.smithi001.stdout: "mimic", 2024-05-09T08:03:32.235 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap-prune", 2024-05-09T08:03:32.235 INFO:teuthology.orchestra.run.smithi001.stdout: "nautilus", 2024-05-09T08:03:32.235 INFO:teuthology.orchestra.run.smithi001.stdout: "octopus", 2024-05-09T08:03:32.236 INFO:teuthology.orchestra.run.smithi001.stdout: "pacific", 2024-05-09T08:03:32.236 INFO:teuthology.orchestra.run.smithi001.stdout: "elector-pinging", 2024-05-09T08:03:32.236 INFO:teuthology.orchestra.run.smithi001.stdout: "quincy", 2024-05-09T08:03:32.236 INFO:teuthology.orchestra.run.smithi001.stdout: "reef", 2024-05-09T08:03:32.236 INFO:teuthology.orchestra.run.smithi001.stdout: "squid" 2024-05-09T08:03:32.236 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:03:32.236 INFO:teuthology.orchestra.run.smithi001.stdout: "quorum_con": "4540701547738038271", 2024-05-09T08:03:32.236 INFO:teuthology.orchestra.run.smithi001.stdout: "quorum_mon": [ 2024-05-09T08:03:32.236 INFO:teuthology.orchestra.run.smithi001.stdout: "kraken", 2024-05-09T08:03:32.236 INFO:teuthology.orchestra.run.smithi001.stdout: "luminous", 2024-05-09T08:03:32.236 INFO:teuthology.orchestra.run.smithi001.stdout: "mimic", 2024-05-09T08:03:32.236 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap-prune", 2024-05-09T08:03:32.236 INFO:teuthology.orchestra.run.smithi001.stdout: "nautilus", 2024-05-09T08:03:32.236 INFO:teuthology.orchestra.run.smithi001.stdout: "octopus", 2024-05-09T08:03:32.236 INFO:teuthology.orchestra.run.smithi001.stdout: "pacific", 2024-05-09T08:03:32.237 INFO:teuthology.orchestra.run.smithi001.stdout: "elector-pinging", 2024-05-09T08:03:32.237 INFO:teuthology.orchestra.run.smithi001.stdout: "quincy", 2024-05-09T08:03:32.237 INFO:teuthology.orchestra.run.smithi001.stdout: "reef", 2024-05-09T08:03:32.237 INFO:teuthology.orchestra.run.smithi001.stdout: "squid" 2024-05-09T08:03:32.237 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:03:32.237 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:03:32.237 INFO:teuthology.orchestra.run.smithi001.stdout: "outside_quorum": [], 2024-05-09T08:03:32.237 INFO:teuthology.orchestra.run.smithi001.stdout: "extra_probe_peers": [], 2024-05-09T08:03:32.237 INFO:teuthology.orchestra.run.smithi001.stdout: "sync_provider": [], 2024-05-09T08:03:32.237 INFO:teuthology.orchestra.run.smithi001.stdout: "monmap": { 2024-05-09T08:03:32.237 INFO:teuthology.orchestra.run.smithi001.stdout: "epoch": 1, 2024-05-09T08:03:32.237 INFO:teuthology.orchestra.run.smithi001.stdout: "fsid": "54329ec8-3270-4043-9a82-c6cd7420aa7f", 2024-05-09T08:03:32.237 INFO:teuthology.orchestra.run.smithi001.stdout: "modified": "2024-05-09T08:01:21.819270Z", 2024-05-09T08:03:32.237 INFO:teuthology.orchestra.run.smithi001.stdout: "created": "2024-05-09T08:01:21.819270Z", 2024-05-09T08:03:32.237 INFO:teuthology.orchestra.run.smithi001.stdout: "min_mon_release": 19, 2024-05-09T08:03:32.238 INFO:teuthology.orchestra.run.smithi001.stdout: "min_mon_release_name": "squid", 2024-05-09T08:03:32.238 INFO:teuthology.orchestra.run.smithi001.stdout: "election_strategy": 1, 2024-05-09T08:03:32.238 INFO:teuthology.orchestra.run.smithi001.stdout: "disallowed_leaders: ": "", 2024-05-09T08:03:32.238 INFO:teuthology.orchestra.run.smithi001.stdout: "stretch_mode": false, 2024-05-09T08:03:32.238 INFO:teuthology.orchestra.run.smithi001.stdout: "tiebreaker_mon": "", 2024-05-09T08:03:32.238 INFO:teuthology.orchestra.run.smithi001.stdout: "removed_ranks: ": "", 2024-05-09T08:03:32.238 INFO:teuthology.orchestra.run.smithi001.stdout: "features": { 2024-05-09T08:03:32.238 INFO:teuthology.orchestra.run.smithi001.stdout: "persistent": [ 2024-05-09T08:03:32.238 INFO:teuthology.orchestra.run.smithi001.stdout: "kraken", 2024-05-09T08:03:32.238 INFO:teuthology.orchestra.run.smithi001.stdout: "luminous", 2024-05-09T08:03:32.238 INFO:teuthology.orchestra.run.smithi001.stdout: "mimic", 2024-05-09T08:03:32.238 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap-prune", 2024-05-09T08:03:32.238 INFO:teuthology.orchestra.run.smithi001.stdout: "nautilus", 2024-05-09T08:03:32.238 INFO:teuthology.orchestra.run.smithi001.stdout: "octopus", 2024-05-09T08:03:32.239 INFO:teuthology.orchestra.run.smithi001.stdout: "pacific", 2024-05-09T08:03:32.239 INFO:teuthology.orchestra.run.smithi001.stdout: "elector-pinging", 2024-05-09T08:03:32.239 INFO:teuthology.orchestra.run.smithi001.stdout: "quincy", 2024-05-09T08:03:32.239 INFO:teuthology.orchestra.run.smithi001.stdout: "reef", 2024-05-09T08:03:32.239 INFO:teuthology.orchestra.run.smithi001.stdout: "squid" 2024-05-09T08:03:32.239 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:03:32.239 INFO:teuthology.orchestra.run.smithi001.stdout: "optional": [] 2024-05-09T08:03:32.239 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:03:32.239 INFO:teuthology.orchestra.run.smithi001.stdout: "mons": [ 2024-05-09T08:03:32.239 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:03:32.239 INFO:teuthology.orchestra.run.smithi001.stdout: "rank": 0, 2024-05-09T08:03:32.239 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "a", 2024-05-09T08:03:32.239 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-05-09T08:03:32.239 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-05-09T08:03:32.240 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:03:32.240 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-05-09T08:03:32.240 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:3300", 2024-05-09T08:03:32.240 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:03:32.240 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:03:32.240 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:03:32.240 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-05-09T08:03:32.240 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6789", 2024-05-09T08:03:32.240 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:03:32.240 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:03:32.240 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:03:32.240 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:03:32.240 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6789/0", 2024-05-09T08:03:32.240 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.1:6789/0", 2024-05-09T08:03:32.240 INFO:teuthology.orchestra.run.smithi001.stdout: "priority": 0, 2024-05-09T08:03:32.241 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 0, 2024-05-09T08:03:32.241 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_location": "{}" 2024-05-09T08:03:32.241 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:03:32.241 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:03:32.241 INFO:teuthology.orchestra.run.smithi001.stdout: "rank": 1, 2024-05-09T08:03:32.241 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "b", 2024-05-09T08:03:32.241 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-05-09T08:03:32.241 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-05-09T08:03:32.241 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:03:32.241 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-05-09T08:03:32.241 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:3300", 2024-05-09T08:03:32.241 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:03:32.241 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:03:32.241 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:03:32.242 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-05-09T08:03:32.242 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:6789", 2024-05-09T08:03:32.242 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:03:32.242 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:03:32.242 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:03:32.242 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:03:32.242 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:6789/0", 2024-05-09T08:03:32.242 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.7:6789/0", 2024-05-09T08:03:32.242 INFO:teuthology.orchestra.run.smithi001.stdout: "priority": 0, 2024-05-09T08:03:32.242 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 0, 2024-05-09T08:03:32.242 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_location": "{}" 2024-05-09T08:03:32.242 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:03:32.242 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:03:32.242 INFO:teuthology.orchestra.run.smithi001.stdout: "rank": 2, 2024-05-09T08:03:32.243 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "c", 2024-05-09T08:03:32.243 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-05-09T08:03:32.243 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-05-09T08:03:32.243 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:03:32.243 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-05-09T08:03:32.243 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:3301", 2024-05-09T08:03:32.243 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:03:32.243 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:03:32.243 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:03:32.243 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-05-09T08:03:32.243 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:6790", 2024-05-09T08:03:32.243 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:03:32.243 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:03:32.243 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:03:32.244 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:03:32.244 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:6790/0", 2024-05-09T08:03:32.244 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.7:6790/0", 2024-05-09T08:03:32.244 INFO:teuthology.orchestra.run.smithi001.stdout: "priority": 0, 2024-05-09T08:03:32.244 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 0, 2024-05-09T08:03:32.244 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_location": "{}" 2024-05-09T08:03:32.244 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:03:32.244 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:03:32.244 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:03:32.244 INFO:teuthology.orchestra.run.smithi001.stdout: "feature_map": { 2024-05-09T08:03:32.244 INFO:teuthology.orchestra.run.smithi001.stdout: "mon": [ 2024-05-09T08:03:32.244 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:03:32.244 INFO:teuthology.orchestra.run.smithi001.stdout: "features": "0x3f03cffffffdffff", 2024-05-09T08:03:32.244 INFO:teuthology.orchestra.run.smithi001.stdout: "release": "luminous", 2024-05-09T08:03:32.244 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 1 2024-05-09T08:03:32.244 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:03:32.245 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:03:32.245 INFO:teuthology.orchestra.run.smithi001.stdout: "mds": [ 2024-05-09T08:03:32.245 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:03:32.245 INFO:teuthology.orchestra.run.smithi001.stdout: "features": "0x3f03cffffffdffff", 2024-05-09T08:03:32.245 INFO:teuthology.orchestra.run.smithi001.stdout: "release": "luminous", 2024-05-09T08:03:32.245 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 6 2024-05-09T08:03:32.245 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:03:32.245 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:03:32.245 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": [ 2024-05-09T08:03:32.245 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:03:32.245 INFO:teuthology.orchestra.run.smithi001.stdout: "features": "0x3f03cffffffdffff", 2024-05-09T08:03:32.245 INFO:teuthology.orchestra.run.smithi001.stdout: "release": "luminous", 2024-05-09T08:03:32.245 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 6 2024-05-09T08:03:32.245 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:03:32.245 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:03:32.246 INFO:teuthology.orchestra.run.smithi001.stdout: "client": [ 2024-05-09T08:03:32.246 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:03:32.246 INFO:teuthology.orchestra.run.smithi001.stdout: "features": "0x2f018fb87aa4aafe", 2024-05-09T08:03:32.246 INFO:teuthology.orchestra.run.smithi001.stdout: "release": "luminous", 2024-05-09T08:03:32.246 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 1 2024-05-09T08:03:32.246 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:03:32.246 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:03:32.246 INFO:teuthology.orchestra.run.smithi001.stdout: "features": "0x3f03cffffffdffff", 2024-05-09T08:03:32.246 INFO:teuthology.orchestra.run.smithi001.stdout: "release": "luminous", 2024-05-09T08:03:32.246 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 5 2024-05-09T08:03:32.246 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:03:32.246 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:03:32.246 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr": [ 2024-05-09T08:03:32.246 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:03:32.246 INFO:teuthology.orchestra.run.smithi001.stdout: "features": "0x3f03cffffffdffff", 2024-05-09T08:03:32.247 INFO:teuthology.orchestra.run.smithi001.stdout: "release": "luminous", 2024-05-09T08:03:32.247 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 2 2024-05-09T08:03:32.247 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:03:32.247 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:03:32.247 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:03:32.247 INFO:teuthology.orchestra.run.smithi001.stdout: "stretch_mode": false 2024-05-09T08:03:32.247 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-05-09T08:03:32.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:32.217+0000 7f0282000640 1 -- 172.21.15.1:0/3603465284 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f028c1549b0 msgr2=0x7f028c1f3780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:32.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:32.217+0000 7f0282000640 1 --2- 172.21.15.1:0/3603465284 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f028c1549b0 0x7f028c1f3780 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f028c0c5ce0 tx=0x7f027800f000 comp rx=0 tx=0).stop 2024-05-09T08:03:32.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:32.217+0000 7f0282000640 1 -- 172.21.15.1:0/3603465284 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f028c1549b0 msgr2=0x7f028c1f3780 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-09T08:03:32.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:32.217+0000 7f0282000640 1 --2- 172.21.15.1:0/3603465284 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f028c1549b0 0x7f028c1f3780 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:32.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:32.217+0000 7f028b200640 1 -- 172.21.15.1:0/3603465284 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f026c06bde0 msgr2=0x7f026c06e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:32.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:32.217+0000 7f028b200640 1 --2- 172.21.15.1:0/3603465284 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f026c06bde0 0x7f026c06e2a0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f0284034000 tx=0x7f028402f000 comp rx=0 tx=0).stop 2024-05-09T08:03:32.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:32.217+0000 7f028b200640 1 -- 172.21.15.1:0/3603465284 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f028c0c54f0 msgr2=0x7f028c0cc680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:32.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:32.217+0000 7f028b200640 1 --2- 172.21.15.1:0/3603465284 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f028c0c54f0 0x7f028c0cc680 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f027c00b580 tx=0x7f027c00ba40 comp rx=0 tx=0).stop 2024-05-09T08:03:32.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:32.217+0000 7f028a800640 1 -- 172.21.15.1:0/3603465284 reap_dead start 2024-05-09T08:03:32.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:32.217+0000 7f028b200640 1 -- 172.21.15.1:0/3603465284 shutdown_connections 2024-05-09T08:03:32.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:32.217+0000 7f028b200640 1 --2- 172.21.15.1:0/3603465284 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f028c0c54f0 0x7f028c0cc680 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:32.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:32.217+0000 7f028b200640 1 -- 172.21.15.1:0/3603465284 >> 172.21.15.1:0/3603465284 conn(0x7f028c06b250 msgr2=0x7f028c14f7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:03:32.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:32.217+0000 7f028b200640 1 -- 172.21.15.1:0/3603465284 shutdown_connections 2024-05-09T08:03:32.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:32.218+0000 7f028b200640 1 -- 172.21.15.1:0/3603465284 wait complete. 2024-05-09T08:03:32.253 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-05-09T08:03:32.253 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2024-05-09T08:03:50.045 DEBUG:teuthology.orchestra.run.smithi001:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T08:03:50.054 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T08:03:52.253 INFO:tasks.mon_thrash.mon_thrasher:killing mon.b 2024-05-09T08:03:52.254 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.b 2024-05-09T08:03:52.254 INFO:tasks.ceph.mon.b:Restarting daemon 2024-05-09T08:03:52.254 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-05-09T08:03:52.257 INFO:tasks.ceph.mon.b:Started 2024-05-09T08:03:52.257 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-05-09T08:03:52.257 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-05-09T08:03:52.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:52.431+0000 7ffb40e00640 1 Processor -- start 2024-05-09T08:03:52.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:52.431+0000 7ffb40e00640 1 -- start start 2024-05-09T08:03:52.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:52.434+0000 7ffb40e00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ffb3c15a540 0x7ffb3c15e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:52.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:52.434+0000 7ffb40e00640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffb3c15aa60 con 0x7ffb3c154530 2024-05-09T08:03:52.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:52.434+0000 7ffb40e00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffb3c15ad40 con 0x7ffb3c15e730 2024-05-09T08:03:52.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:52.434+0000 7ffb40e00640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7ffb3c15b020 con 0x7ffb3c15a540 2024-05-09T08:03:52.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:52.434+0000 7ffb39400640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ffb3c15a540 0x7ffb3c15e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:52.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:52.434+0000 7ffb3a800640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7ffb3c15e730 0x7ffb3c06f3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.1:42202/0 (socket says 172.21.15.1:42202) 2024-05-09T08:03:52.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:52.434+0000 7ffb39e00640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7ffb3c154530 0x7ffb3c070140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:49134/0 (socket says 172.21.15.1:49134) 2024-05-09T08:03:52.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:52.434+0000 7ffb3a800640 1 -- 172.21.15.1:0/646044714 learned_addr learned my addr 172.21.15.1:0/646044714 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T08:03:52.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:52.434+0000 7ffb39400640 1 -- 172.21.15.1:0/646044714 >> v1:172.21.15.7:6789/0 conn(0x7ffb3c15e730 legacy=0x7ffb3c06f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:52.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:52.434+0000 7ffb39400640 1 -- 172.21.15.1:0/646044714 >> v1:172.21.15.1:6789/0 conn(0x7ffb3c154530 legacy=0x7ffb3c070140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:52.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:52.434+0000 7ffb39400640 1 -- 172.21.15.1:0/646044714 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb3c15b300 con 0x7ffb3c15a540 2024-05-09T08:03:52.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:52.436+0000 7ffb39400640 1 --2- 172.21.15.1:0/646044714 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ffb3c15a540 0x7ffb3c15e1f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7ffb280097d0 tx=0x7ffb2802a2a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f8e9445b516c6668 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:03:52.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:52.436+0000 7ffb38a00640 1 -- 172.21.15.1:0/646044714 <== mon.2 v2:172.21.15.7:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb28037070 con 0x7ffb3c15a540 2024-05-09T08:03:52.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:52.436+0000 7ffb38a00640 1 -- 172.21.15.1:0/646044714 <== mon.2 v2:172.21.15.7:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb28032440 con 0x7ffb3c15a540 2024-05-09T08:03:52.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:52.436+0000 7ffb38a00640 1 -- 172.21.15.1:0/646044714 <== mon.2 v2:172.21.15.7:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb28032810 con 0x7ffb3c15a540 2024-05-09T08:03:52.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:52.436+0000 7ffb40e00640 1 -- 172.21.15.1:0/646044714 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ffb3c15a540 msgr2=0x7ffb3c15e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:52.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:52.436+0000 7ffb40e00640 1 --2- 172.21.15.1:0/646044714 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ffb3c15a540 0x7ffb3c15e1f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7ffb280097d0 tx=0x7ffb2802a2a0 comp rx=0 tx=0).stop 2024-05-09T08:03:52.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:52.436+0000 7ffb40e00640 1 -- 172.21.15.1:0/646044714 shutdown_connections 2024-05-09T08:03:52.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:52.436+0000 7ffb40e00640 1 --2- 172.21.15.1:0/646044714 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ffb3c15a540 0x7ffb3c15e1f0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:52.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:52.436+0000 7ffb40e00640 1 -- 172.21.15.1:0/646044714 >> 172.21.15.1:0/646044714 conn(0x7ffb3c0cddd0 msgr2=0x7ffb3c0ce1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:03:52.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:52.437+0000 7ffb40e00640 1 -- 172.21.15.1:0/646044714 shutdown_connections 2024-05-09T08:03:52.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:52.437+0000 7ffb40e00640 1 -- 172.21.15.1:0/646044714 wait complete. 2024-05-09T08:03:52.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:52.439+0000 7ffb40e00640 1 Processor -- start 2024-05-09T08:03:52.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:52.439+0000 7ffb40e00640 1 -- start start 2024-05-09T08:03:52.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:52.440+0000 7ffb40e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffb3c154530 0x7ffb3c150310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:52.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:52.440+0000 7ffb40e00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ffb3c15a540 0x7ffb3c0cd4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:52.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:52.440+0000 7ffb40e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb3c15e730 0x7ffb3c1f4050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:52.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:52.440+0000 7ffb40e00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ffb3c06cde0 con 0x7ffb3c154530 2024-05-09T08:03:52.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:52.440+0000 7ffb40e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ffb3c154910 con 0x7ffb3c15e730 2024-05-09T08:03:52.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:52.440+0000 7ffb40e00640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7ffb3c06cf60 con 0x7ffb3c15a540 2024-05-09T08:03:52.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:52.440+0000 7ffb39e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffb3c154530 0x7ffb3c150310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:52.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:52.440+0000 7ffb39e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffb3c154530 0x7ffb3c150310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:57542/0 (socket says 172.21.15.1:57542) 2024-05-09T08:03:52.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:52.440+0000 7ffb39e00640 1 -- 172.21.15.1:0/1590718794 learned_addr learned my addr 172.21.15.1:0/1590718794 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:03:52.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:52.440+0000 7ffb39400640 1 --2- 172.21.15.1:0/1590718794 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ffb3c15a540 0x7ffb3c0cd4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:52.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:52.440+0000 7ffb3a800640 1 --2- 172.21.15.1:0/1590718794 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb3c15e730 0x7ffb3c1f4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:52.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:52.440+0000 7ffb3a800640 1 -- 172.21.15.1:0/1590718794 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb3c15e730 msgr2=0x7ffb3c1f4050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-05-09T08:03:52.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:52.440+0000 7ffb3a800640 1 -- 172.21.15.1:0/1590718794 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb3c15e730 msgr2=0x7ffb3c1f4050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-09T08:03:52.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:52.440+0000 7ffb3a800640 1 --2- 172.21.15.1:0/1590718794 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb3c15e730 0x7ffb3c1f4050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-09T08:03:52.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:52.440+0000 7ffb3a800640 1 --2- 172.21.15.1:0/1590718794 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb3c15e730 0x7ffb3c1f4050 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T08:03:52.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:52.440+0000 7ffb39e00640 1 -- 172.21.15.1:0/1590718794 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ffb3c15a540 msgr2=0x7ffb3c0cd4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:52.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:52.441+0000 7ffb39e00640 1 --2- 172.21.15.1:0/1590718794 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ffb3c15a540 0x7ffb3c0cd4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:52.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:52.441+0000 7ffb39e00640 1 -- 172.21.15.1:0/1590718794 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb3c15e730 msgr2=0x7ffb3c1f4050 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T08:03:52.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:52.441+0000 7ffb39e00640 1 --2- 172.21.15.1:0/1590718794 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb3c15e730 0x7ffb3c1f4050 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:52.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:52.441+0000 7ffb39e00640 1 -- 172.21.15.1:0/1590718794 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb3c15b950 con 0x7ffb3c154530 2024-05-09T08:03:52.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:52.441+0000 7ffb39e00640 1 --2- 172.21.15.1:0/1590718794 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffb3c154530 0x7ffb3c150310 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ffb2c00c430 tx=0x7ffb2c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:03:52.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:52.441+0000 7ffb32000640 1 -- 172.21.15.1:0/1590718794 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb2c0074e0 con 0x7ffb3c154530 2024-05-09T08:03:52.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:52.441+0000 7ffb32000640 1 -- 172.21.15.1:0/1590718794 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb2c007680 con 0x7ffb3c154530 2024-05-09T08:03:52.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:52.441+0000 7ffb32000640 1 -- 172.21.15.1:0/1590718794 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb2c004a00 con 0x7ffb3c154530 2024-05-09T08:03:52.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:52.441+0000 7ffb40e00640 1 -- 172.21.15.1:0/1590718794 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb3c15bc30 con 0x7ffb3c154530 2024-05-09T08:03:52.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:52.441+0000 7ffb40e00640 1 -- 172.21.15.1:0/1590718794 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb3c150850 con 0x7ffb3c154530 2024-05-09T08:03:52.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:52.442+0000 7ffb40e00640 1 -- 172.21.15.1:0/1590718794 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb3c0ca3b0 con 0x7ffb3c154530 2024-05-09T08:03:52.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:52.443+0000 7ffb32000640 1 -- 172.21.15.1:0/1590718794 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ffb2c01a020 con 0x7ffb3c154530 2024-05-09T08:03:52.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:52.444+0000 7ffb32000640 1 --2- 172.21.15.1:0/1590718794 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7ffb1c06be60 0x7ffb1c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:52.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:52.445+0000 7ffb32000640 1 -- 172.21.15.1:0/1590718794 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7ffb2c08aa60 con 0x7ffb3c154530 2024-05-09T08:03:52.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:52.445+0000 7ffb39400640 1 --2- 172.21.15.1:0/1590718794 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7ffb1c06be60 0x7ffb1c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:52.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:52.446+0000 7ffb39400640 1 --2- 172.21.15.1:0/1590718794 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7ffb1c06be60 0x7ffb1c06e320 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7ffb28034000 tx=0x7ffb2802e000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:03:52.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:52.450+0000 7ffb32000640 1 -- 172.21.15.1:0/1590718794 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ffb3c0ca3b0 con 0x7ffb3c154530 2024-05-09T08:03:52.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:52.740+0000 7ffb40e00640 1 -- 172.21.15.1:0/1590718794 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7ffb3c074d70 con 0x7ffb3c154530 2024-05-09T08:03:52.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:52.740+0000 7ffb32000640 1 -- 172.21.15.1:0/1590718794 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1544 (secure 0 0 0) 0x7ffb2c058f80 con 0x7ffb3c154530 2024-05-09T08:03:52.757 INFO:teuthology.orchestra.run.smithi001.stdout:{"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"54329ec8-3270-4043-9a82-c6cd7420aa7f","modified":"2024-05-09T08:01:21.819270Z","created":"2024-05-09T08:01:21.819270Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:3300","nonce":0},{"type":"v1","addr":"172.21.15.1:6789","nonce":0}]},"addr":"172.21.15.1:6789/0","public_addr":"172.21.15.1:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:3300","nonce":0},{"type":"v1","addr":"172.21.15.7:6789","nonce":0}]},"addr":"172.21.15.7:6789/0","public_addr":"172.21.15.7:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:3301","nonce":0},{"type":"v1","addr":"172.21.15.7:6790","nonce":0}]},"addr":"172.21.15.7:6790/0","public_addr":"172.21.15.7:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-05-09T08:03:52.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:52.743+0000 7ffb40e00640 1 -- 172.21.15.1:0/1590718794 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7ffb1c06be60 msgr2=0x7ffb1c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:52.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:52.743+0000 7ffb40e00640 1 --2- 172.21.15.1:0/1590718794 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7ffb1c06be60 0x7ffb1c06e320 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7ffb28034000 tx=0x7ffb2802e000 comp rx=0 tx=0).stop 2024-05-09T08:03:52.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:52.743+0000 7ffb40e00640 1 -- 172.21.15.1:0/1590718794 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffb3c154530 msgr2=0x7ffb3c150310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:52.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:52.743+0000 7ffb40e00640 1 --2- 172.21.15.1:0/1590718794 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffb3c154530 0x7ffb3c150310 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ffb2c00c430 tx=0x7ffb2c00c8f0 comp rx=0 tx=0).stop 2024-05-09T08:03:52.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:52.743+0000 7ffb40e00640 1 -- 172.21.15.1:0/1590718794 shutdown_connections 2024-05-09T08:03:52.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:52.743+0000 7ffb40e00640 1 --2- 172.21.15.1:0/1590718794 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7ffb1c06be60 0x7ffb1c06e320 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:52.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:52.743+0000 7ffb40e00640 1 --2- 172.21.15.1:0/1590718794 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb3c15e730 0x7ffb3c1f4050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:52.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:52.743+0000 7ffb40e00640 1 --2- 172.21.15.1:0/1590718794 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ffb3c15a540 0x7ffb3c0cd4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:52.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:52.743+0000 7ffb40e00640 1 --2- 172.21.15.1:0/1590718794 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffb3c154530 0x7ffb3c150310 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:52.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:52.743+0000 7ffb40e00640 1 -- 172.21.15.1:0/1590718794 >> 172.21.15.1:0/1590718794 conn(0x7ffb3c0cddd0 msgr2=0x7ffb3c0c95f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:03:52.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:52.744+0000 7ffb40e00640 1 -- 172.21.15.1:0/1590718794 shutdown_connections 2024-05-09T08:03:52.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:52.744+0000 7ffb40e00640 1 -- 172.21.15.1:0/1590718794 wait complete. 2024-05-09T08:03:52.779 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-05-09T08:03:52.779 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-05-09T08:03:52.779 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-05-09T08:03:52.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:52.943+0000 7f1f9cc00640 1 Processor -- start 2024-05-09T08:03:52.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:52.944+0000 7f1f9cc00640 1 -- start start 2024-05-09T08:03:52.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:52.945+0000 7f1f9cc00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f1f980cd490 0x7f1f9806e020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:52.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:52.945+0000 7f1f9cc00640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f1f9815db20 0x7f1f9815df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:52.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:52.946+0000 7f1f9cc00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1f9815b0e0 con 0x7f1f98154490 2024-05-09T08:03:52.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:52.946+0000 7f1f9cc00640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f1f980c5980 con 0x7f1f9815db20 2024-05-09T08:03:52.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:52.946+0000 7f1f9cc00640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f1f9815b3c0 con 0x7f1f980cd490 2024-05-09T08:03:52.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:52.946+0000 7f1f96a00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f1f980cd490 0x7f1f9806e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:52.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:52.946+0000 7f1f96a00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f1f980cd490 0x7f1f9806e020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3301/0 says I am v2:172.21.15.1:39068/0 (socket says 172.21.15.1:39068) 2024-05-09T08:03:52.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:52.946+0000 7f1f97400640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f1f98154490 0x7f1f980c5270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.1:42218/0 (socket says 172.21.15.1:42218) 2024-05-09T08:03:52.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:52.946+0000 7f1f96a00640 1 -- 172.21.15.1:0/2323305675 learned_addr learned my addr 172.21.15.1:0/2323305675 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:03:52.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:52.946+0000 7f1f97e00640 1 --2- 172.21.15.1:0/2323305675 >> v2:172.21.15.1:3300/0 conn(0x7f1f9815db20 0x7f1f9815df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:52.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:52.946+0000 7f1f96a00640 1 -- 172.21.15.1:0/2323305675 >> v2:172.21.15.1:3300/0 conn(0x7f1f9815db20 msgr2=0x7f1f9815df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:03:52.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:52.946+0000 7f1f96a00640 1 --2- 172.21.15.1:0/2323305675 >> v2:172.21.15.1:3300/0 conn(0x7f1f9815db20 0x7f1f9815df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:52.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:52.946+0000 7f1f96a00640 1 -- 172.21.15.1:0/2323305675 >> v1:172.21.15.7:6789/0 conn(0x7f1f98154490 legacy=0x7f1f980c5270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:52.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:52.946+0000 7f1f96a00640 1 -- 172.21.15.1:0/2323305675 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f980c9400 con 0x7f1f980cd490 2024-05-09T08:03:52.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:52.947+0000 7f1f96a00640 1 --2- 172.21.15.1:0/2323305675 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f1f980cd490 0x7f1f9806e020 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f1f84009870 tx=0x7f1f8402a250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d48bafad6b49fb29 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:03:52.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:52.948+0000 7f1f96000640 1 -- 172.21.15.1:0/2323305675 <== mon.2 v2:172.21.15.7:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f84037070 con 0x7f1f980cd490 2024-05-09T08:03:52.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:52.948+0000 7f1f96000640 1 -- 172.21.15.1:0/2323305675 <== mon.2 v2:172.21.15.7:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f84032440 con 0x7f1f980cd490 2024-05-09T08:03:52.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:52.948+0000 7f1f96000640 1 -- 172.21.15.1:0/2323305675 <== mon.2 v2:172.21.15.7:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f84032810 con 0x7f1f980cd490 2024-05-09T08:03:52.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:52.948+0000 7f1f9cc00640 1 -- 172.21.15.1:0/2323305675 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f1f980cd490 msgr2=0x7f1f9806e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:52.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:52.948+0000 7f1f9cc00640 1 --2- 172.21.15.1:0/2323305675 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f1f980cd490 0x7f1f9806e020 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f1f84009870 tx=0x7f1f8402a250 comp rx=0 tx=0).stop 2024-05-09T08:03:52.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:52.949+0000 7f1f9cc00640 1 -- 172.21.15.1:0/2323305675 shutdown_connections 2024-05-09T08:03:52.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:52.949+0000 7f1f9cc00640 1 --2- 172.21.15.1:0/2323305675 >> v2:172.21.15.1:3300/0 conn(0x7f1f9815db20 0x7f1f9815df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:52.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:52.949+0000 7f1f9cc00640 1 --2- 172.21.15.1:0/2323305675 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f1f980cd490 0x7f1f9806e020 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:52.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:52.949+0000 7f1f9cc00640 1 -- 172.21.15.1:0/2323305675 >> 172.21.15.1:0/2323305675 conn(0x7f1f980cc970 msgr2=0x7f1f980ccd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:03:52.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:52.949+0000 7f1f9cc00640 1 -- 172.21.15.1:0/2323305675 shutdown_connections 2024-05-09T08:03:52.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:52.949+0000 7f1f9cc00640 1 -- 172.21.15.1:0/2323305675 wait complete. 2024-05-09T08:03:52.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:52.951+0000 7f1f9cc00640 1 Processor -- start 2024-05-09T08:03:52.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:52.951+0000 7f1f9cc00640 1 -- start start 2024-05-09T08:03:52.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:52.952+0000 7f1f9cc00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f1f98154490 0x7f1f980cbf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:52.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:52.952+0000 7f1f9cc00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f9815db20 0x7f1f981520e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:52.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:52.952+0000 7f1f9cc00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1f981506d0 0x7f1f9814dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:52.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:52.952+0000 7f1f9cc00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f1f980c9ca0 con 0x7f1f981506d0 2024-05-09T08:03:52.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:52.952+0000 7f1f9cc00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1f980c99c0 con 0x7f1f9815db20 2024-05-09T08:03:52.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:52.952+0000 7f1f9cc00640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f1f980ca1c0 con 0x7f1f98154490 2024-05-09T08:03:52.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:52.952+0000 7f1f97400640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f1f98154490 0x7f1f980cbf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:52.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:52.952+0000 7f1f97400640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f1f98154490 0x7f1f980cbf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3301/0 says I am v2:172.21.15.1:39080/0 (socket says 172.21.15.1:39080) 2024-05-09T08:03:52.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:52.952+0000 7f1f97400640 1 -- 172.21.15.1:0/3301647977 learned_addr learned my addr 172.21.15.1:0/3301647977 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:03:52.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:52.952+0000 7f1f97e00640 1 --2- 172.21.15.1:0/3301647977 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1f981506d0 0x7f1f9814dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:52.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:52.952+0000 7f1f96a00640 1 --2- 172.21.15.1:0/3301647977 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f9815db20 0x7f1f981520e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:52.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:52.953+0000 7f1f97400640 1 -- 172.21.15.1:0/3301647977 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f9815db20 msgr2=0x7f1f981520e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:52.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:52.953+0000 7f1f97400640 1 --2- 172.21.15.1:0/3301647977 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f9815db20 0x7f1f981520e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:52.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:52.953+0000 7f1f97400640 1 -- 172.21.15.1:0/3301647977 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1f981506d0 msgr2=0x7f1f9814dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:52.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:52.953+0000 7f1f97400640 1 --2- 172.21.15.1:0/3301647977 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1f981506d0 0x7f1f9814dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:52.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:52.953+0000 7f1f97400640 1 -- 172.21.15.1:0/3301647977 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f981f0290 con 0x7f1f98154490 2024-05-09T08:03:52.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:52.953+0000 7f1f97e00640 1 --2- 172.21.15.1:0/3301647977 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1f981506d0 0x7f1f9814dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:03:52.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:52.953+0000 7f1f97400640 1 --2- 172.21.15.1:0/3301647977 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f1f98154490 0x7f1f980cbf90 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f1f8800c4d0 tx=0x7f1f8800c990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:03:52.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:52.953+0000 7f1f8f400640 1 -- 172.21.15.1:0/3301647977 <== mon.2 v2:172.21.15.7:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f88018070 con 0x7f1f98154490 2024-05-09T08:03:52.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:52.953+0000 7f1f8f400640 1 -- 172.21.15.1:0/3301647977 <== mon.2 v2:172.21.15.7:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f88007480 con 0x7f1f98154490 2024-05-09T08:03:52.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:52.953+0000 7f1f8f400640 1 -- 172.21.15.1:0/3301647977 <== mon.2 v2:172.21.15.7:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f880049e0 con 0x7f1f98154490 2024-05-09T08:03:52.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:52.953+0000 7f1f9cc00640 1 -- 172.21.15.1:0/3301647977 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f981f0570 con 0x7f1f98154490 2024-05-09T08:03:52.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:52.953+0000 7f1f9cc00640 1 -- 172.21.15.1:0/3301647977 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f980cc4d0 con 0x7f1f98154490 2024-05-09T08:03:52.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:52.955+0000 7f1f9cc00640 1 --2- 172.21.15.1:0/3301647977 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1f980cd490 0x7f1f9814e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:52.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:52.955+0000 7f1f9cc00640 1 -- 172.21.15.1:0/3301647977 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f1f981f0970 con 0x7f1f980cd490 2024-05-09T08:03:52.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:52.955+0000 7f1f96a00640 1 --2- 172.21.15.1:0/3301647977 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1f980cd490 0x7f1f9814e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:52.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:52.955+0000 7f1f96a00640 1 -- 172.21.15.1:0/3301647977 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1f981f0740 con 0x7f1f980cd490 2024-05-09T08:03:52.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:52.956+0000 7f1f96a00640 1 --2- 172.21.15.1:0/3301647977 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1f980cd490 0x7f1f9814e010 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f1f9806c310 tx=0x7f1f840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:03:52.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:52.956+0000 7f1f8f400640 1 -- 172.21.15.1:0/3301647977 <== mon.2 v2:172.21.15.7:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1f88004b80 con 0x7f1f98154490 2024-05-09T08:03:52.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:52.957+0000 7f1f8f400640 1 --2- 172.21.15.1:0/3301647977 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f1f7806be30 0x7f1f7806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:52.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:52.957+0000 7f1f8f400640 1 -- 172.21.15.1:0/3301647977 <== mon.2 v2:172.21.15.7:3301/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7f1f8808a4d0 con 0x7f1f98154490 2024-05-09T08:03:52.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:52.957+0000 7f1f8f400640 1 -- 172.21.15.1:0/3301647977 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f84037040 con 0x7f1f980cd490 2024-05-09T08:03:52.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:52.957+0000 7f1f8f400640 1 -- 172.21.15.1:0/3301647977 <== mon.0 v2:172.21.15.1:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f1f84032e20 con 0x7f1f980cd490 2024-05-09T08:03:52.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:52.957+0000 7f1f8f400640 1 -- 172.21.15.1:0/3301647977 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1f980cd490 msgr2=0x7f1f9814e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:52.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:52.957+0000 7f1f8f400640 1 --2- 172.21.15.1:0/3301647977 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1f980cd490 0x7f1f9814e010 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f1f9806c310 tx=0x7f1f840046c0 comp rx=0 tx=0).stop 2024-05-09T08:03:52.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:52.957+0000 7f1f8f400640 1 -- 172.21.15.1:0/3301647977 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1f980cd490 msgr2=0x7f1f9814e010 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-09T08:03:52.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:52.957+0000 7f1f8f400640 1 --2- 172.21.15.1:0/3301647977 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1f980cd490 0x7f1f9814e010 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:52.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:52.957+0000 7f1f97e00640 1 --2- 172.21.15.1:0/3301647977 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f1f7806be30 0x7f1f7806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:52.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:52.958+0000 7f1f97e00640 1 --2- 172.21.15.1:0/3301647977 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f1f7806be30 0x7f1f7806e2f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f1f90007720 tx=0x7f1f90009780 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:03:52.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:52.972+0000 7f1f9cc00640 1 --2- 172.21.15.1:0/3301647977 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1f98157a80 0x7f1f98157e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:52.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:52.972+0000 7f1f9cc00640 1 -- 172.21.15.1:0/3301647977 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f1f9804f660 con 0x7f1f98157a80 2024-05-09T08:03:52.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:52.972+0000 7f1f96a00640 1 --2- 172.21.15.1:0/3301647977 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1f98157a80 0x7f1f98157e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:52.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:52.973+0000 7f1f96a00640 1 -- 172.21.15.1:0/3301647977 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f1f9814d830 con 0x7f1f98157a80 2024-05-09T08:03:52.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:52.973+0000 7f1f96a00640 1 --2- 172.21.15.1:0/3301647977 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1f98157a80 0x7f1f98157e90 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f1f840090b0 tx=0x7f1f84008030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:03:52.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:52.973+0000 7f1f8f400640 1 -- 172.21.15.1:0/3301647977 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f84037070 con 0x7f1f98157a80 2024-05-09T08:03:52.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:52.974+0000 7f1f8f400640 1 -- 172.21.15.1:0/3301647977 <== mon.0 v2:172.21.15.1:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4707 (secure 0 0 0) 0x7f1f8403ce50 con 0x7f1f98157a80 2024-05-09T08:03:52.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:52.974+0000 7f1f8f400640 1 -- 172.21.15.1:0/3301647977 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1f98157a80 msgr2=0x7f1f98157e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:52.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:52.974+0000 7f1f8f400640 1 --2- 172.21.15.1:0/3301647977 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1f98157a80 0x7f1f98157e90 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f1f840090b0 tx=0x7f1f84008030 comp rx=0 tx=0).stop 2024-05-09T08:03:52.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:52.974+0000 7f1f8f400640 1 -- 172.21.15.1:0/3301647977 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1f98157a80 msgr2=0x7f1f98157e90 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-05-09T08:03:52.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:52.974+0000 7f1f8f400640 1 --2- 172.21.15.1:0/3301647977 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1f98157a80 0x7f1f98157e90 secure :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f1f840090b0 tx=0x7f1f84008030 comp rx=0 tx=0).stop 2024-05-09T08:03:52.991 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-05-09T08:03:52.991 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "a", 2024-05-09T08:03:52.991 INFO:teuthology.orchestra.run.smithi001.stdout: "rank": 0, 2024-05-09T08:03:52.991 INFO:teuthology.orchestra.run.smithi001.stdout: "state": "leader", 2024-05-09T08:03:52.991 INFO:teuthology.orchestra.run.smithi001.stdout: "election_epoch": 10, 2024-05-09T08:03:52.991 INFO:teuthology.orchestra.run.smithi001.stdout: "uptime": 116336, 2024-05-09T08:03:52.991 INFO:teuthology.orchestra.run.smithi001.stdout: "quorum": [ 2024-05-09T08:03:52.991 INFO:teuthology.orchestra.run.smithi001.stdout: 0, 2024-05-09T08:03:52.991 INFO:teuthology.orchestra.run.smithi001.stdout: 1, 2024-05-09T08:03:52.991 INFO:teuthology.orchestra.run.smithi001.stdout: 2 2024-05-09T08:03:52.992 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:03:52.992 INFO:teuthology.orchestra.run.smithi001.stdout: "quorum_age": 0, 2024-05-09T08:03:52.992 INFO:teuthology.orchestra.run.smithi001.stdout: "features": { 2024-05-09T08:03:52.992 INFO:teuthology.orchestra.run.smithi001.stdout: "required_con": "2450521708007866372", 2024-05-09T08:03:52.992 INFO:teuthology.orchestra.run.smithi001.stdout: "required_mon": [ 2024-05-09T08:03:52.992 INFO:teuthology.orchestra.run.smithi001.stdout: "kraken", 2024-05-09T08:03:52.992 INFO:teuthology.orchestra.run.smithi001.stdout: "luminous", 2024-05-09T08:03:52.992 INFO:teuthology.orchestra.run.smithi001.stdout: "mimic", 2024-05-09T08:03:52.992 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap-prune", 2024-05-09T08:03:52.992 INFO:teuthology.orchestra.run.smithi001.stdout: "nautilus", 2024-05-09T08:03:52.992 INFO:teuthology.orchestra.run.smithi001.stdout: "octopus", 2024-05-09T08:03:52.992 INFO:teuthology.orchestra.run.smithi001.stdout: "pacific", 2024-05-09T08:03:52.992 INFO:teuthology.orchestra.run.smithi001.stdout: "elector-pinging", 2024-05-09T08:03:52.992 INFO:teuthology.orchestra.run.smithi001.stdout: "quincy", 2024-05-09T08:03:52.992 INFO:teuthology.orchestra.run.smithi001.stdout: "reef", 2024-05-09T08:03:52.993 INFO:teuthology.orchestra.run.smithi001.stdout: "squid" 2024-05-09T08:03:52.993 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:03:52.993 INFO:teuthology.orchestra.run.smithi001.stdout: "quorum_con": "4540701547738038271", 2024-05-09T08:03:52.993 INFO:teuthology.orchestra.run.smithi001.stdout: "quorum_mon": [ 2024-05-09T08:03:52.993 INFO:teuthology.orchestra.run.smithi001.stdout: "kraken", 2024-05-09T08:03:52.993 INFO:teuthology.orchestra.run.smithi001.stdout: "luminous", 2024-05-09T08:03:52.993 INFO:teuthology.orchestra.run.smithi001.stdout: "mimic", 2024-05-09T08:03:52.993 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap-prune", 2024-05-09T08:03:52.993 INFO:teuthology.orchestra.run.smithi001.stdout: "nautilus", 2024-05-09T08:03:52.993 INFO:teuthology.orchestra.run.smithi001.stdout: "octopus", 2024-05-09T08:03:52.993 INFO:teuthology.orchestra.run.smithi001.stdout: "pacific", 2024-05-09T08:03:52.994 INFO:teuthology.orchestra.run.smithi001.stdout: "elector-pinging", 2024-05-09T08:03:52.994 INFO:teuthology.orchestra.run.smithi001.stdout: "quincy", 2024-05-09T08:03:52.994 INFO:teuthology.orchestra.run.smithi001.stdout: "reef", 2024-05-09T08:03:52.994 INFO:teuthology.orchestra.run.smithi001.stdout: "squid" 2024-05-09T08:03:52.994 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:03:52.994 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:03:52.994 INFO:teuthology.orchestra.run.smithi001.stdout: "outside_quorum": [], 2024-05-09T08:03:52.994 INFO:teuthology.orchestra.run.smithi001.stdout: "extra_probe_peers": [], 2024-05-09T08:03:52.994 INFO:teuthology.orchestra.run.smithi001.stdout: "sync_provider": [], 2024-05-09T08:03:52.994 INFO:teuthology.orchestra.run.smithi001.stdout: "monmap": { 2024-05-09T08:03:52.994 INFO:teuthology.orchestra.run.smithi001.stdout: "epoch": 1, 2024-05-09T08:03:52.994 INFO:teuthology.orchestra.run.smithi001.stdout: "fsid": "54329ec8-3270-4043-9a82-c6cd7420aa7f", 2024-05-09T08:03:52.994 INFO:teuthology.orchestra.run.smithi001.stdout: "modified": "2024-05-09T08:01:21.819270Z", 2024-05-09T08:03:52.994 INFO:teuthology.orchestra.run.smithi001.stdout: "created": "2024-05-09T08:01:21.819270Z", 2024-05-09T08:03:52.995 INFO:teuthology.orchestra.run.smithi001.stdout: "min_mon_release": 19, 2024-05-09T08:03:52.995 INFO:teuthology.orchestra.run.smithi001.stdout: "min_mon_release_name": "squid", 2024-05-09T08:03:52.995 INFO:teuthology.orchestra.run.smithi001.stdout: "election_strategy": 1, 2024-05-09T08:03:52.995 INFO:teuthology.orchestra.run.smithi001.stdout: "disallowed_leaders: ": "", 2024-05-09T08:03:52.995 INFO:teuthology.orchestra.run.smithi001.stdout: "stretch_mode": false, 2024-05-09T08:03:52.995 INFO:teuthology.orchestra.run.smithi001.stdout: "tiebreaker_mon": "", 2024-05-09T08:03:52.995 INFO:teuthology.orchestra.run.smithi001.stdout: "removed_ranks: ": "", 2024-05-09T08:03:52.995 INFO:teuthology.orchestra.run.smithi001.stdout: "features": { 2024-05-09T08:03:52.995 INFO:teuthology.orchestra.run.smithi001.stdout: "persistent": [ 2024-05-09T08:03:52.995 INFO:teuthology.orchestra.run.smithi001.stdout: "kraken", 2024-05-09T08:03:52.995 INFO:teuthology.orchestra.run.smithi001.stdout: "luminous", 2024-05-09T08:03:52.995 INFO:teuthology.orchestra.run.smithi001.stdout: "mimic", 2024-05-09T08:03:52.995 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap-prune", 2024-05-09T08:03:52.996 INFO:teuthology.orchestra.run.smithi001.stdout: "nautilus", 2024-05-09T08:03:52.996 INFO:teuthology.orchestra.run.smithi001.stdout: "octopus", 2024-05-09T08:03:52.996 INFO:teuthology.orchestra.run.smithi001.stdout: "pacific", 2024-05-09T08:03:52.996 INFO:teuthology.orchestra.run.smithi001.stdout: "elector-pinging", 2024-05-09T08:03:52.996 INFO:teuthology.orchestra.run.smithi001.stdout: "quincy", 2024-05-09T08:03:52.996 INFO:teuthology.orchestra.run.smithi001.stdout: "reef", 2024-05-09T08:03:52.996 INFO:teuthology.orchestra.run.smithi001.stdout: "squid" 2024-05-09T08:03:52.996 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:03:52.996 INFO:teuthology.orchestra.run.smithi001.stdout: "optional": [] 2024-05-09T08:03:52.996 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:03:52.996 INFO:teuthology.orchestra.run.smithi001.stdout: "mons": [ 2024-05-09T08:03:52.996 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:03:52.997 INFO:teuthology.orchestra.run.smithi001.stdout: "rank": 0, 2024-05-09T08:03:52.997 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "a", 2024-05-09T08:03:52.997 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-05-09T08:03:52.997 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-05-09T08:03:52.997 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:03:52.997 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-05-09T08:03:52.997 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:3300", 2024-05-09T08:03:52.997 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:03:52.997 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:03:52.997 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:03:52.997 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-05-09T08:03:52.997 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6789", 2024-05-09T08:03:52.997 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:03:52.997 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:03:52.998 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:03:52.998 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:03:52.998 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6789/0", 2024-05-09T08:03:52.998 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.1:6789/0", 2024-05-09T08:03:52.998 INFO:teuthology.orchestra.run.smithi001.stdout: "priority": 0, 2024-05-09T08:03:52.998 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 0, 2024-05-09T08:03:52.998 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_location": "{}" 2024-05-09T08:03:52.998 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:03:52.998 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:03:52.998 INFO:teuthology.orchestra.run.smithi001.stdout: "rank": 1, 2024-05-09T08:03:52.998 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "b", 2024-05-09T08:03:52.998 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-05-09T08:03:52.998 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-05-09T08:03:52.998 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:03:52.999 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-05-09T08:03:52.999 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:3300", 2024-05-09T08:03:52.999 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:03:52.999 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:03:52.999 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:03:52.999 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-05-09T08:03:52.999 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:6789", 2024-05-09T08:03:52.999 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:03:52.999 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:03:52.999 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:03:52.999 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:03:52.999 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:6789/0", 2024-05-09T08:03:52.999 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.7:6789/0", 2024-05-09T08:03:52.999 INFO:teuthology.orchestra.run.smithi001.stdout: "priority": 0, 2024-05-09T08:03:53.000 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 0, 2024-05-09T08:03:53.000 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_location": "{}" 2024-05-09T08:03:53.000 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:03:53.000 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:03:53.000 INFO:teuthology.orchestra.run.smithi001.stdout: "rank": 2, 2024-05-09T08:03:53.000 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "c", 2024-05-09T08:03:53.000 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-05-09T08:03:53.000 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-05-09T08:03:53.000 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:03:53.000 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-05-09T08:03:53.000 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:3301", 2024-05-09T08:03:53.000 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:03:53.000 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:03:53.001 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:03:53.001 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-05-09T08:03:53.001 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:6790", 2024-05-09T08:03:53.001 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:03:53.001 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:03:53.001 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:03:53.001 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:03:53.001 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:6790/0", 2024-05-09T08:03:53.001 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.7:6790/0", 2024-05-09T08:03:53.001 INFO:teuthology.orchestra.run.smithi001.stdout: "priority": 0, 2024-05-09T08:03:53.001 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 0, 2024-05-09T08:03:53.001 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_location": "{}" 2024-05-09T08:03:53.001 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:03:53.002 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:03:53.002 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:03:53.002 INFO:teuthology.orchestra.run.smithi001.stdout: "feature_map": { 2024-05-09T08:03:53.002 INFO:teuthology.orchestra.run.smithi001.stdout: "mon": [ 2024-05-09T08:03:53.002 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:03:53.002 INFO:teuthology.orchestra.run.smithi001.stdout: "features": "0x3f03cffffffdffff", 2024-05-09T08:03:53.002 INFO:teuthology.orchestra.run.smithi001.stdout: "release": "luminous", 2024-05-09T08:03:53.002 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 1 2024-05-09T08:03:53.002 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:03:53.002 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:03:53.002 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": [ 2024-05-09T08:03:53.002 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:03:53.002 INFO:teuthology.orchestra.run.smithi001.stdout: "features": "0x3f03cffffffdffff", 2024-05-09T08:03:53.003 INFO:teuthology.orchestra.run.smithi001.stdout: "release": "luminous", 2024-05-09T08:03:53.003 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 2 2024-05-09T08:03:53.003 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:03:53.003 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:03:53.003 INFO:teuthology.orchestra.run.smithi001.stdout: "client": [ 2024-05-09T08:03:53.003 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:03:53.003 INFO:teuthology.orchestra.run.smithi001.stdout: "features": "0x3f03cffffffdffff", 2024-05-09T08:03:53.003 INFO:teuthology.orchestra.run.smithi001.stdout: "release": "luminous", 2024-05-09T08:03:53.003 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 1 2024-05-09T08:03:53.003 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:03:53.003 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:03:53.003 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:03:53.003 INFO:teuthology.orchestra.run.smithi001.stdout: "stretch_mode": false 2024-05-09T08:03:53.003 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-05-09T08:03:53.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:52.974+0000 7f1f9cc00640 1 -- 172.21.15.1:0/3301647977 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f1f7806be30 msgr2=0x7f1f7806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:53.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:52.974+0000 7f1f9cc00640 1 --2- 172.21.15.1:0/3301647977 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f1f7806be30 0x7f1f7806e2f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f1f90007720 tx=0x7f1f90009780 comp rx=0 tx=0).stop 2024-05-09T08:03:53.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:52.974+0000 7f1f9cc00640 1 -- 172.21.15.1:0/3301647977 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f1f98154490 msgr2=0x7f1f980cbf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:53.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:52.974+0000 7f1f9cc00640 1 --2- 172.21.15.1:0/3301647977 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f1f98154490 0x7f1f980cbf90 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f1f8800c4d0 tx=0x7f1f8800c990 comp rx=0 tx=0).stop 2024-05-09T08:03:53.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:52.974+0000 7f1f97e00640 1 -- 172.21.15.1:0/3301647977 reap_dead start 2024-05-09T08:03:53.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:52.974+0000 7f1f97e00640 1 -- 172.21.15.1:0/3301647977 reap_dead start 2024-05-09T08:03:53.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:52.974+0000 7f1f9cc00640 1 -- 172.21.15.1:0/3301647977 shutdown_connections 2024-05-09T08:03:53.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:52.974+0000 7f1f9cc00640 1 -- 172.21.15.1:0/3301647977 >> 172.21.15.1:0/3301647977 conn(0x7f1f980cc970 msgr2=0x7f1f981590b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:03:53.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:52.974+0000 7f1f9cc00640 1 -- 172.21.15.1:0/3301647977 shutdown_connections 2024-05-09T08:03:53.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:52.974+0000 7f1f9cc00640 1 -- 172.21.15.1:0/3301647977 wait complete. 2024-05-09T08:03:53.006 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-05-09T08:03:53.007 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-05-09T08:03:53.007 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-05-09T08:03:53.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.177+0000 7fdf87200640 1 Processor -- start 2024-05-09T08:03:53.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.177+0000 7fdf87200640 1 -- start start 2024-05-09T08:03:53.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.179+0000 7fdf87200640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fdf8806e020 0x7fdf8815e230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:53.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.179+0000 7fdf87200640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf8815b0e0 con 0x7fdf880cd490 2024-05-09T08:03:53.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.179+0000 7fdf87200640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf8815b3c0 con 0x7fdf88154490 2024-05-09T08:03:53.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.179+0000 7fdf87200640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fdf880c5980 con 0x7fdf8806e020 2024-05-09T08:03:53.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.179+0000 7fdf7fe00640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7fdf880cd490 0x7fdf8815db20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:49138/0 (socket says 172.21.15.1:49138) 2024-05-09T08:03:53.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.180+0000 7fdf7fe00640 1 -- 172.21.15.1:0/2900106076 learned_addr learned my addr 172.21.15.1:0/2900106076 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T08:03:53.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.180+0000 7fdf86800640 1 --2- 172.21.15.1:0/2900106076 >> v2:172.21.15.7:3300/0 conn(0x7fdf8806e020 0x7fdf8815e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:53.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.180+0000 7fdf86800640 1 -- 172.21.15.1:0/2900106076 >> v1:172.21.15.7:6789/0 conn(0x7fdf88154490 legacy=0x7fdf880c5270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:53.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.180+0000 7fdf86800640 1 -- 172.21.15.1:0/2900106076 >> v1:172.21.15.1:6789/0 conn(0x7fdf880cd490 legacy=0x7fdf8815db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:53.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.180+0000 7fdf86800640 1 -- 172.21.15.1:0/2900106076 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf880c9400 con 0x7fdf8806e020 2024-05-09T08:03:53.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.180+0000 7fdf85400640 1 -- 172.21.15.1:0/2900106076 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2764104324 0 0) 0x7fdf8815b0e0 con 0x7fdf880cd490 2024-05-09T08:03:53.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.181+0000 7fdf86800640 1 --2- 172.21.15.1:0/2900106076 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf8806e020 0x7fdf8815e230 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fdf80009d90 tx=0x7fdf8002a6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5f16689bf403793a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:03:53.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.181+0000 7fdf85400640 1 -- 172.21.15.1:0/2900106076 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf80007740 con 0x7fdf8806e020 2024-05-09T08:03:53.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.181+0000 7fdf85400640 1 -- 172.21.15.1:0/2900106076 <== mon.1 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf800078e0 con 0x7fdf8806e020 2024-05-09T08:03:53.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.181+0000 7fdf85400640 1 -- 172.21.15.1:0/2900106076 <== mon.1 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf80007cb0 con 0x7fdf8806e020 2024-05-09T08:03:53.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.182+0000 7fdf87200640 1 -- 172.21.15.1:0/2900106076 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf8806e020 msgr2=0x7fdf8815e230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:53.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.182+0000 7fdf87200640 1 --2- 172.21.15.1:0/2900106076 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf8806e020 0x7fdf8815e230 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fdf80009d90 tx=0x7fdf8002a6f0 comp rx=0 tx=0).stop 2024-05-09T08:03:53.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.182+0000 7fdf87200640 1 -- 172.21.15.1:0/2900106076 shutdown_connections 2024-05-09T08:03:53.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.182+0000 7fdf87200640 1 --2- 172.21.15.1:0/2900106076 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf8806e020 0x7fdf8815e230 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:53.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.182+0000 7fdf87200640 1 -- 172.21.15.1:0/2900106076 >> 172.21.15.1:0/2900106076 conn(0x7fdf880cc970 msgr2=0x7fdf880ccd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:03:53.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.182+0000 7fdf87200640 1 -- 172.21.15.1:0/2900106076 shutdown_connections 2024-05-09T08:03:53.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.182+0000 7fdf87200640 1 -- 172.21.15.1:0/2900106076 wait complete. 2024-05-09T08:03:53.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.184+0000 7fdf87200640 1 Processor -- start 2024-05-09T08:03:53.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.184+0000 7fdf87200640 1 -- start start 2024-05-09T08:03:53.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.185+0000 7fdf87200640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fdf8806e020 0x7fdf880c1280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:53.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.185+0000 7fdf87200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf88154490 0x7fdf881f07d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:53.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.185+0000 7fdf87200640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdf88155890 0x7fdf88155d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:53.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.185+0000 7fdf87200640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fdf880c9ca0 con 0x7fdf88155890 2024-05-09T08:03:53.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.185+0000 7fdf87200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fdf880c99c0 con 0x7fdf88154490 2024-05-09T08:03:53.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.185+0000 7fdf87200640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7fdf880ca1c0 con 0x7fdf8806e020 2024-05-09T08:03:53.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.185+0000 7fdf7fe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf88154490 0x7fdf881f07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:53.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.185+0000 7fdf85e00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fdf8806e020 0x7fdf880c1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:53.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.185+0000 7fdf86800640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdf88155890 0x7fdf88155d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:53.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.185+0000 7fdf85e00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fdf8806e020 0x7fdf880c1280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3301/0 says I am v2:172.21.15.1:39092/0 (socket says 172.21.15.1:39092) 2024-05-09T08:03:53.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.185+0000 7fdf7fe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf88154490 0x7fdf881f07d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.1:52558/0 (socket says 172.21.15.1:52558) 2024-05-09T08:03:53.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.185+0000 7fdf86800640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdf88155890 0x7fdf88155d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:57584/0 (socket says 172.21.15.1:57584) 2024-05-09T08:03:53.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.185+0000 7fdf85e00640 1 -- 172.21.15.1:0/2678410323 learned_addr learned my addr 172.21.15.1:0/2678410323 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:03:53.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.186+0000 7fdf85e00640 1 -- 172.21.15.1:0/2678410323 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf88154490 msgr2=0x7fdf881f07d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:53.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.186+0000 7fdf85e00640 1 --2- 172.21.15.1:0/2678410323 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf88154490 0x7fdf881f07d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:53.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.186+0000 7fdf85e00640 1 -- 172.21.15.1:0/2678410323 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdf88155890 msgr2=0x7fdf88155d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:53.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.186+0000 7fdf85e00640 1 --2- 172.21.15.1:0/2678410323 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdf88155890 0x7fdf88155d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:53.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.186+0000 7fdf85e00640 1 -- 172.21.15.1:0/2678410323 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf88155450 con 0x7fdf8806e020 2024-05-09T08:03:53.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.186+0000 7fdf85e00640 1 --2- 172.21.15.1:0/2678410323 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fdf8806e020 0x7fdf880c1280 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fdf7800c400 tx=0x7fdf7800c8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:03:53.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.186+0000 7fdf7e000640 1 -- 172.21.15.1:0/2678410323 <== mon.2 v2:172.21.15.7:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf78017070 con 0x7fdf8806e020 2024-05-09T08:03:53.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.186+0000 7fdf7e000640 1 -- 172.21.15.1:0/2678410323 <== mon.2 v2:172.21.15.7:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf780052d0 con 0x7fdf8806e020 2024-05-09T08:03:53.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.186+0000 7fdf7e000640 1 -- 172.21.15.1:0/2678410323 <== mon.2 v2:172.21.15.7:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf78003830 con 0x7fdf8806e020 2024-05-09T08:03:53.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.186+0000 7fdf87200640 1 -- 172.21.15.1:0/2678410323 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf880c2fb0 con 0x7fdf8806e020 2024-05-09T08:03:53.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.187+0000 7fdf87200640 1 -- 172.21.15.1:0/2678410323 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf880c3550 con 0x7fdf8806e020 2024-05-09T08:03:53.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.187+0000 7fdf87200640 1 --2- 172.21.15.1:0/2678410323 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf880cd490 0x7fdf881efec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:53.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.187+0000 7fdf87200640 1 -- 172.21.15.1:0/2678410323 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fdf8804f660 con 0x7fdf880cd490 2024-05-09T08:03:53.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.188+0000 7fdf7fe00640 1 --2- 172.21.15.1:0/2678410323 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf880cd490 0x7fdf881efec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:53.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.188+0000 7fdf7fe00640 1 -- 172.21.15.1:0/2678410323 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fdf880c4a50 con 0x7fdf880cd490 2024-05-09T08:03:53.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.189+0000 7fdf7fe00640 1 --2- 172.21.15.1:0/2678410323 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf880cd490 0x7fdf881efec0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fdf70000f60 tx=0x7fdf70001970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:03:53.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.189+0000 7fdf7e000640 1 -- 172.21.15.1:0/2678410323 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf7000f5e0 con 0x7fdf880cd490 2024-05-09T08:03:53.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.189+0000 7fdf7e000640 1 -- 172.21.15.1:0/2678410323 <== mon.2 v2:172.21.15.7:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fdf7801a070 con 0x7fdf8806e020 2024-05-09T08:03:53.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.190+0000 7fdf7e000640 1 --2- 172.21.15.1:0/2678410323 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fdf6406bd30 0x7fdf6406e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:53.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.190+0000 7fdf7e000640 1 -- 172.21.15.1:0/2678410323 <== mon.2 v2:172.21.15.7:3301/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7fdf7808a640 con 0x7fdf8806e020 2024-05-09T08:03:53.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.190+0000 7fdf7e000640 1 -- 172.21.15.1:0/2678410323 <== mon.1 v2:172.21.15.7:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fdf70018070 con 0x7fdf880cd490 2024-05-09T08:03:53.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.190+0000 7fdf7e000640 1 -- 172.21.15.1:0/2678410323 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf880cd490 msgr2=0x7fdf881efec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:53.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.190+0000 7fdf7e000640 1 --2- 172.21.15.1:0/2678410323 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf880cd490 0x7fdf881efec0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fdf70000f60 tx=0x7fdf70001970 comp rx=0 tx=0).stop 2024-05-09T08:03:53.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.190+0000 7fdf7e000640 1 -- 172.21.15.1:0/2678410323 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf880cd490 msgr2=0x7fdf881efec0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-05-09T08:03:53.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.190+0000 7fdf7e000640 1 --2- 172.21.15.1:0/2678410323 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf880cd490 0x7fdf881efec0 secure :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fdf70000f60 tx=0x7fdf70001970 comp rx=0 tx=0).stop 2024-05-09T08:03:53.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.190+0000 7fdf86800640 1 --2- 172.21.15.1:0/2678410323 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fdf6406bd30 0x7fdf6406e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:53.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.191+0000 7fdf86800640 1 --2- 172.21.15.1:0/2678410323 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fdf6406bd30 0x7fdf6406e1f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fdf80034000 tx=0x7fdf8002f000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:03:53.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.197+0000 7fdf87200640 1 --2- 172.21.15.1:0/2678410323 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf881f3430 0x7fdf881f3840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:53.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.197+0000 7fdf87200640 1 -- 172.21.15.1:0/2678410323 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fdf880c6e40 con 0x7fdf881f3430 2024-05-09T08:03:53.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.198+0000 7fdf7fe00640 1 --2- 172.21.15.1:0/2678410323 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf881f3430 0x7fdf881f3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:53.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.198+0000 7fdf7fe00640 1 -- 172.21.15.1:0/2678410323 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fdf880734c0 con 0x7fdf881f3430 2024-05-09T08:03:53.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.198+0000 7fdf7fe00640 1 --2- 172.21.15.1:0/2678410323 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf881f3430 0x7fdf881f3840 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fdf70004fd0 tx=0x7fdf70000ed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:03:53.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.199+0000 7fdf7e000640 1 -- 172.21.15.1:0/2678410323 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf70027410 con 0x7fdf881f3430 2024-05-09T08:03:53.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.199+0000 7fdf7e000640 1 -- 172.21.15.1:0/2678410323 <== mon.1 v2:172.21.15.7:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4532 (secure 0 0 0) 0x7fdf700133f0 con 0x7fdf881f3430 2024-05-09T08:03:53.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.199+0000 7fdf7e000640 1 -- 172.21.15.1:0/2678410323 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf881f3430 msgr2=0x7fdf881f3840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:53.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.199+0000 7fdf7e000640 1 --2- 172.21.15.1:0/2678410323 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf881f3430 0x7fdf881f3840 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fdf70004fd0 tx=0x7fdf70000ed0 comp rx=0 tx=0).stop 2024-05-09T08:03:53.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.199+0000 7fdf7e000640 1 -- 172.21.15.1:0/2678410323 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf881f3430 msgr2=0x7fdf881f3840 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-05-09T08:03:53.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.199+0000 7fdf7e000640 1 --2- 172.21.15.1:0/2678410323 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf881f3430 0x7fdf881f3840 secure :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fdf70004fd0 tx=0x7fdf70000ed0 comp rx=0 tx=0).stop 2024-05-09T08:03:53.217 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-05-09T08:03:53.217 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "b", 2024-05-09T08:03:53.217 INFO:teuthology.orchestra.run.smithi001.stdout: "rank": 1, 2024-05-09T08:03:53.217 INFO:teuthology.orchestra.run.smithi001.stdout: "state": "peon", 2024-05-09T08:03:53.217 INFO:teuthology.orchestra.run.smithi001.stdout: "election_epoch": 10, 2024-05-09T08:03:53.217 INFO:teuthology.orchestra.run.smithi001.stdout: "uptime": 860, 2024-05-09T08:03:53.217 INFO:teuthology.orchestra.run.smithi001.stdout: "quorum": [ 2024-05-09T08:03:53.217 INFO:teuthology.orchestra.run.smithi001.stdout: 0, 2024-05-09T08:03:53.217 INFO:teuthology.orchestra.run.smithi001.stdout: 1, 2024-05-09T08:03:53.217 INFO:teuthology.orchestra.run.smithi001.stdout: 2 2024-05-09T08:03:53.217 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:03:53.217 INFO:teuthology.orchestra.run.smithi001.stdout: "quorum_age": 0, 2024-05-09T08:03:53.217 INFO:teuthology.orchestra.run.smithi001.stdout: "features": { 2024-05-09T08:03:53.218 INFO:teuthology.orchestra.run.smithi001.stdout: "required_con": "2450521708007866372", 2024-05-09T08:03:53.218 INFO:teuthology.orchestra.run.smithi001.stdout: "required_mon": [ 2024-05-09T08:03:53.218 INFO:teuthology.orchestra.run.smithi001.stdout: "kraken", 2024-05-09T08:03:53.218 INFO:teuthology.orchestra.run.smithi001.stdout: "luminous", 2024-05-09T08:03:53.218 INFO:teuthology.orchestra.run.smithi001.stdout: "mimic", 2024-05-09T08:03:53.218 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap-prune", 2024-05-09T08:03:53.218 INFO:teuthology.orchestra.run.smithi001.stdout: "nautilus", 2024-05-09T08:03:53.218 INFO:teuthology.orchestra.run.smithi001.stdout: "octopus", 2024-05-09T08:03:53.218 INFO:teuthology.orchestra.run.smithi001.stdout: "pacific", 2024-05-09T08:03:53.218 INFO:teuthology.orchestra.run.smithi001.stdout: "elector-pinging", 2024-05-09T08:03:53.218 INFO:teuthology.orchestra.run.smithi001.stdout: "quincy", 2024-05-09T08:03:53.218 INFO:teuthology.orchestra.run.smithi001.stdout: "reef", 2024-05-09T08:03:53.218 INFO:teuthology.orchestra.run.smithi001.stdout: "squid" 2024-05-09T08:03:53.218 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:03:53.219 INFO:teuthology.orchestra.run.smithi001.stdout: "quorum_con": "4540701547738038271", 2024-05-09T08:03:53.219 INFO:teuthology.orchestra.run.smithi001.stdout: "quorum_mon": [ 2024-05-09T08:03:53.219 INFO:teuthology.orchestra.run.smithi001.stdout: "kraken", 2024-05-09T08:03:53.219 INFO:teuthology.orchestra.run.smithi001.stdout: "luminous", 2024-05-09T08:03:53.219 INFO:teuthology.orchestra.run.smithi001.stdout: "mimic", 2024-05-09T08:03:53.219 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap-prune", 2024-05-09T08:03:53.219 INFO:teuthology.orchestra.run.smithi001.stdout: "nautilus", 2024-05-09T08:03:53.219 INFO:teuthology.orchestra.run.smithi001.stdout: "octopus", 2024-05-09T08:03:53.219 INFO:teuthology.orchestra.run.smithi001.stdout: "pacific", 2024-05-09T08:03:53.219 INFO:teuthology.orchestra.run.smithi001.stdout: "elector-pinging", 2024-05-09T08:03:53.219 INFO:teuthology.orchestra.run.smithi001.stdout: "quincy", 2024-05-09T08:03:53.219 INFO:teuthology.orchestra.run.smithi001.stdout: "reef", 2024-05-09T08:03:53.219 INFO:teuthology.orchestra.run.smithi001.stdout: "squid" 2024-05-09T08:03:53.220 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:03:53.220 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:03:53.220 INFO:teuthology.orchestra.run.smithi001.stdout: "outside_quorum": [], 2024-05-09T08:03:53.220 INFO:teuthology.orchestra.run.smithi001.stdout: "extra_probe_peers": [], 2024-05-09T08:03:53.220 INFO:teuthology.orchestra.run.smithi001.stdout: "sync_provider": [], 2024-05-09T08:03:53.220 INFO:teuthology.orchestra.run.smithi001.stdout: "monmap": { 2024-05-09T08:03:53.220 INFO:teuthology.orchestra.run.smithi001.stdout: "epoch": 1, 2024-05-09T08:03:53.220 INFO:teuthology.orchestra.run.smithi001.stdout: "fsid": "54329ec8-3270-4043-9a82-c6cd7420aa7f", 2024-05-09T08:03:53.220 INFO:teuthology.orchestra.run.smithi001.stdout: "modified": "2024-05-09T08:01:21.819270Z", 2024-05-09T08:03:53.220 INFO:teuthology.orchestra.run.smithi001.stdout: "created": "2024-05-09T08:01:21.819270Z", 2024-05-09T08:03:53.220 INFO:teuthology.orchestra.run.smithi001.stdout: "min_mon_release": 19, 2024-05-09T08:03:53.220 INFO:teuthology.orchestra.run.smithi001.stdout: "min_mon_release_name": "squid", 2024-05-09T08:03:53.221 INFO:teuthology.orchestra.run.smithi001.stdout: "election_strategy": 1, 2024-05-09T08:03:53.221 INFO:teuthology.orchestra.run.smithi001.stdout: "disallowed_leaders: ": "", 2024-05-09T08:03:53.221 INFO:teuthology.orchestra.run.smithi001.stdout: "stretch_mode": false, 2024-05-09T08:03:53.221 INFO:teuthology.orchestra.run.smithi001.stdout: "tiebreaker_mon": "", 2024-05-09T08:03:53.221 INFO:teuthology.orchestra.run.smithi001.stdout: "removed_ranks: ": "", 2024-05-09T08:03:53.221 INFO:teuthology.orchestra.run.smithi001.stdout: "features": { 2024-05-09T08:03:53.221 INFO:teuthology.orchestra.run.smithi001.stdout: "persistent": [ 2024-05-09T08:03:53.222 INFO:teuthology.orchestra.run.smithi001.stdout: "kraken", 2024-05-09T08:03:53.222 INFO:teuthology.orchestra.run.smithi001.stdout: "luminous", 2024-05-09T08:03:53.222 INFO:teuthology.orchestra.run.smithi001.stdout: "mimic", 2024-05-09T08:03:53.222 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap-prune", 2024-05-09T08:03:53.222 INFO:teuthology.orchestra.run.smithi001.stdout: "nautilus", 2024-05-09T08:03:53.222 INFO:teuthology.orchestra.run.smithi001.stdout: "octopus", 2024-05-09T08:03:53.222 INFO:teuthology.orchestra.run.smithi001.stdout: "pacific", 2024-05-09T08:03:53.222 INFO:teuthology.orchestra.run.smithi001.stdout: "elector-pinging", 2024-05-09T08:03:53.222 INFO:teuthology.orchestra.run.smithi001.stdout: "quincy", 2024-05-09T08:03:53.222 INFO:teuthology.orchestra.run.smithi001.stdout: "reef", 2024-05-09T08:03:53.222 INFO:teuthology.orchestra.run.smithi001.stdout: "squid" 2024-05-09T08:03:53.222 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:03:53.222 INFO:teuthology.orchestra.run.smithi001.stdout: "optional": [] 2024-05-09T08:03:53.223 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:03:53.223 INFO:teuthology.orchestra.run.smithi001.stdout: "mons": [ 2024-05-09T08:03:53.223 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:03:53.223 INFO:teuthology.orchestra.run.smithi001.stdout: "rank": 0, 2024-05-09T08:03:53.223 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "a", 2024-05-09T08:03:53.223 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-05-09T08:03:53.223 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-05-09T08:03:53.223 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:03:53.223 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-05-09T08:03:53.224 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:3300", 2024-05-09T08:03:53.224 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:03:53.224 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:03:53.224 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:03:53.224 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-05-09T08:03:53.224 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6789", 2024-05-09T08:03:53.224 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:03:53.224 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:03:53.224 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:03:53.224 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:03:53.224 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6789/0", 2024-05-09T08:03:53.224 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.1:6789/0", 2024-05-09T08:03:53.224 INFO:teuthology.orchestra.run.smithi001.stdout: "priority": 0, 2024-05-09T08:03:53.225 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 0, 2024-05-09T08:03:53.225 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_location": "{}" 2024-05-09T08:03:53.225 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:03:53.225 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:03:53.225 INFO:teuthology.orchestra.run.smithi001.stdout: "rank": 1, 2024-05-09T08:03:53.225 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "b", 2024-05-09T08:03:53.225 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-05-09T08:03:53.225 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-05-09T08:03:53.225 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:03:53.225 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-05-09T08:03:53.225 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:3300", 2024-05-09T08:03:53.225 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:03:53.225 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:03:53.225 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:03:53.225 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-05-09T08:03:53.226 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:6789", 2024-05-09T08:03:53.226 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:03:53.226 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:03:53.226 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:03:53.226 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:03:53.226 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:6789/0", 2024-05-09T08:03:53.226 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.7:6789/0", 2024-05-09T08:03:53.226 INFO:teuthology.orchestra.run.smithi001.stdout: "priority": 0, 2024-05-09T08:03:53.226 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 0, 2024-05-09T08:03:53.226 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_location": "{}" 2024-05-09T08:03:53.226 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:03:53.226 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:03:53.226 INFO:teuthology.orchestra.run.smithi001.stdout: "rank": 2, 2024-05-09T08:03:53.226 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "c", 2024-05-09T08:03:53.227 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-05-09T08:03:53.227 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-05-09T08:03:53.227 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:03:53.227 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-05-09T08:03:53.227 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:3301", 2024-05-09T08:03:53.227 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:03:53.227 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:03:53.227 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:03:53.227 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-05-09T08:03:53.227 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:6790", 2024-05-09T08:03:53.227 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:03:53.227 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:03:53.227 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:03:53.227 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:03:53.227 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:6790/0", 2024-05-09T08:03:53.228 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.7:6790/0", 2024-05-09T08:03:53.228 INFO:teuthology.orchestra.run.smithi001.stdout: "priority": 0, 2024-05-09T08:03:53.228 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 0, 2024-05-09T08:03:53.228 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_location": "{}" 2024-05-09T08:03:53.228 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:03:53.228 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:03:53.228 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:03:53.228 INFO:teuthology.orchestra.run.smithi001.stdout: "feature_map": { 2024-05-09T08:03:53.228 INFO:teuthology.orchestra.run.smithi001.stdout: "mon": [ 2024-05-09T08:03:53.228 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:03:53.228 INFO:teuthology.orchestra.run.smithi001.stdout: "features": "0x3f03cffffffdffff", 2024-05-09T08:03:53.228 INFO:teuthology.orchestra.run.smithi001.stdout: "release": "luminous", 2024-05-09T08:03:53.228 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 1 2024-05-09T08:03:53.228 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:03:53.229 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:03:53.229 INFO:teuthology.orchestra.run.smithi001.stdout: "client": [ 2024-05-09T08:03:53.229 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:03:53.229 INFO:teuthology.orchestra.run.smithi001.stdout: "features": "0x3f03cffffffdffff", 2024-05-09T08:03:53.229 INFO:teuthology.orchestra.run.smithi001.stdout: "release": "luminous", 2024-05-09T08:03:53.229 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 1 2024-05-09T08:03:53.229 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:03:53.229 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:03:53.229 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:03:53.229 INFO:teuthology.orchestra.run.smithi001.stdout: "stretch_mode": false 2024-05-09T08:03:53.229 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-05-09T08:03:53.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.200+0000 7fdf87200640 1 -- 172.21.15.1:0/2678410323 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fdf6406bd30 msgr2=0x7fdf6406e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:53.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.200+0000 7fdf87200640 1 --2- 172.21.15.1:0/2678410323 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fdf6406bd30 0x7fdf6406e1f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fdf80034000 tx=0x7fdf8002f000 comp rx=0 tx=0).stop 2024-05-09T08:03:53.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.200+0000 7fdf87200640 1 -- 172.21.15.1:0/2678410323 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fdf8806e020 msgr2=0x7fdf880c1280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:53.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.200+0000 7fdf87200640 1 --2- 172.21.15.1:0/2678410323 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fdf8806e020 0x7fdf880c1280 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fdf7800c400 tx=0x7fdf7800c8c0 comp rx=0 tx=0).stop 2024-05-09T08:03:53.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.200+0000 7fdf86800640 1 -- 172.21.15.1:0/2678410323 reap_dead start 2024-05-09T08:03:53.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.200+0000 7fdf86800640 1 -- 172.21.15.1:0/2678410323 reap_dead start 2024-05-09T08:03:53.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.200+0000 7fdf87200640 1 -- 172.21.15.1:0/2678410323 shutdown_connections 2024-05-09T08:03:53.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.200+0000 7fdf87200640 1 -- 172.21.15.1:0/2678410323 >> 172.21.15.1:0/2678410323 conn(0x7fdf880cc970 msgr2=0x7fdf88158630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:03:53.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.200+0000 7fdf87200640 1 -- 172.21.15.1:0/2678410323 shutdown_connections 2024-05-09T08:03:53.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.200+0000 7fdf87200640 1 -- 172.21.15.1:0/2678410323 wait complete. 2024-05-09T08:03:53.232 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-05-09T08:03:53.232 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-05-09T08:03:53.232 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-05-09T08:03:53.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.400+0000 7f1942000640 1 Processor -- start 2024-05-09T08:03:53.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.400+0000 7f1942000640 1 -- start start 2024-05-09T08:03:53.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.401+0000 7f1942000640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f193c0cc5f0 0x7f193c15a570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:53.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.401+0000 7f1942000640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f193c0ce4e0 0x7f193c15aab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:53.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.401+0000 7f1942000640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f193c0ccb20 con 0x7f193c0cac60 2024-05-09T08:03:53.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.401+0000 7f1942000640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f193c0cce00 con 0x7f193c0ce4e0 2024-05-09T08:03:53.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.401+0000 7f1942000640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f193c0cf120 con 0x7f193c0cc5f0 2024-05-09T08:03:53.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.402+0000 7f193be00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f193c0cc5f0 0x7f193c15a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:53.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.402+0000 7f1940c00640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f193c0cac60 0x7f193c0cda00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:49146/0 (socket says 172.21.15.1:49146) 2024-05-09T08:03:53.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.402+0000 7f193be00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f193c0cc5f0 0x7f193c15a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.1:52600/0 (socket says 172.21.15.1:52600) 2024-05-09T08:03:53.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.402+0000 7f1940c00640 1 -- 172.21.15.1:0/4042067713 learned_addr learned my addr 172.21.15.1:0/4042067713 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T08:03:53.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.402+0000 7f1941600640 1 --2- 172.21.15.1:0/4042067713 >> v2:172.21.15.1:3300/0 conn(0x7f193c0ce4e0 0x7f193c15aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:53.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.402+0000 7f193be00640 1 -- 172.21.15.1:0/4042067713 >> v2:172.21.15.1:3300/0 conn(0x7f193c0ce4e0 msgr2=0x7f193c15aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:03:53.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.402+0000 7f193be00640 1 --2- 172.21.15.1:0/4042067713 >> v2:172.21.15.1:3300/0 conn(0x7f193c0ce4e0 0x7f193c15aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:53.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.402+0000 7f193be00640 1 -- 172.21.15.1:0/4042067713 >> v1:172.21.15.1:6789/0 conn(0x7f193c0cac60 legacy=0x7f193c0cda00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:53.420 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.402+0000 7f193be00640 1 -- 172.21.15.1:0/4042067713 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f193c0cf400 con 0x7f193c0cc5f0 2024-05-09T08:03:53.420 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.404+0000 7f193be00640 1 --2- 172.21.15.1:0/4042067713 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f193c0cc5f0 0x7f193c15a570 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f193c154dd0 tx=0x7f192802a1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3b85782e011507a6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:03:53.420 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.404+0000 7f193b400640 1 -- 172.21.15.1:0/4042067713 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1928037070 con 0x7f193c0cc5f0 2024-05-09T08:03:53.420 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.404+0000 7f193b400640 1 -- 172.21.15.1:0/4042067713 <== mon.1 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1928032440 con 0x7f193c0cc5f0 2024-05-09T08:03:53.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.404+0000 7f193b400640 1 -- 172.21.15.1:0/4042067713 <== mon.1 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1928032810 con 0x7f193c0cc5f0 2024-05-09T08:03:53.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.405+0000 7f1942000640 1 -- 172.21.15.1:0/4042067713 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f193c0cc5f0 msgr2=0x7f193c15a570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:53.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.405+0000 7f1942000640 1 --2- 172.21.15.1:0/4042067713 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f193c0cc5f0 0x7f193c15a570 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f193c154dd0 tx=0x7f192802a1a0 comp rx=0 tx=0).stop 2024-05-09T08:03:53.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.405+0000 7f1942000640 1 -- 172.21.15.1:0/4042067713 shutdown_connections 2024-05-09T08:03:53.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.405+0000 7f1942000640 1 --2- 172.21.15.1:0/4042067713 >> v2:172.21.15.1:3300/0 conn(0x7f193c0ce4e0 0x7f193c15aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:53.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.405+0000 7f1942000640 1 --2- 172.21.15.1:0/4042067713 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f193c0cc5f0 0x7f193c15a570 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:53.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.405+0000 7f1942000640 1 -- 172.21.15.1:0/4042067713 >> 172.21.15.1:0/4042067713 conn(0x7f193c0cfc20 msgr2=0x7f193c0d0030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:03:53.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.405+0000 7f1942000640 1 -- 172.21.15.1:0/4042067713 shutdown_connections 2024-05-09T08:03:53.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.405+0000 7f1942000640 1 -- 172.21.15.1:0/4042067713 wait complete. 2024-05-09T08:03:53.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.407+0000 7f1942000640 1 Processor -- start 2024-05-09T08:03:53.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.407+0000 7f1942000640 1 -- start start 2024-05-09T08:03:53.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.407+0000 7f1942000640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f193c0cac60 0x7f193c15c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:53.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.408+0000 7f1942000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f193c0cc5f0 0x7f193c14b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:53.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.408+0000 7f1942000640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f193c0ce4e0 0x7f193c14b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:53.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.408+0000 7f1942000640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f193c0ce110 con 0x7f193c0ce4e0 2024-05-09T08:03:53.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.408+0000 7f1942000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f193c0cd550 con 0x7f193c0cc5f0 2024-05-09T08:03:53.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.408+0000 7f1942000640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f193c0ce290 con 0x7f193c0cac60 2024-05-09T08:03:53.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.408+0000 7f1941600640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f193c0ce4e0 0x7f193c14b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:53.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.408+0000 7f1941600640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f193c0ce4e0 0x7f193c14b5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:57592/0 (socket says 172.21.15.1:57592) 2024-05-09T08:03:53.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.408+0000 7f1941600640 1 -- 172.21.15.1:0/1925060871 learned_addr learned my addr 172.21.15.1:0/1925060871 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:03:53.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.408+0000 7f1940c00640 1 --2- 172.21.15.1:0/1925060871 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f193c0cac60 0x7f193c15c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:53.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.408+0000 7f193be00640 1 --2- 172.21.15.1:0/1925060871 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f193c0cc5f0 0x7f193c14b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:53.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.408+0000 7f1941600640 1 -- 172.21.15.1:0/1925060871 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f193c0cac60 msgr2=0x7f193c15c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:53.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.408+0000 7f1941600640 1 --2- 172.21.15.1:0/1925060871 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f193c0cac60 0x7f193c15c640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:53.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.408+0000 7f1941600640 1 -- 172.21.15.1:0/1925060871 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f193c0cc5f0 msgr2=0x7f193c14b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:53.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.408+0000 7f1941600640 1 --2- 172.21.15.1:0/1925060871 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f193c0cc5f0 0x7f193c14b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:53.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.408+0000 7f1941600640 1 -- 172.21.15.1:0/1925060871 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f193c1e6600 con 0x7f193c0ce4e0 2024-05-09T08:03:53.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.408+0000 7f193be00640 1 --2- 172.21.15.1:0/1925060871 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f193c0cc5f0 0x7f193c14b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:03:53.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.409+0000 7f1941600640 1 --2- 172.21.15.1:0/1925060871 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f193c0ce4e0 0x7f193c14b5c0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f193400ec40 tx=0x7f193400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:03:53.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.409+0000 7f1938c00640 1 -- 172.21.15.1:0/1925060871 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1934018020 con 0x7f193c0ce4e0 2024-05-09T08:03:53.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.409+0000 7f1938c00640 1 -- 172.21.15.1:0/1925060871 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f193400ee50 con 0x7f193c0ce4e0 2024-05-09T08:03:53.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.409+0000 7f1942000640 1 -- 172.21.15.1:0/1925060871 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f193c156c60 con 0x7f193c0ce4e0 2024-05-09T08:03:53.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.409+0000 7f1938c00640 1 -- 172.21.15.1:0/1925060871 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19340146c0 con 0x7f193c0ce4e0 2024-05-09T08:03:53.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.409+0000 7f1942000640 1 -- 172.21.15.1:0/1925060871 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f193c155940 con 0x7f193c0ce4e0 2024-05-09T08:03:53.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.410+0000 7f1942000640 1 --2- 172.21.15.1:0/1925060871 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f193c156420 0x7f193c155b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:53.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.410+0000 7f1942000640 1 -- 172.21.15.1:0/1925060871 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f193c1557c0 con 0x7f193c156420 2024-05-09T08:03:53.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.411+0000 7f1940c00640 1 --2- 172.21.15.1:0/1925060871 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f193c156420 0x7f193c155b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:53.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.411+0000 7f1940c00640 1 -- 172.21.15.1:0/1925060871 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f193c157220 con 0x7f193c156420 2024-05-09T08:03:53.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.411+0000 7f1940c00640 1 --2- 172.21.15.1:0/1925060871 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f193c156420 0x7f193c155b80 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f1930000f60 tx=0x7f1930001920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:03:53.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.411+0000 7f1938c00640 1 -- 172.21.15.1:0/1925060871 <== mon.2 v2:172.21.15.7:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f193000f5e0 con 0x7f193c156420 2024-05-09T08:03:53.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.412+0000 7f1938c00640 1 -- 172.21.15.1:0/1925060871 <== mon.2 v2:172.21.15.7:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f1930018070 con 0x7f193c156420 2024-05-09T08:03:53.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.412+0000 7f1938c00640 1 -- 172.21.15.1:0/1925060871 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f193c156420 msgr2=0x7f193c155b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:53.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.412+0000 7f1938c00640 1 --2- 172.21.15.1:0/1925060871 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f193c156420 0x7f193c155b80 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f1930000f60 tx=0x7f1930001920 comp rx=0 tx=0).stop 2024-05-09T08:03:53.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.412+0000 7f1938c00640 1 -- 172.21.15.1:0/1925060871 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f193c156420 msgr2=0x7f193c155b80 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-05-09T08:03:53.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.412+0000 7f1938c00640 1 --2- 172.21.15.1:0/1925060871 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f193c156420 0x7f193c155b80 secure :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f1930000f60 tx=0x7f1930001920 comp rx=0 tx=0).stop 2024-05-09T08:03:53.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.412+0000 7f1938c00640 1 -- 172.21.15.1:0/1925060871 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1934005d10 con 0x7f193c0ce4e0 2024-05-09T08:03:53.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.413+0000 7f1938c00640 1 --2- 172.21.15.1:0/1925060871 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f191c06b910 0x7f191c06ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:53.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.413+0000 7f1938c00640 1 -- 172.21.15.1:0/1925060871 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7f193408bc20 con 0x7f193c0ce4e0 2024-05-09T08:03:53.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.413+0000 7f1940c00640 1 --2- 172.21.15.1:0/1925060871 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f191c06b910 0x7f191c06ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:53.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.414+0000 7f1940c00640 1 --2- 172.21.15.1:0/1925060871 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f191c06b910 0x7f191c06ddd0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f1930001960 tx=0x7f1930009000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:03:53.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.419+0000 7f1942000640 1 --2- 172.21.15.1:0/1925060871 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f193c15e420 0x7f193c1f3880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:53.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.419+0000 7f1942000640 1 -- 172.21.15.1:0/1925060871 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f193c0c34f0 con 0x7f193c15e420 2024-05-09T08:03:53.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.420+0000 7f193be00640 1 --2- 172.21.15.1:0/1925060871 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f193c15e420 0x7f193c1f3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:53.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.420+0000 7f193be00640 1 -- 172.21.15.1:0/1925060871 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f193c04f660 con 0x7f193c15e420 2024-05-09T08:03:53.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.421+0000 7f193be00640 1 --2- 172.21.15.1:0/1925060871 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f193c15e420 0x7f193c1f3880 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f193c0735d0 tx=0x7f19280046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:03:53.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.421+0000 7f1938c00640 1 -- 172.21.15.1:0/1925060871 <== mon.2 v2:172.21.15.7:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1928043070 con 0x7f193c15e420 2024-05-09T08:03:53.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.421+0000 7f1938c00640 1 -- 172.21.15.1:0/1925060871 <== mon.2 v2:172.21.15.7:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5188 (secure 0 0 0) 0x7f1928032440 con 0x7f193c15e420 2024-05-09T08:03:53.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.421+0000 7f1938c00640 1 -- 172.21.15.1:0/1925060871 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f193c15e420 msgr2=0x7f193c1f3880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:53.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.421+0000 7f1938c00640 1 --2- 172.21.15.1:0/1925060871 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f193c15e420 0x7f193c1f3880 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f193c0735d0 tx=0x7f19280046c0 comp rx=0 tx=0).stop 2024-05-09T08:03:53.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.421+0000 7f1938c00640 1 -- 172.21.15.1:0/1925060871 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f193c15e420 msgr2=0x7f193c1f3880 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-05-09T08:03:53.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.421+0000 7f1938c00640 1 --2- 172.21.15.1:0/1925060871 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f193c15e420 0x7f193c1f3880 secure :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f193c0735d0 tx=0x7f19280046c0 comp rx=0 tx=0).stop 2024-05-09T08:03:53.438 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-05-09T08:03:53.438 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "c", 2024-05-09T08:03:53.438 INFO:teuthology.orchestra.run.smithi001.stdout: "rank": 2, 2024-05-09T08:03:53.438 INFO:teuthology.orchestra.run.smithi001.stdout: "state": "peon", 2024-05-09T08:03:53.439 INFO:teuthology.orchestra.run.smithi001.stdout: "election_epoch": 10, 2024-05-09T08:03:53.439 INFO:teuthology.orchestra.run.smithi001.stdout: "uptime": 116837, 2024-05-09T08:03:53.439 INFO:teuthology.orchestra.run.smithi001.stdout: "quorum": [ 2024-05-09T08:03:53.439 INFO:teuthology.orchestra.run.smithi001.stdout: 0, 2024-05-09T08:03:53.439 INFO:teuthology.orchestra.run.smithi001.stdout: 1, 2024-05-09T08:03:53.439 INFO:teuthology.orchestra.run.smithi001.stdout: 2 2024-05-09T08:03:53.439 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:03:53.439 INFO:teuthology.orchestra.run.smithi001.stdout: "quorum_age": 0, 2024-05-09T08:03:53.439 INFO:teuthology.orchestra.run.smithi001.stdout: "features": { 2024-05-09T08:03:53.439 INFO:teuthology.orchestra.run.smithi001.stdout: "required_con": "2450521708007866372", 2024-05-09T08:03:53.439 INFO:teuthology.orchestra.run.smithi001.stdout: "required_mon": [ 2024-05-09T08:03:53.439 INFO:teuthology.orchestra.run.smithi001.stdout: "kraken", 2024-05-09T08:03:53.439 INFO:teuthology.orchestra.run.smithi001.stdout: "luminous", 2024-05-09T08:03:53.439 INFO:teuthology.orchestra.run.smithi001.stdout: "mimic", 2024-05-09T08:03:53.439 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap-prune", 2024-05-09T08:03:53.440 INFO:teuthology.orchestra.run.smithi001.stdout: "nautilus", 2024-05-09T08:03:53.440 INFO:teuthology.orchestra.run.smithi001.stdout: "octopus", 2024-05-09T08:03:53.440 INFO:teuthology.orchestra.run.smithi001.stdout: "pacific", 2024-05-09T08:03:53.440 INFO:teuthology.orchestra.run.smithi001.stdout: "elector-pinging", 2024-05-09T08:03:53.440 INFO:teuthology.orchestra.run.smithi001.stdout: "quincy", 2024-05-09T08:03:53.440 INFO:teuthology.orchestra.run.smithi001.stdout: "reef", 2024-05-09T08:03:53.440 INFO:teuthology.orchestra.run.smithi001.stdout: "squid" 2024-05-09T08:03:53.440 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:03:53.440 INFO:teuthology.orchestra.run.smithi001.stdout: "quorum_con": "4540701547738038271", 2024-05-09T08:03:53.440 INFO:teuthology.orchestra.run.smithi001.stdout: "quorum_mon": [ 2024-05-09T08:03:53.440 INFO:teuthology.orchestra.run.smithi001.stdout: "kraken", 2024-05-09T08:03:53.441 INFO:teuthology.orchestra.run.smithi001.stdout: "luminous", 2024-05-09T08:03:53.441 INFO:teuthology.orchestra.run.smithi001.stdout: "mimic", 2024-05-09T08:03:53.441 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap-prune", 2024-05-09T08:03:53.441 INFO:teuthology.orchestra.run.smithi001.stdout: "nautilus", 2024-05-09T08:03:53.441 INFO:teuthology.orchestra.run.smithi001.stdout: "octopus", 2024-05-09T08:03:53.441 INFO:teuthology.orchestra.run.smithi001.stdout: "pacific", 2024-05-09T08:03:53.441 INFO:teuthology.orchestra.run.smithi001.stdout: "elector-pinging", 2024-05-09T08:03:53.441 INFO:teuthology.orchestra.run.smithi001.stdout: "quincy", 2024-05-09T08:03:53.441 INFO:teuthology.orchestra.run.smithi001.stdout: "reef", 2024-05-09T08:03:53.441 INFO:teuthology.orchestra.run.smithi001.stdout: "squid" 2024-05-09T08:03:53.441 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:03:53.441 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:03:53.441 INFO:teuthology.orchestra.run.smithi001.stdout: "outside_quorum": [], 2024-05-09T08:03:53.441 INFO:teuthology.orchestra.run.smithi001.stdout: "extra_probe_peers": [], 2024-05-09T08:03:53.442 INFO:teuthology.orchestra.run.smithi001.stdout: "sync_provider": [], 2024-05-09T08:03:53.442 INFO:teuthology.orchestra.run.smithi001.stdout: "monmap": { 2024-05-09T08:03:53.442 INFO:teuthology.orchestra.run.smithi001.stdout: "epoch": 1, 2024-05-09T08:03:53.442 INFO:teuthology.orchestra.run.smithi001.stdout: "fsid": "54329ec8-3270-4043-9a82-c6cd7420aa7f", 2024-05-09T08:03:53.442 INFO:teuthology.orchestra.run.smithi001.stdout: "modified": "2024-05-09T08:01:21.819270Z", 2024-05-09T08:03:53.442 INFO:teuthology.orchestra.run.smithi001.stdout: "created": "2024-05-09T08:01:21.819270Z", 2024-05-09T08:03:53.442 INFO:teuthology.orchestra.run.smithi001.stdout: "min_mon_release": 19, 2024-05-09T08:03:53.442 INFO:teuthology.orchestra.run.smithi001.stdout: "min_mon_release_name": "squid", 2024-05-09T08:03:53.442 INFO:teuthology.orchestra.run.smithi001.stdout: "election_strategy": 1, 2024-05-09T08:03:53.442 INFO:teuthology.orchestra.run.smithi001.stdout: "disallowed_leaders: ": "", 2024-05-09T08:03:53.442 INFO:teuthology.orchestra.run.smithi001.stdout: "stretch_mode": false, 2024-05-09T08:03:53.442 INFO:teuthology.orchestra.run.smithi001.stdout: "tiebreaker_mon": "", 2024-05-09T08:03:53.442 INFO:teuthology.orchestra.run.smithi001.stdout: "removed_ranks: ": "", 2024-05-09T08:03:53.443 INFO:teuthology.orchestra.run.smithi001.stdout: "features": { 2024-05-09T08:03:53.443 INFO:teuthology.orchestra.run.smithi001.stdout: "persistent": [ 2024-05-09T08:03:53.443 INFO:teuthology.orchestra.run.smithi001.stdout: "kraken", 2024-05-09T08:03:53.443 INFO:teuthology.orchestra.run.smithi001.stdout: "luminous", 2024-05-09T08:03:53.443 INFO:teuthology.orchestra.run.smithi001.stdout: "mimic", 2024-05-09T08:03:53.443 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap-prune", 2024-05-09T08:03:53.443 INFO:teuthology.orchestra.run.smithi001.stdout: "nautilus", 2024-05-09T08:03:53.443 INFO:teuthology.orchestra.run.smithi001.stdout: "octopus", 2024-05-09T08:03:53.443 INFO:teuthology.orchestra.run.smithi001.stdout: "pacific", 2024-05-09T08:03:53.443 INFO:teuthology.orchestra.run.smithi001.stdout: "elector-pinging", 2024-05-09T08:03:53.443 INFO:teuthology.orchestra.run.smithi001.stdout: "quincy", 2024-05-09T08:03:53.443 INFO:teuthology.orchestra.run.smithi001.stdout: "reef", 2024-05-09T08:03:53.443 INFO:teuthology.orchestra.run.smithi001.stdout: "squid" 2024-05-09T08:03:53.443 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:03:53.443 INFO:teuthology.orchestra.run.smithi001.stdout: "optional": [] 2024-05-09T08:03:53.444 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:03:53.444 INFO:teuthology.orchestra.run.smithi001.stdout: "mons": [ 2024-05-09T08:03:53.444 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:03:53.444 INFO:teuthology.orchestra.run.smithi001.stdout: "rank": 0, 2024-05-09T08:03:53.444 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "a", 2024-05-09T08:03:53.444 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-05-09T08:03:53.444 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-05-09T08:03:53.444 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:03:53.444 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-05-09T08:03:53.444 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:3300", 2024-05-09T08:03:53.444 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:03:53.444 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:03:53.444 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:03:53.445 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-05-09T08:03:53.445 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6789", 2024-05-09T08:03:53.445 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:03:53.445 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:03:53.445 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:03:53.445 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:03:53.445 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6789/0", 2024-05-09T08:03:53.445 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.1:6789/0", 2024-05-09T08:03:53.445 INFO:teuthology.orchestra.run.smithi001.stdout: "priority": 0, 2024-05-09T08:03:53.445 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 0, 2024-05-09T08:03:53.445 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_location": "{}" 2024-05-09T08:03:53.445 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:03:53.445 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:03:53.445 INFO:teuthology.orchestra.run.smithi001.stdout: "rank": 1, 2024-05-09T08:03:53.446 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "b", 2024-05-09T08:03:53.446 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-05-09T08:03:53.446 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-05-09T08:03:53.446 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:03:53.446 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-05-09T08:03:53.446 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:3300", 2024-05-09T08:03:53.446 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:03:53.446 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:03:53.446 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:03:53.446 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-05-09T08:03:53.446 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:6789", 2024-05-09T08:03:53.446 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:03:53.446 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:03:53.447 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:03:53.447 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:03:53.447 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:6789/0", 2024-05-09T08:03:53.447 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.7:6789/0", 2024-05-09T08:03:53.447 INFO:teuthology.orchestra.run.smithi001.stdout: "priority": 0, 2024-05-09T08:03:53.447 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 0, 2024-05-09T08:03:53.447 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_location": "{}" 2024-05-09T08:03:53.447 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:03:53.447 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:03:53.447 INFO:teuthology.orchestra.run.smithi001.stdout: "rank": 2, 2024-05-09T08:03:53.447 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "c", 2024-05-09T08:03:53.447 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-05-09T08:03:53.447 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-05-09T08:03:53.447 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:03:53.448 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-05-09T08:03:53.448 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:3301", 2024-05-09T08:03:53.448 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:03:53.448 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:03:53.448 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:03:53.448 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-05-09T08:03:53.448 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:6790", 2024-05-09T08:03:53.448 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:03:53.448 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:03:53.448 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:03:53.448 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:03:53.448 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:6790/0", 2024-05-09T08:03:53.448 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.7:6790/0", 2024-05-09T08:03:53.448 INFO:teuthology.orchestra.run.smithi001.stdout: "priority": 0, 2024-05-09T08:03:53.449 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 0, 2024-05-09T08:03:53.449 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_location": "{}" 2024-05-09T08:03:53.449 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:03:53.449 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:03:53.449 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:03:53.449 INFO:teuthology.orchestra.run.smithi001.stdout: "feature_map": { 2024-05-09T08:03:53.449 INFO:teuthology.orchestra.run.smithi001.stdout: "mon": [ 2024-05-09T08:03:53.449 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:03:53.449 INFO:teuthology.orchestra.run.smithi001.stdout: "features": "0x3f03cffffffdffff", 2024-05-09T08:03:53.449 INFO:teuthology.orchestra.run.smithi001.stdout: "release": "luminous", 2024-05-09T08:03:53.449 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 1 2024-05-09T08:03:53.449 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:03:53.450 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:03:53.450 INFO:teuthology.orchestra.run.smithi001.stdout: "mds": [ 2024-05-09T08:03:53.450 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:03:53.450 INFO:teuthology.orchestra.run.smithi001.stdout: "features": "0x3f03cffffffdffff", 2024-05-09T08:03:53.450 INFO:teuthology.orchestra.run.smithi001.stdout: "release": "luminous", 2024-05-09T08:03:53.450 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 6 2024-05-09T08:03:53.450 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:03:53.450 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:03:53.450 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": [ 2024-05-09T08:03:53.450 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:03:53.450 INFO:teuthology.orchestra.run.smithi001.stdout: "features": "0x3f03cffffffdffff", 2024-05-09T08:03:53.450 INFO:teuthology.orchestra.run.smithi001.stdout: "release": "luminous", 2024-05-09T08:03:53.450 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 6 2024-05-09T08:03:53.451 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:03:53.451 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:03:53.451 INFO:teuthology.orchestra.run.smithi001.stdout: "client": [ 2024-05-09T08:03:53.451 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:03:53.451 INFO:teuthology.orchestra.run.smithi001.stdout: "features": "0x2f018fb87aa4aafe", 2024-05-09T08:03:53.451 INFO:teuthology.orchestra.run.smithi001.stdout: "release": "luminous", 2024-05-09T08:03:53.451 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 1 2024-05-09T08:03:53.451 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:03:53.451 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:03:53.451 INFO:teuthology.orchestra.run.smithi001.stdout: "features": "0x3f03cffffffdffff", 2024-05-09T08:03:53.451 INFO:teuthology.orchestra.run.smithi001.stdout: "release": "luminous", 2024-05-09T08:03:53.451 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 5 2024-05-09T08:03:53.451 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:03:53.452 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:03:53.452 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr": [ 2024-05-09T08:03:53.452 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:03:53.452 INFO:teuthology.orchestra.run.smithi001.stdout: "features": "0x3f03cffffffdffff", 2024-05-09T08:03:53.452 INFO:teuthology.orchestra.run.smithi001.stdout: "release": "luminous", 2024-05-09T08:03:53.452 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 2 2024-05-09T08:03:53.452 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:03:53.452 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:03:53.452 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:03:53.452 INFO:teuthology.orchestra.run.smithi001.stdout: "stretch_mode": false 2024-05-09T08:03:53.452 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-05-09T08:03:53.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.422+0000 7f1942000640 1 -- 172.21.15.1:0/1925060871 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f191c06b910 msgr2=0x7f191c06ddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:53.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.422+0000 7f1942000640 1 --2- 172.21.15.1:0/1925060871 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f191c06b910 0x7f191c06ddd0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f1930001960 tx=0x7f1930009000 comp rx=0 tx=0).stop 2024-05-09T08:03:53.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.422+0000 7f1942000640 1 -- 172.21.15.1:0/1925060871 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f193c0ce4e0 msgr2=0x7f193c14b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:53.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.422+0000 7f1942000640 1 --2- 172.21.15.1:0/1925060871 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f193c0ce4e0 0x7f193c14b5c0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f193400ec40 tx=0x7f193400c490 comp rx=0 tx=0).stop 2024-05-09T08:03:53.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.422+0000 7f1941600640 1 -- 172.21.15.1:0/1925060871 reap_dead start 2024-05-09T08:03:53.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.422+0000 7f1941600640 1 -- 172.21.15.1:0/1925060871 reap_dead start 2024-05-09T08:03:53.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.422+0000 7f1942000640 1 -- 172.21.15.1:0/1925060871 shutdown_connections 2024-05-09T08:03:53.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.422+0000 7f1942000640 1 -- 172.21.15.1:0/1925060871 >> 172.21.15.1:0/1925060871 conn(0x7f193c0cfc20 msgr2=0x7f193c0cda00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:03:53.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.422+0000 7f1942000640 1 -- 172.21.15.1:0/1925060871 shutdown_connections 2024-05-09T08:03:53.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.422+0000 7f1942000640 1 -- 172.21.15.1:0/1925060871 wait complete. 2024-05-09T08:03:53.455 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-05-09T08:03:53.455 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-05-09T08:03:53.455 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-05-09T08:03:53.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.627+0000 7f1c89600640 1 Processor -- start 2024-05-09T08:03:53.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.627+0000 7f1c89600640 1 -- start start 2024-05-09T08:03:53.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.629+0000 7f1c89600640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f1c8415ba50 0x7f1c8406d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:53.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.629+0000 7f1c89600640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f1c8415a5e0 0x7f1c840ca5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:53.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.629+0000 7f1c89600640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1c8415b550 con 0x7f1c840cd040 2024-05-09T08:03:53.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.629+0000 7f1c89600640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f1c8406cec0 con 0x7f1c8415a5e0 2024-05-09T08:03:53.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.629+0000 7f1c89600640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f1c8406c8b0 con 0x7f1c8415ba50 2024-05-09T08:03:53.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.629+0000 7f1c88c00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f1c8415a5e0 0x7f1c840ca5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:53.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.629+0000 7f1c83e00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f1c840cd040 0x7f1c8406da60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.1:42240/0 (socket says 172.21.15.1:42240) 2024-05-09T08:03:53.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.629+0000 7f1c88c00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f1c8415a5e0 0x7f1c840ca5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.1:52612/0 (socket says 172.21.15.1:52612) 2024-05-09T08:03:53.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.629+0000 7f1c83e00640 1 -- 172.21.15.1:0/583220108 learned_addr learned my addr 172.21.15.1:0/583220108 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T08:03:53.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.629+0000 7f1c83400640 1 --2- 172.21.15.1:0/583220108 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f1c8415ba50 0x7f1c8406d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:53.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.630+0000 7f1c88c00640 1 -- 172.21.15.1:0/583220108 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f1c8415ba50 msgr2=0x7f1c8406d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:03:53.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.630+0000 7f1c88c00640 1 --2- 172.21.15.1:0/583220108 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f1c8415ba50 0x7f1c8406d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:53.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.630+0000 7f1c88c00640 1 -- 172.21.15.1:0/583220108 >> v1:172.21.15.7:6789/0 conn(0x7f1c840cd040 legacy=0x7f1c8406da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:53.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.630+0000 7f1c88c00640 1 -- 172.21.15.1:0/583220108 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c8415af90 con 0x7f1c8415a5e0 2024-05-09T08:03:53.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.630+0000 7f1c82a00640 1 -- 172.21.15.1:0/583220108 <== mon.1 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 350517255 0 0) 0x7f1c8415b550 con 0x7f1c840cd040 2024-05-09T08:03:53.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.631+0000 7f1c88c00640 1 --2- 172.21.15.1:0/583220108 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c8415a5e0 0x7f1c840ca5f0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f1c7c009d90 tx=0x7f1c7c02a530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f0c86118a39f63b3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:03:53.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.632+0000 7f1c82a00640 1 -- 172.21.15.1:0/583220108 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c7c007740 con 0x7f1c8415a5e0 2024-05-09T08:03:53.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.632+0000 7f1c82a00640 1 -- 172.21.15.1:0/583220108 <== mon.1 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c7c0078e0 con 0x7f1c8415a5e0 2024-05-09T08:03:53.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.632+0000 7f1c82a00640 1 -- 172.21.15.1:0/583220108 <== mon.1 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c7c007cb0 con 0x7f1c8415a5e0 2024-05-09T08:03:53.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.632+0000 7f1c89600640 1 -- 172.21.15.1:0/583220108 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c8415a5e0 msgr2=0x7f1c840ca5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:53.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.632+0000 7f1c89600640 1 --2- 172.21.15.1:0/583220108 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c8415a5e0 0x7f1c840ca5f0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f1c7c009d90 tx=0x7f1c7c02a530 comp rx=0 tx=0).stop 2024-05-09T08:03:53.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.632+0000 7f1c89600640 1 -- 172.21.15.1:0/583220108 shutdown_connections 2024-05-09T08:03:53.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.632+0000 7f1c89600640 1 --2- 172.21.15.1:0/583220108 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c8415a5e0 0x7f1c840ca5f0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:53.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.632+0000 7f1c89600640 1 --2- 172.21.15.1:0/583220108 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f1c8415ba50 0x7f1c8406d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:53.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.632+0000 7f1c89600640 1 -- 172.21.15.1:0/583220108 >> 172.21.15.1:0/583220108 conn(0x7f1c840b1780 msgr2=0x7f1c840b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:03:53.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.633+0000 7f1c89600640 1 -- 172.21.15.1:0/583220108 shutdown_connections 2024-05-09T08:03:53.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.633+0000 7f1c89600640 1 -- 172.21.15.1:0/583220108 wait complete. 2024-05-09T08:03:53.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.634+0000 7f1c89600640 1 Processor -- start 2024-05-09T08:03:53.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.635+0000 7f1c89600640 1 -- start start 2024-05-09T08:03:53.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.636+0000 7f1c89600640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f1c840cd040 0x7f1c840b5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:53.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.636+0000 7f1c89600640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1c8415a5e0 0x7f1c840b3e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:53.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.636+0000 7f1c89600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c8415ba50 0x7f1c841f4650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:53.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.636+0000 7f1c89600640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f1c8415aa70 con 0x7f1c8415a5e0 2024-05-09T08:03:53.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.636+0000 7f1c89600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1c8406cb50 con 0x7f1c8415ba50 2024-05-09T08:03:53.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.636+0000 7f1c89600640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f1c8415abf0 con 0x7f1c840cd040 2024-05-09T08:03:53.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.636+0000 7f1c83400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1c8415a5e0 0x7f1c840b3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:53.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.636+0000 7f1c83e00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f1c840cd040 0x7f1c840b5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:53.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.636+0000 7f1c83400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1c8415a5e0 0x7f1c840b3e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:57600/0 (socket says 172.21.15.1:57600) 2024-05-09T08:03:53.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.636+0000 7f1c88c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c8415ba50 0x7f1c841f4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:53.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.636+0000 7f1c83400640 1 -- 172.21.15.1:0/176802644 learned_addr learned my addr 172.21.15.1:0/176802644 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:03:53.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.637+0000 7f1c83400640 1 -- 172.21.15.1:0/176802644 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f1c840cd040 msgr2=0x7f1c840b5210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:53.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.637+0000 7f1c83400640 1 --2- 172.21.15.1:0/176802644 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f1c840cd040 0x7f1c840b5210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:53.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.637+0000 7f1c83400640 1 -- 172.21.15.1:0/176802644 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c8415ba50 msgr2=0x7f1c841f4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:53.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.637+0000 7f1c83400640 1 --2- 172.21.15.1:0/176802644 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c8415ba50 0x7f1c841f4650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:53.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.637+0000 7f1c83400640 1 -- 172.21.15.1:0/176802644 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c840b43b0 con 0x7f1c8415a5e0 2024-05-09T08:03:53.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.637+0000 7f1c88c00640 1 --2- 172.21.15.1:0/176802644 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c8415ba50 0x7f1c841f4650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:03:53.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.637+0000 7f1c83400640 1 --2- 172.21.15.1:0/176802644 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1c8415a5e0 0x7f1c840b3e70 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f1c6c00e870 tx=0x7f1c6c00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:03:53.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.637+0000 7f1c7be00640 1 -- 172.21.15.1:0/176802644 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c6c017070 con 0x7f1c8415a5e0 2024-05-09T08:03:53.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.637+0000 7f1c7be00640 1 -- 172.21.15.1:0/176802644 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c6c00c940 con 0x7f1c8415a5e0 2024-05-09T08:03:53.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.637+0000 7f1c89600640 1 -- 172.21.15.1:0/176802644 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c840b29a0 con 0x7f1c8415a5e0 2024-05-09T08:03:53.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.637+0000 7f1c7be00640 1 -- 172.21.15.1:0/176802644 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c6c00cd30 con 0x7f1c8415a5e0 2024-05-09T08:03:53.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.638+0000 7f1c89600640 1 -- 172.21.15.1:0/176802644 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c840b2f60 con 0x7f1c8415a5e0 2024-05-09T08:03:53.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.639+0000 7f1c89600640 1 -- 172.21.15.1:0/176802644 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c84154d10 con 0x7f1c8415a5e0 2024-05-09T08:03:53.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.640+0000 7f1c7be00640 1 -- 172.21.15.1:0/176802644 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1c6c01a070 con 0x7f1c8415a5e0 2024-05-09T08:03:53.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.641+0000 7f1c7be00640 1 --2- 172.21.15.1:0/176802644 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f1c6006be30 0x7f1c6006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:03:53.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.641+0000 7f1c7be00640 1 -- 172.21.15.1:0/176802644 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7f1c6c08b1a0 con 0x7f1c8415a5e0 2024-05-09T08:03:53.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.641+0000 7f1c83e00640 1 --2- 172.21.15.1:0/176802644 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f1c6006be30 0x7f1c6006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:03:53.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.642+0000 7f1c83e00640 1 --2- 172.21.15.1:0/176802644 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f1c6006be30 0x7f1c6006e2f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f1c74000f60 tx=0x7f1c74007e40 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:03:53.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.646+0000 7f1c7be00640 1 -- 172.21.15.1:0/176802644 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f1c84154d10 con 0x7f1c8415a5e0 2024-05-09T08:03:53.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.938+0000 7f1c89600640 1 -- 172.21.15.1:0/176802644 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "mon scrub"} v 0) -- 0x7f1c8404f660 con 0x7f1c8415a5e0 2024-05-09T08:03:53.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.940+0000 7f1c7be00640 1 -- 172.21.15.1:0/176802644 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) ==== 57+0+0 (secure 0 0 0) 0x7f1c6c058fe0 con 0x7f1c8415a5e0 2024-05-09T08:03:53.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.943+0000 7f1c89600640 1 -- 172.21.15.1:0/176802644 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f1c6006be30 msgr2=0x7f1c6006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:53.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.943+0000 7f1c89600640 1 --2- 172.21.15.1:0/176802644 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f1c6006be30 0x7f1c6006e2f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f1c74000f60 tx=0x7f1c74007e40 comp rx=0 tx=0).stop 2024-05-09T08:03:53.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.943+0000 7f1c89600640 1 -- 172.21.15.1:0/176802644 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1c8415a5e0 msgr2=0x7f1c840b3e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:03:53.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.943+0000 7f1c89600640 1 --2- 172.21.15.1:0/176802644 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1c8415a5e0 0x7f1c840b3e70 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f1c6c00e870 tx=0x7f1c6c00ed30 comp rx=0 tx=0).stop 2024-05-09T08:03:53.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.943+0000 7f1c89600640 1 -- 172.21.15.1:0/176802644 shutdown_connections 2024-05-09T08:03:53.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.943+0000 7f1c89600640 1 --2- 172.21.15.1:0/176802644 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f1c6006be30 0x7f1c6006e2f0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:53.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.943+0000 7f1c89600640 1 --2- 172.21.15.1:0/176802644 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c8415ba50 0x7f1c841f4650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:53.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.943+0000 7f1c89600640 1 --2- 172.21.15.1:0/176802644 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1c8415a5e0 0x7f1c840b3e70 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:53.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.943+0000 7f1c89600640 1 --2- 172.21.15.1:0/176802644 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f1c840cd040 0x7f1c840b5210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:03:53.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.943+0000 7f1c89600640 1 -- 172.21.15.1:0/176802644 >> 172.21.15.1:0/176802644 conn(0x7f1c840b1780 msgr2=0x7f1c840ce400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:03:53.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.943+0000 7f1c89600640 1 -- 172.21.15.1:0/176802644 shutdown_connections 2024-05-09T08:03:53.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:03:53.943+0000 7f1c89600640 1 -- 172.21.15.1:0/176802644 wait complete. 2024-05-09T08:03:53.982 INFO:tasks.mon_thrash.mon_thrasher:waiting for 10.0 secs before continuing thrashing 2024-05-09T08:04:03.983 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-05-09T08:04:03.983 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-05-09T08:04:04.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:04.181+0000 7fe497e00640 1 Processor -- start 2024-05-09T08:04:04.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:04.181+0000 7fe497e00640 1 -- start start 2024-05-09T08:04:04.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:04.183+0000 7fe497e00640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fe498158cf0 0x7fe498162850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:04:04.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:04.183+0000 7fe497e00640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe49806bfd0 con 0x7fe49815f610 2024-05-09T08:04:04.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:04.183+0000 7fe497e00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe498162d90 con 0x7fe49815f1f0 2024-05-09T08:04:04.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:04.183+0000 7fe497e00640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7fe49815ed60 con 0x7fe498158cf0 2024-05-09T08:04:04.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:04.184+0000 7fe497400640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7fe49815f610 0x7fe4981691d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:44500/0 (socket says 172.21.15.1:44500) 2024-05-09T08:04:04.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:04.184+0000 7fe497400640 1 -- 172.21.15.1:0/3052497585 learned_addr learned my addr 172.21.15.1:0/3052497585 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T08:04:04.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:04.184+0000 7fe496a00640 1 --2- 172.21.15.1:0/3052497585 >> v2:172.21.15.1:3300/0 conn(0x7fe498158cf0 0x7fe498162850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:04:04.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:04.184+0000 7fe495600640 1 -- 172.21.15.1:0/3052497585 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 705485436 0 0) 0x7fe49806bfd0 con 0x7fe49815f610 2024-05-09T08:04:04.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:04.184+0000 7fe495600640 1 -- 172.21.15.1:0/3052497585 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe47c003520 con 0x7fe49815f610 2024-05-09T08:04:04.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:04.184+0000 7fe495600640 1 -- 172.21.15.1:0/3052497585 <== mon.1 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2723755838 0 0) 0x7fe498162d90 con 0x7fe49815f1f0 2024-05-09T08:04:04.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:04.184+0000 7fe495600640 1 -- 172.21.15.1:0/3052497585 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe49806bfd0 con 0x7fe49815f1f0 2024-05-09T08:04:04.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:04.184+0000 7fe496a00640 1 -- 172.21.15.1:0/3052497585 >> v1:172.21.15.7:6789/0 conn(0x7fe49815f1f0 legacy=0x7fe49806b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:04:04.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:04.184+0000 7fe496a00640 1 -- 172.21.15.1:0/3052497585 >> v1:172.21.15.1:6789/0 conn(0x7fe49815f610 legacy=0x7fe4981691d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:04:04.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:04.184+0000 7fe496a00640 1 -- 172.21.15.1:0/3052497585 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4980c76b0 con 0x7fe498158cf0 2024-05-09T08:04:04.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:04.186+0000 7fe496a00640 1 --2- 172.21.15.1:0/3052497585 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe498158cf0 0x7fe498162850 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fe49815eee0 tx=0x7fe48802a2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=309d5522100bb6fe server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:04:04.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:04.186+0000 7fe495600640 1 -- 172.21.15.1:0/3052497585 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe488037070 con 0x7fe498158cf0 2024-05-09T08:04:04.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:04.186+0000 7fe495600640 1 -- 172.21.15.1:0/3052497585 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe488032640 con 0x7fe498158cf0 2024-05-09T08:04:04.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:04.186+0000 7fe495600640 1 -- 172.21.15.1:0/3052497585 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe488032a10 con 0x7fe498158cf0 2024-05-09T08:04:04.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:04.187+0000 7fe497e00640 1 -- 172.21.15.1:0/3052497585 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe498158cf0 msgr2=0x7fe498162850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:04:04.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:04.187+0000 7fe497e00640 1 --2- 172.21.15.1:0/3052497585 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe498158cf0 0x7fe498162850 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fe49815eee0 tx=0x7fe48802a2c0 comp rx=0 tx=0).stop 2024-05-09T08:04:04.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:04.187+0000 7fe497e00640 1 -- 172.21.15.1:0/3052497585 shutdown_connections 2024-05-09T08:04:04.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:04.187+0000 7fe497e00640 1 --2- 172.21.15.1:0/3052497585 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe498158cf0 0x7fe498162850 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:04:04.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:04.187+0000 7fe497e00640 1 -- 172.21.15.1:0/3052497585 >> 172.21.15.1:0/3052497585 conn(0x7fe4980cbc10 msgr2=0x7fe4980cc020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:04:04.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:04.187+0000 7fe497e00640 1 -- 172.21.15.1:0/3052497585 shutdown_connections 2024-05-09T08:04:04.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:04.187+0000 7fe497e00640 1 -- 172.21.15.1:0/3052497585 wait complete. 2024-05-09T08:04:04.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:04.190+0000 7fe497e00640 1 Processor -- start 2024-05-09T08:04:04.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:04.190+0000 7fe497e00640 1 -- start start 2024-05-09T08:04:04.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:04.191+0000 7fe497e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe498158cf0 0x7fe4980c9a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:04:04.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:04.191+0000 7fe497e00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fe49815f1f0 0x7fe498155830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:04:04.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:04.191+0000 7fe497e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe49815f610 0x7fe49815d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:04:04.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:04.191+0000 7fe497e00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fe4980c7c50 con 0x7fe498158cf0 2024-05-09T08:04:04.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:04.191+0000 7fe497e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe4980c8210 con 0x7fe49815f610 2024-05-09T08:04:04.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:04.191+0000 7fe497e00640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7fe4980c7f30 con 0x7fe49815f1f0 2024-05-09T08:04:04.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:04.192+0000 7fe496000640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fe49815f1f0 0x7fe498155830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:04:04.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:04.192+0000 7fe497400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe49815f610 0x7fe49815d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:04:04.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:04.192+0000 7fe496000640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fe49815f1f0 0x7fe498155830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3301/0 says I am v2:172.21.15.1:57718/0 (socket says 172.21.15.1:57718) 2024-05-09T08:04:04.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:04.192+0000 7fe496000640 1 -- 172.21.15.1:0/2663085500 learned_addr learned my addr 172.21.15.1:0/2663085500 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:04:04.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:04.192+0000 7fe496a00640 1 --2- 172.21.15.1:0/2663085500 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe498158cf0 0x7fe4980c9a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:04:04.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:04.192+0000 7fe497400640 1 -- 172.21.15.1:0/2663085500 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fe49815f1f0 msgr2=0x7fe498155830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:04:04.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:04.192+0000 7fe497400640 1 --2- 172.21.15.1:0/2663085500 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fe49815f1f0 0x7fe498155830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:04:04.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:04.192+0000 7fe497400640 1 -- 172.21.15.1:0/2663085500 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe498158cf0 msgr2=0x7fe4980c9a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:04:04.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:04.193+0000 7fe497400640 1 --2- 172.21.15.1:0/2663085500 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe498158cf0 0x7fe4980c9a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:04:04.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:04.193+0000 7fe497400640 1 -- 172.21.15.1:0/2663085500 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe498159ba0 con 0x7fe49815f610 2024-05-09T08:04:04.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:04.193+0000 7fe496000640 1 --2- 172.21.15.1:0/2663085500 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fe49815f1f0 0x7fe498155830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:04:04.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:04.193+0000 7fe496a00640 1 --2- 172.21.15.1:0/2663085500 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe498158cf0 0x7fe4980c9a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T08:04:04.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:04.193+0000 7fe497400640 1 --2- 172.21.15.1:0/2663085500 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe49815f610 0x7fe49815d000 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fe49000ca70 tx=0x7fe49000cf30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:04:04.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:04.193+0000 7fe48ea00640 1 -- 172.21.15.1:0/2663085500 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe490017070 con 0x7fe49815f610 2024-05-09T08:04:04.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:04.193+0000 7fe48ea00640 1 -- 172.21.15.1:0/2663085500 <== mon.1 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe49000ab70 con 0x7fe49815f610 2024-05-09T08:04:04.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:04.193+0000 7fe48ea00640 1 -- 172.21.15.1:0/2663085500 <== mon.1 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe490012400 con 0x7fe49815f610 2024-05-09T08:04:04.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:04.193+0000 7fe497e00640 1 -- 172.21.15.1:0/2663085500 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4980ca4a0 con 0x7fe49815f610 2024-05-09T08:04:04.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:04.193+0000 7fe497e00640 1 -- 172.21.15.1:0/2663085500 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4980caa60 con 0x7fe49815f610 2024-05-09T08:04:04.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:04.195+0000 7fe497e00640 1 -- 172.21.15.1:0/2663085500 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe49804f660 con 0x7fe49815f610 2024-05-09T08:04:04.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:04.197+0000 7fe48ea00640 1 -- 172.21.15.1:0/2663085500 <== mon.1 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe49001a070 con 0x7fe49815f610 2024-05-09T08:04:04.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:04.198+0000 7fe48ea00640 1 --2- 172.21.15.1:0/2663085500 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fe47806be30 0x7fe47806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:04:04.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:04.199+0000 7fe48ea00640 1 -- 172.21.15.1:0/2663085500 <== mon.1 v2:172.21.15.7:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7fe49008ae20 con 0x7fe49815f610 2024-05-09T08:04:04.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:04.199+0000 7fe496a00640 1 --2- 172.21.15.1:0/2663085500 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fe47806be30 0x7fe47806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:04:04.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:04.200+0000 7fe496a00640 1 --2- 172.21.15.1:0/2663085500 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fe47806be30 0x7fe47806e2f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fe488034000 tx=0x7fe48802e000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:04:04.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:04.203+0000 7fe48ea00640 1 -- 172.21.15.1:0/2663085500 <== mon.1 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fe498159ba0 con 0x7fe49815f610 2024-05-09T08:04:04.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:04.579+0000 7fe497e00640 1 -- 172.21.15.1:0/2663085500 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fe49815c270 con 0x7fe49815f610 2024-05-09T08:04:04.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:04.580+0000 7fe48ea00640 1 -- 172.21.15.1:0/2663085500 <== mon.1 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1545 (secure 0 0 0) 0x7fe490058cf0 con 0x7fe49815f610 2024-05-09T08:04:04.596 INFO:teuthology.orchestra.run.smithi001.stdout:{"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":12,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"54329ec8-3270-4043-9a82-c6cd7420aa7f","modified":"2024-05-09T08:01:21.819270Z","created":"2024-05-09T08:01:21.819270Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:3300","nonce":0},{"type":"v1","addr":"172.21.15.1:6789","nonce":0}]},"addr":"172.21.15.1:6789/0","public_addr":"172.21.15.1:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:3300","nonce":0},{"type":"v1","addr":"172.21.15.7:6789","nonce":0}]},"addr":"172.21.15.7:6789/0","public_addr":"172.21.15.7:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:3301","nonce":0},{"type":"v1","addr":"172.21.15.7:6790","nonce":0}]},"addr":"172.21.15.7:6790/0","public_addr":"172.21.15.7:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-05-09T08:04:04.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:04.583+0000 7fe497e00640 1 -- 172.21.15.1:0/2663085500 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fe47806be30 msgr2=0x7fe47806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:04:04.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:04.583+0000 7fe497e00640 1 --2- 172.21.15.1:0/2663085500 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fe47806be30 0x7fe47806e2f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fe488034000 tx=0x7fe48802e000 comp rx=0 tx=0).stop 2024-05-09T08:04:04.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:04.583+0000 7fe497e00640 1 -- 172.21.15.1:0/2663085500 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe49815f610 msgr2=0x7fe49815d000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:04:04.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:04.583+0000 7fe497e00640 1 --2- 172.21.15.1:0/2663085500 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe49815f610 0x7fe49815d000 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fe49000ca70 tx=0x7fe49000cf30 comp rx=0 tx=0).stop 2024-05-09T08:04:04.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:04.583+0000 7fe497e00640 1 -- 172.21.15.1:0/2663085500 shutdown_connections 2024-05-09T08:04:04.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:04.583+0000 7fe497e00640 1 --2- 172.21.15.1:0/2663085500 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fe47806be30 0x7fe47806e2f0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:04:04.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:04.583+0000 7fe497e00640 1 --2- 172.21.15.1:0/2663085500 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe49815f610 0x7fe49815d000 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:04:04.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:04.583+0000 7fe497e00640 1 --2- 172.21.15.1:0/2663085500 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fe49815f1f0 0x7fe498155830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:04:04.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:04.583+0000 7fe497e00640 1 --2- 172.21.15.1:0/2663085500 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe498158cf0 0x7fe4980c9a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:04:04.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:04.583+0000 7fe497e00640 1 -- 172.21.15.1:0/2663085500 >> 172.21.15.1:0/2663085500 conn(0x7fe4980cbc10 msgr2=0x7fe498072910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:04:04.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:04.584+0000 7fe497e00640 1 -- 172.21.15.1:0/2663085500 shutdown_connections 2024-05-09T08:04:04.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:04.584+0000 7fe497e00640 1 -- 172.21.15.1:0/2663085500 wait complete. 2024-05-09T08:04:04.624 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-05-09T08:04:04.625 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-05-09T08:04:04.625 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-05-09T08:04:04.625 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-05-09T08:04:04.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:04.822+0000 7f64da800640 1 Processor -- start 2024-05-09T08:04:04.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:04.823+0000 7f64da800640 1 -- start start 2024-05-09T08:04:04.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:04.824+0000 7f64da800640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f64d40c54f0 0x7f64d406f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:04:04.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:04.824+0000 7f64da800640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f64d40c58d0 0x7f64d41589e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:04:04.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:04.824+0000 7f64da800640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f64d40718e0 con 0x7f64d40c6470 2024-05-09T08:04:04.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:04.824+0000 7f64da800640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f64d40726e0 con 0x7f64d40c58d0 2024-05-09T08:04:04.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:04.824+0000 7f64da800640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f64d4071b60 con 0x7f64d40c54f0 2024-05-09T08:04:04.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:04.824+0000 7f64d9400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f64d40c54f0 0x7f64d406f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:04:04.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:04.825+0000 7f64d9400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f64d40c54f0 0x7f64d406f930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.1:33776/0 (socket says 172.21.15.1:33776) 2024-05-09T08:04:04.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:04.825+0000 7f64d9400640 1 -- 172.21.15.1:0/3820429882 learned_addr learned my addr 172.21.15.1:0/3820429882 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:04:04.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:04.825+0000 7f64d8a00640 1 --2- 172.21.15.1:0/3820429882 >> v2:172.21.15.1:3300/0 conn(0x7f64d40c58d0 0x7f64d41589e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:04:04.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:04.825+0000 7f64d9400640 1 -- 172.21.15.1:0/3820429882 >> v2:172.21.15.1:3300/0 conn(0x7f64d40c58d0 msgr2=0x7f64d41589e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:04:04.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:04.825+0000 7f64d9400640 1 --2- 172.21.15.1:0/3820429882 >> v2:172.21.15.1:3300/0 conn(0x7f64d40c58d0 0x7f64d41589e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:04:04.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:04.825+0000 7f64d9400640 1 -- 172.21.15.1:0/3820429882 >> v1:172.21.15.1:6789/0 conn(0x7f64d40c6470 legacy=0x7f64d4151d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:04:04.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:04.825+0000 7f64d9400640 1 -- 172.21.15.1:0/3820429882 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64d4071e40 con 0x7f64d40c54f0 2024-05-09T08:04:04.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:04.825+0000 7f64d8a00640 1 --2- 172.21.15.1:0/3820429882 >> v2:172.21.15.1:3300/0 conn(0x7f64d40c58d0 0x7f64d41589e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:04:04.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:04.826+0000 7f64d9400640 1 --2- 172.21.15.1:0/3820429882 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f64d40c54f0 0x7f64d406f930 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f64d40c96b0 tx=0x7f64c802a280 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c06e3aa107e0ab6a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:04:04.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:04.827+0000 7f64d3e00640 1 -- 172.21.15.1:0/3820429882 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64c8037070 con 0x7f64d40c54f0 2024-05-09T08:04:04.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:04.827+0000 7f64d3e00640 1 -- 172.21.15.1:0/3820429882 <== mon.1 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64c8032440 con 0x7f64d40c54f0 2024-05-09T08:04:04.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:04.827+0000 7f64d3e00640 1 -- 172.21.15.1:0/3820429882 <== mon.1 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64c8032810 con 0x7f64d40c54f0 2024-05-09T08:04:04.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:04.827+0000 7f64da800640 1 -- 172.21.15.1:0/3820429882 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f64d40c54f0 msgr2=0x7f64d406f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:04:04.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:04.827+0000 7f64da800640 1 --2- 172.21.15.1:0/3820429882 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f64d40c54f0 0x7f64d406f930 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f64d40c96b0 tx=0x7f64c802a280 comp rx=0 tx=0).stop 2024-05-09T08:04:04.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:04.827+0000 7f64da800640 1 -- 172.21.15.1:0/3820429882 shutdown_connections 2024-05-09T08:04:04.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:04.827+0000 7f64da800640 1 --2- 172.21.15.1:0/3820429882 >> v2:172.21.15.1:3300/0 conn(0x7f64d40c58d0 0x7f64d41589e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:04:04.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:04.827+0000 7f64da800640 1 --2- 172.21.15.1:0/3820429882 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f64d40c54f0 0x7f64d406f930 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:04:04.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:04.827+0000 7f64da800640 1 -- 172.21.15.1:0/3820429882 >> 172.21.15.1:0/3820429882 conn(0x7f64d406b250 msgr2=0x7f64d406b630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:04:04.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:04.828+0000 7f64da800640 1 -- 172.21.15.1:0/3820429882 shutdown_connections 2024-05-09T08:04:04.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:04.828+0000 7f64da800640 1 -- 172.21.15.1:0/3820429882 wait complete. 2024-05-09T08:04:04.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:04.830+0000 7f64da800640 1 Processor -- start 2024-05-09T08:04:04.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:04.830+0000 7f64da800640 1 -- start start 2024-05-09T08:04:04.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:04.832+0000 7f64da800640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f64d40c54f0 0x7f64d41eed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:04:04.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:04.832+0000 7f64da800640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f64d40c58d0 0x7f64d41f20d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:04:04.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:04.832+0000 7f64da800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f64d40c6470 0x7f64d40caf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:04:04.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:04.832+0000 7f64da800640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f64d41584c0 con 0x7f64d40c58d0 2024-05-09T08:04:04.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:04.832+0000 7f64da800640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f64d4072400 con 0x7f64d40c6470 2024-05-09T08:04:04.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:04.832+0000 7f64da800640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f64d406fe70 con 0x7f64d40c54f0 2024-05-09T08:04:04.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:04.832+0000 7f64d9400640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f64d40c54f0 0x7f64d41eed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:04:04.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:04.832+0000 7f64d9e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f64d40c6470 0x7f64d40caf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:04:04.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:04.832+0000 7f64d9400640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f64d40c54f0 0x7f64d41eed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3301/0 says I am v2:172.21.15.1:57726/0 (socket says 172.21.15.1:57726) 2024-05-09T08:04:04.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:04.832+0000 7f64d9400640 1 -- 172.21.15.1:0/3388864361 learned_addr learned my addr 172.21.15.1:0/3388864361 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:04:04.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:04.832+0000 7f64d8a00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f64d40c58d0 0x7f64d41f20d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:04:04.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:04.833+0000 7f64d9e00640 1 -- 172.21.15.1:0/3388864361 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f64d40c54f0 msgr2=0x7f64d41eed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:04:04.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:04.833+0000 7f64d9e00640 1 --2- 172.21.15.1:0/3388864361 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f64d40c54f0 0x7f64d41eed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:04:04.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:04.833+0000 7f64d9e00640 1 -- 172.21.15.1:0/3388864361 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f64d40c58d0 msgr2=0x7f64d41f20d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:04:04.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:04.833+0000 7f64d9e00640 1 --2- 172.21.15.1:0/3388864361 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f64d40c58d0 0x7f64d41f20d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:04:04.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:04.833+0000 7f64d9e00640 1 -- 172.21.15.1:0/3388864361 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64d41582a0 con 0x7f64d40c6470 2024-05-09T08:04:04.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:04.833+0000 7f64d9400640 1 --2- 172.21.15.1:0/3388864361 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f64d40c54f0 0x7f64d41eed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T08:04:04.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:04.833+0000 7f64d9e00640 1 --2- 172.21.15.1:0/3388864361 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f64d40c6470 0x7f64d40caf50 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f64cc001d40 tx=0x7f64cc00ee70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:04:04.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:04.834+0000 7f64d1600640 1 -- 172.21.15.1:0/3388864361 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64cc003970 con 0x7f64d40c6470 2024-05-09T08:04:04.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:04.834+0000 7f64d1600640 1 -- 172.21.15.1:0/3388864361 <== mon.1 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64cc003b10 con 0x7f64d40c6470 2024-05-09T08:04:04.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:04.834+0000 7f64d1600640 1 -- 172.21.15.1:0/3388864361 <== mon.1 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64cc0050d0 con 0x7f64d40c6470 2024-05-09T08:04:04.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:04.834+0000 7f64da800640 1 -- 172.21.15.1:0/3388864361 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64d4157fc0 con 0x7f64d40c6470 2024-05-09T08:04:04.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:04.834+0000 7f64da800640 1 -- 172.21.15.1:0/3388864361 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64d41ee540 con 0x7f64d40c6470 2024-05-09T08:04:04.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:04.835+0000 7f64da800640 1 --2- 172.21.15.1:0/3388864361 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f64d4157790 0x7f64d40bc550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:04:04.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:04.835+0000 7f64da800640 1 -- 172.21.15.1:0/3388864361 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f64d40cc9f0 con 0x7f64d4157790 2024-05-09T08:04:04.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:04.835+0000 7f64d9400640 1 --2- 172.21.15.1:0/3388864361 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f64d4157790 0x7f64d40bc550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:04:04.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:04.836+0000 7f64d9400640 1 -- 172.21.15.1:0/3388864361 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f64d404f660 con 0x7f64d4157790 2024-05-09T08:04:04.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:04.837+0000 7f64d9400640 1 --2- 172.21.15.1:0/3388864361 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f64d4157790 0x7f64d40bc550 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f64c802a2c0 tx=0x7f64c80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:04:04.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:04.837+0000 7f64d1600640 1 -- 172.21.15.1:0/3388864361 <== mon.1 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f64cc005270 con 0x7f64d40c6470 2024-05-09T08:04:04.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:04.838+0000 7f64d1600640 1 --2- 172.21.15.1:0/3388864361 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f64b406be30 0x7f64b406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:04:04.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:04.838+0000 7f64d1600640 1 -- 172.21.15.1:0/3388864361 <== mon.1 v2:172.21.15.7:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7f64cc08eff0 con 0x7f64d40c6470 2024-05-09T08:04:04.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:04.839+0000 7f64d1600640 1 -- 172.21.15.1:0/3388864361 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64c8042070 con 0x7f64d4157790 2024-05-09T08:04:04.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:04.839+0000 7f64d1600640 1 -- 172.21.15.1:0/3388864361 <== mon.0 v2:172.21.15.1:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f64c803e460 con 0x7f64d4157790 2024-05-09T08:04:04.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:04.839+0000 7f64d1600640 1 -- 172.21.15.1:0/3388864361 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f64d4157790 msgr2=0x7f64d40bc550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:04:04.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:04.839+0000 7f64d8a00640 1 --2- 172.21.15.1:0/3388864361 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f64b406be30 0x7f64b406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:04:04.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:04.839+0000 7f64d1600640 1 --2- 172.21.15.1:0/3388864361 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f64d4157790 0x7f64d40bc550 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f64c802a2c0 tx=0x7f64c80046c0 comp rx=0 tx=0).stop 2024-05-09T08:04:04.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:04.839+0000 7f64d1600640 1 -- 172.21.15.1:0/3388864361 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f64d4157790 msgr2=0x7f64d40bc550 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-05-09T08:04:04.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:04.839+0000 7f64d1600640 1 --2- 172.21.15.1:0/3388864361 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f64d4157790 0x7f64d40bc550 secure :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f64c802a2c0 tx=0x7f64c80046c0 comp rx=0 tx=0).stop 2024-05-09T08:04:04.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:04.839+0000 7f64d8a00640 1 --2- 172.21.15.1:0/3388864361 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f64b406be30 0x7f64b406e2f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f64d414f4f0 tx=0x7f64c0008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:04:04.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:04.852+0000 7f64da800640 1 --2- 172.21.15.1:0/3388864361 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f64d40bca90 0x7f64d40cb7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:04:04.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:04.852+0000 7f64da800640 1 -- 172.21.15.1:0/3388864361 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f64d40bf850 con 0x7f64d40bca90 2024-05-09T08:04:04.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:04.853+0000 7f64d9400640 1 --2- 172.21.15.1:0/3388864361 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f64d40bca90 0x7f64d40cb7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:04:04.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:04.853+0000 7f64d9400640 1 -- 172.21.15.1:0/3388864361 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f64d414f520 con 0x7f64d40bca90 2024-05-09T08:04:04.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:04.854+0000 7f64d9400640 1 --2- 172.21.15.1:0/3388864361 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f64d40bca90 0x7f64d40cb7e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f64c80090b0 tx=0x7f64c8009240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:04:04.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:04.854+0000 7f64d1600640 1 -- 172.21.15.1:0/3388864361 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64c8037040 con 0x7f64d40bca90 2024-05-09T08:04:04.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:04.854+0000 7f64d1600640 1 -- 172.21.15.1:0/3388864361 <== mon.0 v2:172.21.15.1:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4708 (secure 0 0 0) 0x7f64c8003e70 con 0x7f64d40bca90 2024-05-09T08:04:04.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:04.854+0000 7f64d1600640 1 -- 172.21.15.1:0/3388864361 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f64d40bca90 msgr2=0x7f64d40cb7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:04:04.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:04.854+0000 7f64d1600640 1 --2- 172.21.15.1:0/3388864361 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f64d40bca90 0x7f64d40cb7e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f64c80090b0 tx=0x7f64c8009240 comp rx=0 tx=0).stop 2024-05-09T08:04:04.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:04.854+0000 7f64d1600640 1 -- 172.21.15.1:0/3388864361 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f64d40bca90 msgr2=0x7f64d40cb7e0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-05-09T08:04:04.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:04.854+0000 7f64d1600640 1 --2- 172.21.15.1:0/3388864361 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f64d40bca90 0x7f64d40cb7e0 secure :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f64c80090b0 tx=0x7f64c8009240 comp rx=0 tx=0).stop 2024-05-09T08:04:04.871 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-05-09T08:04:04.871 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "a", 2024-05-09T08:04:04.871 INFO:teuthology.orchestra.run.smithi001.stdout: "rank": 0, 2024-05-09T08:04:04.871 INFO:teuthology.orchestra.run.smithi001.stdout: "state": "leader", 2024-05-09T08:04:04.871 INFO:teuthology.orchestra.run.smithi001.stdout: "election_epoch": 10, 2024-05-09T08:04:04.872 INFO:teuthology.orchestra.run.smithi001.stdout: "uptime": 128217, 2024-05-09T08:04:04.872 INFO:teuthology.orchestra.run.smithi001.stdout: "quorum": [ 2024-05-09T08:04:04.872 INFO:teuthology.orchestra.run.smithi001.stdout: 0, 2024-05-09T08:04:04.872 INFO:teuthology.orchestra.run.smithi001.stdout: 1, 2024-05-09T08:04:04.872 INFO:teuthology.orchestra.run.smithi001.stdout: 2 2024-05-09T08:04:04.872 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:04:04.872 INFO:teuthology.orchestra.run.smithi001.stdout: "quorum_age": 12, 2024-05-09T08:04:04.872 INFO:teuthology.orchestra.run.smithi001.stdout: "features": { 2024-05-09T08:04:04.872 INFO:teuthology.orchestra.run.smithi001.stdout: "required_con": "2450521708007866372", 2024-05-09T08:04:04.872 INFO:teuthology.orchestra.run.smithi001.stdout: "required_mon": [ 2024-05-09T08:04:04.872 INFO:teuthology.orchestra.run.smithi001.stdout: "kraken", 2024-05-09T08:04:04.872 INFO:teuthology.orchestra.run.smithi001.stdout: "luminous", 2024-05-09T08:04:04.872 INFO:teuthology.orchestra.run.smithi001.stdout: "mimic", 2024-05-09T08:04:04.872 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap-prune", 2024-05-09T08:04:04.873 INFO:teuthology.orchestra.run.smithi001.stdout: "nautilus", 2024-05-09T08:04:04.873 INFO:teuthology.orchestra.run.smithi001.stdout: "octopus", 2024-05-09T08:04:04.873 INFO:teuthology.orchestra.run.smithi001.stdout: "pacific", 2024-05-09T08:04:04.873 INFO:teuthology.orchestra.run.smithi001.stdout: "elector-pinging", 2024-05-09T08:04:04.873 INFO:teuthology.orchestra.run.smithi001.stdout: "quincy", 2024-05-09T08:04:04.873 INFO:teuthology.orchestra.run.smithi001.stdout: "reef", 2024-05-09T08:04:04.873 INFO:teuthology.orchestra.run.smithi001.stdout: "squid" 2024-05-09T08:04:04.873 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:04:04.873 INFO:teuthology.orchestra.run.smithi001.stdout: "quorum_con": "4540701547738038271", 2024-05-09T08:04:04.873 INFO:teuthology.orchestra.run.smithi001.stdout: "quorum_mon": [ 2024-05-09T08:04:04.874 INFO:teuthology.orchestra.run.smithi001.stdout: "kraken", 2024-05-09T08:04:04.874 INFO:teuthology.orchestra.run.smithi001.stdout: "luminous", 2024-05-09T08:04:04.874 INFO:teuthology.orchestra.run.smithi001.stdout: "mimic", 2024-05-09T08:04:04.874 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap-prune", 2024-05-09T08:04:04.874 INFO:teuthology.orchestra.run.smithi001.stdout: "nautilus", 2024-05-09T08:04:04.874 INFO:teuthology.orchestra.run.smithi001.stdout: "octopus", 2024-05-09T08:04:04.874 INFO:teuthology.orchestra.run.smithi001.stdout: "pacific", 2024-05-09T08:04:04.874 INFO:teuthology.orchestra.run.smithi001.stdout: "elector-pinging", 2024-05-09T08:04:04.874 INFO:teuthology.orchestra.run.smithi001.stdout: "quincy", 2024-05-09T08:04:04.874 INFO:teuthology.orchestra.run.smithi001.stdout: "reef", 2024-05-09T08:04:04.874 INFO:teuthology.orchestra.run.smithi001.stdout: "squid" 2024-05-09T08:04:04.874 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:04:04.874 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:04:04.874 INFO:teuthology.orchestra.run.smithi001.stdout: "outside_quorum": [], 2024-05-09T08:04:04.875 INFO:teuthology.orchestra.run.smithi001.stdout: "extra_probe_peers": [], 2024-05-09T08:04:04.875 INFO:teuthology.orchestra.run.smithi001.stdout: "sync_provider": [], 2024-05-09T08:04:04.875 INFO:teuthology.orchestra.run.smithi001.stdout: "monmap": { 2024-05-09T08:04:04.875 INFO:teuthology.orchestra.run.smithi001.stdout: "epoch": 1, 2024-05-09T08:04:04.875 INFO:teuthology.orchestra.run.smithi001.stdout: "fsid": "54329ec8-3270-4043-9a82-c6cd7420aa7f", 2024-05-09T08:04:04.875 INFO:teuthology.orchestra.run.smithi001.stdout: "modified": "2024-05-09T08:01:21.819270Z", 2024-05-09T08:04:04.875 INFO:teuthology.orchestra.run.smithi001.stdout: "created": "2024-05-09T08:01:21.819270Z", 2024-05-09T08:04:04.875 INFO:teuthology.orchestra.run.smithi001.stdout: "min_mon_release": 19, 2024-05-09T08:04:04.875 INFO:teuthology.orchestra.run.smithi001.stdout: "min_mon_release_name": "squid", 2024-05-09T08:04:04.875 INFO:teuthology.orchestra.run.smithi001.stdout: "election_strategy": 1, 2024-05-09T08:04:04.875 INFO:teuthology.orchestra.run.smithi001.stdout: "disallowed_leaders: ": "", 2024-05-09T08:04:04.875 INFO:teuthology.orchestra.run.smithi001.stdout: "stretch_mode": false, 2024-05-09T08:04:04.875 INFO:teuthology.orchestra.run.smithi001.stdout: "tiebreaker_mon": "", 2024-05-09T08:04:04.876 INFO:teuthology.orchestra.run.smithi001.stdout: "removed_ranks: ": "", 2024-05-09T08:04:04.876 INFO:teuthology.orchestra.run.smithi001.stdout: "features": { 2024-05-09T08:04:04.876 INFO:teuthology.orchestra.run.smithi001.stdout: "persistent": [ 2024-05-09T08:04:04.876 INFO:teuthology.orchestra.run.smithi001.stdout: "kraken", 2024-05-09T08:04:04.876 INFO:teuthology.orchestra.run.smithi001.stdout: "luminous", 2024-05-09T08:04:04.876 INFO:teuthology.orchestra.run.smithi001.stdout: "mimic", 2024-05-09T08:04:04.876 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap-prune", 2024-05-09T08:04:04.876 INFO:teuthology.orchestra.run.smithi001.stdout: "nautilus", 2024-05-09T08:04:04.876 INFO:teuthology.orchestra.run.smithi001.stdout: "octopus", 2024-05-09T08:04:04.876 INFO:teuthology.orchestra.run.smithi001.stdout: "pacific", 2024-05-09T08:04:04.876 INFO:teuthology.orchestra.run.smithi001.stdout: "elector-pinging", 2024-05-09T08:04:04.876 INFO:teuthology.orchestra.run.smithi001.stdout: "quincy", 2024-05-09T08:04:04.876 INFO:teuthology.orchestra.run.smithi001.stdout: "reef", 2024-05-09T08:04:04.876 INFO:teuthology.orchestra.run.smithi001.stdout: "squid" 2024-05-09T08:04:04.877 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:04:04.877 INFO:teuthology.orchestra.run.smithi001.stdout: "optional": [] 2024-05-09T08:04:04.877 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:04:04.877 INFO:teuthology.orchestra.run.smithi001.stdout: "mons": [ 2024-05-09T08:04:04.877 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:04:04.877 INFO:teuthology.orchestra.run.smithi001.stdout: "rank": 0, 2024-05-09T08:04:04.877 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "a", 2024-05-09T08:04:04.877 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-05-09T08:04:04.877 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-05-09T08:04:04.878 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:04:04.878 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-05-09T08:04:04.878 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:3300", 2024-05-09T08:04:04.878 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:04:04.878 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:04:04.878 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:04:04.878 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-05-09T08:04:04.878 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6789", 2024-05-09T08:04:04.878 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:04:04.878 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:04:04.878 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:04:04.878 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:04:04.878 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6789/0", 2024-05-09T08:04:04.879 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.1:6789/0", 2024-05-09T08:04:04.879 INFO:teuthology.orchestra.run.smithi001.stdout: "priority": 0, 2024-05-09T08:04:04.879 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 0, 2024-05-09T08:04:04.879 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_location": "{}" 2024-05-09T08:04:04.879 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:04:04.879 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:04:04.879 INFO:teuthology.orchestra.run.smithi001.stdout: "rank": 1, 2024-05-09T08:04:04.879 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "b", 2024-05-09T08:04:04.879 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-05-09T08:04:04.879 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-05-09T08:04:04.879 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:04:04.879 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-05-09T08:04:04.879 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:3300", 2024-05-09T08:04:04.879 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:04:04.879 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:04:04.880 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:04:04.880 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-05-09T08:04:04.880 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:6789", 2024-05-09T08:04:04.880 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:04:04.880 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:04:04.880 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:04:04.880 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:04:04.880 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:6789/0", 2024-05-09T08:04:04.880 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.7:6789/0", 2024-05-09T08:04:04.880 INFO:teuthology.orchestra.run.smithi001.stdout: "priority": 0, 2024-05-09T08:04:04.880 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 0, 2024-05-09T08:04:04.881 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_location": "{}" 2024-05-09T08:04:04.881 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:04:04.881 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:04:04.881 INFO:teuthology.orchestra.run.smithi001.stdout: "rank": 2, 2024-05-09T08:04:04.881 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "c", 2024-05-09T08:04:04.881 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-05-09T08:04:04.881 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-05-09T08:04:04.881 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:04:04.881 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-05-09T08:04:04.881 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:3301", 2024-05-09T08:04:04.881 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:04:04.881 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:04:04.881 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:04:04.881 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-05-09T08:04:04.882 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:6790", 2024-05-09T08:04:04.882 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:04:04.882 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:04:04.882 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:04:04.882 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:04:04.882 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:6790/0", 2024-05-09T08:04:04.882 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.7:6790/0", 2024-05-09T08:04:04.882 INFO:teuthology.orchestra.run.smithi001.stdout: "priority": 0, 2024-05-09T08:04:04.882 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 0, 2024-05-09T08:04:04.882 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_location": "{}" 2024-05-09T08:04:04.882 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:04:04.882 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:04:04.882 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:04:04.883 INFO:teuthology.orchestra.run.smithi001.stdout: "feature_map": { 2024-05-09T08:04:04.883 INFO:teuthology.orchestra.run.smithi001.stdout: "mon": [ 2024-05-09T08:04:04.883 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:04:04.883 INFO:teuthology.orchestra.run.smithi001.stdout: "features": "0x3f03cffffffdffff", 2024-05-09T08:04:04.883 INFO:teuthology.orchestra.run.smithi001.stdout: "release": "luminous", 2024-05-09T08:04:04.883 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 1 2024-05-09T08:04:04.883 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:04:04.883 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:04:04.883 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": [ 2024-05-09T08:04:04.883 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:04:04.883 INFO:teuthology.orchestra.run.smithi001.stdout: "features": "0x3f03cffffffdffff", 2024-05-09T08:04:04.883 INFO:teuthology.orchestra.run.smithi001.stdout: "release": "luminous", 2024-05-09T08:04:04.883 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 2 2024-05-09T08:04:04.883 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:04:04.884 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:04:04.884 INFO:teuthology.orchestra.run.smithi001.stdout: "client": [ 2024-05-09T08:04:04.884 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:04:04.884 INFO:teuthology.orchestra.run.smithi001.stdout: "features": "0x3f03cffffffdffff", 2024-05-09T08:04:04.884 INFO:teuthology.orchestra.run.smithi001.stdout: "release": "luminous", 2024-05-09T08:04:04.884 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 1 2024-05-09T08:04:04.884 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:04:04.884 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:04:04.884 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:04:04.884 INFO:teuthology.orchestra.run.smithi001.stdout: "stretch_mode": false 2024-05-09T08:04:04.884 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-05-09T08:04:04.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:04.855+0000 7f64da800640 1 -- 172.21.15.1:0/3388864361 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f64b406be30 msgr2=0x7f64b406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:04:04.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:04.855+0000 7f64da800640 1 --2- 172.21.15.1:0/3388864361 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f64b406be30 0x7f64b406e2f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f64d414f4f0 tx=0x7f64c0008040 comp rx=0 tx=0).stop 2024-05-09T08:04:04.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:04.855+0000 7f64da800640 1 -- 172.21.15.1:0/3388864361 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f64d40c6470 msgr2=0x7f64d40caf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:04:04.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:04.855+0000 7f64da800640 1 --2- 172.21.15.1:0/3388864361 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f64d40c6470 0x7f64d40caf50 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f64cc001d40 tx=0x7f64cc00ee70 comp rx=0 tx=0).stop 2024-05-09T08:04:04.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:04.855+0000 7f64d9e00640 1 -- 172.21.15.1:0/3388864361 reap_dead start 2024-05-09T08:04:04.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:04.855+0000 7f64d9e00640 1 -- 172.21.15.1:0/3388864361 reap_dead start 2024-05-09T08:04:04.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:04.855+0000 7f64da800640 1 -- 172.21.15.1:0/3388864361 shutdown_connections 2024-05-09T08:04:04.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:04.855+0000 7f64da800640 1 -- 172.21.15.1:0/3388864361 >> 172.21.15.1:0/3388864361 conn(0x7f64d406b250 msgr2=0x7f64d414e400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:04:04.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:04.855+0000 7f64da800640 1 -- 172.21.15.1:0/3388864361 shutdown_connections 2024-05-09T08:04:04.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:04.855+0000 7f64da800640 1 -- 172.21.15.1:0/3388864361 wait complete. 2024-05-09T08:04:04.887 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-05-09T08:04:04.887 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-05-09T08:04:04.887 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-05-09T08:04:05.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.110+0000 7fb0c1c00640 1 Processor -- start 2024-05-09T08:04:05.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.110+0000 7fb0c1c00640 1 -- start start 2024-05-09T08:04:05.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.112+0000 7fb0c1c00640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fb0bc1513a0 0x7fb0bc0c3070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:04:05.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.112+0000 7fb0c1c00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb0bc0c5df0 0x7fb0bc157240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:04:05.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.112+0000 7fb0c1c00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb0bc0c5350 con 0x7fb0bc0c7b10 2024-05-09T08:04:05.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.112+0000 7fb0c1c00640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7fb0bc0c3670 con 0x7fb0bc1513a0 2024-05-09T08:04:05.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.112+0000 7fb0c1c00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fb0bc0c3950 con 0x7fb0bc0c5df0 2024-05-09T08:04:05.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.112+0000 7fb0bbe00640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fb0bc1513a0 0x7fb0bc0c3070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:04:05.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.112+0000 7fb0bbe00640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fb0bc1513a0 0x7fb0bc0c3070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:35828/0 (socket says 172.21.15.1:35828) 2024-05-09T08:04:05.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.112+0000 7fb0bbe00640 1 -- 172.21.15.1:0/1382187088 learned_addr learned my addr 172.21.15.1:0/1382187088 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:04:05.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.112+0000 7fb0bb400640 1 --2- 172.21.15.1:0/1382187088 >> v2:172.21.15.7:3300/0 conn(0x7fb0bc0c5df0 0x7fb0bc157240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:04:05.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.113+0000 7fb0bbe00640 1 -- 172.21.15.1:0/1382187088 >> v2:172.21.15.7:3300/0 conn(0x7fb0bc0c5df0 msgr2=0x7fb0bc157240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:04:05.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.113+0000 7fb0bbe00640 1 --2- 172.21.15.1:0/1382187088 >> v2:172.21.15.7:3300/0 conn(0x7fb0bc0c5df0 0x7fb0bc157240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:04:05.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.113+0000 7fb0bbe00640 1 -- 172.21.15.1:0/1382187088 >> v1:172.21.15.7:6789/0 conn(0x7fb0bc0c7b10 legacy=0x7fb0bc0c8600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:04:05.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.113+0000 7fb0bbe00640 1 -- 172.21.15.1:0/1382187088 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0bc0c55f0 con 0x7fb0bc1513a0 2024-05-09T08:04:05.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.114+0000 7fb0bbe00640 1 --2- 172.21.15.1:0/1382187088 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb0bc1513a0 0x7fb0bc0c3070 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fb0bc073980 tx=0x7fb0ac02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=596199f648010709 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:04:05.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.114+0000 7fb0baa00640 1 -- 172.21.15.1:0/1382187088 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0ac037070 con 0x7fb0bc1513a0 2024-05-09T08:04:05.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.114+0000 7fb0baa00640 1 -- 172.21.15.1:0/1382187088 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0ac032440 con 0x7fb0bc1513a0 2024-05-09T08:04:05.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.114+0000 7fb0baa00640 1 -- 172.21.15.1:0/1382187088 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0ac032810 con 0x7fb0bc1513a0 2024-05-09T08:04:05.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.114+0000 7fb0c1c00640 1 -- 172.21.15.1:0/1382187088 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb0bc1513a0 msgr2=0x7fb0bc0c3070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:04:05.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.114+0000 7fb0c1c00640 1 --2- 172.21.15.1:0/1382187088 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb0bc1513a0 0x7fb0bc0c3070 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fb0bc073980 tx=0x7fb0ac02a270 comp rx=0 tx=0).stop 2024-05-09T08:04:05.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.114+0000 7fb0c1c00640 1 -- 172.21.15.1:0/1382187088 shutdown_connections 2024-05-09T08:04:05.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.114+0000 7fb0c1c00640 1 --2- 172.21.15.1:0/1382187088 >> v2:172.21.15.7:3300/0 conn(0x7fb0bc0c5df0 0x7fb0bc157240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:04:05.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.114+0000 7fb0c1c00640 1 --2- 172.21.15.1:0/1382187088 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb0bc1513a0 0x7fb0bc0c3070 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:04:05.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.115+0000 7fb0c1c00640 1 -- 172.21.15.1:0/1382187088 >> 172.21.15.1:0/1382187088 conn(0x7fb0bc06b250 msgr2=0x7fb0bc06b630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:04:05.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.115+0000 7fb0c1c00640 1 -- 172.21.15.1:0/1382187088 shutdown_connections 2024-05-09T08:04:05.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.115+0000 7fb0c1c00640 1 -- 172.21.15.1:0/1382187088 wait complete. 2024-05-09T08:04:05.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.116+0000 7fb0c1c00640 1 Processor -- start 2024-05-09T08:04:05.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.116+0000 7fb0c1c00640 1 -- start start 2024-05-09T08:04:05.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.117+0000 7fb0c1c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0bc0c5df0 0x7fb0bc1effd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:04:05.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.117+0000 7fb0c1c00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb0bc0c7b10 0x7fb0bc0cbe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:04:05.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.117+0000 7fb0c1c00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fb0bc1513a0 0x7fb0bc149c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:04:05.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.117+0000 7fb0c1c00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fb0bc0c3db0 con 0x7fb0bc0c7b10 2024-05-09T08:04:05.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.117+0000 7fb0c1c00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb0bc0c3c30 con 0x7fb0bc0c5df0 2024-05-09T08:04:05.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.117+0000 7fb0c1c00640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7fb0bc0c8d10 con 0x7fb0bc1513a0 2024-05-09T08:04:05.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.118+0000 7fb0bbe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0bc0c5df0 0x7fb0bc1effd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:04:05.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.118+0000 7fb0bbe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0bc0c5df0 0x7fb0bc1effd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.1:33792/0 (socket says 172.21.15.1:33792) 2024-05-09T08:04:05.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.118+0000 7fb0bbe00640 1 -- 172.21.15.1:0/3067514634 learned_addr learned my addr 172.21.15.1:0/3067514634 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:04:05.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.118+0000 7fb0c1200640 1 --2- 172.21.15.1:0/3067514634 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fb0bc1513a0 0x7fb0bc149c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:04:05.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.118+0000 7fb0bb400640 1 --2- 172.21.15.1:0/3067514634 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb0bc0c7b10 0x7fb0bc0cbe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:04:05.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.118+0000 7fb0bbe00640 1 -- 172.21.15.1:0/3067514634 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fb0bc1513a0 msgr2=0x7fb0bc149c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:04:05.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.118+0000 7fb0bbe00640 1 --2- 172.21.15.1:0/3067514634 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fb0bc1513a0 0x7fb0bc149c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:04:05.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.118+0000 7fb0bbe00640 1 -- 172.21.15.1:0/3067514634 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb0bc0c7b10 msgr2=0x7fb0bc0cbe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:04:05.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.118+0000 7fb0bbe00640 1 --2- 172.21.15.1:0/3067514634 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb0bc0c7b10 0x7fb0bc0cbe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:04:05.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.118+0000 7fb0bbe00640 1 -- 172.21.15.1:0/3067514634 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0bc1f0510 con 0x7fb0bc0c5df0 2024-05-09T08:04:05.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.118+0000 7fb0bb400640 1 --2- 172.21.15.1:0/3067514634 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb0bc0c7b10 0x7fb0bc0cbe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T08:04:05.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.119+0000 7fb0bbe00640 1 --2- 172.21.15.1:0/3067514634 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0bc0c5df0 0x7fb0bc1effd0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fb0ac02a2b0 tx=0x7fb0ac0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:04:05.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.119+0000 7fb0b3e00640 1 -- 172.21.15.1:0/3067514634 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0ac037040 con 0x7fb0bc0c5df0 2024-05-09T08:04:05.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.119+0000 7fb0b3e00640 1 -- 172.21.15.1:0/3067514634 <== mon.1 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0ac042070 con 0x7fb0bc0c5df0 2024-05-09T08:04:05.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.119+0000 7fb0b3e00640 1 -- 172.21.15.1:0/3067514634 <== mon.1 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0ac030860 con 0x7fb0bc0c5df0 2024-05-09T08:04:05.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.119+0000 7fb0c1c00640 1 -- 172.21.15.1:0/3067514634 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0bc157510 con 0x7fb0bc0c5df0 2024-05-09T08:04:05.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.119+0000 7fb0c1c00640 1 -- 172.21.15.1:0/3067514634 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0bc1ec190 con 0x7fb0bc0c5df0 2024-05-09T08:04:05.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.120+0000 7fb0c1c00640 1 --2- 172.21.15.1:0/3067514634 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0bc14a150 0x7fb0bc1ec360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:04:05.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.120+0000 7fb0c1c00640 1 -- 172.21.15.1:0/3067514634 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb0bc156fe0 con 0x7fb0bc14a150 2024-05-09T08:04:05.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.121+0000 7fb0bb400640 1 --2- 172.21.15.1:0/3067514634 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0bc14a150 0x7fb0bc1ec360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:04:05.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.121+0000 7fb0bb400640 1 -- 172.21.15.1:0/3067514634 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb0bc151780 con 0x7fb0bc14a150 2024-05-09T08:04:05.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.121+0000 7fb0bb400640 1 --2- 172.21.15.1:0/3067514634 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0bc14a150 0x7fb0bc1ec360 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fb0bc1568c0 tx=0x7fb0a8010f50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:04:05.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.122+0000 7fb0b3e00640 1 -- 172.21.15.1:0/3067514634 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0a8004030 con 0x7fb0bc14a150 2024-05-09T08:04:05.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.122+0000 7fb0b3e00640 1 -- 172.21.15.1:0/3067514634 <== mon.1 v2:172.21.15.7:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fb0a80041d0 con 0x7fb0bc14a150 2024-05-09T08:04:05.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.122+0000 7fb0b3e00640 1 -- 172.21.15.1:0/3067514634 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0bc14a150 msgr2=0x7fb0bc1ec360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:04:05.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.122+0000 7fb0b3e00640 1 --2- 172.21.15.1:0/3067514634 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0bc14a150 0x7fb0bc1ec360 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fb0bc1568c0 tx=0x7fb0a8010f50 comp rx=0 tx=0).stop 2024-05-09T08:04:05.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.122+0000 7fb0b3e00640 1 -- 172.21.15.1:0/3067514634 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0bc14a150 msgr2=0x7fb0bc1ec360 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-05-09T08:04:05.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.122+0000 7fb0b3e00640 1 --2- 172.21.15.1:0/3067514634 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0bc14a150 0x7fb0bc1ec360 secure :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fb0bc1568c0 tx=0x7fb0a8010f50 comp rx=0 tx=0).stop 2024-05-09T08:04:05.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.122+0000 7fb0b3e00640 1 -- 172.21.15.1:0/3067514634 <== mon.1 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb0ac030a00 con 0x7fb0bc0c5df0 2024-05-09T08:04:05.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.124+0000 7fb0b3e00640 1 --2- 172.21.15.1:0/3067514634 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fb09c06b940 0x7fb09c06de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:04:05.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.124+0000 7fb0b3e00640 1 -- 172.21.15.1:0/3067514634 <== mon.1 v2:172.21.15.7:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7fb0ac0abd90 con 0x7fb0bc0c5df0 2024-05-09T08:04:05.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.124+0000 7fb0bb400640 1 --2- 172.21.15.1:0/3067514634 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fb09c06b940 0x7fb09c06de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:04:05.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.125+0000 7fb0bb400640 1 --2- 172.21.15.1:0/3067514634 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fb09c06b940 0x7fb09c06de00 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fb0a800bfd0 tx=0x7fb0a8015000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:04:05.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.130+0000 7fb0c1c00640 1 --2- 172.21.15.1:0/3067514634 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0bc156930 0x7fb0bc1ec8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:04:05.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.130+0000 7fb0c1c00640 1 -- 172.21.15.1:0/3067514634 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb0bc04f660 con 0x7fb0bc156930 2024-05-09T08:04:05.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.131+0000 7fb0c1200640 1 --2- 172.21.15.1:0/3067514634 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0bc156930 0x7fb0bc1ec8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:04:05.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.131+0000 7fb0c1200640 1 -- 172.21.15.1:0/3067514634 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fb0bc0718e0 con 0x7fb0bc156930 2024-05-09T08:04:05.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.131+0000 7fb0c1200640 1 --2- 172.21.15.1:0/3067514634 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0bc156930 0x7fb0bc1ec8a0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fb0b400f830 tx=0x7fb0b400fcf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:04:05.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.132+0000 7fb0b3e00640 1 -- 172.21.15.1:0/3067514634 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0b4006d60 con 0x7fb0bc156930 2024-05-09T08:04:05.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.132+0000 7fb0b3e00640 1 -- 172.21.15.1:0/3067514634 <== mon.1 v2:172.21.15.7:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4535 (secure 0 0 0) 0x7fb0b4006d60 con 0x7fb0bc156930 2024-05-09T08:04:05.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.132+0000 7fb0b3e00640 1 -- 172.21.15.1:0/3067514634 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0bc156930 msgr2=0x7fb0bc1ec8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:04:05.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.132+0000 7fb0b3e00640 1 --2- 172.21.15.1:0/3067514634 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0bc156930 0x7fb0bc1ec8a0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fb0b400f830 tx=0x7fb0b400fcf0 comp rx=0 tx=0).stop 2024-05-09T08:04:05.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.132+0000 7fb0b3e00640 1 -- 172.21.15.1:0/3067514634 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0bc156930 msgr2=0x7fb0bc1ec8a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-05-09T08:04:05.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.132+0000 7fb0b3e00640 1 --2- 172.21.15.1:0/3067514634 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0bc156930 0x7fb0bc1ec8a0 secure :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fb0b400f830 tx=0x7fb0b400fcf0 comp rx=0 tx=0).stop 2024-05-09T08:04:05.153 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-05-09T08:04:05.154 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "b", 2024-05-09T08:04:05.154 INFO:teuthology.orchestra.run.smithi001.stdout: "rank": 1, 2024-05-09T08:04:05.154 INFO:teuthology.orchestra.run.smithi001.stdout: "state": "peon", 2024-05-09T08:04:05.154 INFO:teuthology.orchestra.run.smithi001.stdout: "election_epoch": 10, 2024-05-09T08:04:05.154 INFO:teuthology.orchestra.run.smithi001.stdout: "uptime": 12793, 2024-05-09T08:04:05.155 INFO:teuthology.orchestra.run.smithi001.stdout: "quorum": [ 2024-05-09T08:04:05.155 INFO:teuthology.orchestra.run.smithi001.stdout: 0, 2024-05-09T08:04:05.155 INFO:teuthology.orchestra.run.smithi001.stdout: 1, 2024-05-09T08:04:05.155 INFO:teuthology.orchestra.run.smithi001.stdout: 2 2024-05-09T08:04:05.156 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:04:05.156 INFO:teuthology.orchestra.run.smithi001.stdout: "quorum_age": 12, 2024-05-09T08:04:05.156 INFO:teuthology.orchestra.run.smithi001.stdout: "features": { 2024-05-09T08:04:05.156 INFO:teuthology.orchestra.run.smithi001.stdout: "required_con": "2450521708007866372", 2024-05-09T08:04:05.156 INFO:teuthology.orchestra.run.smithi001.stdout: "required_mon": [ 2024-05-09T08:04:05.156 INFO:teuthology.orchestra.run.smithi001.stdout: "kraken", 2024-05-09T08:04:05.156 INFO:teuthology.orchestra.run.smithi001.stdout: "luminous", 2024-05-09T08:04:05.156 INFO:teuthology.orchestra.run.smithi001.stdout: "mimic", 2024-05-09T08:04:05.156 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap-prune", 2024-05-09T08:04:05.156 INFO:teuthology.orchestra.run.smithi001.stdout: "nautilus", 2024-05-09T08:04:05.156 INFO:teuthology.orchestra.run.smithi001.stdout: "octopus", 2024-05-09T08:04:05.156 INFO:teuthology.orchestra.run.smithi001.stdout: "pacific", 2024-05-09T08:04:05.156 INFO:teuthology.orchestra.run.smithi001.stdout: "elector-pinging", 2024-05-09T08:04:05.157 INFO:teuthology.orchestra.run.smithi001.stdout: "quincy", 2024-05-09T08:04:05.157 INFO:teuthology.orchestra.run.smithi001.stdout: "reef", 2024-05-09T08:04:05.157 INFO:teuthology.orchestra.run.smithi001.stdout: "squid" 2024-05-09T08:04:05.157 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:04:05.157 INFO:teuthology.orchestra.run.smithi001.stdout: "quorum_con": "4540701547738038271", 2024-05-09T08:04:05.157 INFO:teuthology.orchestra.run.smithi001.stdout: "quorum_mon": [ 2024-05-09T08:04:05.157 INFO:teuthology.orchestra.run.smithi001.stdout: "kraken", 2024-05-09T08:04:05.157 INFO:teuthology.orchestra.run.smithi001.stdout: "luminous", 2024-05-09T08:04:05.157 INFO:teuthology.orchestra.run.smithi001.stdout: "mimic", 2024-05-09T08:04:05.157 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap-prune", 2024-05-09T08:04:05.157 INFO:teuthology.orchestra.run.smithi001.stdout: "nautilus", 2024-05-09T08:04:05.157 INFO:teuthology.orchestra.run.smithi001.stdout: "octopus", 2024-05-09T08:04:05.157 INFO:teuthology.orchestra.run.smithi001.stdout: "pacific", 2024-05-09T08:04:05.158 INFO:teuthology.orchestra.run.smithi001.stdout: "elector-pinging", 2024-05-09T08:04:05.158 INFO:teuthology.orchestra.run.smithi001.stdout: "quincy", 2024-05-09T08:04:05.158 INFO:teuthology.orchestra.run.smithi001.stdout: "reef", 2024-05-09T08:04:05.158 INFO:teuthology.orchestra.run.smithi001.stdout: "squid" 2024-05-09T08:04:05.158 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:04:05.158 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:04:05.158 INFO:teuthology.orchestra.run.smithi001.stdout: "outside_quorum": [], 2024-05-09T08:04:05.158 INFO:teuthology.orchestra.run.smithi001.stdout: "extra_probe_peers": [], 2024-05-09T08:04:05.158 INFO:teuthology.orchestra.run.smithi001.stdout: "sync_provider": [], 2024-05-09T08:04:05.158 INFO:teuthology.orchestra.run.smithi001.stdout: "monmap": { 2024-05-09T08:04:05.158 INFO:teuthology.orchestra.run.smithi001.stdout: "epoch": 1, 2024-05-09T08:04:05.158 INFO:teuthology.orchestra.run.smithi001.stdout: "fsid": "54329ec8-3270-4043-9a82-c6cd7420aa7f", 2024-05-09T08:04:05.158 INFO:teuthology.orchestra.run.smithi001.stdout: "modified": "2024-05-09T08:01:21.819270Z", 2024-05-09T08:04:05.159 INFO:teuthology.orchestra.run.smithi001.stdout: "created": "2024-05-09T08:01:21.819270Z", 2024-05-09T08:04:05.159 INFO:teuthology.orchestra.run.smithi001.stdout: "min_mon_release": 19, 2024-05-09T08:04:05.159 INFO:teuthology.orchestra.run.smithi001.stdout: "min_mon_release_name": "squid", 2024-05-09T08:04:05.159 INFO:teuthology.orchestra.run.smithi001.stdout: "election_strategy": 1, 2024-05-09T08:04:05.159 INFO:teuthology.orchestra.run.smithi001.stdout: "disallowed_leaders: ": "", 2024-05-09T08:04:05.159 INFO:teuthology.orchestra.run.smithi001.stdout: "stretch_mode": false, 2024-05-09T08:04:05.159 INFO:teuthology.orchestra.run.smithi001.stdout: "tiebreaker_mon": "", 2024-05-09T08:04:05.159 INFO:teuthology.orchestra.run.smithi001.stdout: "removed_ranks: ": "", 2024-05-09T08:04:05.159 INFO:teuthology.orchestra.run.smithi001.stdout: "features": { 2024-05-09T08:04:05.159 INFO:teuthology.orchestra.run.smithi001.stdout: "persistent": [ 2024-05-09T08:04:05.159 INFO:teuthology.orchestra.run.smithi001.stdout: "kraken", 2024-05-09T08:04:05.159 INFO:teuthology.orchestra.run.smithi001.stdout: "luminous", 2024-05-09T08:04:05.160 INFO:teuthology.orchestra.run.smithi001.stdout: "mimic", 2024-05-09T08:04:05.160 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap-prune", 2024-05-09T08:04:05.160 INFO:teuthology.orchestra.run.smithi001.stdout: "nautilus", 2024-05-09T08:04:05.160 INFO:teuthology.orchestra.run.smithi001.stdout: "octopus", 2024-05-09T08:04:05.160 INFO:teuthology.orchestra.run.smithi001.stdout: "pacific", 2024-05-09T08:04:05.160 INFO:teuthology.orchestra.run.smithi001.stdout: "elector-pinging", 2024-05-09T08:04:05.160 INFO:teuthology.orchestra.run.smithi001.stdout: "quincy", 2024-05-09T08:04:05.160 INFO:teuthology.orchestra.run.smithi001.stdout: "reef", 2024-05-09T08:04:05.160 INFO:teuthology.orchestra.run.smithi001.stdout: "squid" 2024-05-09T08:04:05.160 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:04:05.160 INFO:teuthology.orchestra.run.smithi001.stdout: "optional": [] 2024-05-09T08:04:05.160 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:04:05.160 INFO:teuthology.orchestra.run.smithi001.stdout: "mons": [ 2024-05-09T08:04:05.160 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:04:05.161 INFO:teuthology.orchestra.run.smithi001.stdout: "rank": 0, 2024-05-09T08:04:05.161 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "a", 2024-05-09T08:04:05.161 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-05-09T08:04:05.161 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-05-09T08:04:05.161 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:04:05.161 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-05-09T08:04:05.161 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:3300", 2024-05-09T08:04:05.161 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:04:05.161 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:04:05.161 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:04:05.161 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-05-09T08:04:05.161 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6789", 2024-05-09T08:04:05.161 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:04:05.161 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:04:05.162 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:04:05.162 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:04:05.162 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6789/0", 2024-05-09T08:04:05.162 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.1:6789/0", 2024-05-09T08:04:05.162 INFO:teuthology.orchestra.run.smithi001.stdout: "priority": 0, 2024-05-09T08:04:05.162 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 0, 2024-05-09T08:04:05.162 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_location": "{}" 2024-05-09T08:04:05.162 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:04:05.162 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:04:05.162 INFO:teuthology.orchestra.run.smithi001.stdout: "rank": 1, 2024-05-09T08:04:05.162 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "b", 2024-05-09T08:04:05.162 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-05-09T08:04:05.162 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-05-09T08:04:05.162 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:04:05.163 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-05-09T08:04:05.163 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:3300", 2024-05-09T08:04:05.163 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:04:05.163 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:04:05.163 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:04:05.163 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-05-09T08:04:05.163 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:6789", 2024-05-09T08:04:05.163 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:04:05.163 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:04:05.163 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:04:05.163 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:04:05.163 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:6789/0", 2024-05-09T08:04:05.164 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.7:6789/0", 2024-05-09T08:04:05.164 INFO:teuthology.orchestra.run.smithi001.stdout: "priority": 0, 2024-05-09T08:04:05.165 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 0, 2024-05-09T08:04:05.165 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_location": "{}" 2024-05-09T08:04:05.165 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:04:05.165 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:04:05.165 INFO:teuthology.orchestra.run.smithi001.stdout: "rank": 2, 2024-05-09T08:04:05.165 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "c", 2024-05-09T08:04:05.165 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-05-09T08:04:05.165 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-05-09T08:04:05.165 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:04:05.165 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-05-09T08:04:05.165 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:3301", 2024-05-09T08:04:05.165 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:04:05.165 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:04:05.166 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:04:05.166 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-05-09T08:04:05.166 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:6790", 2024-05-09T08:04:05.166 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:04:05.166 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:04:05.166 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:04:05.166 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:04:05.166 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:6790/0", 2024-05-09T08:04:05.166 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.7:6790/0", 2024-05-09T08:04:05.166 INFO:teuthology.orchestra.run.smithi001.stdout: "priority": 0, 2024-05-09T08:04:05.166 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 0, 2024-05-09T08:04:05.166 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_location": "{}" 2024-05-09T08:04:05.166 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:04:05.167 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:04:05.167 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:04:05.167 INFO:teuthology.orchestra.run.smithi001.stdout: "feature_map": { 2024-05-09T08:04:05.167 INFO:teuthology.orchestra.run.smithi001.stdout: "mon": [ 2024-05-09T08:04:05.167 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:04:05.167 INFO:teuthology.orchestra.run.smithi001.stdout: "features": "0x3f03cffffffdffff", 2024-05-09T08:04:05.167 INFO:teuthology.orchestra.run.smithi001.stdout: "release": "luminous", 2024-05-09T08:04:05.167 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 1 2024-05-09T08:04:05.167 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:04:05.167 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:04:05.167 INFO:teuthology.orchestra.run.smithi001.stdout: "client": [ 2024-05-09T08:04:05.167 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:04:05.167 INFO:teuthology.orchestra.run.smithi001.stdout: "features": "0x3f03cffffffdffff", 2024-05-09T08:04:05.167 INFO:teuthology.orchestra.run.smithi001.stdout: "release": "luminous", 2024-05-09T08:04:05.168 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 2 2024-05-09T08:04:05.168 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:04:05.168 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:04:05.168 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:04:05.168 INFO:teuthology.orchestra.run.smithi001.stdout: "stretch_mode": false 2024-05-09T08:04:05.168 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-05-09T08:04:05.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.132+0000 7fb0c1c00640 1 -- 172.21.15.1:0/3067514634 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fb09c06b940 msgr2=0x7fb09c06de00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:04:05.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.132+0000 7fb0c1c00640 1 --2- 172.21.15.1:0/3067514634 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fb09c06b940 0x7fb09c06de00 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fb0a800bfd0 tx=0x7fb0a8015000 comp rx=0 tx=0).stop 2024-05-09T08:04:05.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.133+0000 7fb0c1c00640 1 -- 172.21.15.1:0/3067514634 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0bc0c5df0 msgr2=0x7fb0bc1effd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:04:05.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.133+0000 7fb0c1c00640 1 --2- 172.21.15.1:0/3067514634 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0bc0c5df0 0x7fb0bc1effd0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fb0ac02a2b0 tx=0x7fb0ac0046c0 comp rx=0 tx=0).stop 2024-05-09T08:04:05.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.133+0000 7fb0c1200640 1 -- 172.21.15.1:0/3067514634 reap_dead start 2024-05-09T08:04:05.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.133+0000 7fb0c1200640 1 -- 172.21.15.1:0/3067514634 reap_dead start 2024-05-09T08:04:05.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.133+0000 7fb0c1c00640 1 -- 172.21.15.1:0/3067514634 shutdown_connections 2024-05-09T08:04:05.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.133+0000 7fb0c1c00640 1 -- 172.21.15.1:0/3067514634 >> 172.21.15.1:0/3067514634 conn(0x7fb0bc06b250 msgr2=0x7fb0bc0c8600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:04:05.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.133+0000 7fb0c1c00640 1 -- 172.21.15.1:0/3067514634 shutdown_connections 2024-05-09T08:04:05.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.133+0000 7fb0c1c00640 1 -- 172.21.15.1:0/3067514634 wait complete. 2024-05-09T08:04:05.170 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-05-09T08:04:05.170 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-05-09T08:04:05.171 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-05-09T08:04:05.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.323+0000 7f254e800640 1 Processor -- start 2024-05-09T08:04:05.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.324+0000 7f254e800640 1 -- start start 2024-05-09T08:04:05.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.325+0000 7f254e800640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f25480c5890 0x7f254815b050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:04:05.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.325+0000 7f254e800640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f25480718e0 0x7f25480c69e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:04:05.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.325+0000 7f254e800640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f254806f060 con 0x7f25480c6f20 2024-05-09T08:04:05.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.325+0000 7f254e800640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f254815b590 con 0x7f25480718e0 2024-05-09T08:04:05.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.325+0000 7f254e800640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f2548072150 con 0x7f25480c5890 2024-05-09T08:04:05.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.325+0000 7f254d400640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f25480c5890 0x7f254815b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:04:05.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.325+0000 7f254ca00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f25480718e0 0x7f25480c69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:04:05.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.325+0000 7f254ca00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f25480718e0 0x7f25480c69e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.1:33832/0 (socket says 172.21.15.1:33832) 2024-05-09T08:04:05.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.325+0000 7f254d400640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f25480c5890 0x7f254815b050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3301/0 says I am v2:172.21.15.1:57748/0 (socket says 172.21.15.1:57748) 2024-05-09T08:04:05.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.325+0000 7f254ca00640 1 -- 172.21.15.1:0/3429316044 learned_addr learned my addr 172.21.15.1:0/3429316044 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:04:05.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.326+0000 7f254ca00640 1 -- 172.21.15.1:0/3429316044 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f25480c5890 msgr2=0x7f254815b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:04:05.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.326+0000 7f254ca00640 1 --2- 172.21.15.1:0/3429316044 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f25480c5890 0x7f254815b050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:04:05.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.326+0000 7f254ca00640 1 -- 172.21.15.1:0/3429316044 >> v1:172.21.15.7:6789/0 conn(0x7f25480c6f20 legacy=0x7f2548072940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:04:05.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.326+0000 7f254ca00640 1 -- 172.21.15.1:0/3429316044 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f254806fe80 con 0x7f25480718e0 2024-05-09T08:04:05.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.327+0000 7f254ca00640 1 --2- 172.21.15.1:0/3429316044 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f25480718e0 0x7f25480c69e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f2534009870 tx=0x7f253402a2e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=54e0fd70aabb31ae server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:04:05.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.327+0000 7f2547e00640 1 -- 172.21.15.1:0/3429316044 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2534037070 con 0x7f25480718e0 2024-05-09T08:04:05.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.327+0000 7f2547e00640 1 -- 172.21.15.1:0/3429316044 <== mon.1 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2534032640 con 0x7f25480718e0 2024-05-09T08:04:05.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.327+0000 7f2547e00640 1 -- 172.21.15.1:0/3429316044 <== mon.1 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2534032a10 con 0x7f25480718e0 2024-05-09T08:04:05.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.328+0000 7f254e800640 1 -- 172.21.15.1:0/3429316044 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f25480718e0 msgr2=0x7f25480c69e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:04:05.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.328+0000 7f254e800640 1 --2- 172.21.15.1:0/3429316044 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f25480718e0 0x7f25480c69e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f2534009870 tx=0x7f253402a2e0 comp rx=0 tx=0).stop 2024-05-09T08:04:05.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.328+0000 7f254e800640 1 -- 172.21.15.1:0/3429316044 shutdown_connections 2024-05-09T08:04:05.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.328+0000 7f254e800640 1 --2- 172.21.15.1:0/3429316044 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f25480718e0 0x7f25480c69e0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:04:05.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.328+0000 7f254e800640 1 --2- 172.21.15.1:0/3429316044 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f25480c5890 0x7f254815b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:04:05.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.328+0000 7f254e800640 1 -- 172.21.15.1:0/3429316044 >> 172.21.15.1:0/3429316044 conn(0x7f254806b250 msgr2=0x7f254806b630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:04:05.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.328+0000 7f254e800640 1 -- 172.21.15.1:0/3429316044 shutdown_connections 2024-05-09T08:04:05.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.328+0000 7f254e800640 1 -- 172.21.15.1:0/3429316044 wait complete. 2024-05-09T08:04:05.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.331+0000 7f254e800640 1 Processor -- start 2024-05-09T08:04:05.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.331+0000 7f254e800640 1 -- start start 2024-05-09T08:04:05.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.332+0000 7f254e800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f25480718e0 0x7f2548156f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:04:05.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.332+0000 7f254e800640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f25480c5890 0x7f254814e170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:04:05.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.332+0000 7f254e800640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f25480c6f20 0x7f254814f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:04:05.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.332+0000 7f254e800640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f254806f8c0 con 0x7f25480c6f20 2024-05-09T08:04:05.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.332+0000 7f254e800640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f254806f5e0 con 0x7f25480718e0 2024-05-09T08:04:05.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.332+0000 7f254e800640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f254806fba0 con 0x7f25480c5890 2024-05-09T08:04:05.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.333+0000 7f254d400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f25480718e0 0x7f2548156f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:04:05.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.333+0000 7f254de00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f25480c6f20 0x7f254814f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:04:05.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.333+0000 7f254ca00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f25480c5890 0x7f254814e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:04:05.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.333+0000 7f254de00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f25480c6f20 0x7f254814f150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:35854/0 (socket says 172.21.15.1:35854) 2024-05-09T08:04:05.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.333+0000 7f254de00640 1 -- 172.21.15.1:0/2246223855 learned_addr learned my addr 172.21.15.1:0/2246223855 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:04:05.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.333+0000 7f254ca00640 1 -- 172.21.15.1:0/2246223855 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f25480718e0 msgr2=0x7f2548156f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:04:05.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.333+0000 7f254ca00640 1 --2- 172.21.15.1:0/2246223855 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f25480718e0 0x7f2548156f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:04:05.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.333+0000 7f254ca00640 1 -- 172.21.15.1:0/2246223855 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f25480c6f20 msgr2=0x7f254814f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:04:05.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.333+0000 7f254ca00640 1 --2- 172.21.15.1:0/2246223855 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f25480c6f20 0x7f254814f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:04:05.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.333+0000 7f254ca00640 1 -- 172.21.15.1:0/2246223855 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25481576e0 con 0x7f25480c5890 2024-05-09T08:04:05.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.334+0000 7f254ca00640 1 --2- 172.21.15.1:0/2246223855 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f25480c5890 0x7f254814e170 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f25480701c0 tx=0x7f25340046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:04:05.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.334+0000 7f2545600640 1 -- 172.21.15.1:0/2246223855 <== mon.2 v2:172.21.15.7:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2534037040 con 0x7f25480c5890 2024-05-09T08:04:05.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.334+0000 7f2545600640 1 -- 172.21.15.1:0/2246223855 <== mon.2 v2:172.21.15.7:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2534042070 con 0x7f25480c5890 2024-05-09T08:04:05.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.334+0000 7f254e800640 1 -- 172.21.15.1:0/2246223855 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2548151090 con 0x7f25480c5890 2024-05-09T08:04:05.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.334+0000 7f2545600640 1 -- 172.21.15.1:0/2246223855 <== mon.2 v2:172.21.15.7:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25340309b0 con 0x7f25480c5890 2024-05-09T08:04:05.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.334+0000 7f254e800640 1 -- 172.21.15.1:0/2246223855 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f2548151650 con 0x7f25480c5890 2024-05-09T08:04:05.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.335+0000 7f254e800640 1 --2- 172.21.15.1:0/2246223855 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f2548151820 0x7f254814f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:04:05.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.335+0000 7f254e800640 1 -- 172.21.15.1:0/2246223855 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f25481574a0 con 0x7f2548151820 2024-05-09T08:04:05.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.336+0000 7f254d400640 1 --2- 172.21.15.1:0/2246223855 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f2548151820 0x7f254814f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:04:05.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.336+0000 7f254d400640 1 -- 172.21.15.1:0/2246223855 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f254804f660 con 0x7f2548151820 2024-05-09T08:04:05.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.336+0000 7f2545600640 1 -- 172.21.15.1:0/2246223855 <== mon.2 v2:172.21.15.7:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2534030b50 con 0x7f25480c5890 2024-05-09T08:04:05.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.337+0000 7f254d400640 1 --2- 172.21.15.1:0/2246223855 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f2548151820 0x7f254814f690 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f253c0118e0 tx=0x7f253c011da0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:04:05.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.337+0000 7f2545600640 1 --2- 172.21.15.1:0/2246223855 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f252806be60 0x7f252806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:04:05.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.337+0000 7f2545600640 1 -- 172.21.15.1:0/2246223855 <== mon.2 v2:172.21.15.7:3301/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7f25340ab810 con 0x7f25480c5890 2024-05-09T08:04:05.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.338+0000 7f254de00640 1 --2- 172.21.15.1:0/2246223855 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f252806be60 0x7f252806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:04:05.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.338+0000 7f2545600640 1 -- 172.21.15.1:0/2246223855 <== mon.2 v2:172.21.15.7:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f253c019070 con 0x7f2548151820 2024-05-09T08:04:05.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.338+0000 7f2545600640 1 -- 172.21.15.1:0/2246223855 <== mon.2 v2:172.21.15.7:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f253c004300 con 0x7f2548151820 2024-05-09T08:04:05.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.338+0000 7f2545600640 1 -- 172.21.15.1:0/2246223855 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f2548151820 msgr2=0x7f254814f690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:04:05.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.338+0000 7f2545600640 1 --2- 172.21.15.1:0/2246223855 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f2548151820 0x7f254814f690 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f253c0118e0 tx=0x7f253c011da0 comp rx=0 tx=0).stop 2024-05-09T08:04:05.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.338+0000 7f2545600640 1 -- 172.21.15.1:0/2246223855 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f2548151820 msgr2=0x7f254814f690 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-05-09T08:04:05.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.338+0000 7f2545600640 1 --2- 172.21.15.1:0/2246223855 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f2548151820 0x7f254814f690 secure :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f253c0118e0 tx=0x7f253c011da0 comp rx=0 tx=0).stop 2024-05-09T08:04:05.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.338+0000 7f254de00640 1 --2- 172.21.15.1:0/2246223855 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f252806be60 0x7f252806e320 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f2540007fd0 tx=0x7f2540009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:04:05.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.351+0000 7f254e800640 1 --2- 172.21.15.1:0/2246223855 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f25481f07c0 0x7f25481f0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:04:05.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.351+0000 7f254e800640 1 -- 172.21.15.1:0/2246223855 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f25480bf020 con 0x7f25481f07c0 2024-05-09T08:04:05.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.352+0000 7f254d400640 1 --2- 172.21.15.1:0/2246223855 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f25481f07c0 0x7f25481f0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:04:05.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.352+0000 7f254d400640 1 -- 172.21.15.1:0/2246223855 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f25480c7520 con 0x7f25481f07c0 2024-05-09T08:04:05.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.352+0000 7f254d400640 1 --2- 172.21.15.1:0/2246223855 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f25481f07c0 0x7f25481f0ba0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f253c0118e0 tx=0x7f253c0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:04:05.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.353+0000 7f2545600640 1 -- 172.21.15.1:0/2246223855 <== mon.2 v2:172.21.15.7:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f253c019040 con 0x7f25481f07c0 2024-05-09T08:04:05.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.353+0000 7f2545600640 1 -- 172.21.15.1:0/2246223855 <== mon.2 v2:172.21.15.7:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5189 (secure 0 0 0) 0x7f253c003da0 con 0x7f25481f07c0 2024-05-09T08:04:05.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.353+0000 7f2545600640 1 -- 172.21.15.1:0/2246223855 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f25481f07c0 msgr2=0x7f25481f0ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:04:05.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.353+0000 7f2545600640 1 --2- 172.21.15.1:0/2246223855 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f25481f07c0 0x7f25481f0ba0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f253c0118e0 tx=0x7f253c0046c0 comp rx=0 tx=0).stop 2024-05-09T08:04:05.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.353+0000 7f2545600640 1 -- 172.21.15.1:0/2246223855 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f25481f07c0 msgr2=0x7f25481f0ba0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-05-09T08:04:05.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.353+0000 7f2545600640 1 --2- 172.21.15.1:0/2246223855 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f25481f07c0 0x7f25481f0ba0 secure :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f253c0118e0 tx=0x7f253c0046c0 comp rx=0 tx=0).stop 2024-05-09T08:04:05.370 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-05-09T08:04:05.370 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "c", 2024-05-09T08:04:05.370 INFO:teuthology.orchestra.run.smithi001.stdout: "rank": 2, 2024-05-09T08:04:05.370 INFO:teuthology.orchestra.run.smithi001.stdout: "state": "peon", 2024-05-09T08:04:05.370 INFO:teuthology.orchestra.run.smithi001.stdout: "election_epoch": 10, 2024-05-09T08:04:05.370 INFO:teuthology.orchestra.run.smithi001.stdout: "uptime": 128769, 2024-05-09T08:04:05.371 INFO:teuthology.orchestra.run.smithi001.stdout: "quorum": [ 2024-05-09T08:04:05.371 INFO:teuthology.orchestra.run.smithi001.stdout: 0, 2024-05-09T08:04:05.371 INFO:teuthology.orchestra.run.smithi001.stdout: 1, 2024-05-09T08:04:05.371 INFO:teuthology.orchestra.run.smithi001.stdout: 2 2024-05-09T08:04:05.371 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:04:05.371 INFO:teuthology.orchestra.run.smithi001.stdout: "quorum_age": 12, 2024-05-09T08:04:05.371 INFO:teuthology.orchestra.run.smithi001.stdout: "features": { 2024-05-09T08:04:05.371 INFO:teuthology.orchestra.run.smithi001.stdout: "required_con": "2450521708007866372", 2024-05-09T08:04:05.371 INFO:teuthology.orchestra.run.smithi001.stdout: "required_mon": [ 2024-05-09T08:04:05.371 INFO:teuthology.orchestra.run.smithi001.stdout: "kraken", 2024-05-09T08:04:05.371 INFO:teuthology.orchestra.run.smithi001.stdout: "luminous", 2024-05-09T08:04:05.371 INFO:teuthology.orchestra.run.smithi001.stdout: "mimic", 2024-05-09T08:04:05.371 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap-prune", 2024-05-09T08:04:05.371 INFO:teuthology.orchestra.run.smithi001.stdout: "nautilus", 2024-05-09T08:04:05.371 INFO:teuthology.orchestra.run.smithi001.stdout: "octopus", 2024-05-09T08:04:05.372 INFO:teuthology.orchestra.run.smithi001.stdout: "pacific", 2024-05-09T08:04:05.372 INFO:teuthology.orchestra.run.smithi001.stdout: "elector-pinging", 2024-05-09T08:04:05.372 INFO:teuthology.orchestra.run.smithi001.stdout: "quincy", 2024-05-09T08:04:05.372 INFO:teuthology.orchestra.run.smithi001.stdout: "reef", 2024-05-09T08:04:05.372 INFO:teuthology.orchestra.run.smithi001.stdout: "squid" 2024-05-09T08:04:05.372 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:04:05.372 INFO:teuthology.orchestra.run.smithi001.stdout: "quorum_con": "4540701547738038271", 2024-05-09T08:04:05.372 INFO:teuthology.orchestra.run.smithi001.stdout: "quorum_mon": [ 2024-05-09T08:04:05.372 INFO:teuthology.orchestra.run.smithi001.stdout: "kraken", 2024-05-09T08:04:05.372 INFO:teuthology.orchestra.run.smithi001.stdout: "luminous", 2024-05-09T08:04:05.372 INFO:teuthology.orchestra.run.smithi001.stdout: "mimic", 2024-05-09T08:04:05.372 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap-prune", 2024-05-09T08:04:05.372 INFO:teuthology.orchestra.run.smithi001.stdout: "nautilus", 2024-05-09T08:04:05.372 INFO:teuthology.orchestra.run.smithi001.stdout: "octopus", 2024-05-09T08:04:05.373 INFO:teuthology.orchestra.run.smithi001.stdout: "pacific", 2024-05-09T08:04:05.373 INFO:teuthology.orchestra.run.smithi001.stdout: "elector-pinging", 2024-05-09T08:04:05.373 INFO:teuthology.orchestra.run.smithi001.stdout: "quincy", 2024-05-09T08:04:05.373 INFO:teuthology.orchestra.run.smithi001.stdout: "reef", 2024-05-09T08:04:05.373 INFO:teuthology.orchestra.run.smithi001.stdout: "squid" 2024-05-09T08:04:05.373 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:04:05.373 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:04:05.373 INFO:teuthology.orchestra.run.smithi001.stdout: "outside_quorum": [], 2024-05-09T08:04:05.373 INFO:teuthology.orchestra.run.smithi001.stdout: "extra_probe_peers": [], 2024-05-09T08:04:05.373 INFO:teuthology.orchestra.run.smithi001.stdout: "sync_provider": [], 2024-05-09T08:04:05.373 INFO:teuthology.orchestra.run.smithi001.stdout: "monmap": { 2024-05-09T08:04:05.373 INFO:teuthology.orchestra.run.smithi001.stdout: "epoch": 1, 2024-05-09T08:04:05.373 INFO:teuthology.orchestra.run.smithi001.stdout: "fsid": "54329ec8-3270-4043-9a82-c6cd7420aa7f", 2024-05-09T08:04:05.373 INFO:teuthology.orchestra.run.smithi001.stdout: "modified": "2024-05-09T08:01:21.819270Z", 2024-05-09T08:04:05.373 INFO:teuthology.orchestra.run.smithi001.stdout: "created": "2024-05-09T08:01:21.819270Z", 2024-05-09T08:04:05.374 INFO:teuthology.orchestra.run.smithi001.stdout: "min_mon_release": 19, 2024-05-09T08:04:05.374 INFO:teuthology.orchestra.run.smithi001.stdout: "min_mon_release_name": "squid", 2024-05-09T08:04:05.374 INFO:teuthology.orchestra.run.smithi001.stdout: "election_strategy": 1, 2024-05-09T08:04:05.374 INFO:teuthology.orchestra.run.smithi001.stdout: "disallowed_leaders: ": "", 2024-05-09T08:04:05.374 INFO:teuthology.orchestra.run.smithi001.stdout: "stretch_mode": false, 2024-05-09T08:04:05.374 INFO:teuthology.orchestra.run.smithi001.stdout: "tiebreaker_mon": "", 2024-05-09T08:04:05.374 INFO:teuthology.orchestra.run.smithi001.stdout: "removed_ranks: ": "", 2024-05-09T08:04:05.374 INFO:teuthology.orchestra.run.smithi001.stdout: "features": { 2024-05-09T08:04:05.374 INFO:teuthology.orchestra.run.smithi001.stdout: "persistent": [ 2024-05-09T08:04:05.374 INFO:teuthology.orchestra.run.smithi001.stdout: "kraken", 2024-05-09T08:04:05.374 INFO:teuthology.orchestra.run.smithi001.stdout: "luminous", 2024-05-09T08:04:05.374 INFO:teuthology.orchestra.run.smithi001.stdout: "mimic", 2024-05-09T08:04:05.374 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap-prune", 2024-05-09T08:04:05.375 INFO:teuthology.orchestra.run.smithi001.stdout: "nautilus", 2024-05-09T08:04:05.375 INFO:teuthology.orchestra.run.smithi001.stdout: "octopus", 2024-05-09T08:04:05.375 INFO:teuthology.orchestra.run.smithi001.stdout: "pacific", 2024-05-09T08:04:05.375 INFO:teuthology.orchestra.run.smithi001.stdout: "elector-pinging", 2024-05-09T08:04:05.375 INFO:teuthology.orchestra.run.smithi001.stdout: "quincy", 2024-05-09T08:04:05.375 INFO:teuthology.orchestra.run.smithi001.stdout: "reef", 2024-05-09T08:04:05.375 INFO:teuthology.orchestra.run.smithi001.stdout: "squid" 2024-05-09T08:04:05.375 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:04:05.375 INFO:teuthology.orchestra.run.smithi001.stdout: "optional": [] 2024-05-09T08:04:05.375 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:04:05.375 INFO:teuthology.orchestra.run.smithi001.stdout: "mons": [ 2024-05-09T08:04:05.375 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:04:05.375 INFO:teuthology.orchestra.run.smithi001.stdout: "rank": 0, 2024-05-09T08:04:05.375 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "a", 2024-05-09T08:04:05.376 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-05-09T08:04:05.376 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-05-09T08:04:05.376 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:04:05.376 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-05-09T08:04:05.376 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:3300", 2024-05-09T08:04:05.376 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:04:05.376 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:04:05.376 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:04:05.376 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-05-09T08:04:05.376 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6789", 2024-05-09T08:04:05.376 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:04:05.376 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:04:05.376 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:04:05.377 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:04:05.377 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6789/0", 2024-05-09T08:04:05.377 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.1:6789/0", 2024-05-09T08:04:05.377 INFO:teuthology.orchestra.run.smithi001.stdout: "priority": 0, 2024-05-09T08:04:05.377 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 0, 2024-05-09T08:04:05.377 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_location": "{}" 2024-05-09T08:04:05.377 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:04:05.377 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:04:05.377 INFO:teuthology.orchestra.run.smithi001.stdout: "rank": 1, 2024-05-09T08:04:05.377 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "b", 2024-05-09T08:04:05.377 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-05-09T08:04:05.377 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-05-09T08:04:05.377 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:04:05.378 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-05-09T08:04:05.378 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:3300", 2024-05-09T08:04:05.378 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:04:05.378 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:04:05.378 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:04:05.378 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-05-09T08:04:05.378 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:6789", 2024-05-09T08:04:05.378 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:04:05.378 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:04:05.378 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:04:05.378 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:04:05.378 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:6789/0", 2024-05-09T08:04:05.378 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.7:6789/0", 2024-05-09T08:04:05.378 INFO:teuthology.orchestra.run.smithi001.stdout: "priority": 0, 2024-05-09T08:04:05.379 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 0, 2024-05-09T08:04:05.379 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_location": "{}" 2024-05-09T08:04:05.379 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:04:05.379 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:04:05.379 INFO:teuthology.orchestra.run.smithi001.stdout: "rank": 2, 2024-05-09T08:04:05.379 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "c", 2024-05-09T08:04:05.379 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-05-09T08:04:05.379 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-05-09T08:04:05.379 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:04:05.379 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-05-09T08:04:05.379 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:3301", 2024-05-09T08:04:05.379 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:04:05.379 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:04:05.380 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:04:05.380 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-05-09T08:04:05.380 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:6790", 2024-05-09T08:04:05.380 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:04:05.380 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:04:05.380 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:04:05.380 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:04:05.380 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:6790/0", 2024-05-09T08:04:05.380 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.7:6790/0", 2024-05-09T08:04:05.380 INFO:teuthology.orchestra.run.smithi001.stdout: "priority": 0, 2024-05-09T08:04:05.380 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 0, 2024-05-09T08:04:05.380 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_location": "{}" 2024-05-09T08:04:05.380 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:04:05.380 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:04:05.381 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:04:05.381 INFO:teuthology.orchestra.run.smithi001.stdout: "feature_map": { 2024-05-09T08:04:05.381 INFO:teuthology.orchestra.run.smithi001.stdout: "mon": [ 2024-05-09T08:04:05.381 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:04:05.381 INFO:teuthology.orchestra.run.smithi001.stdout: "features": "0x3f03cffffffdffff", 2024-05-09T08:04:05.381 INFO:teuthology.orchestra.run.smithi001.stdout: "release": "luminous", 2024-05-09T08:04:05.381 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 1 2024-05-09T08:04:05.381 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:04:05.381 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:04:05.381 INFO:teuthology.orchestra.run.smithi001.stdout: "mds": [ 2024-05-09T08:04:05.381 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:04:05.381 INFO:teuthology.orchestra.run.smithi001.stdout: "features": "0x3f03cffffffdffff", 2024-05-09T08:04:05.381 INFO:teuthology.orchestra.run.smithi001.stdout: "release": "luminous", 2024-05-09T08:04:05.382 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 6 2024-05-09T08:04:05.382 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:04:05.382 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:04:05.382 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": [ 2024-05-09T08:04:05.382 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:04:05.382 INFO:teuthology.orchestra.run.smithi001.stdout: "features": "0x3f03cffffffdffff", 2024-05-09T08:04:05.382 INFO:teuthology.orchestra.run.smithi001.stdout: "release": "luminous", 2024-05-09T08:04:05.382 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 6 2024-05-09T08:04:05.382 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:04:05.382 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:04:05.382 INFO:teuthology.orchestra.run.smithi001.stdout: "client": [ 2024-05-09T08:04:05.382 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:04:05.382 INFO:teuthology.orchestra.run.smithi001.stdout: "features": "0x2f018fb87aa4aafe", 2024-05-09T08:04:05.382 INFO:teuthology.orchestra.run.smithi001.stdout: "release": "luminous", 2024-05-09T08:04:05.383 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 1 2024-05-09T08:04:05.383 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:04:05.383 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:04:05.383 INFO:teuthology.orchestra.run.smithi001.stdout: "features": "0x3f03cffffffdffff", 2024-05-09T08:04:05.383 INFO:teuthology.orchestra.run.smithi001.stdout: "release": "luminous", 2024-05-09T08:04:05.383 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 6 2024-05-09T08:04:05.383 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:04:05.383 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:04:05.383 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr": [ 2024-05-09T08:04:05.383 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:04:05.383 INFO:teuthology.orchestra.run.smithi001.stdout: "features": "0x3f03cffffffdffff", 2024-05-09T08:04:05.383 INFO:teuthology.orchestra.run.smithi001.stdout: "release": "luminous", 2024-05-09T08:04:05.383 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 2 2024-05-09T08:04:05.384 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:04:05.384 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:04:05.384 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:04:05.384 INFO:teuthology.orchestra.run.smithi001.stdout: "stretch_mode": false 2024-05-09T08:04:05.384 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-05-09T08:04:05.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.354+0000 7f254e800640 1 -- 172.21.15.1:0/2246223855 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f252806be60 msgr2=0x7f252806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:04:05.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.354+0000 7f254e800640 1 --2- 172.21.15.1:0/2246223855 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f252806be60 0x7f252806e320 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f2540007fd0 tx=0x7f2540009040 comp rx=0 tx=0).stop 2024-05-09T08:04:05.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.354+0000 7f254e800640 1 -- 172.21.15.1:0/2246223855 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f25480c5890 msgr2=0x7f254814e170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:04:05.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.354+0000 7f254e800640 1 --2- 172.21.15.1:0/2246223855 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f25480c5890 0x7f254814e170 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f25480701c0 tx=0x7f25340046c0 comp rx=0 tx=0).stop 2024-05-09T08:04:05.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.354+0000 7f254de00640 1 -- 172.21.15.1:0/2246223855 reap_dead start 2024-05-09T08:04:05.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.354+0000 7f254de00640 1 -- 172.21.15.1:0/2246223855 reap_dead start 2024-05-09T08:04:05.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.354+0000 7f254e800640 1 -- 172.21.15.1:0/2246223855 shutdown_connections 2024-05-09T08:04:05.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.354+0000 7f254e800640 1 -- 172.21.15.1:0/2246223855 >> 172.21.15.1:0/2246223855 conn(0x7f254806b250 msgr2=0x7f254806b630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:04:05.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.354+0000 7f254e800640 1 -- 172.21.15.1:0/2246223855 shutdown_connections 2024-05-09T08:04:05.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.354+0000 7f254e800640 1 -- 172.21.15.1:0/2246223855 wait complete. 2024-05-09T08:04:05.391 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-05-09T08:04:05.391 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['a'] 2024-05-09T08:04:05.391 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2024-05-09T08:04:05.391 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.a 2024-05-09T08:04:05.391 INFO:tasks.mon_thrash.mon_thrasher:killing mon.a 2024-05-09T08:04:05.391 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-05-09T08:04:05.392 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T08:04:05.417 INFO:tasks.ceph.mon.a:Stopped 2024-05-09T08:04:05.417 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2024-05-09T08:04:05.418 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-05-09T08:04:05.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.620+0000 7f9bd7a00640 1 Processor -- start 2024-05-09T08:04:05.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.620+0000 7f9bd7a00640 1 -- start start 2024-05-09T08:04:05.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.622+0000 7f9bd7a00640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f9bd00b1550 0x7f9bd00aeb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:04:05.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.622+0000 7f9bd7a00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f9bd00cc070 0x7f9bd00cc500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:04:05.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.622+0000 7f9bd7a00640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9bd015a520 con 0x7f9bd0154a10 2024-05-09T08:04:05.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.622+0000 7f9bd7a00640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f9bd00cce60 con 0x7f9bd00b1550 2024-05-09T08:04:05.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.622+0000 7f9bd7a00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f9bd015a7e0 con 0x7f9bd00cc070 2024-05-09T08:04:05.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.622+0000 7f9bd5000640 1 -- >> v1:172.21.15.1:6789/0 conn(0x7f9bd0154a10 legacy=0x7f9bd0072570 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.1:6789/0 2024-05-09T08:04:05.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.622+0000 7f9bcfe00640 1 -- >> v2:172.21.15.1:3300/0 conn(0x7f9bd00b1550 msgr2=0x7f9bd00aeb20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.1:3300/0 2024-05-09T08:04:05.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.622+0000 7f9bcfe00640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f9bd00b1550 0x7f9bd00aeb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T08:04:05.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.623+0000 7f9bd5a00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f9bd00cc070 0x7f9bd00cc500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:04:05.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.623+0000 7f9bd5a00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f9bd00cc070 0x7f9bd00cc500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.1:33868/0 (socket says 172.21.15.1:33868) 2024-05-09T08:04:05.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.623+0000 7f9bd5a00640 1 -- 172.21.15.1:0/974717186 learned_addr learned my addr 172.21.15.1:0/974717186 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:04:05.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.623+0000 7f9bd5a00640 1 -- 172.21.15.1:0/974717186 >> v2:172.21.15.1:3300/0 conn(0x7f9bd00b1550 msgr2=0x7f9bd00aeb20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T08:04:05.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.623+0000 7f9bd5a00640 1 --2- 172.21.15.1:0/974717186 >> v2:172.21.15.1:3300/0 conn(0x7f9bd00b1550 0x7f9bd00aeb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:04:05.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.623+0000 7f9bd5a00640 1 -- 172.21.15.1:0/974717186 >> v1:172.21.15.1:6789/0 conn(0x7f9bd0154a10 legacy=0x7f9bd0072570 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T08:04:05.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.623+0000 7f9bd5a00640 1 -- 172.21.15.1:0/974717186 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bd015aac0 con 0x7f9bd00cc070 2024-05-09T08:04:05.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.625+0000 7f9bd5a00640 1 --2- 172.21.15.1:0/974717186 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9bd00cc070 0x7f9bd00cc500 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f9bd0070a20 tx=0x7f9bc802a7a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=923b5effa6c895e5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:04:05.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.625+0000 7f9bcf400640 1 -- 172.21.15.1:0/974717186 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bc8036070 con 0x7f9bd00cc070 2024-05-09T08:04:05.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.625+0000 7f9bcf400640 1 -- 172.21.15.1:0/974717186 <== mon.1 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9bc8031600 con 0x7f9bd00cc070 2024-05-09T08:04:05.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.625+0000 7f9bcf400640 1 -- 172.21.15.1:0/974717186 <== mon.1 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bc80319d0 con 0x7f9bd00cc070 2024-05-09T08:04:05.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.626+0000 7f9bd7a00640 1 -- 172.21.15.1:0/974717186 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9bd00cc070 msgr2=0x7f9bd00cc500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:04:05.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.626+0000 7f9bd7a00640 1 --2- 172.21.15.1:0/974717186 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9bd00cc070 0x7f9bd00cc500 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f9bd0070a20 tx=0x7f9bc802a7a0 comp rx=0 tx=0).stop 2024-05-09T08:04:05.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.626+0000 7f9bd7a00640 1 -- 172.21.15.1:0/974717186 shutdown_connections 2024-05-09T08:04:05.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.626+0000 7f9bd7a00640 1 --2- 172.21.15.1:0/974717186 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9bd00cc070 0x7f9bd00cc500 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:04:05.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.626+0000 7f9bd7a00640 1 --2- 172.21.15.1:0/974717186 >> v2:172.21.15.1:3300/0 conn(0x7f9bd00b1550 0x7f9bd00aeb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:04:05.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.626+0000 7f9bd7a00640 1 -- 172.21.15.1:0/974717186 >> 172.21.15.1:0/974717186 conn(0x7f9bd006f9b0 msgr2=0x7f9bd006fdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:04:05.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.626+0000 7f9bd7a00640 1 -- 172.21.15.1:0/974717186 shutdown_connections 2024-05-09T08:04:05.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.626+0000 7f9bd7a00640 1 -- 172.21.15.1:0/974717186 wait complete. 2024-05-09T08:04:05.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.628+0000 7f9bd7a00640 1 Processor -- start 2024-05-09T08:04:05.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.628+0000 7f9bd7a00640 1 -- start start 2024-05-09T08:04:05.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.629+0000 7f9bd7a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9bd00b1550 0x7f9bd01f4150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:04:05.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.630+0000 7f9bd7a00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9bd00cc070 0x7f9bd00b0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:04:05.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.630+0000 7f9bd7a00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f9bd0154a10 0x7f9bd0151b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:04:05.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.630+0000 7f9bd7a00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f9bd00b1b10 con 0x7f9bd00cc070 2024-05-09T08:04:05.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.630+0000 7f9bd7a00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9bd015b080 con 0x7f9bd00b1550 2024-05-09T08:04:05.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.630+0000 7f9bd7a00640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f9bd0072c80 con 0x7f9bd0154a10 2024-05-09T08:04:05.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.630+0000 7f9bcfe00640 1 -- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9bd00cc070 msgr2=0x7f9bd00b0bd0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.1:3300/0 2024-05-09T08:04:05.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.630+0000 7f9bcfe00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9bd00cc070 0x7f9bd00b0bd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T08:04:05.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.630+0000 7f9bd5000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9bd00b1550 0x7f9bd01f4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:04:05.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.630+0000 7f9bd5a00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f9bd0154a10 0x7f9bd0151b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:04:05.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.630+0000 7f9bd5000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9bd00b1550 0x7f9bd01f4150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.1:33882/0 (socket says 172.21.15.1:33882) 2024-05-09T08:04:05.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.630+0000 7f9bd5a00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f9bd0154a10 0x7f9bd0151b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3301/0 says I am v2:172.21.15.1:57792/0 (socket says 172.21.15.1:57792) 2024-05-09T08:04:05.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.630+0000 7f9bd5000640 1 -- 172.21.15.1:0/1762038063 learned_addr learned my addr 172.21.15.1:0/1762038063 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:04:05.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.631+0000 7f9bd5000640 1 -- 172.21.15.1:0/1762038063 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f9bd0154a10 msgr2=0x7f9bd0151b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:04:05.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.631+0000 7f9bd5000640 1 --2- 172.21.15.1:0/1762038063 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f9bd0154a10 0x7f9bd0151b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:04:05.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.631+0000 7f9bd5000640 1 -- 172.21.15.1:0/1762038063 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9bd00cc070 msgr2=0x7f9bd00b0bd0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T08:04:05.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.631+0000 7f9bd5000640 1 --2- 172.21.15.1:0/1762038063 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9bd00cc070 0x7f9bd00b0bd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:04:05.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.631+0000 7f9bd5000640 1 -- 172.21.15.1:0/1762038063 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bd00b0610 con 0x7f9bd00b1550 2024-05-09T08:04:05.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.631+0000 7f9bd5000640 1 --2- 172.21.15.1:0/1762038063 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9bd00b1550 0x7f9bd01f4150 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f9bc400b550 tx=0x7f9bc400ba10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:04:05.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.631+0000 7f9bccc00640 1 -- 172.21.15.1:0/1762038063 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bc4003dc0 con 0x7f9bd00b1550 2024-05-09T08:04:05.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.631+0000 7f9bccc00640 1 -- 172.21.15.1:0/1762038063 <== mon.1 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9bc4015070 con 0x7f9bd00b1550 2024-05-09T08:04:05.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.631+0000 7f9bd7a00640 1 -- 172.21.15.1:0/1762038063 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9bd00b08f0 con 0x7f9bd00b1550 2024-05-09T08:04:05.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.631+0000 7f9bccc00640 1 -- 172.21.15.1:0/1762038063 <== mon.1 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bc40052b0 con 0x7f9bd00b1550 2024-05-09T08:04:05.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.632+0000 7f9bd7a00640 1 -- 172.21.15.1:0/1762038063 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9bd01e4bc0 con 0x7f9bd00b1550 2024-05-09T08:04:05.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.633+0000 7f9bd7a00640 1 -- 172.21.15.1:0/1762038063 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9bd0074d70 con 0x7f9bd00b1550 2024-05-09T08:04:05.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.634+0000 7f9bccc00640 1 -- 172.21.15.1:0/1762038063 <== mon.1 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9bc4005450 con 0x7f9bd00b1550 2024-05-09T08:04:05.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.635+0000 7f9bccc00640 1 --2- 172.21.15.1:0/1762038063 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f9bb006be30 0x7f9bb006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:04:05.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.635+0000 7f9bccc00640 1 -- 172.21.15.1:0/1762038063 <== mon.1 v2:172.21.15.7:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7f9bc408af80 con 0x7f9bd00b1550 2024-05-09T08:04:05.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.635+0000 7f9bcfe00640 1 --2- 172.21.15.1:0/1762038063 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f9bb006be30 0x7f9bb006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:04:05.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.636+0000 7f9bcfe00640 1 --2- 172.21.15.1:0/1762038063 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f9bb006be30 0x7f9bb006e2f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f9bd00b0570 tx=0x7f9bbc006d30 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:04:05.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.639+0000 7f9bccc00640 1 -- 172.21.15.1:0/1762038063 <== mon.1 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f9bc4059520 con 0x7f9bd00b1550 2024-05-09T08:04:05.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.942+0000 7f9bd7a00640 1 -- 172.21.15.1:0/1762038063 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f9bd00ad8d0 con 0x7f9bd00b1550 2024-05-09T08:04:05.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.943+0000 7f9bccc00640 1 -- 172.21.15.1:0/1762038063 <== mon.1 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1545 (secure 0 0 0) 0x7f9bc408a070 con 0x7f9bd00b1550 2024-05-09T08:04:05.959 INFO:teuthology.orchestra.run.smithi001.stdout:{"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":13,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"54329ec8-3270-4043-9a82-c6cd7420aa7f","modified":"2024-05-09T08:01:21.819270Z","created":"2024-05-09T08:01:21.819270Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:3300","nonce":0},{"type":"v1","addr":"172.21.15.1:6789","nonce":0}]},"addr":"172.21.15.1:6789/0","public_addr":"172.21.15.1:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:3300","nonce":0},{"type":"v1","addr":"172.21.15.7:6789","nonce":0}]},"addr":"172.21.15.7:6789/0","public_addr":"172.21.15.7:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:3301","nonce":0},{"type":"v1","addr":"172.21.15.7:6790","nonce":0}]},"addr":"172.21.15.7:6790/0","public_addr":"172.21.15.7:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-05-09T08:04:05.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.945+0000 7f9bd7a00640 1 -- 172.21.15.1:0/1762038063 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f9bb006be30 msgr2=0x7f9bb006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:04:05.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.945+0000 7f9bd7a00640 1 --2- 172.21.15.1:0/1762038063 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f9bb006be30 0x7f9bb006e2f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f9bd00b0570 tx=0x7f9bbc006d30 comp rx=0 tx=0).stop 2024-05-09T08:04:05.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.945+0000 7f9bd7a00640 1 -- 172.21.15.1:0/1762038063 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9bd00b1550 msgr2=0x7f9bd01f4150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:04:05.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.945+0000 7f9bd7a00640 1 --2- 172.21.15.1:0/1762038063 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9bd00b1550 0x7f9bd01f4150 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f9bc400b550 tx=0x7f9bc400ba10 comp rx=0 tx=0).stop 2024-05-09T08:04:05.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.945+0000 7f9bd7a00640 1 -- 172.21.15.1:0/1762038063 shutdown_connections 2024-05-09T08:04:05.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.945+0000 7f9bd7a00640 1 --2- 172.21.15.1:0/1762038063 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f9bb006be30 0x7f9bb006e2f0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:04:05.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.945+0000 7f9bd7a00640 1 --2- 172.21.15.1:0/1762038063 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f9bd0154a10 0x7f9bd0151b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:04:05.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.945+0000 7f9bd7a00640 1 --2- 172.21.15.1:0/1762038063 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9bd00cc070 0x7f9bd00b0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:04:05.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.945+0000 7f9bd7a00640 1 --2- 172.21.15.1:0/1762038063 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9bd00b1550 0x7f9bd01f4150 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:04:05.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.945+0000 7f9bd7a00640 1 -- 172.21.15.1:0/1762038063 >> 172.21.15.1:0/1762038063 conn(0x7f9bd006f9b0 msgr2=0x7f9bd0153f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:04:05.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.945+0000 7f9bd7a00640 1 -- 172.21.15.1:0/1762038063 shutdown_connections 2024-05-09T08:04:05.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:05.945+0000 7f9bd7a00640 1 -- 172.21.15.1:0/1762038063 wait complete. 2024-05-09T08:04:08.983 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-05-09T08:04:09.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:09.174+0000 7fab0a800640 1 Processor -- start 2024-05-09T08:04:09.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:09.174+0000 7fab0a800640 1 -- start start 2024-05-09T08:04:09.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:09.176+0000 7fab0a800640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fab0415ba50 0x7fab0406d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:04:09.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:09.176+0000 7fab0a800640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fab0415a5e0 0x7fab040ca5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:04:09.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:09.176+0000 7fab0a800640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fab0415b550 con 0x7fab040cd040 2024-05-09T08:04:09.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:09.176+0000 7fab0a800640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7fab0406cec0 con 0x7fab0415a5e0 2024-05-09T08:04:09.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:09.176+0000 7fab0a800640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fab0406c8b0 con 0x7fab0415ba50 2024-05-09T08:04:09.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:09.176+0000 7fab09400640 1 -- >> v1:172.21.15.1:6789/0 conn(0x7fab040cd040 legacy=0x7fab0406da60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.1:6789/0 2024-05-09T08:04:09.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:09.176+0000 7fab09e00640 1 -- >> v2:172.21.15.1:3300/0 conn(0x7fab0415a5e0 msgr2=0x7fab040ca5f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.1:3300/0 2024-05-09T08:04:09.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:09.176+0000 7fab08a00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fab0415ba50 0x7fab0406d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:04:09.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:09.176+0000 7fab09e00640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fab0415a5e0 0x7fab040ca5f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T08:04:09.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:09.177+0000 7fab08a00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fab0415ba50 0x7fab0406d350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.1:33890/0 (socket says 172.21.15.1:33890) 2024-05-09T08:04:09.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:09.177+0000 7fab08a00640 1 -- 172.21.15.1:0/3526104712 learned_addr learned my addr 172.21.15.1:0/3526104712 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:04:09.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:09.177+0000 7fab08a00640 1 -- 172.21.15.1:0/3526104712 >> v2:172.21.15.1:3300/0 conn(0x7fab0415a5e0 msgr2=0x7fab040ca5f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T08:04:09.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:09.177+0000 7fab08a00640 1 --2- 172.21.15.1:0/3526104712 >> v2:172.21.15.1:3300/0 conn(0x7fab0415a5e0 0x7fab040ca5f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:04:09.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:09.177+0000 7fab08a00640 1 -- 172.21.15.1:0/3526104712 >> v1:172.21.15.1:6789/0 conn(0x7fab040cd040 legacy=0x7fab0406da60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T08:04:09.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:09.177+0000 7fab08a00640 1 -- 172.21.15.1:0/3526104712 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab0415af90 con 0x7fab0415ba50 2024-05-09T08:04:09.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:09.179+0000 7fab08a00640 1 --2- 172.21.15.1:0/3526104712 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab0415ba50 0x7fab0406d350 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fab0415ec10 tx=0x7faaf002a130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bb625ab8feddb1e0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:04:09.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:09.179+0000 7fab03e00640 1 -- 172.21.15.1:0/3526104712 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaf0037070 con 0x7fab0415ba50 2024-05-09T08:04:09.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:09.179+0000 7fab03e00640 1 -- 172.21.15.1:0/3526104712 <== mon.1 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faaf002ae40 con 0x7fab0415ba50 2024-05-09T08:04:09.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:09.179+0000 7fab03e00640 1 -- 172.21.15.1:0/3526104712 <== mon.1 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaf0032660 con 0x7fab0415ba50 2024-05-09T08:04:09.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:09.180+0000 7fab0a800640 1 -- 172.21.15.1:0/3526104712 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab0415ba50 msgr2=0x7fab0406d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:04:09.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:09.180+0000 7fab0a800640 1 --2- 172.21.15.1:0/3526104712 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab0415ba50 0x7fab0406d350 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fab0415ec10 tx=0x7faaf002a130 comp rx=0 tx=0).stop 2024-05-09T08:04:09.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:09.180+0000 7fab0a800640 1 -- 172.21.15.1:0/3526104712 shutdown_connections 2024-05-09T08:04:09.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:09.180+0000 7fab0a800640 1 --2- 172.21.15.1:0/3526104712 >> v2:172.21.15.1:3300/0 conn(0x7fab0415a5e0 0x7fab040ca5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:04:09.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:09.180+0000 7fab0a800640 1 --2- 172.21.15.1:0/3526104712 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab0415ba50 0x7fab0406d350 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:04:09.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:09.180+0000 7fab0a800640 1 -- 172.21.15.1:0/3526104712 >> 172.21.15.1:0/3526104712 conn(0x7fab040b1780 msgr2=0x7fab040b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:04:09.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:09.180+0000 7fab0a800640 1 -- 172.21.15.1:0/3526104712 shutdown_connections 2024-05-09T08:04:09.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:09.180+0000 7fab0a800640 1 -- 172.21.15.1:0/3526104712 wait complete. 2024-05-09T08:04:09.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:09.182+0000 7fab0a800640 1 Processor -- start 2024-05-09T08:04:09.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:09.183+0000 7fab0a800640 1 -- start start 2024-05-09T08:04:09.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:09.183+0000 7fab0a800640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fab040cd040 0x7fab040cdda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:04:09.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:09.183+0000 7fab0a800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab0415a5e0 0x7fab040ce2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:04:09.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:09.184+0000 7fab0a800640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fab0415ba50 0x7fab040c43c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:04:09.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:09.184+0000 7fab0a800640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fab0415aa50 con 0x7fab0415ba50 2024-05-09T08:04:09.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:09.184+0000 7fab0a800640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fab0406cb50 con 0x7fab0415a5e0 2024-05-09T08:04:09.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:09.184+0000 7fab0a800640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7fab0415abd0 con 0x7fab040cd040 2024-05-09T08:04:09.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:09.184+0000 7fab09e00640 1 -- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fab0415ba50 msgr2=0x7fab040c43c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.1:3300/0 2024-05-09T08:04:09.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:09.184+0000 7fab09e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fab0415ba50 0x7fab040c43c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T08:04:09.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:09.184+0000 7fab09400640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fab040cd040 0x7fab040cdda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:04:09.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:09.184+0000 7fab09400640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fab040cd040 0x7fab040cdda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3301/0 says I am v2:172.21.15.1:57806/0 (socket says 172.21.15.1:57806) 2024-05-09T08:04:09.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:09.184+0000 7fab09400640 1 -- 172.21.15.1:0/3682821909 learned_addr learned my addr 172.21.15.1:0/3682821909 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:04:09.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:09.184+0000 7fab08a00640 1 --2- 172.21.15.1:0/3682821909 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab0415a5e0 0x7fab040ce2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:04:09.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:09.184+0000 7fab09400640 1 -- 172.21.15.1:0/3682821909 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab0415a5e0 msgr2=0x7fab040ce2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:04:09.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:09.184+0000 7fab09400640 1 --2- 172.21.15.1:0/3682821909 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab0415a5e0 0x7fab040ce2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:04:09.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:09.184+0000 7fab09400640 1 -- 172.21.15.1:0/3682821909 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fab0415ba50 msgr2=0x7fab040c43c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T08:04:09.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:09.184+0000 7fab09400640 1 --2- 172.21.15.1:0/3682821909 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fab0415ba50 0x7fab040c43c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:04:09.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:09.184+0000 7fab09400640 1 -- 172.21.15.1:0/3682821909 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab04158020 con 0x7fab040cd040 2024-05-09T08:04:09.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:09.185+0000 7fab09400640 1 --2- 172.21.15.1:0/3682821909 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fab040cd040 0x7fab040cdda0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7faaf800ba10 tx=0x7faaf800bed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:04:09.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:09.185+0000 7fab01600640 1 -- 172.21.15.1:0/3682821909 <== mon.2 v2:172.21.15.7:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaf800cbd0 con 0x7fab040cd040 2024-05-09T08:04:09.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:09.185+0000 7fab01600640 1 -- 172.21.15.1:0/3682821909 <== mon.2 v2:172.21.15.7:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faaf8010070 con 0x7fab040cd040 2024-05-09T08:04:09.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:09.185+0000 7fab01600640 1 -- 172.21.15.1:0/3682821909 <== mon.2 v2:172.21.15.7:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaf8015470 con 0x7fab040cd040 2024-05-09T08:04:09.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:09.185+0000 7fab0a800640 1 -- 172.21.15.1:0/3682821909 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab041f1360 con 0x7fab040cd040 2024-05-09T08:04:09.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:09.185+0000 7fab0a800640 1 -- 172.21.15.1:0/3682821909 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fab04157d40 con 0x7fab040cd040 2024-05-09T08:04:09.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:09.187+0000 7fab0a800640 1 -- 172.21.15.1:0/3682821909 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab0404f660 con 0x7fab040cd040 2024-05-09T08:04:09.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:09.188+0000 7fab01600640 1 -- 172.21.15.1:0/3682821909 <== mon.2 v2:172.21.15.7:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7faaf801e3f0 con 0x7fab040cd040 2024-05-09T08:04:09.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:09.190+0000 7fab01600640 1 --2- 172.21.15.1:0/3682821909 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7faae406be10 0x7faae406e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:04:09.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:09.190+0000 7fab01600640 1 -- 172.21.15.1:0/3682821909 <== mon.2 v2:172.21.15.7:3301/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7faaf808b420 con 0x7fab040cd040 2024-05-09T08:04:09.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:09.190+0000 7fab08a00640 1 --2- 172.21.15.1:0/3682821909 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7faae406be10 0x7faae406e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:04:09.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:09.191+0000 7fab08a00640 1 --2- 172.21.15.1:0/3682821909 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7faae406be10 0x7faae406e2d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7faaf0034000 tx=0x7faaf002e000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:04:09.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:09.194+0000 7fab01600640 1 -- 172.21.15.1:0/3682821909 <== mon.2 v2:172.21.15.7:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7faaf80599d0 con 0x7fab040cd040 2024-05-09T08:04:09.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:09.520+0000 7fab0a800640 1 -- 172.21.15.1:0/3682821909 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fab040cda50 con 0x7fab040cd040 2024-05-09T08:04:09.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:09.521+0000 7fab01600640 1 -- 172.21.15.1:0/3682821909 <== mon.2 v2:172.21.15.7:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1545 (secure 0 0 0) 0x7faaf80592d0 con 0x7fab040cd040 2024-05-09T08:04:09.537 INFO:teuthology.orchestra.run.smithi001.stdout:{"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":16,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"54329ec8-3270-4043-9a82-c6cd7420aa7f","modified":"2024-05-09T08:01:21.819270Z","created":"2024-05-09T08:01:21.819270Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:3300","nonce":0},{"type":"v1","addr":"172.21.15.1:6789","nonce":0}]},"addr":"172.21.15.1:6789/0","public_addr":"172.21.15.1:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:3300","nonce":0},{"type":"v1","addr":"172.21.15.7:6789","nonce":0}]},"addr":"172.21.15.7:6789/0","public_addr":"172.21.15.7:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:3301","nonce":0},{"type":"v1","addr":"172.21.15.7:6790","nonce":0}]},"addr":"172.21.15.7:6790/0","public_addr":"172.21.15.7:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-05-09T08:04:09.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:09.524+0000 7fab0a800640 1 -- 172.21.15.1:0/3682821909 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7faae406be10 msgr2=0x7faae406e2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:04:09.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:09.524+0000 7fab0a800640 1 --2- 172.21.15.1:0/3682821909 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7faae406be10 0x7faae406e2d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7faaf0034000 tx=0x7faaf002e000 comp rx=0 tx=0).stop 2024-05-09T08:04:09.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:09.524+0000 7fab0a800640 1 -- 172.21.15.1:0/3682821909 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fab040cd040 msgr2=0x7fab040cdda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:04:09.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:09.524+0000 7fab0a800640 1 --2- 172.21.15.1:0/3682821909 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fab040cd040 0x7fab040cdda0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7faaf800ba10 tx=0x7faaf800bed0 comp rx=0 tx=0).stop 2024-05-09T08:04:09.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:09.525+0000 7fab0a800640 1 -- 172.21.15.1:0/3682821909 shutdown_connections 2024-05-09T08:04:09.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:09.525+0000 7fab0a800640 1 --2- 172.21.15.1:0/3682821909 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7faae406be10 0x7faae406e2d0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:04:09.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:09.525+0000 7fab0a800640 1 --2- 172.21.15.1:0/3682821909 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab0415a5e0 0x7fab040ce2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:04:09.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:09.525+0000 7fab0a800640 1 --2- 172.21.15.1:0/3682821909 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fab0415ba50 0x7fab040c43c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:04:09.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:09.525+0000 7fab0a800640 1 --2- 172.21.15.1:0/3682821909 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fab040cd040 0x7fab040cdda0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:04:09.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:09.525+0000 7fab0a800640 1 -- 172.21.15.1:0/3682821909 >> 172.21.15.1:0/3682821909 conn(0x7fab040b1780 msgr2=0x7fab040c6600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:04:09.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:09.525+0000 7fab0a800640 1 -- 172.21.15.1:0/3682821909 shutdown_connections 2024-05-09T08:04:09.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:09.525+0000 7fab0a800640 1 -- 172.21.15.1:0/3682821909 wait complete. 2024-05-09T08:04:12.567 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-05-09T08:04:12.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:12.748+0000 7ff37a800640 1 Processor -- start 2024-05-09T08:04:12.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:12.748+0000 7ff37a800640 1 -- start start 2024-05-09T08:04:12.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:12.749+0000 7ff37a800640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff374152500 0x7ff3740c8180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:04:12.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:12.750+0000 7ff37a800640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7ff3740c7660 0x7ff3740c7ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:04:12.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:12.750+0000 7ff37a800640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff37415c790 con 0x7ff3740cc730 2024-05-09T08:04:12.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:12.750+0000 7ff37a800640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7ff3740c5e50 con 0x7ff3740c7660 2024-05-09T08:04:12.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:12.750+0000 7ff37a800640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7ff3740c22d0 con 0x7ff374152500 2024-05-09T08:04:12.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:12.750+0000 7ff379e00640 1 -- >> v2:172.21.15.1:3300/0 conn(0x7ff3740c7660 msgr2=0x7ff3740c7ab0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.1:3300/0 2024-05-09T08:04:12.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:12.750+0000 7ff379e00640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7ff3740c7660 0x7ff3740c7ab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T08:04:12.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:12.750+0000 7ff379400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff374152500 0x7ff3740c8180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:04:12.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:12.750+0000 7ff378a00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7ff3740cc730 0x7ff37415c080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.1:36402/0 (socket says 172.21.15.1:36402) 2024-05-09T08:04:12.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:12.750+0000 7ff378a00640 1 -- 172.21.15.1:0/744852471 learned_addr learned my addr 172.21.15.1:0/744852471 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T08:04:12.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:12.750+0000 7ff379400640 1 -- 172.21.15.1:0/744852471 >> v2:172.21.15.1:3300/0 conn(0x7ff3740c7660 msgr2=0x7ff3740c7ab0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T08:04:12.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:12.750+0000 7ff379400640 1 --2- 172.21.15.1:0/744852471 >> v2:172.21.15.1:3300/0 conn(0x7ff3740c7660 0x7ff3740c7ab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:04:12.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:12.750+0000 7ff379400640 1 -- 172.21.15.1:0/744852471 >> v1:172.21.15.7:6789/0 conn(0x7ff3740cc730 legacy=0x7ff37415c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:04:12.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:12.751+0000 7ff379400640 1 -- 172.21.15.1:0/744852471 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3740c6a30 con 0x7ff374152500 2024-05-09T08:04:12.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:12.752+0000 7ff379400640 1 --2- 172.21.15.1:0/744852471 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff374152500 0x7ff3740c8180 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7ff37406eb20 tx=0x7ff36802a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c4c6270a4744c138 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:04:12.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:12.752+0000 7ff373e00640 1 -- 172.21.15.1:0/744852471 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff368037070 con 0x7ff374152500 2024-05-09T08:04:12.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:12.752+0000 7ff373e00640 1 -- 172.21.15.1:0/744852471 <== mon.1 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff368032440 con 0x7ff374152500 2024-05-09T08:04:12.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:12.752+0000 7ff373e00640 1 -- 172.21.15.1:0/744852471 <== mon.1 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff368032810 con 0x7ff374152500 2024-05-09T08:04:12.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:12.753+0000 7ff37a800640 1 -- 172.21.15.1:0/744852471 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff374152500 msgr2=0x7ff3740c8180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:04:12.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:12.753+0000 7ff37a800640 1 --2- 172.21.15.1:0/744852471 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff374152500 0x7ff3740c8180 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7ff37406eb20 tx=0x7ff36802a270 comp rx=0 tx=0).stop 2024-05-09T08:04:12.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:12.753+0000 7ff37a800640 1 -- 172.21.15.1:0/744852471 shutdown_connections 2024-05-09T08:04:12.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:12.753+0000 7ff37a800640 1 --2- 172.21.15.1:0/744852471 >> v2:172.21.15.1:3300/0 conn(0x7ff3740c7660 0x7ff3740c7ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:04:12.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:12.753+0000 7ff37a800640 1 --2- 172.21.15.1:0/744852471 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff374152500 0x7ff3740c8180 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:04:12.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:12.753+0000 7ff37a800640 1 -- 172.21.15.1:0/744852471 >> 172.21.15.1:0/744852471 conn(0x7ff3740cbc10 msgr2=0x7ff3740cc020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:04:12.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:12.753+0000 7ff37a800640 1 -- 172.21.15.1:0/744852471 shutdown_connections 2024-05-09T08:04:12.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:12.753+0000 7ff37a800640 1 -- 172.21.15.1:0/744852471 wait complete. 2024-05-09T08:04:12.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:12.754+0000 7ff37a800640 1 Processor -- start 2024-05-09T08:04:12.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:12.755+0000 7ff37a800640 1 -- start start 2024-05-09T08:04:12.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:12.755+0000 7ff37a800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff3740c7660 0x7ff374148240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:04:12.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:12.755+0000 7ff37a800640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ff3740cc730 0x7ff3741f07b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:04:12.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:12.755+0000 7ff37a800640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff374152500 0x7ff374149050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:04:12.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:12.756+0000 7ff37a800640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ff3740c6fd0 con 0x7ff374152500 2024-05-09T08:04:12.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:12.756+0000 7ff37a800640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff3740ccb50 con 0x7ff3740c7660 2024-05-09T08:04:12.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:12.756+0000 7ff37a800640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7ff3740c72b0 con 0x7ff3740cc730 2024-05-09T08:04:12.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:12.756+0000 7ff379e00640 1 -- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff374152500 msgr2=0x7ff374149050 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.1:3300/0 2024-05-09T08:04:12.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:12.756+0000 7ff379e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff374152500 0x7ff374149050 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T08:04:12.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:12.756+0000 7ff379400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff3740c7660 0x7ff374148240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:04:12.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:12.756+0000 7ff379400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff3740c7660 0x7ff374148240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.1:33926/0 (socket says 172.21.15.1:33926) 2024-05-09T08:04:12.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:12.756+0000 7ff379400640 1 -- 172.21.15.1:0/1630470807 learned_addr learned my addr 172.21.15.1:0/1630470807 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:04:12.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:12.756+0000 7ff378a00640 1 --2- 172.21.15.1:0/1630470807 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ff3740cc730 0x7ff3741f07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:04:12.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:12.756+0000 7ff379400640 1 -- 172.21.15.1:0/1630470807 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ff3740cc730 msgr2=0x7ff3741f07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:04:12.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:12.756+0000 7ff379400640 1 --2- 172.21.15.1:0/1630470807 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ff3740cc730 0x7ff3741f07b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:04:12.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:12.756+0000 7ff379400640 1 -- 172.21.15.1:0/1630470807 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff374152500 msgr2=0x7ff374149050 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T08:04:12.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:12.756+0000 7ff379400640 1 --2- 172.21.15.1:0/1630470807 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff374152500 0x7ff374149050 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:04:12.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:12.756+0000 7ff379400640 1 -- 172.21.15.1:0/1630470807 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff374158c20 con 0x7ff3740c7660 2024-05-09T08:04:12.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:12.757+0000 7ff379400640 1 --2- 172.21.15.1:0/1630470807 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff3740c7660 0x7ff374148240 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ff368009890 tx=0x7ff3680046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:04:12.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:12.757+0000 7ff371600640 1 -- 172.21.15.1:0/1630470807 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff368041070 con 0x7ff3740c7660 2024-05-09T08:04:12.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:12.757+0000 7ff371600640 1 -- 172.21.15.1:0/1630470807 <== mon.1 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff36803c950 con 0x7ff3740c7660 2024-05-09T08:04:12.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:12.757+0000 7ff371600640 1 -- 172.21.15.1:0/1630470807 <== mon.1 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff368037040 con 0x7ff3740c7660 2024-05-09T08:04:12.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:12.757+0000 7ff37a800640 1 -- 172.21.15.1:0/1630470807 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff374158ee0 con 0x7ff3740c7660 2024-05-09T08:04:12.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:12.757+0000 7ff37a800640 1 -- 172.21.15.1:0/1630470807 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3741effb0 con 0x7ff3740c7660 2024-05-09T08:04:12.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:12.759+0000 7ff37a800640 1 -- 172.21.15.1:0/1630470807 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3740c2a40 con 0x7ff3740c7660 2024-05-09T08:04:12.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:12.760+0000 7ff371600640 1 -- 172.21.15.1:0/1630470807 <== mon.1 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff368044070 con 0x7ff3740c7660 2024-05-09T08:04:12.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:12.760+0000 7ff371600640 1 --2- 172.21.15.1:0/1630470807 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7ff35406bde0 0x7ff35406e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:04:12.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:12.761+0000 7ff371600640 1 -- 172.21.15.1:0/1630470807 <== mon.1 v2:172.21.15.7:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7ff3680abf80 con 0x7ff3740c7660 2024-05-09T08:04:12.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:12.761+0000 7ff378a00640 1 --2- 172.21.15.1:0/1630470807 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7ff35406bde0 0x7ff35406e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:04:12.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:12.761+0000 7ff378a00640 1 --2- 172.21.15.1:0/1630470807 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7ff35406bde0 0x7ff35406e2a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7ff3740cb920 tx=0x7ff360007e80 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:04:12.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:12.765+0000 7ff371600640 1 -- 172.21.15.1:0/1630470807 <== mon.1 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ff36807a000 con 0x7ff3740c7660 2024-05-09T08:04:13.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:13.034+0000 7ff37a800640 1 -- 172.21.15.1:0/1630470807 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7ff37404f640 con 0x7ff3740c7660 2024-05-09T08:04:13.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:13.034+0000 7ff371600640 1 -- 172.21.15.1:0/1630470807 <== mon.1 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1545 (secure 0 0 0) 0x7ff36807e300 con 0x7ff3740c7660 2024-05-09T08:04:13.050 INFO:teuthology.orchestra.run.smithi001.stdout:{"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":20,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"54329ec8-3270-4043-9a82-c6cd7420aa7f","modified":"2024-05-09T08:01:21.819270Z","created":"2024-05-09T08:01:21.819270Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:3300","nonce":0},{"type":"v1","addr":"172.21.15.1:6789","nonce":0}]},"addr":"172.21.15.1:6789/0","public_addr":"172.21.15.1:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:3300","nonce":0},{"type":"v1","addr":"172.21.15.7:6789","nonce":0}]},"addr":"172.21.15.7:6789/0","public_addr":"172.21.15.7:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:3301","nonce":0},{"type":"v1","addr":"172.21.15.7:6790","nonce":0}]},"addr":"172.21.15.7:6790/0","public_addr":"172.21.15.7:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-05-09T08:04:13.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:13.037+0000 7ff37a800640 1 -- 172.21.15.1:0/1630470807 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7ff35406bde0 msgr2=0x7ff35406e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:04:13.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:13.037+0000 7ff37a800640 1 --2- 172.21.15.1:0/1630470807 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7ff35406bde0 0x7ff35406e2a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7ff3740cb920 tx=0x7ff360007e80 comp rx=0 tx=0).stop 2024-05-09T08:04:13.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:13.037+0000 7ff37a800640 1 -- 172.21.15.1:0/1630470807 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff3740c7660 msgr2=0x7ff374148240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:04:13.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:13.037+0000 7ff37a800640 1 --2- 172.21.15.1:0/1630470807 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff3740c7660 0x7ff374148240 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ff368009890 tx=0x7ff3680046c0 comp rx=0 tx=0).stop 2024-05-09T08:04:13.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:13.037+0000 7ff37a800640 1 -- 172.21.15.1:0/1630470807 shutdown_connections 2024-05-09T08:04:13.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:13.037+0000 7ff37a800640 1 --2- 172.21.15.1:0/1630470807 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7ff35406bde0 0x7ff35406e2a0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:04:13.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:13.037+0000 7ff37a800640 1 --2- 172.21.15.1:0/1630470807 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff374152500 0x7ff374149050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:04:13.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:13.037+0000 7ff37a800640 1 --2- 172.21.15.1:0/1630470807 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ff3740cc730 0x7ff3741f07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:04:13.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:13.037+0000 7ff37a800640 1 --2- 172.21.15.1:0/1630470807 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff3740c7660 0x7ff374148240 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:04:13.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:13.037+0000 7ff37a800640 1 -- 172.21.15.1:0/1630470807 >> 172.21.15.1:0/1630470807 conn(0x7ff3740cbc10 msgr2=0x7ff37415b380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:04:13.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:13.037+0000 7ff37a800640 1 -- 172.21.15.1:0/1630470807 shutdown_connections 2024-05-09T08:04:13.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:13.037+0000 7ff37a800640 1 -- 172.21.15.1:0/1630470807 wait complete. 2024-05-09T08:04:16.069 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-05-09T08:04:16.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:16.230+0000 7fe103200640 1 Processor -- start 2024-05-09T08:04:16.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:16.230+0000 7fe103200640 1 -- start start 2024-05-09T08:04:16.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:16.232+0000 7fe103200640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fe1040cad60 0x7fe10415ba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:04:16.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:16.232+0000 7fe103200640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fe10415a610 0x7fe1040cd030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:04:16.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:16.232+0000 7fe103200640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe10415ac00 con 0x7fe10406d7f0 2024-05-09T08:04:16.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:16.232+0000 7fe103200640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7fe104154dd0 con 0x7fe1040cad60 2024-05-09T08:04:16.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:16.232+0000 7fe103200640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7fe10415aec0 con 0x7fe10415a610 2024-05-09T08:04:16.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:16.232+0000 7fe101e00640 1 -- >> v2:172.21.15.1:3300/0 conn(0x7fe1040cad60 msgr2=0x7fe10415ba80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.1:3300/0 2024-05-09T08:04:16.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:16.233+0000 7fe101e00640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fe1040cad60 0x7fe10415ba80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T08:04:16.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:16.233+0000 7fe101400640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fe10415a610 0x7fe1040cd030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:04:16.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:16.233+0000 7fe101400640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fe10415a610 0x7fe1040cd030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3301/0 says I am v2:172.21.15.1:40008/0 (socket says 172.21.15.1:40008) 2024-05-09T08:04:16.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:16.233+0000 7fe101400640 1 -- 172.21.15.1:0/699210614 learned_addr learned my addr 172.21.15.1:0/699210614 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:04:16.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:16.233+0000 7fe102800640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fe10406d7f0 0x7fe1040cff10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.1:38176/0 (socket says 172.21.15.1:38176) 2024-05-09T08:04:16.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:16.233+0000 7fe101400640 1 -- 172.21.15.1:0/699210614 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fe10415a610 msgr2=0x7fe1040cd030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-05-09T08:04:16.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:16.233+0000 7fe101400640 1 -- 172.21.15.1:0/699210614 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fe10415a610 msgr2=0x7fe1040cd030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-09T08:04:16.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:16.233+0000 7fe101400640 1 --2- 172.21.15.1:0/699210614 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fe10415a610 0x7fe1040cd030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-09T08:04:16.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:16.233+0000 7fe101400640 1 --2- 172.21.15.1:0/699210614 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fe10415a610 0x7fe1040cd030 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T08:04:16.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:16.434+0000 7fe101e00640 1 -- 172.21.15.1:0/699210614 >> v2:172.21.15.1:3300/0 conn(0x7fe1040cad60 msgr2=0x7fe10415ba80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.1:3300/0 2024-05-09T08:04:16.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:16.434+0000 7fe101e00640 1 --2- 172.21.15.1:0/699210614 >> v2:172.21.15.1:3300/0 conn(0x7fe1040cad60 0x7fe10415ba80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-09T08:04:16.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:16.434+0000 7fe101400640 1 --2- 172.21.15.1:0/699210614 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fe10415a610 0x7fe1040cd030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:04:16.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:16.434+0000 7fe101400640 1 -- 172.21.15.1:0/699210614 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fe10415a610 msgr2=0x7fe1040cd030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-09T08:04:16.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:16.434+0000 7fe101400640 1 -- 172.21.15.1:0/699210614 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fe10415a610 msgr2=0x7fe1040cd030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-09T08:04:16.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:16.434+0000 7fe101400640 1 --2- 172.21.15.1:0/699210614 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fe10415a610 0x7fe1040cd030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-09T08:04:16.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:16.434+0000 7fe101400640 1 --2- 172.21.15.1:0/699210614 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fe10415a610 0x7fe1040cd030 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-09T08:04:16.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:16.835+0000 7fe101e00640 1 -- 172.21.15.1:0/699210614 >> v2:172.21.15.1:3300/0 conn(0x7fe1040cad60 msgr2=0x7fe10415ba80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.1:3300/0 2024-05-09T08:04:16.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:16.835+0000 7fe101e00640 1 --2- 172.21.15.1:0/699210614 >> v2:172.21.15.1:3300/0 conn(0x7fe1040cad60 0x7fe10415ba80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-09T08:04:16.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:16.835+0000 7fe101400640 1 --2- 172.21.15.1:0/699210614 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fe10415a610 0x7fe1040cd030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:04:16.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:16.835+0000 7fe101400640 1 -- 172.21.15.1:0/699210614 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fe10415a610 msgr2=0x7fe1040cd030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-05-09T08:04:16.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:16.835+0000 7fe101400640 1 -- 172.21.15.1:0/699210614 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fe10415a610 msgr2=0x7fe1040cd030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-09T08:04:16.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:16.835+0000 7fe101400640 1 --2- 172.21.15.1:0/699210614 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fe10415a610 0x7fe1040cd030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-09T08:04:16.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:16.835+0000 7fe101400640 1 --2- 172.21.15.1:0/699210614 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fe10415a610 0x7fe1040cd030 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-09T08:04:17.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:17.636+0000 7fe101e00640 1 -- 172.21.15.1:0/699210614 >> v2:172.21.15.1:3300/0 conn(0x7fe1040cad60 msgr2=0x7fe10415ba80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.1:3300/0 2024-05-09T08:04:17.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:17.636+0000 7fe101e00640 1 --2- 172.21.15.1:0/699210614 >> v2:172.21.15.1:3300/0 conn(0x7fe1040cad60 0x7fe10415ba80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-09T08:04:17.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:17.637+0000 7fe101400640 1 --2- 172.21.15.1:0/699210614 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fe10415a610 0x7fe1040cd030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:04:17.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:17.637+0000 7fe101400640 1 -- 172.21.15.1:0/699210614 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fe10415a610 msgr2=0x7fe1040cd030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-09T08:04:17.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:17.637+0000 7fe101400640 1 -- 172.21.15.1:0/699210614 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fe10415a610 msgr2=0x7fe1040cd030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-09T08:04:17.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:17.637+0000 7fe101400640 1 --2- 172.21.15.1:0/699210614 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fe10415a610 0x7fe1040cd030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-09T08:04:17.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:17.637+0000 7fe101400640 1 --2- 172.21.15.1:0/699210614 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fe10415a610 0x7fe1040cd030 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-09T08:04:19.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:19.232+0000 7fe0fb400640 1 -- 172.21.15.1:0/699210614 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fe10415a610 msgr2=0x7fe1040cd030 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T08:04:19.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:19.232+0000 7fe0fb400640 1 --2- 172.21.15.1:0/699210614 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fe10415a610 0x7fe1040cd030 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:04:19.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:19.232+0000 7fe0fb400640 1 -- 172.21.15.1:0/699210614 >> v2:172.21.15.1:3300/0 conn(0x7fe1040cad60 msgr2=0x7fe10415ba80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T08:04:19.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:19.232+0000 7fe0fb400640 1 --2- 172.21.15.1:0/699210614 >> v2:172.21.15.1:3300/0 conn(0x7fe1040cad60 0x7fe10415ba80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:04:19.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:19.232+0000 7fe0fb400640 1 -- 172.21.15.1:0/699210614 >> v1:172.21.15.7:6789/0 conn(0x7fe10406d7f0 legacy=0x7fe1040cff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:04:19.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:19.232+0000 7fe0fb400640 1 --2- 172.21.15.1:0/699210614 >> v2:172.21.15.7:3300/0 conn(0x7fe0e40036c0 0x7fe0e4005b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:04:19.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:19.233+0000 7fe0fb400640 1 -- 172.21.15.1:0/699210614 --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe10415b1a0 con 0x7fe0e4006220 2024-05-09T08:04:19.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:19.233+0000 7fe0fb400640 1 -- 172.21.15.1:0/699210614 --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fe104154dd0 con 0x7fe0e40036c0 2024-05-09T08:04:19.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:19.233+0000 7fe101400640 1 -- 172.21.15.1:0/699210614 >> v1:172.21.15.1:6789/0 conn(0x7fe0e4006220 legacy=0x7fe0e40086e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.1:6789/0 2024-05-09T08:04:19.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:19.233+0000 7fe101e00640 1 --2- 172.21.15.1:0/699210614 >> v2:172.21.15.7:3300/0 conn(0x7fe0e40036c0 0x7fe0e4005b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:04:19.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:19.233+0000 7fe101e00640 1 -- 172.21.15.1:0/699210614 >> v2:172.21.15.7:3300/0 conn(0x7fe0e40036c0 msgr2=0x7fe0e4005b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-09T08:04:19.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:19.233+0000 7fe101e00640 1 -- 172.21.15.1:0/699210614 >> v2:172.21.15.7:3300/0 conn(0x7fe0e40036c0 msgr2=0x7fe0e4005b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-09T08:04:19.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:19.233+0000 7fe101e00640 1 --2- 172.21.15.1:0/699210614 >> v2:172.21.15.7:3300/0 conn(0x7fe0e40036c0 0x7fe0e4005b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-09T08:04:19.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:19.233+0000 7fe101e00640 1 --2- 172.21.15.1:0/699210614 >> v2:172.21.15.7:3300/0 conn(0x7fe0e40036c0 0x7fe0e4005b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T08:04:19.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:19.434+0000 7fe101400640 1 -- 172.21.15.1:0/699210614 >> v1:172.21.15.1:6789/0 conn(0x7fe0e4006220 legacy=0x7fe0e40086e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.1:6789/0 2024-05-09T08:04:19.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:19.434+0000 7fe101e00640 1 --2- 172.21.15.1:0/699210614 >> v2:172.21.15.7:3300/0 conn(0x7fe0e40036c0 0x7fe0e4005b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:04:19.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:19.434+0000 7fe101e00640 1 -- 172.21.15.1:0/699210614 >> v2:172.21.15.7:3300/0 conn(0x7fe0e40036c0 msgr2=0x7fe0e4005b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-09T08:04:19.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:19.434+0000 7fe101e00640 1 -- 172.21.15.1:0/699210614 >> v2:172.21.15.7:3300/0 conn(0x7fe0e40036c0 msgr2=0x7fe0e4005b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-09T08:04:19.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:19.434+0000 7fe101e00640 1 --2- 172.21.15.1:0/699210614 >> v2:172.21.15.7:3300/0 conn(0x7fe0e40036c0 0x7fe0e4005b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-09T08:04:19.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:19.434+0000 7fe101e00640 1 --2- 172.21.15.1:0/699210614 >> v2:172.21.15.7:3300/0 conn(0x7fe0e40036c0 0x7fe0e4005b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-09T08:04:19.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:19.835+0000 7fe101400640 1 -- 172.21.15.1:0/699210614 >> v1:172.21.15.1:6789/0 conn(0x7fe0e4006220 legacy=0x7fe0e40086e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.1:6789/0 2024-05-09T08:04:19.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:19.835+0000 7fe101e00640 1 --2- 172.21.15.1:0/699210614 >> v2:172.21.15.7:3300/0 conn(0x7fe0e40036c0 0x7fe0e4005b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:04:19.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:19.836+0000 7fe101e00640 1 -- 172.21.15.1:0/699210614 >> v1:172.21.15.1:6789/0 conn(0x7fe0e4006220 legacy=0x7fe0e40086e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T08:04:19.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:19.836+0000 7fe101e00640 1 -- 172.21.15.1:0/699210614 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0e400afc0 con 0x7fe0e40036c0 2024-05-09T08:04:19.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:19.837+0000 7fe101e00640 1 --2- 172.21.15.1:0/699210614 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe0e40036c0 0x7fe0e4005b40 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fe0f4003980 tx=0x7fe0f402b030 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5284a01238411817 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:04:19.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:19.837+0000 7fe100a00640 1 -- 172.21.15.1:0/699210614 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0f4031070 con 0x7fe0e40036c0 2024-05-09T08:04:19.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:19.838+0000 7fe100a00640 1 -- 172.21.15.1:0/699210614 <== mon.1 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0f402bd40 con 0x7fe0e40036c0 2024-05-09T08:04:19.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:19.838+0000 7fe100a00640 1 -- 172.21.15.1:0/699210614 <== mon.1 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0f4035630 con 0x7fe0e40036c0 2024-05-09T08:04:19.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:19.839+0000 7fe103200640 1 -- 172.21.15.1:0/699210614 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe0e40036c0 msgr2=0x7fe0e4005b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:04:19.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:19.839+0000 7fe103200640 1 --2- 172.21.15.1:0/699210614 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe0e40036c0 0x7fe0e4005b40 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fe0f4003980 tx=0x7fe0f402b030 comp rx=0 tx=0).stop 2024-05-09T08:04:19.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:19.839+0000 7fe102800640 1 -- 172.21.15.1:0/699210614 reap_dead start 2024-05-09T08:04:19.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:19.839+0000 7fe103200640 1 -- 172.21.15.1:0/699210614 shutdown_connections 2024-05-09T08:04:19.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:19.839+0000 7fe103200640 1 --2- 172.21.15.1:0/699210614 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe0e40036c0 0x7fe0e4005b40 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:04:19.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:19.839+0000 7fe103200640 1 -- 172.21.15.1:0/699210614 >> 172.21.15.1:0/699210614 conn(0x7fe1040670b0 msgr2=0x7fe1040674c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:04:19.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:19.839+0000 7fe103200640 1 -- 172.21.15.1:0/699210614 shutdown_connections 2024-05-09T08:04:19.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:19.839+0000 7fe103200640 1 -- 172.21.15.1:0/699210614 wait complete. 2024-05-09T08:04:19.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:19.842+0000 7fe103200640 1 Processor -- start 2024-05-09T08:04:19.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:19.842+0000 7fe103200640 1 -- start start 2024-05-09T08:04:19.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:19.843+0000 7fe103200640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fe0e40036c0 0x7fe1041e7440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:04:19.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:19.843+0000 7fe103200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe1040b58c0 0x7fe1040c5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:04:19.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:19.843+0000 7fe103200640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe1040c6100 0x7fe1041f3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:04:19.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:19.843+0000 7fe103200640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fe1040cc830 con 0x7fe1040c6100 2024-05-09T08:04:19.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:19.843+0000 7fe103200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe1040cc550 con 0x7fe1040b58c0 2024-05-09T08:04:19.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:19.843+0000 7fe103200640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7fe1040ccb10 con 0x7fe0e40036c0 2024-05-09T08:04:19.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:19.844+0000 7fe102800640 1 -- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe1040c6100 msgr2=0x7fe1041f3890 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.1:3300/0 2024-05-09T08:04:19.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:19.844+0000 7fe101e00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fe0e40036c0 0x7fe1041e7440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:04:19.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:19.844+0000 7fe102800640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe1040c6100 0x7fe1041f3890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T08:04:19.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:19.844+0000 7fe101e00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fe0e40036c0 0x7fe1041e7440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3301/0 says I am v2:172.21.15.1:40040/0 (socket says 172.21.15.1:40040) 2024-05-09T08:04:19.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:19.844+0000 7fe101e00640 1 -- 172.21.15.1:0/3736675413 learned_addr learned my addr 172.21.15.1:0/3736675413 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:04:19.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:19.844+0000 7fe101400640 1 --2- 172.21.15.1:0/3736675413 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe1040b58c0 0x7fe1040c5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:04:19.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:19.844+0000 7fe101e00640 1 -- 172.21.15.1:0/3736675413 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe1040b58c0 msgr2=0x7fe1040c5bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:04:19.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:19.844+0000 7fe101e00640 1 --2- 172.21.15.1:0/3736675413 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe1040b58c0 0x7fe1040c5bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:04:19.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:19.844+0000 7fe101e00640 1 -- 172.21.15.1:0/3736675413 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe1040c6100 msgr2=0x7fe1041f3890 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T08:04:19.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:19.844+0000 7fe101e00640 1 --2- 172.21.15.1:0/3736675413 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe1040c6100 0x7fe1041f3890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:04:19.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:19.844+0000 7fe101e00640 1 -- 172.21.15.1:0/3736675413 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1040c6950 con 0x7fe0e40036c0 2024-05-09T08:04:19.862 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:19.845+0000 7fe101e00640 1 --2- 172.21.15.1:0/3736675413 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fe0e40036c0 0x7fe1041e7440 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fe0f402b620 tx=0x7fe0f4005db0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:04:19.862 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:19.845+0000 7fe0fa000640 1 -- 172.21.15.1:0/3736675413 <== mon.2 v2:172.21.15.7:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0f402bb40 con 0x7fe0e40036c0 2024-05-09T08:04:19.862 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:19.845+0000 7fe0fa000640 1 -- 172.21.15.1:0/3736675413 <== mon.2 v2:172.21.15.7:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0f40345c0 con 0x7fe0e40036c0 2024-05-09T08:04:19.862 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:19.845+0000 7fe103200640 1 -- 172.21.15.1:0/3736675413 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1040c36d0 con 0x7fe0e40036c0 2024-05-09T08:04:19.862 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:19.845+0000 7fe0fa000640 1 -- 172.21.15.1:0/3736675413 <== mon.2 v2:172.21.15.7:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0f4034770 con 0x7fe0e40036c0 2024-05-09T08:04:19.862 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:19.845+0000 7fe103200640 1 -- 172.21.15.1:0/3736675413 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1040cf220 con 0x7fe0e40036c0 2024-05-09T08:04:19.863 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:19.847+0000 7fe103200640 1 -- 172.21.15.1:0/3736675413 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe10404f660 con 0x7fe0e40036c0 2024-05-09T08:04:19.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:19.849+0000 7fe0fa000640 1 -- 172.21.15.1:0/3736675413 <== mon.2 v2:172.21.15.7:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe0f4046030 con 0x7fe0e40036c0 2024-05-09T08:04:19.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:19.850+0000 7fe0fa000640 1 --2- 172.21.15.1:0/3736675413 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fe0e406cbf0 0x7fe0e406f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:04:19.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:19.850+0000 7fe0fa000640 1 -- 172.21.15.1:0/3736675413 <== mon.2 v2:172.21.15.7:3301/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7fe0f40ac200 con 0x7fe0e40036c0 2024-05-09T08:04:19.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:19.851+0000 7fe101400640 1 --2- 172.21.15.1:0/3736675413 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fe0e406cbf0 0x7fe0e406f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:04:19.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:19.851+0000 7fe101400640 1 --2- 172.21.15.1:0/3736675413 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fe0e406cbf0 0x7fe0e406f0b0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fe104153160 tx=0x7fe0f0008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:04:19.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:19.853+0000 7fe0fa000640 1 -- 172.21.15.1:0/3736675413 <== mon.2 v2:172.21.15.7:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fe0f404d020 con 0x7fe0e40036c0 2024-05-09T08:04:20.153 DEBUG:teuthology.orchestra.run.smithi001:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T08:04:20.159 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T08:04:20.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.339+0000 7fe103200640 1 -- 172.21.15.1:0/3736675413 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fe1040cbff0 con 0x7fe0e40036c0 2024-05-09T08:04:20.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.340+0000 7fe0fa000640 1 -- 172.21.15.1:0/3736675413 <== mon.2 v2:172.21.15.7:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1538 (secure 0 0 0) 0x7fe0f4005e20 con 0x7fe0e40036c0 2024-05-09T08:04:20.355 INFO:teuthology.orchestra.run.smithi001.stdout:{"election_epoch":12,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"54329ec8-3270-4043-9a82-c6cd7420aa7f","modified":"2024-05-09T08:01:21.819270Z","created":"2024-05-09T08:01:21.819270Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:3300","nonce":0},{"type":"v1","addr":"172.21.15.1:6789","nonce":0}]},"addr":"172.21.15.1:6789/0","public_addr":"172.21.15.1:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:3300","nonce":0},{"type":"v1","addr":"172.21.15.7:6789","nonce":0}]},"addr":"172.21.15.7:6789/0","public_addr":"172.21.15.7:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:3301","nonce":0},{"type":"v1","addr":"172.21.15.7:6790","nonce":0}]},"addr":"172.21.15.7:6790/0","public_addr":"172.21.15.7:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-05-09T08:04:20.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.342+0000 7fe103200640 1 -- 172.21.15.1:0/3736675413 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fe0e406cbf0 msgr2=0x7fe0e406f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:04:20.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.342+0000 7fe103200640 1 --2- 172.21.15.1:0/3736675413 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fe0e406cbf0 0x7fe0e406f0b0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fe104153160 tx=0x7fe0f0008040 comp rx=0 tx=0).stop 2024-05-09T08:04:20.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.342+0000 7fe103200640 1 -- 172.21.15.1:0/3736675413 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fe0e40036c0 msgr2=0x7fe1041e7440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:04:20.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.342+0000 7fe103200640 1 --2- 172.21.15.1:0/3736675413 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fe0e40036c0 0x7fe1041e7440 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fe0f402b620 tx=0x7fe0f4005db0 comp rx=0 tx=0).stop 2024-05-09T08:04:20.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.343+0000 7fe103200640 1 -- 172.21.15.1:0/3736675413 shutdown_connections 2024-05-09T08:04:20.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.343+0000 7fe103200640 1 --2- 172.21.15.1:0/3736675413 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fe0e406cbf0 0x7fe0e406f0b0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:04:20.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.343+0000 7fe103200640 1 --2- 172.21.15.1:0/3736675413 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe1040b58c0 0x7fe1040c5bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:04:20.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.343+0000 7fe103200640 1 --2- 172.21.15.1:0/3736675413 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe1040c6100 0x7fe1041f3890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:04:20.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.343+0000 7fe103200640 1 --2- 172.21.15.1:0/3736675413 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fe0e40036c0 0x7fe1041e7440 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:04:20.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.343+0000 7fe103200640 1 -- 172.21.15.1:0/3736675413 >> 172.21.15.1:0/3736675413 conn(0x7fe1040670b0 msgr2=0x7fe1040cfef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:04:20.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.343+0000 7fe103200640 1 -- 172.21.15.1:0/3736675413 shutdown_connections 2024-05-09T08:04:20.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.343+0000 7fe103200640 1 -- 172.21.15.1:0/3736675413 wait complete. 2024-05-09T08:04:20.376 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2024-05-09T08:04:20.376 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon b 2024-05-09T08:04:20.376 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-05-09T08:04:20.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.529+0000 7f65ea800640 1 Processor -- start 2024-05-09T08:04:20.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.530+0000 7f65ea800640 1 -- start start 2024-05-09T08:04:20.547 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.531+0000 7f65ea800640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f65e40c5e40 0x7f65e40c9570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:04:20.547 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.531+0000 7f65ea800640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f65e4071c80 con 0x7f65e415c360 2024-05-09T08:04:20.547 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.532+0000 7f65ea800640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f65e4071f20 con 0x7f65e4072c20 2024-05-09T08:04:20.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.532+0000 7f65ea800640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f65e415b980 con 0x7f65e40c5e40 2024-05-09T08:04:20.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.532+0000 7f65e3400640 1 -- >> v1:172.21.15.1:6789/0 conn(0x7f65e415c360 legacy=0x7f65e40cc5f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.1:6789/0 2024-05-09T08:04:20.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.532+0000 7f65e2a00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f65e40c5e40 0x7f65e40c9570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:04:20.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.532+0000 7f65e3e00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f65e4072c20 0x7f65e4073050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.1:38184/0 (socket says 172.21.15.1:38184) 2024-05-09T08:04:20.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.532+0000 7f65e3e00640 1 -- 172.21.15.1:0/166762606 learned_addr learned my addr 172.21.15.1:0/166762606 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T08:04:20.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.532+0000 7f65e2a00640 1 -- 172.21.15.1:0/166762606 >> v1:172.21.15.7:6789/0 conn(0x7f65e4072c20 legacy=0x7f65e4073050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:04:20.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.532+0000 7f65e2a00640 1 -- 172.21.15.1:0/166762606 >> v1:172.21.15.1:6789/0 conn(0x7f65e415c360 legacy=0x7f65e40cc5f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T08:04:20.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.532+0000 7f65e2a00640 1 -- 172.21.15.1:0/166762606 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65e40721e0 con 0x7f65e40c5e40 2024-05-09T08:04:20.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.532+0000 7f65e2000640 1 -- 172.21.15.1:0/166762606 <== mon.1 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 486653699 0 0) 0x7f65e4071f20 con 0x7f65e4072c20 2024-05-09T08:04:20.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.534+0000 7f65e2a00640 1 --2- 172.21.15.1:0/166762606 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65e40c5e40 0x7f65e40c9570 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f65e40c6d30 tx=0x7f65d002a0a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8a628f43efcf8f26 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:04:20.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.535+0000 7f65e2000640 1 -- 172.21.15.1:0/166762606 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65d0037070 con 0x7f65e40c5e40 2024-05-09T08:04:20.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.535+0000 7f65e2000640 1 -- 172.21.15.1:0/166762606 <== mon.1 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65d002adb0 con 0x7f65e40c5e40 2024-05-09T08:04:20.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.535+0000 7f65e2000640 1 -- 172.21.15.1:0/166762606 <== mon.1 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65d0032660 con 0x7f65e40c5e40 2024-05-09T08:04:20.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.535+0000 7f65ea800640 1 -- 172.21.15.1:0/166762606 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65e40c5e40 msgr2=0x7f65e40c9570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:04:20.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.535+0000 7f65ea800640 1 --2- 172.21.15.1:0/166762606 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65e40c5e40 0x7f65e40c9570 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f65e40c6d30 tx=0x7f65d002a0a0 comp rx=0 tx=0).stop 2024-05-09T08:04:20.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.535+0000 7f65ea800640 1 -- 172.21.15.1:0/166762606 shutdown_connections 2024-05-09T08:04:20.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.535+0000 7f65ea800640 1 --2- 172.21.15.1:0/166762606 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65e40c5e40 0x7f65e40c9570 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:04:20.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.535+0000 7f65ea800640 1 -- 172.21.15.1:0/166762606 >> 172.21.15.1:0/166762606 conn(0x7f65e406dad0 msgr2=0x7f65e406deb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:04:20.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.536+0000 7f65ea800640 1 -- 172.21.15.1:0/166762606 shutdown_connections 2024-05-09T08:04:20.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.536+0000 7f65ea800640 1 -- 172.21.15.1:0/166762606 wait complete. 2024-05-09T08:04:20.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.538+0000 7f65ea800640 1 Processor -- start 2024-05-09T08:04:20.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.538+0000 7f65ea800640 1 -- start start 2024-05-09T08:04:20.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.539+0000 7f65ea800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65e4072c20 0x7f65e4156f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:04:20.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.539+0000 7f65ea800640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f65e40c5e40 0x7f65e415a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:04:20.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.539+0000 7f65ea800640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f65e415c360 0x7f65e41f0f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:04:20.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.539+0000 7f65ea800640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f65e40727a0 con 0x7f65e40c5e40 2024-05-09T08:04:20.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.539+0000 7f65ea800640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f65e4073760 con 0x7f65e4072c20 2024-05-09T08:04:20.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.539+0000 7f65ea800640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f65e40cc0d0 con 0x7f65e415c360 2024-05-09T08:04:20.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.539+0000 7f65e2a00640 1 -- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f65e40c5e40 msgr2=0x7f65e415a600 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.1:3300/0 2024-05-09T08:04:20.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.539+0000 7f65e2a00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f65e40c5e40 0x7f65e415a600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T08:04:20.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.539+0000 7f65e3400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65e4072c20 0x7f65e4156f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:04:20.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.540+0000 7f65e3e00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f65e415c360 0x7f65e41f0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:04:20.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.540+0000 7f65e3400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65e4072c20 0x7f65e4156f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.1:40294/0 (socket says 172.21.15.1:40294) 2024-05-09T08:04:20.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.540+0000 7f65e3400640 1 -- 172.21.15.1:0/2368849374 learned_addr learned my addr 172.21.15.1:0/2368849374 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:04:20.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.540+0000 7f65e3400640 1 -- 172.21.15.1:0/2368849374 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f65e415c360 msgr2=0x7f65e41f0f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:04:20.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.540+0000 7f65e3400640 1 --2- 172.21.15.1:0/2368849374 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f65e415c360 0x7f65e41f0f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:04:20.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.540+0000 7f65e3400640 1 -- 172.21.15.1:0/2368849374 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f65e40c5e40 msgr2=0x7f65e415a600 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T08:04:20.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.540+0000 7f65e3400640 1 --2- 172.21.15.1:0/2368849374 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f65e40c5e40 0x7f65e415a600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:04:20.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.540+0000 7f65e3400640 1 -- 172.21.15.1:0/2368849374 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65e415ab40 con 0x7f65e4072c20 2024-05-09T08:04:20.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.540+0000 7f65e3400640 1 --2- 172.21.15.1:0/2368849374 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65e4072c20 0x7f65e4156f90 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f65d400b550 tx=0x7f65d400ba10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:04:20.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.540+0000 7f65db400640 1 -- 172.21.15.1:0/2368849374 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65d4003dc0 con 0x7f65e4072c20 2024-05-09T08:04:20.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.541+0000 7f65db400640 1 -- 172.21.15.1:0/2368849374 <== mon.1 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65d4015070 con 0x7f65e4072c20 2024-05-09T08:04:20.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.541+0000 7f65db400640 1 -- 172.21.15.1:0/2368849374 <== mon.1 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65d40052b0 con 0x7f65e4072c20 2024-05-09T08:04:20.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.541+0000 7f65ea800640 1 -- 172.21.15.1:0/2368849374 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65e41f1450 con 0x7f65e4072c20 2024-05-09T08:04:20.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.541+0000 7f65ea800640 1 -- 172.21.15.1:0/2368849374 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65e41578e0 con 0x7f65e4072c20 2024-05-09T08:04:20.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.542+0000 7f65ea800640 1 --2- 172.21.15.1:0/2368849374 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65e40ca250 0x7f65e4149180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:04:20.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.542+0000 7f65ea800640 1 -- 172.21.15.1:0/2368849374 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f65e40706f0 con 0x7f65e40ca250 2024-05-09T08:04:20.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.542+0000 7f65e2a00640 1 --2- 172.21.15.1:0/2368849374 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65e40ca250 0x7f65e4149180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:04:20.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.543+0000 7f65e2a00640 1 -- 172.21.15.1:0/2368849374 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f65e4074d70 con 0x7f65e40ca250 2024-05-09T08:04:20.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.543+0000 7f65e2a00640 1 --2- 172.21.15.1:0/2368849374 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65e40ca250 0x7f65e4149180 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f65e40c8c60 tx=0x7f65d00046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:04:20.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.543+0000 7f65db400640 1 -- 172.21.15.1:0/2368849374 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65d0042070 con 0x7f65e40ca250 2024-05-09T08:04:20.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.544+0000 7f65db400640 1 -- 172.21.15.1:0/2368849374 <== mon.1 v2:172.21.15.7:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f65d0032d10 con 0x7f65e40ca250 2024-05-09T08:04:20.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.544+0000 7f65db400640 1 -- 172.21.15.1:0/2368849374 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65e40ca250 msgr2=0x7f65e4149180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:04:20.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.544+0000 7f65db400640 1 --2- 172.21.15.1:0/2368849374 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65e40ca250 0x7f65e4149180 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f65e40c8c60 tx=0x7f65d00046c0 comp rx=0 tx=0).stop 2024-05-09T08:04:20.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.544+0000 7f65db400640 1 -- 172.21.15.1:0/2368849374 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65e40ca250 msgr2=0x7f65e4149180 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-05-09T08:04:20.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.544+0000 7f65db400640 1 --2- 172.21.15.1:0/2368849374 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65e40ca250 0x7f65e4149180 secure :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f65e40c8c60 tx=0x7f65d00046c0 comp rx=0 tx=0).stop 2024-05-09T08:04:20.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.544+0000 7f65db400640 1 -- 172.21.15.1:0/2368849374 <== mon.1 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f65d4005450 con 0x7f65e4072c20 2024-05-09T08:04:20.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.545+0000 7f65db400640 1 --2- 172.21.15.1:0/2368849374 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f65c406b940 0x7f65c406de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:04:20.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.545+0000 7f65db400640 1 -- 172.21.15.1:0/2368849374 <== mon.1 v2:172.21.15.7:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7f65d408afb0 con 0x7f65e4072c20 2024-05-09T08:04:20.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.545+0000 7f65e2a00640 1 --2- 172.21.15.1:0/2368849374 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f65c406b940 0x7f65c406de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:04:20.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.546+0000 7f65e2a00640 1 --2- 172.21.15.1:0/2368849374 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f65c406b940 0x7f65c406de00 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f65e40b24e0 tx=0x7f65d003e000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:04:20.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.557+0000 7f65ea800640 1 --2- 172.21.15.1:0/2368849374 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65e4159a40 0x7f65e4159e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:04:20.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.557+0000 7f65ea800640 1 -- 172.21.15.1:0/2368849374 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f65e4159290 con 0x7f65e4159a40 2024-05-09T08:04:20.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.558+0000 7f65e3e00640 1 --2- 172.21.15.1:0/2368849374 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65e4159a40 0x7f65e4159e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:04:20.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.558+0000 7f65e3e00640 1 -- 172.21.15.1:0/2368849374 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f65e414d730 con 0x7f65e4159a40 2024-05-09T08:04:20.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.558+0000 7f65e3e00640 1 --2- 172.21.15.1:0/2368849374 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65e4159a40 0x7f65e4159e20 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f65dc000f80 tx=0x7f65dc007a90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:04:20.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.558+0000 7f65db400640 1 -- 172.21.15.1:0/2368849374 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65dc00f7a0 con 0x7f65e4159a40 2024-05-09T08:04:20.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.559+0000 7f65db400640 1 -- 172.21.15.1:0/2368849374 <== mon.1 v2:172.21.15.7:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4695 (secure 0 0 0) 0x7f65dc00f7a0 con 0x7f65e4159a40 2024-05-09T08:04:20.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.559+0000 7f65db400640 1 -- 172.21.15.1:0/2368849374 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65e4159a40 msgr2=0x7f65e4159e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:04:20.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.559+0000 7f65db400640 1 --2- 172.21.15.1:0/2368849374 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65e4159a40 0x7f65e4159e20 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f65dc000f80 tx=0x7f65dc007a90 comp rx=0 tx=0).stop 2024-05-09T08:04:20.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.559+0000 7f65db400640 1 -- 172.21.15.1:0/2368849374 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65e4159a40 msgr2=0x7f65e4159e20 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-05-09T08:04:20.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.559+0000 7f65db400640 1 --2- 172.21.15.1:0/2368849374 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65e4159a40 0x7f65e4159e20 secure :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f65dc000f80 tx=0x7f65dc007a90 comp rx=0 tx=0).stop 2024-05-09T08:04:20.575 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-05-09T08:04:20.575 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "b", 2024-05-09T08:04:20.575 INFO:teuthology.orchestra.run.smithi001.stdout: "rank": 1, 2024-05-09T08:04:20.575 INFO:teuthology.orchestra.run.smithi001.stdout: "state": "leader", 2024-05-09T08:04:20.575 INFO:teuthology.orchestra.run.smithi001.stdout: "election_epoch": 12, 2024-05-09T08:04:20.575 INFO:teuthology.orchestra.run.smithi001.stdout: "uptime": 28221, 2024-05-09T08:04:20.575 INFO:teuthology.orchestra.run.smithi001.stdout: "quorum": [ 2024-05-09T08:04:20.575 INFO:teuthology.orchestra.run.smithi001.stdout: 1, 2024-05-09T08:04:20.576 INFO:teuthology.orchestra.run.smithi001.stdout: 2 2024-05-09T08:04:20.576 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:04:20.576 INFO:teuthology.orchestra.run.smithi001.stdout: "quorum_age": 1, 2024-05-09T08:04:20.576 INFO:teuthology.orchestra.run.smithi001.stdout: "features": { 2024-05-09T08:04:20.576 INFO:teuthology.orchestra.run.smithi001.stdout: "required_con": "2450521708007866372", 2024-05-09T08:04:20.576 INFO:teuthology.orchestra.run.smithi001.stdout: "required_mon": [ 2024-05-09T08:04:20.576 INFO:teuthology.orchestra.run.smithi001.stdout: "kraken", 2024-05-09T08:04:20.576 INFO:teuthology.orchestra.run.smithi001.stdout: "luminous", 2024-05-09T08:04:20.576 INFO:teuthology.orchestra.run.smithi001.stdout: "mimic", 2024-05-09T08:04:20.576 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap-prune", 2024-05-09T08:04:20.576 INFO:teuthology.orchestra.run.smithi001.stdout: "nautilus", 2024-05-09T08:04:20.576 INFO:teuthology.orchestra.run.smithi001.stdout: "octopus", 2024-05-09T08:04:20.576 INFO:teuthology.orchestra.run.smithi001.stdout: "pacific", 2024-05-09T08:04:20.576 INFO:teuthology.orchestra.run.smithi001.stdout: "elector-pinging", 2024-05-09T08:04:20.577 INFO:teuthology.orchestra.run.smithi001.stdout: "quincy", 2024-05-09T08:04:20.577 INFO:teuthology.orchestra.run.smithi001.stdout: "reef", 2024-05-09T08:04:20.577 INFO:teuthology.orchestra.run.smithi001.stdout: "squid" 2024-05-09T08:04:20.577 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:04:20.577 INFO:teuthology.orchestra.run.smithi001.stdout: "quorum_con": "4540701547738038271", 2024-05-09T08:04:20.577 INFO:teuthology.orchestra.run.smithi001.stdout: "quorum_mon": [ 2024-05-09T08:04:20.577 INFO:teuthology.orchestra.run.smithi001.stdout: "kraken", 2024-05-09T08:04:20.577 INFO:teuthology.orchestra.run.smithi001.stdout: "luminous", 2024-05-09T08:04:20.577 INFO:teuthology.orchestra.run.smithi001.stdout: "mimic", 2024-05-09T08:04:20.577 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap-prune", 2024-05-09T08:04:20.577 INFO:teuthology.orchestra.run.smithi001.stdout: "nautilus", 2024-05-09T08:04:20.577 INFO:teuthology.orchestra.run.smithi001.stdout: "octopus", 2024-05-09T08:04:20.577 INFO:teuthology.orchestra.run.smithi001.stdout: "pacific", 2024-05-09T08:04:20.578 INFO:teuthology.orchestra.run.smithi001.stdout: "elector-pinging", 2024-05-09T08:04:20.578 INFO:teuthology.orchestra.run.smithi001.stdout: "quincy", 2024-05-09T08:04:20.578 INFO:teuthology.orchestra.run.smithi001.stdout: "reef", 2024-05-09T08:04:20.578 INFO:teuthology.orchestra.run.smithi001.stdout: "squid" 2024-05-09T08:04:20.578 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:04:20.578 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:04:20.578 INFO:teuthology.orchestra.run.smithi001.stdout: "outside_quorum": [], 2024-05-09T08:04:20.578 INFO:teuthology.orchestra.run.smithi001.stdout: "extra_probe_peers": [], 2024-05-09T08:04:20.578 INFO:teuthology.orchestra.run.smithi001.stdout: "sync_provider": [], 2024-05-09T08:04:20.578 INFO:teuthology.orchestra.run.smithi001.stdout: "monmap": { 2024-05-09T08:04:20.578 INFO:teuthology.orchestra.run.smithi001.stdout: "epoch": 1, 2024-05-09T08:04:20.578 INFO:teuthology.orchestra.run.smithi001.stdout: "fsid": "54329ec8-3270-4043-9a82-c6cd7420aa7f", 2024-05-09T08:04:20.578 INFO:teuthology.orchestra.run.smithi001.stdout: "modified": "2024-05-09T08:01:21.819270Z", 2024-05-09T08:04:20.579 INFO:teuthology.orchestra.run.smithi001.stdout: "created": "2024-05-09T08:01:21.819270Z", 2024-05-09T08:04:20.579 INFO:teuthology.orchestra.run.smithi001.stdout: "min_mon_release": 19, 2024-05-09T08:04:20.579 INFO:teuthology.orchestra.run.smithi001.stdout: "min_mon_release_name": "squid", 2024-05-09T08:04:20.579 INFO:teuthology.orchestra.run.smithi001.stdout: "election_strategy": 1, 2024-05-09T08:04:20.579 INFO:teuthology.orchestra.run.smithi001.stdout: "disallowed_leaders: ": "", 2024-05-09T08:04:20.579 INFO:teuthology.orchestra.run.smithi001.stdout: "stretch_mode": false, 2024-05-09T08:04:20.579 INFO:teuthology.orchestra.run.smithi001.stdout: "tiebreaker_mon": "", 2024-05-09T08:04:20.579 INFO:teuthology.orchestra.run.smithi001.stdout: "removed_ranks: ": "", 2024-05-09T08:04:20.579 INFO:teuthology.orchestra.run.smithi001.stdout: "features": { 2024-05-09T08:04:20.579 INFO:teuthology.orchestra.run.smithi001.stdout: "persistent": [ 2024-05-09T08:04:20.579 INFO:teuthology.orchestra.run.smithi001.stdout: "kraken", 2024-05-09T08:04:20.579 INFO:teuthology.orchestra.run.smithi001.stdout: "luminous", 2024-05-09T08:04:20.580 INFO:teuthology.orchestra.run.smithi001.stdout: "mimic", 2024-05-09T08:04:20.580 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap-prune", 2024-05-09T08:04:20.580 INFO:teuthology.orchestra.run.smithi001.stdout: "nautilus", 2024-05-09T08:04:20.580 INFO:teuthology.orchestra.run.smithi001.stdout: "octopus", 2024-05-09T08:04:20.580 INFO:teuthology.orchestra.run.smithi001.stdout: "pacific", 2024-05-09T08:04:20.580 INFO:teuthology.orchestra.run.smithi001.stdout: "elector-pinging", 2024-05-09T08:04:20.580 INFO:teuthology.orchestra.run.smithi001.stdout: "quincy", 2024-05-09T08:04:20.580 INFO:teuthology.orchestra.run.smithi001.stdout: "reef", 2024-05-09T08:04:20.580 INFO:teuthology.orchestra.run.smithi001.stdout: "squid" 2024-05-09T08:04:20.580 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:04:20.580 INFO:teuthology.orchestra.run.smithi001.stdout: "optional": [] 2024-05-09T08:04:20.580 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:04:20.580 INFO:teuthology.orchestra.run.smithi001.stdout: "mons": [ 2024-05-09T08:04:20.581 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:04:20.581 INFO:teuthology.orchestra.run.smithi001.stdout: "rank": 0, 2024-05-09T08:04:20.581 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "a", 2024-05-09T08:04:20.581 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-05-09T08:04:20.581 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-05-09T08:04:20.581 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:04:20.581 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-05-09T08:04:20.581 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:3300", 2024-05-09T08:04:20.581 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:04:20.581 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:04:20.581 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:04:20.581 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-05-09T08:04:20.581 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6789", 2024-05-09T08:04:20.582 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:04:20.582 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:04:20.582 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:04:20.582 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:04:20.582 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6789/0", 2024-05-09T08:04:20.582 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.1:6789/0", 2024-05-09T08:04:20.582 INFO:teuthology.orchestra.run.smithi001.stdout: "priority": 0, 2024-05-09T08:04:20.582 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 0, 2024-05-09T08:04:20.582 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_location": "{}" 2024-05-09T08:04:20.582 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:04:20.582 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:04:20.582 INFO:teuthology.orchestra.run.smithi001.stdout: "rank": 1, 2024-05-09T08:04:20.582 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "b", 2024-05-09T08:04:20.583 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-05-09T08:04:20.583 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-05-09T08:04:20.583 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:04:20.583 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-05-09T08:04:20.583 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:3300", 2024-05-09T08:04:20.583 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:04:20.583 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:04:20.583 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:04:20.583 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-05-09T08:04:20.583 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:6789", 2024-05-09T08:04:20.583 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:04:20.583 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:04:20.583 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:04:20.584 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:04:20.584 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:6789/0", 2024-05-09T08:04:20.584 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.7:6789/0", 2024-05-09T08:04:20.584 INFO:teuthology.orchestra.run.smithi001.stdout: "priority": 0, 2024-05-09T08:04:20.584 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 0, 2024-05-09T08:04:20.584 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_location": "{}" 2024-05-09T08:04:20.584 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:04:20.584 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:04:20.584 INFO:teuthology.orchestra.run.smithi001.stdout: "rank": 2, 2024-05-09T08:04:20.584 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "c", 2024-05-09T08:04:20.584 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-05-09T08:04:20.584 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-05-09T08:04:20.584 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:04:20.584 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-05-09T08:04:20.585 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:3301", 2024-05-09T08:04:20.585 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:04:20.585 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:04:20.585 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:04:20.585 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-05-09T08:04:20.585 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:6790", 2024-05-09T08:04:20.585 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:04:20.585 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:04:20.585 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:04:20.585 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:04:20.585 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:6790/0", 2024-05-09T08:04:20.585 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.7:6790/0", 2024-05-09T08:04:20.585 INFO:teuthology.orchestra.run.smithi001.stdout: "priority": 0, 2024-05-09T08:04:20.585 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 0, 2024-05-09T08:04:20.586 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_location": "{}" 2024-05-09T08:04:20.586 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:04:20.586 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:04:20.586 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:04:20.586 INFO:teuthology.orchestra.run.smithi001.stdout: "feature_map": { 2024-05-09T08:04:20.586 INFO:teuthology.orchestra.run.smithi001.stdout: "mon": [ 2024-05-09T08:04:20.586 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:04:20.586 INFO:teuthology.orchestra.run.smithi001.stdout: "features": "0x3f03cffffffdffff", 2024-05-09T08:04:20.586 INFO:teuthology.orchestra.run.smithi001.stdout: "release": "luminous", 2024-05-09T08:04:20.586 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 1 2024-05-09T08:04:20.586 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:04:20.586 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:04:20.586 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": [ 2024-05-09T08:04:20.586 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:04:20.587 INFO:teuthology.orchestra.run.smithi001.stdout: "features": "0x3f03cffffffdffff", 2024-05-09T08:04:20.587 INFO:teuthology.orchestra.run.smithi001.stdout: "release": "luminous", 2024-05-09T08:04:20.587 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 1 2024-05-09T08:04:20.587 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:04:20.587 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:04:20.587 INFO:teuthology.orchestra.run.smithi001.stdout: "client": [ 2024-05-09T08:04:20.587 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:04:20.587 INFO:teuthology.orchestra.run.smithi001.stdout: "features": "0x3f03cffffffdffff", 2024-05-09T08:04:20.587 INFO:teuthology.orchestra.run.smithi001.stdout: "release": "luminous", 2024-05-09T08:04:20.587 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 2 2024-05-09T08:04:20.587 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:04:20.587 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:04:20.587 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:04:20.587 INFO:teuthology.orchestra.run.smithi001.stdout: "stretch_mode": false 2024-05-09T08:04:20.588 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-05-09T08:04:20.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.559+0000 7f65ea800640 1 -- 172.21.15.1:0/2368849374 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f65c406b940 msgr2=0x7f65c406de00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:04:20.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.559+0000 7f65ea800640 1 --2- 172.21.15.1:0/2368849374 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f65c406b940 0x7f65c406de00 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f65e40b24e0 tx=0x7f65d003e000 comp rx=0 tx=0).stop 2024-05-09T08:04:20.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.559+0000 7f65ea800640 1 -- 172.21.15.1:0/2368849374 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65e4072c20 msgr2=0x7f65e4156f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:04:20.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.559+0000 7f65ea800640 1 --2- 172.21.15.1:0/2368849374 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65e4072c20 0x7f65e4156f90 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f65d400b550 tx=0x7f65d400ba10 comp rx=0 tx=0).stop 2024-05-09T08:04:20.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.559+0000 7f65e3e00640 1 -- 172.21.15.1:0/2368849374 reap_dead start 2024-05-09T08:04:20.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.559+0000 7f65ea800640 1 -- 172.21.15.1:0/2368849374 shutdown_connections 2024-05-09T08:04:20.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.559+0000 7f65ea800640 1 -- 172.21.15.1:0/2368849374 >> 172.21.15.1:0/2368849374 conn(0x7f65e406dad0 msgr2=0x7f65e4149db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:04:20.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.560+0000 7f65ea800640 1 -- 172.21.15.1:0/2368849374 shutdown_connections 2024-05-09T08:04:20.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.560+0000 7f65ea800640 1 -- 172.21.15.1:0/2368849374 wait complete. 2024-05-09T08:04:20.592 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-05-09T08:04:20.592 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon c 2024-05-09T08:04:20.592 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-05-09T08:04:20.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.741+0000 7ff88b200640 1 Processor -- start 2024-05-09T08:04:20.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.742+0000 7ff88b200640 1 -- start start 2024-05-09T08:04:20.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.743+0000 7ff88b200640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff88c0c6c50 0x7ff88c071af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:04:20.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.743+0000 7ff88b200640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7ff88c0c7030 0x7ff88c0c8280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:04:20.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.743+0000 7ff88b200640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff88c0c8f60 con 0x7ff88c06fbd0 2024-05-09T08:04:20.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.743+0000 7ff88b200640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7ff88c0c89a0 con 0x7ff88c0c7030 2024-05-09T08:04:20.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.743+0000 7ff88b200640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7ff88c0c8c80 con 0x7ff88c0c6c50 2024-05-09T08:04:20.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.743+0000 7ff889400640 1 -- >> v2:172.21.15.1:3300/0 conn(0x7ff88c0c7030 msgr2=0x7ff88c0c8280 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.1:3300/0 2024-05-09T08:04:20.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.743+0000 7ff889400640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7ff88c0c7030 0x7ff88c0c8280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T08:04:20.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.743+0000 7ff889e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff88c0c6c50 0x7ff88c071af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:04:20.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.743+0000 7ff88a800640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7ff88c06fbd0 0x7ff88c15beb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.1:38188/0 (socket says 172.21.15.1:38188) 2024-05-09T08:04:20.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.743+0000 7ff88a800640 1 -- 172.21.15.1:0/1824885729 learned_addr learned my addr 172.21.15.1:0/1824885729 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T08:04:20.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.744+0000 7ff888a00640 1 -- 172.21.15.1:0/1824885729 <== mon.1 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2969330483 0 0) 0x7ff88c0c8f60 con 0x7ff88c06fbd0 2024-05-09T08:04:20.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.744+0000 7ff888a00640 1 -- 172.21.15.1:0/1824885729 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff870003520 con 0x7ff88c06fbd0 2024-05-09T08:04:20.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.744+0000 7ff889e00640 1 -- 172.21.15.1:0/1824885729 >> v2:172.21.15.1:3300/0 conn(0x7ff88c0c7030 msgr2=0x7ff88c0c8280 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T08:04:20.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.744+0000 7ff889e00640 1 --2- 172.21.15.1:0/1824885729 >> v2:172.21.15.1:3300/0 conn(0x7ff88c0c7030 0x7ff88c0c8280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:04:20.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.744+0000 7ff889e00640 1 -- 172.21.15.1:0/1824885729 >> v1:172.21.15.7:6789/0 conn(0x7ff88c06fbd0 legacy=0x7ff88c15beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:04:20.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.744+0000 7ff889e00640 1 -- 172.21.15.1:0/1824885729 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff88c06f0f0 con 0x7ff88c0c6c50 2024-05-09T08:04:20.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.745+0000 7ff889e00640 1 --2- 172.21.15.1:0/1824885729 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff88c0c6c50 0x7ff88c071af0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ff88c0c6700 tx=0x7ff87c02a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=55382e3761e5208e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:04:20.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.746+0000 7ff888a00640 1 -- 172.21.15.1:0/1824885729 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff87c037070 con 0x7ff88c0c6c50 2024-05-09T08:04:20.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.746+0000 7ff888a00640 1 -- 172.21.15.1:0/1824885729 <== mon.1 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff87c032440 con 0x7ff88c0c6c50 2024-05-09T08:04:20.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.746+0000 7ff888a00640 1 -- 172.21.15.1:0/1824885729 <== mon.1 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff87c032810 con 0x7ff88c0c6c50 2024-05-09T08:04:20.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.746+0000 7ff88b200640 1 -- 172.21.15.1:0/1824885729 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff88c0c6c50 msgr2=0x7ff88c071af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:04:20.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.746+0000 7ff88b200640 1 --2- 172.21.15.1:0/1824885729 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff88c0c6c50 0x7ff88c071af0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ff88c0c6700 tx=0x7ff87c02a1c0 comp rx=0 tx=0).stop 2024-05-09T08:04:20.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.746+0000 7ff88b200640 1 -- 172.21.15.1:0/1824885729 shutdown_connections 2024-05-09T08:04:20.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.746+0000 7ff88b200640 1 --2- 172.21.15.1:0/1824885729 >> v2:172.21.15.1:3300/0 conn(0x7ff88c0c7030 0x7ff88c0c8280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:04:20.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.746+0000 7ff88b200640 1 --2- 172.21.15.1:0/1824885729 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff88c0c6c50 0x7ff88c071af0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:04:20.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.746+0000 7ff88b200640 1 -- 172.21.15.1:0/1824885729 >> 172.21.15.1:0/1824885729 conn(0x7ff88c06b250 msgr2=0x7ff88c06b630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:04:20.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.746+0000 7ff88b200640 1 -- 172.21.15.1:0/1824885729 shutdown_connections 2024-05-09T08:04:20.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.747+0000 7ff88b200640 1 -- 172.21.15.1:0/1824885729 wait complete. 2024-05-09T08:04:20.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.749+0000 7ff88b200640 1 Processor -- start 2024-05-09T08:04:20.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.749+0000 7ff88b200640 1 -- start start 2024-05-09T08:04:20.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.751+0000 7ff88b200640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ff88c06fbd0 0x7ff88c15a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:04:20.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.751+0000 7ff88b200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff88c0c6c50 0x7ff88c1f7cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:04:20.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.751+0000 7ff88b200640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff88c0c7030 0x7ff88c1f8230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:04:20.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.751+0000 7ff88b200640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ff88c0c7740 con 0x7ff88c0c7030 2024-05-09T08:04:20.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.751+0000 7ff88b200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff88c06f6b0 con 0x7ff88c0c6c50 2024-05-09T08:04:20.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.751+0000 7ff88b200640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7ff88c15c5c0 con 0x7ff88c06fbd0 2024-05-09T08:04:20.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.751+0000 7ff88a800640 1 -- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff88c0c7030 msgr2=0x7ff88c1f8230 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.1:3300/0 2024-05-09T08:04:20.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.751+0000 7ff88a800640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff88c0c7030 0x7ff88c1f8230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T08:04:20.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.751+0000 7ff889e00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ff88c06fbd0 0x7ff88c15a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:04:20.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.751+0000 7ff889e00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ff88c06fbd0 0x7ff88c15a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3301/0 says I am v2:172.21.15.1:40064/0 (socket says 172.21.15.1:40064) 2024-05-09T08:04:20.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.751+0000 7ff889e00640 1 -- 172.21.15.1:0/3375378063 learned_addr learned my addr 172.21.15.1:0/3375378063 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:04:20.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.751+0000 7ff889400640 1 --2- 172.21.15.1:0/3375378063 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff88c0c6c50 0x7ff88c1f7cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:04:20.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.752+0000 7ff889e00640 1 -- 172.21.15.1:0/3375378063 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff88c0c6c50 msgr2=0x7ff88c1f7cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:04:20.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.752+0000 7ff889e00640 1 --2- 172.21.15.1:0/3375378063 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff88c0c6c50 0x7ff88c1f7cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:04:20.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.752+0000 7ff889e00640 1 -- 172.21.15.1:0/3375378063 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff88c0c7030 msgr2=0x7ff88c1f8230 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T08:04:20.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.752+0000 7ff889e00640 1 --2- 172.21.15.1:0/3375378063 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff88c0c7030 0x7ff88c1f8230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:04:20.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.752+0000 7ff889e00640 1 -- 172.21.15.1:0/3375378063 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff88c0c0030 con 0x7ff88c06fbd0 2024-05-09T08:04:20.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.752+0000 7ff889e00640 1 --2- 172.21.15.1:0/3375378063 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ff88c06fbd0 0x7ff88c15a630 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7ff87c009890 tx=0x7ff87c0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:04:20.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.752+0000 7ff882000640 1 -- 172.21.15.1:0/3375378063 <== mon.2 v2:172.21.15.7:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff87c041070 con 0x7ff88c06fbd0 2024-05-09T08:04:20.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.752+0000 7ff882000640 1 -- 172.21.15.1:0/3375378063 <== mon.2 v2:172.21.15.7:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff87c03c870 con 0x7ff88c06fbd0 2024-05-09T08:04:20.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.752+0000 7ff882000640 1 -- 172.21.15.1:0/3375378063 <== mon.2 v2:172.21.15.7:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff87c037040 con 0x7ff88c06fbd0 2024-05-09T08:04:20.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.752+0000 7ff88b200640 1 -- 172.21.15.1:0/3375378063 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff88c1f4e20 con 0x7ff88c06fbd0 2024-05-09T08:04:20.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.753+0000 7ff88b200640 1 -- 172.21.15.1:0/3375378063 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7ff88c158440 con 0x7ff88c06fbd0 2024-05-09T08:04:20.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.754+0000 7ff88b200640 1 --2- 172.21.15.1:0/3375378063 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ff88c1590e0 0x7ff88c152340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:04:20.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.754+0000 7ff88b200640 1 -- 172.21.15.1:0/3375378063 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7ff88c0cb980 con 0x7ff88c1590e0 2024-05-09T08:04:20.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.754+0000 7ff889400640 1 --2- 172.21.15.1:0/3375378063 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ff88c1590e0 0x7ff88c152340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:04:20.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.755+0000 7ff889400640 1 -- 172.21.15.1:0/3375378063 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff88c04f660 con 0x7ff88c1590e0 2024-05-09T08:04:20.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.755+0000 7ff882000640 1 -- 172.21.15.1:0/3375378063 <== mon.2 v2:172.21.15.7:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff87c044070 con 0x7ff88c06fbd0 2024-05-09T08:04:20.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.755+0000 7ff889400640 1 --2- 172.21.15.1:0/3375378063 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ff88c1590e0 0x7ff88c152340 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7ff88c0c9100 tx=0x7ff878009b00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:04:20.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.755+0000 7ff882000640 1 --2- 172.21.15.1:0/3375378063 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7ff86c06bde0 0x7ff86c06e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:04:20.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.756+0000 7ff882000640 1 -- 172.21.15.1:0/3375378063 <== mon.2 v2:172.21.15.7:3301/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7ff87c0abbd0 con 0x7ff88c06fbd0 2024-05-09T08:04:20.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.756+0000 7ff882000640 1 -- 172.21.15.1:0/3375378063 <== mon.2 v2:172.21.15.7:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff878016070 con 0x7ff88c1590e0 2024-05-09T08:04:20.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.756+0000 7ff88a800640 1 --2- 172.21.15.1:0/3375378063 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7ff86c06bde0 0x7ff86c06e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:04:20.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.756+0000 7ff882000640 1 -- 172.21.15.1:0/3375378063 <== mon.2 v2:172.21.15.7:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7ff878012600 con 0x7ff88c1590e0 2024-05-09T08:04:20.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.756+0000 7ff882000640 1 -- 172.21.15.1:0/3375378063 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ff88c1590e0 msgr2=0x7ff88c152340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:04:20.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.756+0000 7ff882000640 1 --2- 172.21.15.1:0/3375378063 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ff88c1590e0 0x7ff88c152340 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7ff88c0c9100 tx=0x7ff878009b00 comp rx=0 tx=0).stop 2024-05-09T08:04:20.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.756+0000 7ff882000640 1 -- 172.21.15.1:0/3375378063 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ff88c1590e0 msgr2=0x7ff88c152340 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-05-09T08:04:20.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.756+0000 7ff882000640 1 --2- 172.21.15.1:0/3375378063 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ff88c1590e0 0x7ff88c152340 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:04:20.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.757+0000 7ff88a800640 1 --2- 172.21.15.1:0/3375378063 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7ff86c06bde0 0x7ff86c06e2a0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7ff884004b90 tx=0x7ff884001710 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:04:20.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.763+0000 7ff88b200640 1 --2- 172.21.15.1:0/3375378063 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ff88c0bfa70 0x7ff88c152880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:04:20.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.763+0000 7ff88b200640 1 -- 172.21.15.1:0/3375378063 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7ff88c1f4c80 con 0x7ff88c0bfa70 2024-05-09T08:04:20.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.763+0000 7ff889400640 1 --2- 172.21.15.1:0/3375378063 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ff88c0bfa70 0x7ff88c152880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:04:20.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.764+0000 7ff889400640 1 -- 172.21.15.1:0/3375378063 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7ff88c0c7520 con 0x7ff88c0bfa70 2024-05-09T08:04:20.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.764+0000 7ff889400640 1 --2- 172.21.15.1:0/3375378063 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ff88c0bfa70 0x7ff88c152880 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7ff878015000 tx=0x7ff8780059d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:04:20.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.765+0000 7ff882000640 1 -- 172.21.15.1:0/3375378063 <== mon.2 v2:172.21.15.7:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff878016040 con 0x7ff88c0bfa70 2024-05-09T08:04:20.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.765+0000 7ff882000640 1 -- 172.21.15.1:0/3375378063 <== mon.2 v2:172.21.15.7:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5177 (secure 0 0 0) 0x7ff878005a40 con 0x7ff88c0bfa70 2024-05-09T08:04:20.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.765+0000 7ff882000640 1 -- 172.21.15.1:0/3375378063 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ff88c0bfa70 msgr2=0x7ff88c152880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:04:20.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.765+0000 7ff882000640 1 --2- 172.21.15.1:0/3375378063 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ff88c0bfa70 0x7ff88c152880 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7ff878015000 tx=0x7ff8780059d0 comp rx=0 tx=0).stop 2024-05-09T08:04:20.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.765+0000 7ff882000640 1 -- 172.21.15.1:0/3375378063 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ff88c0bfa70 msgr2=0x7ff88c152880 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-05-09T08:04:20.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.765+0000 7ff882000640 1 --2- 172.21.15.1:0/3375378063 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ff88c0bfa70 0x7ff88c152880 secure :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7ff878015000 tx=0x7ff8780059d0 comp rx=0 tx=0).stop 2024-05-09T08:04:20.781 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-05-09T08:04:20.781 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "c", 2024-05-09T08:04:20.781 INFO:teuthology.orchestra.run.smithi001.stdout: "rank": 2, 2024-05-09T08:04:20.781 INFO:teuthology.orchestra.run.smithi001.stdout: "state": "peon", 2024-05-09T08:04:20.781 INFO:teuthology.orchestra.run.smithi001.stdout: "election_epoch": 12, 2024-05-09T08:04:20.782 INFO:teuthology.orchestra.run.smithi001.stdout: "uptime": 144181, 2024-05-09T08:04:20.782 INFO:teuthology.orchestra.run.smithi001.stdout: "quorum": [ 2024-05-09T08:04:20.782 INFO:teuthology.orchestra.run.smithi001.stdout: 1, 2024-05-09T08:04:20.782 INFO:teuthology.orchestra.run.smithi001.stdout: 2 2024-05-09T08:04:20.782 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:04:20.782 INFO:teuthology.orchestra.run.smithi001.stdout: "quorum_age": 1, 2024-05-09T08:04:20.782 INFO:teuthology.orchestra.run.smithi001.stdout: "features": { 2024-05-09T08:04:20.782 INFO:teuthology.orchestra.run.smithi001.stdout: "required_con": "2450521708007866372", 2024-05-09T08:04:20.782 INFO:teuthology.orchestra.run.smithi001.stdout: "required_mon": [ 2024-05-09T08:04:20.782 INFO:teuthology.orchestra.run.smithi001.stdout: "kraken", 2024-05-09T08:04:20.782 INFO:teuthology.orchestra.run.smithi001.stdout: "luminous", 2024-05-09T08:04:20.782 INFO:teuthology.orchestra.run.smithi001.stdout: "mimic", 2024-05-09T08:04:20.782 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap-prune", 2024-05-09T08:04:20.782 INFO:teuthology.orchestra.run.smithi001.stdout: "nautilus", 2024-05-09T08:04:20.783 INFO:teuthology.orchestra.run.smithi001.stdout: "octopus", 2024-05-09T08:04:20.783 INFO:teuthology.orchestra.run.smithi001.stdout: "pacific", 2024-05-09T08:04:20.783 INFO:teuthology.orchestra.run.smithi001.stdout: "elector-pinging", 2024-05-09T08:04:20.783 INFO:teuthology.orchestra.run.smithi001.stdout: "quincy", 2024-05-09T08:04:20.783 INFO:teuthology.orchestra.run.smithi001.stdout: "reef", 2024-05-09T08:04:20.783 INFO:teuthology.orchestra.run.smithi001.stdout: "squid" 2024-05-09T08:04:20.783 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:04:20.783 INFO:teuthology.orchestra.run.smithi001.stdout: "quorum_con": "4540701547738038271", 2024-05-09T08:04:20.783 INFO:teuthology.orchestra.run.smithi001.stdout: "quorum_mon": [ 2024-05-09T08:04:20.783 INFO:teuthology.orchestra.run.smithi001.stdout: "kraken", 2024-05-09T08:04:20.783 INFO:teuthology.orchestra.run.smithi001.stdout: "luminous", 2024-05-09T08:04:20.783 INFO:teuthology.orchestra.run.smithi001.stdout: "mimic", 2024-05-09T08:04:20.783 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap-prune", 2024-05-09T08:04:20.783 INFO:teuthology.orchestra.run.smithi001.stdout: "nautilus", 2024-05-09T08:04:20.784 INFO:teuthology.orchestra.run.smithi001.stdout: "octopus", 2024-05-09T08:04:20.784 INFO:teuthology.orchestra.run.smithi001.stdout: "pacific", 2024-05-09T08:04:20.784 INFO:teuthology.orchestra.run.smithi001.stdout: "elector-pinging", 2024-05-09T08:04:20.784 INFO:teuthology.orchestra.run.smithi001.stdout: "quincy", 2024-05-09T08:04:20.784 INFO:teuthology.orchestra.run.smithi001.stdout: "reef", 2024-05-09T08:04:20.784 INFO:teuthology.orchestra.run.smithi001.stdout: "squid" 2024-05-09T08:04:20.784 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:04:20.784 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:04:20.784 INFO:teuthology.orchestra.run.smithi001.stdout: "outside_quorum": [], 2024-05-09T08:04:20.784 INFO:teuthology.orchestra.run.smithi001.stdout: "extra_probe_peers": [], 2024-05-09T08:04:20.784 INFO:teuthology.orchestra.run.smithi001.stdout: "sync_provider": [], 2024-05-09T08:04:20.784 INFO:teuthology.orchestra.run.smithi001.stdout: "monmap": { 2024-05-09T08:04:20.784 INFO:teuthology.orchestra.run.smithi001.stdout: "epoch": 1, 2024-05-09T08:04:20.784 INFO:teuthology.orchestra.run.smithi001.stdout: "fsid": "54329ec8-3270-4043-9a82-c6cd7420aa7f", 2024-05-09T08:04:20.785 INFO:teuthology.orchestra.run.smithi001.stdout: "modified": "2024-05-09T08:01:21.819270Z", 2024-05-09T08:04:20.785 INFO:teuthology.orchestra.run.smithi001.stdout: "created": "2024-05-09T08:01:21.819270Z", 2024-05-09T08:04:20.785 INFO:teuthology.orchestra.run.smithi001.stdout: "min_mon_release": 19, 2024-05-09T08:04:20.785 INFO:teuthology.orchestra.run.smithi001.stdout: "min_mon_release_name": "squid", 2024-05-09T08:04:20.785 INFO:teuthology.orchestra.run.smithi001.stdout: "election_strategy": 1, 2024-05-09T08:04:20.785 INFO:teuthology.orchestra.run.smithi001.stdout: "disallowed_leaders: ": "", 2024-05-09T08:04:20.785 INFO:teuthology.orchestra.run.smithi001.stdout: "stretch_mode": false, 2024-05-09T08:04:20.785 INFO:teuthology.orchestra.run.smithi001.stdout: "tiebreaker_mon": "", 2024-05-09T08:04:20.785 INFO:teuthology.orchestra.run.smithi001.stdout: "removed_ranks: ": "", 2024-05-09T08:04:20.785 INFO:teuthology.orchestra.run.smithi001.stdout: "features": { 2024-05-09T08:04:20.785 INFO:teuthology.orchestra.run.smithi001.stdout: "persistent": [ 2024-05-09T08:04:20.785 INFO:teuthology.orchestra.run.smithi001.stdout: "kraken", 2024-05-09T08:04:20.785 INFO:teuthology.orchestra.run.smithi001.stdout: "luminous", 2024-05-09T08:04:20.786 INFO:teuthology.orchestra.run.smithi001.stdout: "mimic", 2024-05-09T08:04:20.786 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap-prune", 2024-05-09T08:04:20.786 INFO:teuthology.orchestra.run.smithi001.stdout: "nautilus", 2024-05-09T08:04:20.786 INFO:teuthology.orchestra.run.smithi001.stdout: "octopus", 2024-05-09T08:04:20.786 INFO:teuthology.orchestra.run.smithi001.stdout: "pacific", 2024-05-09T08:04:20.786 INFO:teuthology.orchestra.run.smithi001.stdout: "elector-pinging", 2024-05-09T08:04:20.786 INFO:teuthology.orchestra.run.smithi001.stdout: "quincy", 2024-05-09T08:04:20.786 INFO:teuthology.orchestra.run.smithi001.stdout: "reef", 2024-05-09T08:04:20.786 INFO:teuthology.orchestra.run.smithi001.stdout: "squid" 2024-05-09T08:04:20.786 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:04:20.786 INFO:teuthology.orchestra.run.smithi001.stdout: "optional": [] 2024-05-09T08:04:20.786 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:04:20.786 INFO:teuthology.orchestra.run.smithi001.stdout: "mons": [ 2024-05-09T08:04:20.787 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:04:20.787 INFO:teuthology.orchestra.run.smithi001.stdout: "rank": 0, 2024-05-09T08:04:20.787 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "a", 2024-05-09T08:04:20.787 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-05-09T08:04:20.787 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-05-09T08:04:20.787 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:04:20.787 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-05-09T08:04:20.787 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:3300", 2024-05-09T08:04:20.787 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:04:20.787 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:04:20.787 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:04:20.787 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-05-09T08:04:20.787 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6789", 2024-05-09T08:04:20.787 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:04:20.788 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:04:20.788 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:04:20.788 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:04:20.788 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6789/0", 2024-05-09T08:04:20.788 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.1:6789/0", 2024-05-09T08:04:20.788 INFO:teuthology.orchestra.run.smithi001.stdout: "priority": 0, 2024-05-09T08:04:20.788 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 0, 2024-05-09T08:04:20.788 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_location": "{}" 2024-05-09T08:04:20.788 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:04:20.788 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:04:20.788 INFO:teuthology.orchestra.run.smithi001.stdout: "rank": 1, 2024-05-09T08:04:20.788 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "b", 2024-05-09T08:04:20.788 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-05-09T08:04:20.789 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-05-09T08:04:20.789 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:04:20.789 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-05-09T08:04:20.789 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:3300", 2024-05-09T08:04:20.789 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:04:20.789 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:04:20.789 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:04:20.789 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-05-09T08:04:20.789 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:6789", 2024-05-09T08:04:20.789 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:04:20.789 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:04:20.789 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:04:20.789 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:04:20.790 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:6789/0", 2024-05-09T08:04:20.790 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.7:6789/0", 2024-05-09T08:04:20.790 INFO:teuthology.orchestra.run.smithi001.stdout: "priority": 0, 2024-05-09T08:04:20.790 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 0, 2024-05-09T08:04:20.790 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_location": "{}" 2024-05-09T08:04:20.790 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:04:20.790 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:04:20.790 INFO:teuthology.orchestra.run.smithi001.stdout: "rank": 2, 2024-05-09T08:04:20.790 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "c", 2024-05-09T08:04:20.790 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-05-09T08:04:20.790 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-05-09T08:04:20.790 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:04:20.790 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-05-09T08:04:20.791 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:3301", 2024-05-09T08:04:20.791 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:04:20.791 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:04:20.791 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:04:20.791 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-05-09T08:04:20.791 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:6790", 2024-05-09T08:04:20.791 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:04:20.791 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:04:20.791 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:04:20.791 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:04:20.791 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:6790/0", 2024-05-09T08:04:20.791 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.7:6790/0", 2024-05-09T08:04:20.791 INFO:teuthology.orchestra.run.smithi001.stdout: "priority": 0, 2024-05-09T08:04:20.792 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 0, 2024-05-09T08:04:20.792 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_location": "{}" 2024-05-09T08:04:20.792 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:04:20.792 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:04:20.792 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:04:20.792 INFO:teuthology.orchestra.run.smithi001.stdout: "feature_map": { 2024-05-09T08:04:20.792 INFO:teuthology.orchestra.run.smithi001.stdout: "mon": [ 2024-05-09T08:04:20.792 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:04:20.792 INFO:teuthology.orchestra.run.smithi001.stdout: "features": "0x3f03cffffffdffff", 2024-05-09T08:04:20.792 INFO:teuthology.orchestra.run.smithi001.stdout: "release": "luminous", 2024-05-09T08:04:20.792 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 1 2024-05-09T08:04:20.792 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:04:20.792 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:04:20.792 INFO:teuthology.orchestra.run.smithi001.stdout: "mds": [ 2024-05-09T08:04:20.793 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:04:20.793 INFO:teuthology.orchestra.run.smithi001.stdout: "features": "0x3f03cffffffdffff", 2024-05-09T08:04:20.793 INFO:teuthology.orchestra.run.smithi001.stdout: "release": "luminous", 2024-05-09T08:04:20.793 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 6 2024-05-09T08:04:20.793 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:04:20.793 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:04:20.793 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": [ 2024-05-09T08:04:20.793 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:04:20.793 INFO:teuthology.orchestra.run.smithi001.stdout: "features": "0x3f03cffffffdffff", 2024-05-09T08:04:20.793 INFO:teuthology.orchestra.run.smithi001.stdout: "release": "luminous", 2024-05-09T08:04:20.793 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 7 2024-05-09T08:04:20.793 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:04:20.793 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:04:20.794 INFO:teuthology.orchestra.run.smithi001.stdout: "client": [ 2024-05-09T08:04:20.794 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:04:20.794 INFO:teuthology.orchestra.run.smithi001.stdout: "features": "0x2f018fb87aa4aafe", 2024-05-09T08:04:20.794 INFO:teuthology.orchestra.run.smithi001.stdout: "release": "luminous", 2024-05-09T08:04:20.794 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 1 2024-05-09T08:04:20.794 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:04:20.794 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:04:20.794 INFO:teuthology.orchestra.run.smithi001.stdout: "features": "0x3f03cffffffdffff", 2024-05-09T08:04:20.794 INFO:teuthology.orchestra.run.smithi001.stdout: "release": "luminous", 2024-05-09T08:04:20.794 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 6 2024-05-09T08:04:20.794 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:04:20.794 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:04:20.794 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr": [ 2024-05-09T08:04:20.795 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:04:20.795 INFO:teuthology.orchestra.run.smithi001.stdout: "features": "0x3f03cffffffdffff", 2024-05-09T08:04:20.795 INFO:teuthology.orchestra.run.smithi001.stdout: "release": "luminous", 2024-05-09T08:04:20.795 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 2 2024-05-09T08:04:20.795 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:04:20.795 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:04:20.795 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:04:20.795 INFO:teuthology.orchestra.run.smithi001.stdout: "stretch_mode": false 2024-05-09T08:04:20.795 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-05-09T08:04:20.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.765+0000 7ff88b200640 1 -- 172.21.15.1:0/3375378063 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7ff86c06bde0 msgr2=0x7ff86c06e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:04:20.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.765+0000 7ff88b200640 1 --2- 172.21.15.1:0/3375378063 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7ff86c06bde0 0x7ff86c06e2a0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7ff884004b90 tx=0x7ff884001710 comp rx=0 tx=0).stop 2024-05-09T08:04:20.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.765+0000 7ff88b200640 1 -- 172.21.15.1:0/3375378063 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ff88c06fbd0 msgr2=0x7ff88c15a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:04:20.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.765+0000 7ff88b200640 1 --2- 172.21.15.1:0/3375378063 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ff88c06fbd0 0x7ff88c15a630 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7ff87c009890 tx=0x7ff87c0046c0 comp rx=0 tx=0).stop 2024-05-09T08:04:20.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.765+0000 7ff88a800640 1 -- 172.21.15.1:0/3375378063 reap_dead start 2024-05-09T08:04:20.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.765+0000 7ff88a800640 1 -- 172.21.15.1:0/3375378063 reap_dead start 2024-05-09T08:04:20.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.765+0000 7ff88b200640 1 -- 172.21.15.1:0/3375378063 shutdown_connections 2024-05-09T08:04:20.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.765+0000 7ff88b200640 1 -- 172.21.15.1:0/3375378063 >> 172.21.15.1:0/3375378063 conn(0x7ff88c06b250 msgr2=0x7ff88c0c5ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:04:20.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.766+0000 7ff88b200640 1 -- 172.21.15.1:0/3375378063 shutdown_connections 2024-05-09T08:04:20.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:20.766+0000 7ff88b200640 1 -- 172.21.15.1:0/3375378063 wait complete. 2024-05-09T08:04:20.797 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-05-09T08:04:20.797 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2024-05-09T08:04:34.006 INFO:tasks.workunit.client.0.smithi001.stderr:Updating files: 61% (7771/12535) Updating files: 62% (7772/12535) Updating files: 63% (7898/12535) Updating files: 64% (8023/12535) Updating files: 65% (8148/12535) Updating files: 66% (8274/12535) Updating files: 67% (8399/12535) Updating files: 68% (8524/12535) Updating files: 69% (8650/12535) Updating files: 70% (8775/12535) Updating files: 71% (8900/12535) Updating files: 72% (9026/12535) Updating files: 73% (9151/12535) Updating files: 74% (9276/12535) Updating files: 75% (9402/12535) Updating files: 76% (9527/12535) Updating files: 77% (9652/12535) Updating files: 78% (9778/12535) Updating files: 79% (9903/12535) Updating files: 80% (10028/12535) Updating files: 81% (10154/12535) Updating files: 82% (10279/12535) Updating files: 83% (10405/12535) Updating files: 84% (10530/12535) Updating files: 85% (10655/12535) Updating files: 86% (10781/12535) Updating files: 87% (10906/12535) Updating files: 88% (11031/12535) Updating files: 89% (11157/12535) Updating files: 90% (11282/12535) Updating files: 91% (11407/12535) Updating files: 92% (11533/12535) Updating files: 93% (11658/12535) Updating files: 94% (11783/12535) Updating files: 95% (11909/12535) Updating files: 96% (12034/12535) Updating files: 97% (12159/12535) Updating files: 98% (12285/12535) Updating files: 98% (12371/12535) Updating files: 99% (12410/12535) Updating files: 100% (12535/12535) Updating files: 100% (12535/12535), done. 2024-05-09T08:04:34.390 INFO:tasks.workunit.client.0.smithi001.stderr:Note: switching to 'be85c202969d25af6e130678e7d73a20e9c4ca11'. 2024-05-09T08:04:34.390 INFO:tasks.workunit.client.0.smithi001.stderr: 2024-05-09T08:04:34.390 INFO:tasks.workunit.client.0.smithi001.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-05-09T08:04:34.391 INFO:tasks.workunit.client.0.smithi001.stderr:changes and commit them, and you can discard any commits you make in this 2024-05-09T08:04:34.391 INFO:tasks.workunit.client.0.smithi001.stderr:state without impacting any branches by switching back to a branch. 2024-05-09T08:04:34.391 INFO:tasks.workunit.client.0.smithi001.stderr: 2024-05-09T08:04:34.391 INFO:tasks.workunit.client.0.smithi001.stderr:If you want to create a new branch to retain commits you create, you may 2024-05-09T08:04:34.391 INFO:tasks.workunit.client.0.smithi001.stderr:do so (now or later) by using -c with the switch command. Example: 2024-05-09T08:04:34.391 INFO:tasks.workunit.client.0.smithi001.stderr: 2024-05-09T08:04:34.391 INFO:tasks.workunit.client.0.smithi001.stderr: git switch -c 2024-05-09T08:04:34.391 INFO:tasks.workunit.client.0.smithi001.stderr: 2024-05-09T08:04:34.391 INFO:tasks.workunit.client.0.smithi001.stderr:Or undo this operation with: 2024-05-09T08:04:34.391 INFO:tasks.workunit.client.0.smithi001.stderr: 2024-05-09T08:04:34.391 INFO:tasks.workunit.client.0.smithi001.stderr: git switch - 2024-05-09T08:04:34.391 INFO:tasks.workunit.client.0.smithi001.stderr: 2024-05-09T08:04:34.391 INFO:tasks.workunit.client.0.smithi001.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-05-09T08:04:34.391 INFO:tasks.workunit.client.0.smithi001.stderr: 2024-05-09T08:04:34.392 INFO:tasks.workunit.client.0.smithi001.stderr:HEAD is now at be85c202969 Merge pull request #57242 from cbodley/wip-65767-squid 2024-05-09T08:04:34.400 DEBUG:teuthology.orchestra.run.smithi001:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-05-09T08:04:34.460 INFO:tasks.workunit.client.0.smithi001.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-05-09T08:04:34.468 INFO:tasks.workunit.client.0.smithi001.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-05-09T08:04:34.468 INFO:tasks.workunit.client.0.smithi001.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-05-09T08:04:35.236 INFO:tasks.workunit.client.0.smithi001.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-05-09T08:04:35.346 INFO:tasks.workunit.client.0.smithi001.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-05-09T08:04:35.430 INFO:tasks.workunit.client.0.smithi001.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-05-09T08:04:35.437 INFO:tasks.workunit.client.0.smithi001.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-05-09T08:04:35.438 INFO:tasks.workunit.client.0.smithi001.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-05-09T08:04:35.517 INFO:tasks.workunit.client.0.smithi001.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-05-09T08:04:35.527 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-05-09T08:04:35.527 DEBUG:teuthology.orchestra.run.smithi001:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-05-09T08:04:35.577 INFO:tasks.workunit:Running workunits matching suites/iozone.sh on client.0... 2024-05-09T08:04:35.579 INFO:tasks.workunit:Running workunit suites/iozone.sh... 2024-05-09T08:04:35.579 DEBUG:teuthology.orchestra.run.smithi001:workunit test suites/iozone.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=be85c202969d25af6e130678e7d73a20e9c4ca11 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/iozone.sh 2024-05-09T08:04:35.701 INFO:tasks.workunit.client.0.smithi001.stderr:+ iozone -c -e -s 1024M -r 16K -t 1 -F f1 -i 0 -i 1 2024-05-09T08:04:35.803 INFO:tasks.workunit.client.0.smithi001.stdout: Iozone: Performance Test of File I/O 2024-05-09T08:04:35.804 INFO:tasks.workunit.client.0.smithi001.stdout: Version $Revision: 3.494 $ 2024-05-09T08:04:35.804 INFO:tasks.workunit.client.0.smithi001.stdout: Compiled for 64 bit mode. 2024-05-09T08:04:35.804 INFO:tasks.workunit.client.0.smithi001.stdout: Build: linux-AMD64 2024-05-09T08:04:35.804 INFO:tasks.workunit.client.0.smithi001.stdout: 2024-05-09T08:04:35.804 INFO:tasks.workunit.client.0.smithi001.stdout: Contributors:William Norcott, Don Capps, Isom Crawford, Kirby Collins 2024-05-09T08:04:35.804 INFO:tasks.workunit.client.0.smithi001.stdout: Al Slater, Scott Rhine, Mike Wisner, Ken Goss 2024-05-09T08:04:35.804 INFO:tasks.workunit.client.0.smithi001.stdout: Steve Landherr, Brad Smith, Mark Kelly, Dr. Alain CYR, 2024-05-09T08:04:35.804 INFO:tasks.workunit.client.0.smithi001.stdout: Randy Dunlap, Mark Montague, Dan Million, Gavin Brebner, 2024-05-09T08:04:35.804 INFO:tasks.workunit.client.0.smithi001.stdout: Jean-Marc Zucconi, Jeff Blomberg, Benny Halevy, Dave Boone, 2024-05-09T08:04:35.804 INFO:tasks.workunit.client.0.smithi001.stdout: Erik Habbinga, Kris Strecker, Walter Wong, Joshua Root, 2024-05-09T08:04:35.804 INFO:tasks.workunit.client.0.smithi001.stdout: Fabrice Bacchella, Zhenghua Xue, Qin Li, Darren Sawyer, 2024-05-09T08:04:35.805 INFO:tasks.workunit.client.0.smithi001.stdout: Vangel Bojaxhi, Ben England, Vikentsi Lapa, 2024-05-09T08:04:35.805 INFO:tasks.workunit.client.0.smithi001.stdout: Alexey Skidanov, Sudhir Kumar. 2024-05-09T08:04:35.805 INFO:tasks.workunit.client.0.smithi001.stdout: 2024-05-09T08:04:35.805 INFO:tasks.workunit.client.0.smithi001.stdout: Run began: Thu May 9 08:04:35 2024 2024-05-09T08:04:35.805 INFO:tasks.workunit.client.0.smithi001.stdout: 2024-05-09T08:04:35.805 INFO:tasks.workunit.client.0.smithi001.stdout: Include close in write timing 2024-05-09T08:04:35.805 INFO:tasks.workunit.client.0.smithi001.stdout: Include fsync in write timing 2024-05-09T08:04:35.805 INFO:tasks.workunit.client.0.smithi001.stdout: File size set to 1048576 kB 2024-05-09T08:04:35.805 INFO:tasks.workunit.client.0.smithi001.stdout: Record Size 16 kB 2024-05-09T08:04:35.805 INFO:tasks.workunit.client.0.smithi001.stdout: Command line used: iozone -c -e -s 1024M -r 16K -t 1 -F f1 -i 0 -i 1 2024-05-09T08:04:35.805 INFO:tasks.workunit.client.0.smithi001.stdout: Output is in kBytes/sec 2024-05-09T08:04:35.805 INFO:tasks.workunit.client.0.smithi001.stdout: Time Resolution = 0.000001 seconds. 2024-05-09T08:04:35.805 INFO:tasks.workunit.client.0.smithi001.stdout: Processor cache size set to 1024 kBytes. 2024-05-09T08:04:35.805 INFO:tasks.workunit.client.0.smithi001.stdout: Processor cache line size set to 32 bytes. 2024-05-09T08:04:35.806 INFO:tasks.workunit.client.0.smithi001.stdout: File stride size set to 17 * record size. 2024-05-09T08:04:35.806 INFO:tasks.workunit.client.0.smithi001.stdout: Throughput test with 1 process 2024-05-09T08:04:35.806 INFO:tasks.workunit.client.0.smithi001.stdout: Each process writes a 1048576 kByte file in 16 kByte records 2024-05-09T08:04:40.538 INFO:tasks.workunit.client.0.smithi001.stdout: 2024-05-09T08:04:40.539 INFO:tasks.workunit.client.0.smithi001.stdout: Children see throughput for 1 initial writers = 221756.88 kB/sec 2024-05-09T08:04:40.539 INFO:tasks.workunit.client.0.smithi001.stdout: Parent sees throughput for 1 initial writers = 221719.73 kB/sec 2024-05-09T08:04:40.539 INFO:tasks.workunit.client.0.smithi001.stdout: Min throughput per process = 221756.88 kB/sec 2024-05-09T08:04:40.539 INFO:tasks.workunit.client.0.smithi001.stdout: Max throughput per process = 221756.88 kB/sec 2024-05-09T08:04:40.539 INFO:tasks.workunit.client.0.smithi001.stdout: Avg throughput per process = 221756.88 kB/sec 2024-05-09T08:04:40.539 INFO:tasks.workunit.client.0.smithi001.stdout: Min xfer = 1048576.00 kB 2024-05-09T08:04:40.798 INFO:tasks.mon_thrash.mon_thrasher:killing mon.a 2024-05-09T08:04:40.798 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.a 2024-05-09T08:04:40.798 INFO:tasks.ceph.mon.a:Restarting daemon 2024-05-09T08:04:40.799 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-05-09T08:04:40.807 INFO:tasks.ceph.mon.a:Started 2024-05-09T08:04:40.807 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-05-09T08:04:40.807 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-05-09T08:04:40.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:40.970+0000 7fcfcb400640 1 Processor -- start 2024-05-09T08:04:40.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:40.970+0000 7fcfcb400640 1 -- start start 2024-05-09T08:04:40.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:40.971+0000 7fcfcb400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fcfc406dca0 0x7fcfc4072900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:04:40.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:40.971+0000 7fcfcb400640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fcfc406ece0 0x7fcfc4159010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:04:40.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:40.971+0000 7fcfcb400640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcfc4073200 con 0x7fcfc40c6ea0 2024-05-09T08:04:40.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:40.971+0000 7fcfcb400640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fcfc4072e40 con 0x7fcfc406dca0 2024-05-09T08:04:40.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:40.971+0000 7fcfcb400640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7fcfc40737c0 con 0x7fcfc406ece0 2024-05-09T08:04:40.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:40.972+0000 7fcfc8a00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fcfc406dca0 0x7fcfc4072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:04:40.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:40.972+0000 7fcfc9400640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fcfc406ece0 0x7fcfc4159010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:04:40.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:40.972+0000 7fcfc3e00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fcfc40c6ea0 0x7fcfc406e080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.1:44342/0 (socket says 172.21.15.1:44342) 2024-05-09T08:04:40.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:40.972+0000 7fcfc3e00640 1 -- 172.21.15.1:0/1436945746 learned_addr learned my addr 172.21.15.1:0/1436945746 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T08:04:40.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:40.972+0000 7fcfc3400640 1 -- 172.21.15.1:0/1436945746 <== mon.1 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3633349323 0 0) 0x7fcfc4073200 con 0x7fcfc40c6ea0 2024-05-09T08:04:40.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:40.972+0000 7fcfc3400640 1 -- 172.21.15.1:0/1436945746 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcfac003520 con 0x7fcfc40c6ea0 2024-05-09T08:04:40.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:40.972+0000 7fcfc8a00640 1 -- 172.21.15.1:0/1436945746 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fcfc406ece0 msgr2=0x7fcfc4159010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:04:40.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:40.972+0000 7fcfc8a00640 1 --2- 172.21.15.1:0/1436945746 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fcfc406ece0 0x7fcfc4159010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:04:40.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:40.972+0000 7fcfc8a00640 1 -- 172.21.15.1:0/1436945746 >> v1:172.21.15.7:6789/0 conn(0x7fcfc40c6ea0 legacy=0x7fcfc406e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:04:40.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:40.972+0000 7fcfc8a00640 1 -- 172.21.15.1:0/1436945746 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfc40734e0 con 0x7fcfc406dca0 2024-05-09T08:04:40.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:40.974+0000 7fcfc8a00640 1 --2- 172.21.15.1:0/1436945746 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcfc406dca0 0x7fcfc4072900 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fcfb80097a0 tx=0x7fcfb802a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dfc2ca59dbde0207 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:04:40.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:40.974+0000 7fcfc3400640 1 -- 172.21.15.1:0/1436945746 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfb8037070 con 0x7fcfc406dca0 2024-05-09T08:04:40.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:40.974+0000 7fcfc3400640 1 -- 172.21.15.1:0/1436945746 <== mon.1 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcfb8032440 con 0x7fcfc406dca0 2024-05-09T08:04:40.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:40.974+0000 7fcfc3400640 1 -- 172.21.15.1:0/1436945746 <== mon.1 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfb8032810 con 0x7fcfc406dca0 2024-05-09T08:04:40.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:40.975+0000 7fcfcb400640 1 -- 172.21.15.1:0/1436945746 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcfc406dca0 msgr2=0x7fcfc4072900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:04:40.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:40.975+0000 7fcfcb400640 1 --2- 172.21.15.1:0/1436945746 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcfc406dca0 0x7fcfc4072900 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fcfb80097a0 tx=0x7fcfb802a180 comp rx=0 tx=0).stop 2024-05-09T08:04:40.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:40.975+0000 7fcfcb400640 1 -- 172.21.15.1:0/1436945746 shutdown_connections 2024-05-09T08:04:40.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:40.975+0000 7fcfcb400640 1 --2- 172.21.15.1:0/1436945746 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fcfc406ece0 0x7fcfc4159010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:04:40.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:40.975+0000 7fcfcb400640 1 --2- 172.21.15.1:0/1436945746 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcfc406dca0 0x7fcfc4072900 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:04:40.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:40.975+0000 7fcfcb400640 1 -- 172.21.15.1:0/1436945746 >> 172.21.15.1:0/1436945746 conn(0x7fcfc40cc070 msgr2=0x7fcfc40cc480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:04:40.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:40.976+0000 7fcfcb400640 1 -- 172.21.15.1:0/1436945746 shutdown_connections 2024-05-09T08:04:40.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:40.976+0000 7fcfcb400640 1 -- 172.21.15.1:0/1436945746 wait complete. 2024-05-09T08:04:40.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:40.978+0000 7fcfcb400640 1 Processor -- start 2024-05-09T08:04:40.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:40.978+0000 7fcfcb400640 1 -- start start 2024-05-09T08:04:40.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:40.980+0000 7fcfcb400640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fcfc406dca0 0x7fcfc40c2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:04:40.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:40.980+0000 7fcfcb400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcfc406ece0 0x7fcfc41f4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:04:40.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:40.980+0000 7fcfcb400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcfc40c6ea0 0x7fcfc40c5f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:04:40.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:40.980+0000 7fcfcb400640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fcfc4072470 con 0x7fcfc40c6ea0 2024-05-09T08:04:40.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:40.980+0000 7fcfcb400640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fcfc4159550 con 0x7fcfc406ece0 2024-05-09T08:04:40.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:40.980+0000 7fcfcb400640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7fcfc4072100 con 0x7fcfc406dca0 2024-05-09T08:04:40.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:40.980+0000 7fcfc8a00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fcfc406dca0 0x7fcfc40c2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:04:40.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:40.980+0000 7fcfc8a00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fcfc406dca0 0x7fcfc40c2a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3301/0 says I am v2:172.21.15.1:35444/0 (socket says 172.21.15.1:35444) 2024-05-09T08:04:40.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:40.980+0000 7fcfc8a00640 1 -- 172.21.15.1:0/2882039316 learned_addr learned my addr 172.21.15.1:0/2882039316 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:04:40.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:40.980+0000 7fcfc3e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcfc406ece0 0x7fcfc41f4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:04:40.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:40.980+0000 7fcfc9400640 1 --2- 172.21.15.1:0/2882039316 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcfc40c6ea0 0x7fcfc40c5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:04:40.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:40.980+0000 7fcfc9400640 1 -- 172.21.15.1:0/2882039316 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcfc40c6ea0 msgr2=0x7fcfc40c5f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-05-09T08:04:40.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:40.980+0000 7fcfc9400640 1 -- 172.21.15.1:0/2882039316 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcfc40c6ea0 msgr2=0x7fcfc40c5f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-09T08:04:40.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:40.980+0000 7fcfc9400640 1 --2- 172.21.15.1:0/2882039316 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcfc40c6ea0 0x7fcfc40c5f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-09T08:04:40.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:40.980+0000 7fcfc9400640 1 --2- 172.21.15.1:0/2882039316 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcfc40c6ea0 0x7fcfc40c5f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T08:04:40.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:40.981+0000 7fcfc8a00640 1 -- 172.21.15.1:0/2882039316 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcfc406ece0 msgr2=0x7fcfc41f4f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:04:40.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:40.981+0000 7fcfc8a00640 1 --2- 172.21.15.1:0/2882039316 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcfc406ece0 0x7fcfc41f4f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:04:40.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:40.981+0000 7fcfc8a00640 1 -- 172.21.15.1:0/2882039316 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcfc40c6ea0 msgr2=0x7fcfc40c5f20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T08:04:40.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:40.981+0000 7fcfc8a00640 1 --2- 172.21.15.1:0/2882039316 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcfc40c6ea0 0x7fcfc40c5f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:04:40.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:40.981+0000 7fcfc8a00640 1 -- 172.21.15.1:0/2882039316 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfc4158280 con 0x7fcfc406dca0 2024-05-09T08:04:40.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:40.981+0000 7fcfc8a00640 1 --2- 172.21.15.1:0/2882039316 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fcfc406dca0 0x7fcfc40c2a60 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fcfb8035040 tx=0x7fcfb80046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:04:40.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:40.981+0000 7fcfc0c00640 1 -- 172.21.15.1:0/2882039316 <== mon.2 v2:172.21.15.7:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfb8041030 con 0x7fcfc406dca0 2024-05-09T08:04:40.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:40.981+0000 7fcfc0c00640 1 -- 172.21.15.1:0/2882039316 <== mon.2 v2:172.21.15.7:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcfb803d4e0 con 0x7fcfc406dca0 2024-05-09T08:04:40.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:40.981+0000 7fcfc0c00640 1 -- 172.21.15.1:0/2882039316 <== mon.2 v2:172.21.15.7:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfb8037040 con 0x7fcfc406dca0 2024-05-09T08:04:40.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:40.982+0000 7fcfcb400640 1 -- 172.21.15.1:0/2882039316 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfc4158520 con 0x7fcfc406dca0 2024-05-09T08:04:40.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:40.982+0000 7fcfcb400640 1 -- 172.21.15.1:0/2882039316 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfc4158ae0 con 0x7fcfc406dca0 2024-05-09T08:04:40.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:40.983+0000 7fcfcb400640 1 -- 172.21.15.1:0/2882039316 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcfc4074d70 con 0x7fcfc406dca0 2024-05-09T08:04:41.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:40.985+0000 7fcfc0c00640 1 -- 172.21.15.1:0/2882039316 <== mon.2 v2:172.21.15.7:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fcfb8044070 con 0x7fcfc406dca0 2024-05-09T08:04:41.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:40.986+0000 7fcfc0c00640 1 --2- 172.21.15.1:0/2882039316 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fcfa406be30 0x7fcfa406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:04:41.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:40.986+0000 7fcfc0c00640 1 -- 172.21.15.1:0/2882039316 <== mon.2 v2:172.21.15.7:3301/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7fcfb80ab8f0 con 0x7fcfc406dca0 2024-05-09T08:04:41.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:40.986+0000 7fcfc3e00640 1 --2- 172.21.15.1:0/2882039316 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fcfa406be30 0x7fcfa406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:04:41.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:40.987+0000 7fcfc3e00640 1 --2- 172.21.15.1:0/2882039316 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fcfa406be30 0x7fcfa406e2f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fcfc414c8b0 tx=0x7fcfb0009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:04:41.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:40.991+0000 7fcfc0c00640 1 -- 172.21.15.1:0/2882039316 <== mon.2 v2:172.21.15.7:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fcfb8079d90 con 0x7fcfc406dca0 2024-05-09T08:04:41.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:41.282+0000 7fcfcb400640 1 -- 172.21.15.1:0/2882039316 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fcfc41515a0 con 0x7fcfc406dca0 2024-05-09T08:04:46.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:45.992+0000 7fcfc0c00640 1 -- 172.21.15.1:0/2882039316 <== mon.2 v2:172.21.15.7:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1538 (secure 0 0 0) 0x7fcfb8079690 con 0x7fcfc406dca0 2024-05-09T08:04:46.007 INFO:teuthology.orchestra.run.smithi001.stdout:{"election_epoch":16,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"54329ec8-3270-4043-9a82-c6cd7420aa7f","modified":"2024-05-09T08:01:21.819270Z","created":"2024-05-09T08:01:21.819270Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:3300","nonce":0},{"type":"v1","addr":"172.21.15.1:6789","nonce":0}]},"addr":"172.21.15.1:6789/0","public_addr":"172.21.15.1:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:3300","nonce":0},{"type":"v1","addr":"172.21.15.7:6789","nonce":0}]},"addr":"172.21.15.7:6789/0","public_addr":"172.21.15.7:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:3301","nonce":0},{"type":"v1","addr":"172.21.15.7:6790","nonce":0}]},"addr":"172.21.15.7:6790/0","public_addr":"172.21.15.7:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-05-09T08:04:46.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:45.994+0000 7fcfcb400640 1 -- 172.21.15.1:0/2882039316 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fcfa406be30 msgr2=0x7fcfa406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:04:46.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:45.994+0000 7fcfcb400640 1 --2- 172.21.15.1:0/2882039316 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fcfa406be30 0x7fcfa406e2f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fcfc414c8b0 tx=0x7fcfb0009040 comp rx=0 tx=0).stop 2024-05-09T08:04:46.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:45.994+0000 7fcfcb400640 1 -- 172.21.15.1:0/2882039316 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fcfc406dca0 msgr2=0x7fcfc40c2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:04:46.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:45.994+0000 7fcfcb400640 1 --2- 172.21.15.1:0/2882039316 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fcfc406dca0 0x7fcfc40c2a60 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fcfb8035040 tx=0x7fcfb80046c0 comp rx=0 tx=0).stop 2024-05-09T08:04:46.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:45.995+0000 7fcfcb400640 1 -- 172.21.15.1:0/2882039316 shutdown_connections 2024-05-09T08:04:46.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:45.995+0000 7fcfcb400640 1 --2- 172.21.15.1:0/2882039316 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fcfa406be30 0x7fcfa406e2f0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:04:46.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:45.995+0000 7fcfcb400640 1 --2- 172.21.15.1:0/2882039316 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcfc406ece0 0x7fcfc41f4f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:04:46.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:45.995+0000 7fcfcb400640 1 --2- 172.21.15.1:0/2882039316 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcfc40c6ea0 0x7fcfc40c5f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:04:46.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:45.995+0000 7fcfcb400640 1 --2- 172.21.15.1:0/2882039316 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fcfc406dca0 0x7fcfc40c2a60 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:04:46.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:45.995+0000 7fcfcb400640 1 -- 172.21.15.1:0/2882039316 >> 172.21.15.1:0/2882039316 conn(0x7fcfc40cc070 msgr2=0x7fcfc40c7350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:04:46.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:45.995+0000 7fcfcb400640 1 -- 172.21.15.1:0/2882039316 shutdown_connections 2024-05-09T08:04:46.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:45.995+0000 7fcfcb400640 1 -- 172.21.15.1:0/2882039316 wait complete. 2024-05-09T08:04:49.029 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-05-09T08:04:49.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:49.338+0000 7fb538c00640 1 Processor -- start 2024-05-09T08:04:49.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:49.338+0000 7fb538c00640 1 -- start start 2024-05-09T08:04:49.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:49.339+0000 7fb538c00640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fb5340c6ea0 0x7fb534072900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:04:49.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:49.340+0000 7fb538c00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fb53406ece0 0x7fb534159010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:04:49.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:49.340+0000 7fb538c00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb534073200 con 0x7fb53406dca0 2024-05-09T08:04:49.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:49.340+0000 7fb538c00640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7fb534072e40 con 0x7fb5340c6ea0 2024-05-09T08:04:49.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:49.340+0000 7fb538c00640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7fb5340737c0 con 0x7fb53406ece0 2024-05-09T08:04:49.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:49.340+0000 7fb531e00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fb53406dca0 0x7fb53406e080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.1:42680/0 (socket says 172.21.15.1:42680) 2024-05-09T08:04:49.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:49.340+0000 7fb531e00640 1 -- 172.21.15.1:0/4166538042 learned_addr learned my addr 172.21.15.1:0/4166538042 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T08:04:49.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:49.340+0000 7fb532800640 1 --2- 172.21.15.1:0/4166538042 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fb53406ece0 0x7fb534159010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:04:49.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:49.340+0000 7fb52be00640 1 --2- 172.21.15.1:0/4166538042 >> v2:172.21.15.1:3300/0 conn(0x7fb5340c6ea0 0x7fb534072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:04:49.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:49.340+0000 7fb531400640 1 -- 172.21.15.1:0/4166538042 <== mon.1 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2805208336 0 0) 0x7fb534073200 con 0x7fb53406dca0 2024-05-09T08:04:49.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:49.340+0000 7fb532800640 1 -- 172.21.15.1:0/4166538042 >> v2:172.21.15.1:3300/0 conn(0x7fb5340c6ea0 msgr2=0x7fb534072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:04:49.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:49.340+0000 7fb532800640 1 --2- 172.21.15.1:0/4166538042 >> v2:172.21.15.1:3300/0 conn(0x7fb5340c6ea0 0x7fb534072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:04:49.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:49.340+0000 7fb532800640 1 -- 172.21.15.1:0/4166538042 >> v1:172.21.15.7:6789/0 conn(0x7fb53406dca0 legacy=0x7fb53406e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:04:49.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:49.340+0000 7fb532800640 1 -- 172.21.15.1:0/4166538042 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5340734e0 con 0x7fb53406ece0 2024-05-09T08:04:49.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:49.342+0000 7fb532800640 1 --2- 172.21.15.1:0/4166538042 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fb53406ece0 0x7fb534159010 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fb52c009d90 tx=0x7fb52c02ea70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9f6d1528e75e87ed server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:04:49.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:49.343+0000 7fb531400640 1 -- 172.21.15.1:0/4166538042 <== mon.2 v2:172.21.15.7:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb52c032830 con 0x7fb53406ece0 2024-05-09T08:04:49.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:49.343+0000 7fb531400640 1 -- 172.21.15.1:0/4166538042 <== mon.2 v2:172.21.15.7:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb52c0329d0 con 0x7fb53406ece0 2024-05-09T08:04:49.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:49.343+0000 7fb531400640 1 -- 172.21.15.1:0/4166538042 <== mon.2 v2:172.21.15.7:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb52c032da0 con 0x7fb53406ece0 2024-05-09T08:04:49.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:49.344+0000 7fb538c00640 1 -- 172.21.15.1:0/4166538042 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fb53406ece0 msgr2=0x7fb534159010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:04:49.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:49.344+0000 7fb538c00640 1 --2- 172.21.15.1:0/4166538042 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fb53406ece0 0x7fb534159010 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fb52c009d90 tx=0x7fb52c02ea70 comp rx=0 tx=0).stop 2024-05-09T08:04:49.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:49.344+0000 7fb538c00640 1 -- 172.21.15.1:0/4166538042 shutdown_connections 2024-05-09T08:04:49.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:49.344+0000 7fb538c00640 1 --2- 172.21.15.1:0/4166538042 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fb53406ece0 0x7fb534159010 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:04:49.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:49.344+0000 7fb538c00640 1 --2- 172.21.15.1:0/4166538042 >> v2:172.21.15.1:3300/0 conn(0x7fb5340c6ea0 0x7fb534072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:04:49.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:49.344+0000 7fb538c00640 1 -- 172.21.15.1:0/4166538042 >> 172.21.15.1:0/4166538042 conn(0x7fb5340cc070 msgr2=0x7fb5340cc480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:04:49.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:49.344+0000 7fb538c00640 1 -- 172.21.15.1:0/4166538042 shutdown_connections 2024-05-09T08:04:49.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:49.344+0000 7fb538c00640 1 -- 172.21.15.1:0/4166538042 wait complete. 2024-05-09T08:04:49.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:49.346+0000 7fb538c00640 1 Processor -- start 2024-05-09T08:04:49.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:49.346+0000 7fb538c00640 1 -- start start 2024-05-09T08:04:49.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:49.347+0000 7fb538c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb53406dca0 0x7fb5341ec220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:04:49.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:49.347+0000 7fb538c00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb53406ece0 0x7fb5340ca6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:04:49.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:49.347+0000 7fb538c00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fb5340c6ea0 0x7fb5340c19c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:04:49.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:49.347+0000 7fb538c00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fb534072470 con 0x7fb53406ece0 2024-05-09T08:04:49.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:49.347+0000 7fb538c00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb534159550 con 0x7fb53406dca0 2024-05-09T08:04:49.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:49.347+0000 7fb538c00640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7fb534072100 con 0x7fb5340c6ea0 2024-05-09T08:04:49.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:49.348+0000 7fb531e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb53406dca0 0x7fb5341ec220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:04:49.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:49.348+0000 7fb52be00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb53406ece0 0x7fb5340ca6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:04:49.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:49.348+0000 7fb531e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb53406dca0 0x7fb5341ec220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.1:45096/0 (socket says 172.21.15.1:45096) 2024-05-09T08:04:49.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:49.348+0000 7fb531e00640 1 -- 172.21.15.1:0/1313091117 learned_addr learned my addr 172.21.15.1:0/1313091117 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:04:49.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:49.348+0000 7fb532800640 1 --2- 172.21.15.1:0/1313091117 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fb5340c6ea0 0x7fb5340c19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:04:49.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:49.348+0000 7fb52be00640 1 -- 172.21.15.1:0/1313091117 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fb5340c6ea0 msgr2=0x7fb5340c19c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:04:49.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:49.348+0000 7fb52be00640 1 --2- 172.21.15.1:0/1313091117 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fb5340c6ea0 0x7fb5340c19c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:04:49.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:49.348+0000 7fb52be00640 1 -- 172.21.15.1:0/1313091117 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb53406dca0 msgr2=0x7fb5341ec220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:04:49.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:49.348+0000 7fb52be00640 1 --2- 172.21.15.1:0/1313091117 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb53406dca0 0x7fb5341ec220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:04:49.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:49.348+0000 7fb52be00640 1 -- 172.21.15.1:0/1313091117 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb53414e2f0 con 0x7fb53406ece0 2024-05-09T08:04:49.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:49.348+0000 7fb52be00640 1 --2- 172.21.15.1:0/1313091117 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb53406ece0 0x7fb5340ca6a0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fb51c00e8a0 tx=0x7fb51c00ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:04:49.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:49.349+0000 7fb52a000640 1 -- 172.21.15.1:0/1313091117 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb51c017070 con 0x7fb53406ece0 2024-05-09T08:04:49.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:49.349+0000 7fb52a000640 1 -- 172.21.15.1:0/1313091117 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb51c00c940 con 0x7fb53406ece0 2024-05-09T08:04:49.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:49.349+0000 7fb52a000640 1 -- 172.21.15.1:0/1313091117 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb51c009e60 con 0x7fb53406ece0 2024-05-09T08:04:49.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:49.349+0000 7fb538c00640 1 -- 172.21.15.1:0/1313091117 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5341ec920 con 0x7fb53406ece0 2024-05-09T08:04:49.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:49.350+0000 7fb538c00640 1 -- 172.21.15.1:0/1313091117 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5341ecee0 con 0x7fb53406ece0 2024-05-09T08:04:49.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:49.352+0000 7fb52a000640 1 -- 172.21.15.1:0/1313091117 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb51c01a070 con 0x7fb53406ece0 2024-05-09T08:04:49.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:49.352+0000 7fb538c00640 1 -- 172.21.15.1:0/1313091117 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb534074d70 con 0x7fb53406ece0 2024-05-09T08:04:49.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:49.353+0000 7fb52a000640 1 --2- 172.21.15.1:0/1313091117 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fb51006be60 0x7fb51006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:04:49.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:49.353+0000 7fb52a000640 1 -- 172.21.15.1:0/1313091117 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7fb51c08a660 con 0x7fb53406ece0 2024-05-09T08:04:49.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:49.353+0000 7fb531e00640 1 --2- 172.21.15.1:0/1313091117 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fb51006be60 0x7fb51006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:04:49.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:49.354+0000 7fb531e00640 1 --2- 172.21.15.1:0/1313091117 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fb51006be60 0x7fb51006e320 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fb524000f60 tx=0x7fb524007e10 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:04:49.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:49.363+0000 7fb52a000640 1 -- 172.21.15.1:0/1313091117 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fb51c058660 con 0x7fb53406ece0 2024-05-09T08:04:49.391 INFO:tasks.workunit.client.0.smithi001.stdout: 2024-05-09T08:04:49.391 INFO:tasks.workunit.client.0.smithi001.stdout: Children see throughput for 1 rewriters = 234374.50 kB/sec 2024-05-09T08:04:49.391 INFO:tasks.workunit.client.0.smithi001.stdout: Parent sees throughput for 1 rewriters = 234328.13 kB/sec 2024-05-09T08:04:49.391 INFO:tasks.workunit.client.0.smithi001.stdout: Min throughput per process = 234374.50 kB/sec 2024-05-09T08:04:49.391 INFO:tasks.workunit.client.0.smithi001.stdout: Max throughput per process = 234374.50 kB/sec 2024-05-09T08:04:49.391 INFO:tasks.workunit.client.0.smithi001.stdout: Avg throughput per process = 234374.50 kB/sec 2024-05-09T08:04:49.391 INFO:tasks.workunit.client.0.smithi001.stdout: Min xfer = 1048576.00 kB 2024-05-09T08:04:49.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:49.632+0000 7fb538c00640 1 -- 172.21.15.1:0/1313091117 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fb5340ccd20 con 0x7fb53406ece0 2024-05-09T08:04:49.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:49.633+0000 7fb52a000640 1 -- 172.21.15.1:0/1313091117 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1544 (secure 0 0 0) 0x7fb51c05c960 con 0x7fb53406ece0 2024-05-09T08:04:49.648 INFO:teuthology.orchestra.run.smithi001.stdout:{"election_epoch":20,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":2,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"54329ec8-3270-4043-9a82-c6cd7420aa7f","modified":"2024-05-09T08:01:21.819270Z","created":"2024-05-09T08:01:21.819270Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:3300","nonce":0},{"type":"v1","addr":"172.21.15.1:6789","nonce":0}]},"addr":"172.21.15.1:6789/0","public_addr":"172.21.15.1:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:3300","nonce":0},{"type":"v1","addr":"172.21.15.7:6789","nonce":0}]},"addr":"172.21.15.7:6789/0","public_addr":"172.21.15.7:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:3301","nonce":0},{"type":"v1","addr":"172.21.15.7:6790","nonce":0}]},"addr":"172.21.15.7:6790/0","public_addr":"172.21.15.7:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-05-09T08:04:49.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:49.635+0000 7fb538c00640 1 -- 172.21.15.1:0/1313091117 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fb51006be60 msgr2=0x7fb51006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:04:49.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:49.635+0000 7fb538c00640 1 --2- 172.21.15.1:0/1313091117 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fb51006be60 0x7fb51006e320 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fb524000f60 tx=0x7fb524007e10 comp rx=0 tx=0).stop 2024-05-09T08:04:49.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:49.635+0000 7fb538c00640 1 -- 172.21.15.1:0/1313091117 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb53406ece0 msgr2=0x7fb5340ca6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:04:49.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:49.635+0000 7fb538c00640 1 --2- 172.21.15.1:0/1313091117 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb53406ece0 0x7fb5340ca6a0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fb51c00e8a0 tx=0x7fb51c00ed60 comp rx=0 tx=0).stop 2024-05-09T08:04:49.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:49.635+0000 7fb538c00640 1 -- 172.21.15.1:0/1313091117 shutdown_connections 2024-05-09T08:04:49.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:49.635+0000 7fb538c00640 1 --2- 172.21.15.1:0/1313091117 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fb51006be60 0x7fb51006e320 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:04:49.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:49.635+0000 7fb538c00640 1 --2- 172.21.15.1:0/1313091117 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fb5340c6ea0 0x7fb5340c19c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:04:49.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:49.635+0000 7fb538c00640 1 --2- 172.21.15.1:0/1313091117 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb53406ece0 0x7fb5340ca6a0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:04:49.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:49.635+0000 7fb538c00640 1 --2- 172.21.15.1:0/1313091117 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb53406dca0 0x7fb5341ec220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:04:49.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:49.635+0000 7fb538c00640 1 -- 172.21.15.1:0/1313091117 >> 172.21.15.1:0/1313091117 conn(0x7fb5340cc070 msgr2=0x7fb5341515c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:04:49.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:49.636+0000 7fb538c00640 1 -- 172.21.15.1:0/1313091117 shutdown_connections 2024-05-09T08:04:49.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:49.636+0000 7fb538c00640 1 -- 172.21.15.1:0/1313091117 wait complete. 2024-05-09T08:04:49.668 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-05-09T08:04:49.668 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-05-09T08:04:49.668 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-05-09T08:04:49.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:49.851+0000 7f5229000640 1 Processor -- start 2024-05-09T08:04:49.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:49.851+0000 7f5229000640 1 -- start start 2024-05-09T08:04:49.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:49.853+0000 7f5229000640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f52240c5890 0x7f522415b050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:04:49.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:49.853+0000 7f5229000640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f52240718e0 0x7f52240c69e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:04:49.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:49.853+0000 7f5229000640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f522406f060 con 0x7f52240c6f20 2024-05-09T08:04:49.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:49.853+0000 7f5229000640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f522415b590 con 0x7f52240718e0 2024-05-09T08:04:49.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:49.853+0000 7f5229000640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f5224072150 con 0x7f52240c5890 2024-05-09T08:04:49.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:49.853+0000 7f521aa00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f52240718e0 0x7f52240c69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:04:49.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:49.853+0000 7f5223400640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f52240c5890 0x7f522415b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:04:49.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:49.853+0000 7f521aa00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f52240718e0 0x7f52240c69e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.1:45108/0 (socket says 172.21.15.1:45108) 2024-05-09T08:04:49.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:49.853+0000 7f521aa00640 1 -- 172.21.15.1:0/3012458253 learned_addr learned my addr 172.21.15.1:0/3012458253 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:04:49.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:49.853+0000 7f5223e00640 1 --1- 172.21.15.1:0/3012458253 >> v1:172.21.15.7:6789/0 conn(0x7f52240c6f20 0x7f5224072940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.1:42682/0 (socket says 172.21.15.1:42682) 2024-05-09T08:04:49.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:49.854+0000 7f521aa00640 1 -- 172.21.15.1:0/3012458253 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f52240c5890 msgr2=0x7f522415b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:04:49.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:49.854+0000 7f521aa00640 1 --2- 172.21.15.1:0/3012458253 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f52240c5890 0x7f522415b050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:04:49.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:49.854+0000 7f521aa00640 1 -- 172.21.15.1:0/3012458253 >> v1:172.21.15.7:6789/0 conn(0x7f52240c6f20 legacy=0x7f5224072940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:04:49.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:49.854+0000 7f521aa00640 1 -- 172.21.15.1:0/3012458253 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f522406fe80 con 0x7f52240718e0 2024-05-09T08:04:49.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:49.854+0000 7f5222a00640 1 -- 172.21.15.1:0/3012458253 <== mon.1 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3615574264 0 0) 0x7f522406f060 con 0x7f52240c6f20 2024-05-09T08:04:49.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:49.855+0000 7f521aa00640 1 --2- 172.21.15.1:0/3012458253 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52240718e0 0x7f52240c69e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f520c009870 tx=0x7f520c02a300 comp rx=0 tx=0).ready entity=mon.1 client_cookie=33d6d615ebddb8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:04:49.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:49.855+0000 7f5222a00640 1 -- 172.21.15.1:0/3012458253 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f520c037070 con 0x7f52240718e0 2024-05-09T08:04:49.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:49.855+0000 7f5222a00640 1 -- 172.21.15.1:0/3012458253 <== mon.1 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f520c02ae50 con 0x7f52240718e0 2024-05-09T08:04:49.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:49.855+0000 7f5222a00640 1 -- 172.21.15.1:0/3012458253 <== mon.1 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f520c032860 con 0x7f52240718e0 2024-05-09T08:04:49.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:49.855+0000 7f5229000640 1 -- 172.21.15.1:0/3012458253 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52240718e0 msgr2=0x7f52240c69e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:04:49.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:49.855+0000 7f5229000640 1 --2- 172.21.15.1:0/3012458253 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52240718e0 0x7f52240c69e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f520c009870 tx=0x7f520c02a300 comp rx=0 tx=0).stop 2024-05-09T08:04:49.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:49.856+0000 7f5229000640 1 -- 172.21.15.1:0/3012458253 shutdown_connections 2024-05-09T08:04:49.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:49.856+0000 7f5229000640 1 --2- 172.21.15.1:0/3012458253 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52240718e0 0x7f52240c69e0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:04:49.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:49.856+0000 7f5229000640 1 --2- 172.21.15.1:0/3012458253 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f52240c5890 0x7f522415b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:04:49.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:49.856+0000 7f5229000640 1 -- 172.21.15.1:0/3012458253 >> 172.21.15.1:0/3012458253 conn(0x7f522406b250 msgr2=0x7f522406b630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:04:49.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:49.856+0000 7f5229000640 1 -- 172.21.15.1:0/3012458253 shutdown_connections 2024-05-09T08:04:49.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:49.856+0000 7f5229000640 1 -- 172.21.15.1:0/3012458253 wait complete. 2024-05-09T08:04:49.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:49.857+0000 7f5229000640 1 Processor -- start 2024-05-09T08:04:49.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:49.857+0000 7f5229000640 1 -- start start 2024-05-09T08:04:49.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:49.858+0000 7f5229000640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f52240718e0 0x7f522415b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:04:49.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:49.858+0000 7f5229000640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f52240c5890 0x7f52241514a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:04:49.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:49.858+0000 7f5229000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52240c6f20 0x7f52241f0730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:04:49.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:49.858+0000 7f5229000640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f522406f8c0 con 0x7f52240718e0 2024-05-09T08:04:49.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:49.858+0000 7f5229000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f522406f5e0 con 0x7f52240c6f20 2024-05-09T08:04:49.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:49.858+0000 7f5229000640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f522406fba0 con 0x7f52240c5890 2024-05-09T08:04:49.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:49.858+0000 7f5223400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f52240718e0 0x7f522415b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:04:49.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:49.858+0000 7f5223400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f52240718e0 0x7f522415b730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45218/0 (socket says 172.21.15.1:45218) 2024-05-09T08:04:49.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:49.859+0000 7f5223400640 1 -- 172.21.15.1:0/1602073768 learned_addr learned my addr 172.21.15.1:0/1602073768 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:04:49.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:49.859+0000 7f521aa00640 1 --2- 172.21.15.1:0/1602073768 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f52240c5890 0x7f52241514a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:04:49.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:49.859+0000 7f5223e00640 1 --2- 172.21.15.1:0/1602073768 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52240c6f20 0x7f52241f0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:04:49.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:49.859+0000 7f5223400640 1 -- 172.21.15.1:0/1602073768 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f52240c5890 msgr2=0x7f52241514a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:04:49.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:49.859+0000 7f5223400640 1 --2- 172.21.15.1:0/1602073768 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f52240c5890 0x7f52241514a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:04:49.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:49.859+0000 7f5223400640 1 -- 172.21.15.1:0/1602073768 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52240c6f20 msgr2=0x7f52241f0730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:04:49.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:49.859+0000 7f5223400640 1 --2- 172.21.15.1:0/1602073768 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52240c6f20 0x7f52241f0730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:04:49.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:49.859+0000 7f5223400640 1 -- 172.21.15.1:0/1602073768 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f522415be90 con 0x7f52240718e0 2024-05-09T08:04:49.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:49.859+0000 7f5223400640 1 --2- 172.21.15.1:0/1602073768 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f52240718e0 0x7f522415b730 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f5214009db0 tx=0x7f521400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:04:49.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:49.859+0000 7f521be00640 1 -- 172.21.15.1:0/1602073768 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5214018020 con 0x7f52240718e0 2024-05-09T08:04:49.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:49.859+0000 7f521be00640 1 -- 172.21.15.1:0/1602073768 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5214014480 con 0x7f52240718e0 2024-05-09T08:04:49.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:49.859+0000 7f5229000640 1 -- 172.21.15.1:0/1602073768 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f522414edd0 con 0x7f52240718e0 2024-05-09T08:04:49.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:49.859+0000 7f521be00640 1 -- 172.21.15.1:0/1602073768 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5214014850 con 0x7f52240718e0 2024-05-09T08:04:49.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:49.860+0000 7f5229000640 1 -- 172.21.15.1:0/1602073768 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f522414f390 con 0x7f52240718e0 2024-05-09T08:04:49.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:49.860+0000 7f5229000640 1 --2- 172.21.15.1:0/1602073768 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f522414e730 0x7f52241f0c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:04:49.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:49.860+0000 7f5229000640 1 -- 172.21.15.1:0/1602073768 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f522415bc70 con 0x7f522414e730 2024-05-09T08:04:49.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:49.861+0000 7f521aa00640 1 --2- 172.21.15.1:0/1602073768 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f522414e730 0x7f52241f0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:04:49.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:49.861+0000 7f521aa00640 1 -- 172.21.15.1:0/1602073768 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f52241586e0 con 0x7f522414e730 2024-05-09T08:04:49.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:49.862+0000 7f521aa00640 1 --2- 172.21.15.1:0/1602073768 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f522414e730 0x7f52241f0c70 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f5224071240 tx=0x7f520c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:04:49.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:49.862+0000 7f521be00640 1 -- 172.21.15.1:0/1602073768 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5214005d10 con 0x7f52240718e0 2024-05-09T08:04:49.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:49.863+0000 7f521be00640 1 --2- 172.21.15.1:0/1602073768 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f520006be30 0x7f520006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:04:49.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:49.863+0000 7f521be00640 1 -- 172.21.15.1:0/1602073768 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f520c037040 con 0x7f522414e730 2024-05-09T08:04:49.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:49.863+0000 7f521be00640 1 -- 172.21.15.1:0/1602073768 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7f521408bda0 con 0x7f52240718e0 2024-05-09T08:04:49.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:49.863+0000 7f521be00640 1 -- 172.21.15.1:0/1602073768 <== mon.0 v2:172.21.15.1:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f520c02ae50 con 0x7f522414e730 2024-05-09T08:04:49.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:49.863+0000 7f521be00640 1 -- 172.21.15.1:0/1602073768 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f522414e730 msgr2=0x7f52241f0c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:04:49.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:49.863+0000 7f521be00640 1 --2- 172.21.15.1:0/1602073768 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f522414e730 0x7f52241f0c70 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f5224071240 tx=0x7f520c0046c0 comp rx=0 tx=0).stop 2024-05-09T08:04:49.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:49.863+0000 7f521be00640 1 -- 172.21.15.1:0/1602073768 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f522414e730 msgr2=0x7f52241f0c70 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-05-09T08:04:49.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:49.863+0000 7f521be00640 1 --2- 172.21.15.1:0/1602073768 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f522414e730 0x7f52241f0c70 secure :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f5224071240 tx=0x7f520c0046c0 comp rx=0 tx=0).stop 2024-05-09T08:04:49.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:49.863+0000 7f5223e00640 1 --2- 172.21.15.1:0/1602073768 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f520006be30 0x7f520006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:04:49.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:49.864+0000 7f5223e00640 1 --2- 172.21.15.1:0/1602073768 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f520006be30 0x7f520006e2f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f521c007fd0 tx=0x7f521c007f60 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:04:49.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:49.870+0000 7f5229000640 1 --2- 172.21.15.1:0/1602073768 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f52240c2880 0x7f52241f3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:04:49.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:49.870+0000 7f5229000640 1 -- 172.21.15.1:0/1602073768 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f52240c7520 con 0x7f52240c2880 2024-05-09T08:04:49.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:49.870+0000 7f521aa00640 1 --2- 172.21.15.1:0/1602073768 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f52240c2880 0x7f52241f3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:04:49.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:49.871+0000 7f521aa00640 1 -- 172.21.15.1:0/1602073768 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f52240c9cf0 con 0x7f52240c2880 2024-05-09T08:04:49.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:49.871+0000 7f521aa00640 1 --2- 172.21.15.1:0/1602073768 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f52240c2880 0x7f52241f3670 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f520c0090b0 tx=0x7f520c008030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:04:49.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:49.871+0000 7f521be00640 1 -- 172.21.15.1:0/1602073768 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f520c037030 con 0x7f52240c2880 2024-05-09T08:04:49.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:49.872+0000 7f521be00640 1 -- 172.21.15.1:0/1602073768 <== mon.0 v2:172.21.15.1:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4535 (secure 0 0 0) 0x7f520c047750 con 0x7f52240c2880 2024-05-09T08:04:49.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:49.872+0000 7f521be00640 1 -- 172.21.15.1:0/1602073768 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f52240c2880 msgr2=0x7f52241f3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:04:49.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:49.872+0000 7f521be00640 1 --2- 172.21.15.1:0/1602073768 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f52240c2880 0x7f52241f3670 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f520c0090b0 tx=0x7f520c008030 comp rx=0 tx=0).stop 2024-05-09T08:04:49.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:49.872+0000 7f521be00640 1 -- 172.21.15.1:0/1602073768 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f52240c2880 msgr2=0x7f52241f3670 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-05-09T08:04:49.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:49.872+0000 7f521be00640 1 --2- 172.21.15.1:0/1602073768 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f52240c2880 0x7f52241f3670 secure :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f520c0090b0 tx=0x7f520c008030 comp rx=0 tx=0).stop 2024-05-09T08:04:49.887 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-05-09T08:04:49.887 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "a", 2024-05-09T08:04:49.887 INFO:teuthology.orchestra.run.smithi001.stdout: "rank": 0, 2024-05-09T08:04:49.887 INFO:teuthology.orchestra.run.smithi001.stdout: "state": "leader", 2024-05-09T08:04:49.887 INFO:teuthology.orchestra.run.smithi001.stdout: "election_epoch": 20, 2024-05-09T08:04:49.888 INFO:teuthology.orchestra.run.smithi001.stdout: "uptime": 8904, 2024-05-09T08:04:49.888 INFO:teuthology.orchestra.run.smithi001.stdout: "quorum": [ 2024-05-09T08:04:49.888 INFO:teuthology.orchestra.run.smithi001.stdout: 0, 2024-05-09T08:04:49.888 INFO:teuthology.orchestra.run.smithi001.stdout: 1, 2024-05-09T08:04:49.888 INFO:teuthology.orchestra.run.smithi001.stdout: 2 2024-05-09T08:04:49.888 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:04:49.888 INFO:teuthology.orchestra.run.smithi001.stdout: "quorum_age": 2, 2024-05-09T08:04:49.888 INFO:teuthology.orchestra.run.smithi001.stdout: "features": { 2024-05-09T08:04:49.888 INFO:teuthology.orchestra.run.smithi001.stdout: "required_con": "2450521708007866372", 2024-05-09T08:04:49.888 INFO:teuthology.orchestra.run.smithi001.stdout: "required_mon": [ 2024-05-09T08:04:49.888 INFO:teuthology.orchestra.run.smithi001.stdout: "kraken", 2024-05-09T08:04:49.888 INFO:teuthology.orchestra.run.smithi001.stdout: "luminous", 2024-05-09T08:04:49.888 INFO:teuthology.orchestra.run.smithi001.stdout: "mimic", 2024-05-09T08:04:49.888 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap-prune", 2024-05-09T08:04:49.889 INFO:teuthology.orchestra.run.smithi001.stdout: "nautilus", 2024-05-09T08:04:49.889 INFO:teuthology.orchestra.run.smithi001.stdout: "octopus", 2024-05-09T08:04:49.889 INFO:teuthology.orchestra.run.smithi001.stdout: "pacific", 2024-05-09T08:04:49.889 INFO:teuthology.orchestra.run.smithi001.stdout: "elector-pinging", 2024-05-09T08:04:49.889 INFO:teuthology.orchestra.run.smithi001.stdout: "quincy", 2024-05-09T08:04:49.889 INFO:teuthology.orchestra.run.smithi001.stdout: "reef", 2024-05-09T08:04:49.889 INFO:teuthology.orchestra.run.smithi001.stdout: "squid" 2024-05-09T08:04:49.889 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:04:49.889 INFO:teuthology.orchestra.run.smithi001.stdout: "quorum_con": "4540701547738038271", 2024-05-09T08:04:49.889 INFO:teuthology.orchestra.run.smithi001.stdout: "quorum_mon": [ 2024-05-09T08:04:49.889 INFO:teuthology.orchestra.run.smithi001.stdout: "kraken", 2024-05-09T08:04:49.889 INFO:teuthology.orchestra.run.smithi001.stdout: "luminous", 2024-05-09T08:04:49.889 INFO:teuthology.orchestra.run.smithi001.stdout: "mimic", 2024-05-09T08:04:49.889 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap-prune", 2024-05-09T08:04:49.890 INFO:teuthology.orchestra.run.smithi001.stdout: "nautilus", 2024-05-09T08:04:49.890 INFO:teuthology.orchestra.run.smithi001.stdout: "octopus", 2024-05-09T08:04:49.890 INFO:teuthology.orchestra.run.smithi001.stdout: "pacific", 2024-05-09T08:04:49.890 INFO:teuthology.orchestra.run.smithi001.stdout: "elector-pinging", 2024-05-09T08:04:49.890 INFO:teuthology.orchestra.run.smithi001.stdout: "quincy", 2024-05-09T08:04:49.890 INFO:teuthology.orchestra.run.smithi001.stdout: "reef", 2024-05-09T08:04:49.890 INFO:teuthology.orchestra.run.smithi001.stdout: "squid" 2024-05-09T08:04:49.890 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:04:49.890 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:04:49.890 INFO:teuthology.orchestra.run.smithi001.stdout: "outside_quorum": [], 2024-05-09T08:04:49.890 INFO:teuthology.orchestra.run.smithi001.stdout: "extra_probe_peers": [], 2024-05-09T08:04:49.890 INFO:teuthology.orchestra.run.smithi001.stdout: "sync_provider": [], 2024-05-09T08:04:49.890 INFO:teuthology.orchestra.run.smithi001.stdout: "monmap": { 2024-05-09T08:04:49.890 INFO:teuthology.orchestra.run.smithi001.stdout: "epoch": 1, 2024-05-09T08:04:49.891 INFO:teuthology.orchestra.run.smithi001.stdout: "fsid": "54329ec8-3270-4043-9a82-c6cd7420aa7f", 2024-05-09T08:04:49.891 INFO:teuthology.orchestra.run.smithi001.stdout: "modified": "2024-05-09T08:01:21.819270Z", 2024-05-09T08:04:49.891 INFO:teuthology.orchestra.run.smithi001.stdout: "created": "2024-05-09T08:01:21.819270Z", 2024-05-09T08:04:49.891 INFO:teuthology.orchestra.run.smithi001.stdout: "min_mon_release": 19, 2024-05-09T08:04:49.891 INFO:teuthology.orchestra.run.smithi001.stdout: "min_mon_release_name": "squid", 2024-05-09T08:04:49.891 INFO:teuthology.orchestra.run.smithi001.stdout: "election_strategy": 1, 2024-05-09T08:04:49.891 INFO:teuthology.orchestra.run.smithi001.stdout: "disallowed_leaders: ": "", 2024-05-09T08:04:49.891 INFO:teuthology.orchestra.run.smithi001.stdout: "stretch_mode": false, 2024-05-09T08:04:49.891 INFO:teuthology.orchestra.run.smithi001.stdout: "tiebreaker_mon": "", 2024-05-09T08:04:49.891 INFO:teuthology.orchestra.run.smithi001.stdout: "removed_ranks: ": "", 2024-05-09T08:04:49.891 INFO:teuthology.orchestra.run.smithi001.stdout: "features": { 2024-05-09T08:04:49.891 INFO:teuthology.orchestra.run.smithi001.stdout: "persistent": [ 2024-05-09T08:04:49.891 INFO:teuthology.orchestra.run.smithi001.stdout: "kraken", 2024-05-09T08:04:49.891 INFO:teuthology.orchestra.run.smithi001.stdout: "luminous", 2024-05-09T08:04:49.892 INFO:teuthology.orchestra.run.smithi001.stdout: "mimic", 2024-05-09T08:04:49.892 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap-prune", 2024-05-09T08:04:49.892 INFO:teuthology.orchestra.run.smithi001.stdout: "nautilus", 2024-05-09T08:04:49.892 INFO:teuthology.orchestra.run.smithi001.stdout: "octopus", 2024-05-09T08:04:49.892 INFO:teuthology.orchestra.run.smithi001.stdout: "pacific", 2024-05-09T08:04:49.892 INFO:teuthology.orchestra.run.smithi001.stdout: "elector-pinging", 2024-05-09T08:04:49.892 INFO:teuthology.orchestra.run.smithi001.stdout: "quincy", 2024-05-09T08:04:49.892 INFO:teuthology.orchestra.run.smithi001.stdout: "reef", 2024-05-09T08:04:49.892 INFO:teuthology.orchestra.run.smithi001.stdout: "squid" 2024-05-09T08:04:49.892 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:04:49.892 INFO:teuthology.orchestra.run.smithi001.stdout: "optional": [] 2024-05-09T08:04:49.892 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:04:49.892 INFO:teuthology.orchestra.run.smithi001.stdout: "mons": [ 2024-05-09T08:04:49.892 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:04:49.892 INFO:teuthology.orchestra.run.smithi001.stdout: "rank": 0, 2024-05-09T08:04:49.893 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "a", 2024-05-09T08:04:49.893 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-05-09T08:04:49.893 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-05-09T08:04:49.893 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:04:49.893 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-05-09T08:04:49.893 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:3300", 2024-05-09T08:04:49.893 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:04:49.893 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:04:49.893 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:04:49.893 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-05-09T08:04:49.893 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6789", 2024-05-09T08:04:49.893 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:04:49.893 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:04:49.893 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:04:49.894 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:04:49.894 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6789/0", 2024-05-09T08:04:49.894 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.1:6789/0", 2024-05-09T08:04:49.894 INFO:teuthology.orchestra.run.smithi001.stdout: "priority": 0, 2024-05-09T08:04:49.894 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 0, 2024-05-09T08:04:49.894 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_location": "{}" 2024-05-09T08:04:49.894 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:04:49.894 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:04:49.894 INFO:teuthology.orchestra.run.smithi001.stdout: "rank": 1, 2024-05-09T08:04:49.894 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "b", 2024-05-09T08:04:49.894 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-05-09T08:04:49.894 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-05-09T08:04:49.894 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:04:49.894 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-05-09T08:04:49.895 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:3300", 2024-05-09T08:04:49.895 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:04:49.895 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:04:49.895 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:04:49.895 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-05-09T08:04:49.895 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:6789", 2024-05-09T08:04:49.895 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:04:49.895 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:04:49.895 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:04:49.895 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:04:49.895 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:6789/0", 2024-05-09T08:04:49.895 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.7:6789/0", 2024-05-09T08:04:49.895 INFO:teuthology.orchestra.run.smithi001.stdout: "priority": 0, 2024-05-09T08:04:49.895 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 0, 2024-05-09T08:04:49.895 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_location": "{}" 2024-05-09T08:04:49.896 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:04:49.896 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:04:49.896 INFO:teuthology.orchestra.run.smithi001.stdout: "rank": 2, 2024-05-09T08:04:49.896 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "c", 2024-05-09T08:04:49.896 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-05-09T08:04:49.896 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-05-09T08:04:49.896 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:04:49.896 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-05-09T08:04:49.896 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:3301", 2024-05-09T08:04:49.896 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:04:49.896 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:04:49.896 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:04:49.896 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-05-09T08:04:49.896 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:6790", 2024-05-09T08:04:49.897 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:04:49.897 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:04:49.897 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:04:49.897 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:04:49.897 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:6790/0", 2024-05-09T08:04:49.897 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.7:6790/0", 2024-05-09T08:04:49.897 INFO:teuthology.orchestra.run.smithi001.stdout: "priority": 0, 2024-05-09T08:04:49.897 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 0, 2024-05-09T08:04:49.897 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_location": "{}" 2024-05-09T08:04:49.897 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:04:49.897 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:04:49.897 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:04:49.897 INFO:teuthology.orchestra.run.smithi001.stdout: "feature_map": { 2024-05-09T08:04:49.897 INFO:teuthology.orchestra.run.smithi001.stdout: "mon": [ 2024-05-09T08:04:49.898 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:04:49.898 INFO:teuthology.orchestra.run.smithi001.stdout: "features": "0x3f03cffffffdffff", 2024-05-09T08:04:49.898 INFO:teuthology.orchestra.run.smithi001.stdout: "release": "luminous", 2024-05-09T08:04:49.898 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 1 2024-05-09T08:04:49.898 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:04:49.898 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:04:49.898 INFO:teuthology.orchestra.run.smithi001.stdout: "client": [ 2024-05-09T08:04:49.898 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:04:49.898 INFO:teuthology.orchestra.run.smithi001.stdout: "features": "0x3f03cffffffdffff", 2024-05-09T08:04:49.898 INFO:teuthology.orchestra.run.smithi001.stdout: "release": "luminous", 2024-05-09T08:04:49.898 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 2 2024-05-09T08:04:49.898 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:04:49.898 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:04:49.898 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:04:49.898 INFO:teuthology.orchestra.run.smithi001.stdout: "stretch_mode": false 2024-05-09T08:04:49.899 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-05-09T08:04:49.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:49.872+0000 7f5229000640 1 -- 172.21.15.1:0/1602073768 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f520006be30 msgr2=0x7f520006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:04:49.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:49.872+0000 7f5229000640 1 --2- 172.21.15.1:0/1602073768 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f520006be30 0x7f520006e2f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f521c007fd0 tx=0x7f521c007f60 comp rx=0 tx=0).stop 2024-05-09T08:04:49.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:49.872+0000 7f5229000640 1 -- 172.21.15.1:0/1602073768 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f52240718e0 msgr2=0x7f522415b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:04:49.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:49.872+0000 7f5229000640 1 --2- 172.21.15.1:0/1602073768 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f52240718e0 0x7f522415b730 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f5214009db0 tx=0x7f521400c490 comp rx=0 tx=0).stop 2024-05-09T08:04:49.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:49.872+0000 7f5223e00640 1 -- 172.21.15.1:0/1602073768 reap_dead start 2024-05-09T08:04:49.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:49.872+0000 7f5229000640 1 -- 172.21.15.1:0/1602073768 shutdown_connections 2024-05-09T08:04:49.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:49.872+0000 7f5229000640 1 --2- 172.21.15.1:0/1602073768 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f52240718e0 0x7f522415b730 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:04:49.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:49.872+0000 7f5229000640 1 -- 172.21.15.1:0/1602073768 >> 172.21.15.1:0/1602073768 conn(0x7f522406b250 msgr2=0x7f52240c3fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:04:49.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:49.873+0000 7f5229000640 1 -- 172.21.15.1:0/1602073768 shutdown_connections 2024-05-09T08:04:49.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:49.873+0000 7f5229000640 1 -- 172.21.15.1:0/1602073768 wait complete. 2024-05-09T08:04:49.903 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-05-09T08:04:49.903 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-05-09T08:04:49.904 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-05-09T08:04:50.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.091+0000 7ff19f800640 1 Processor -- start 2024-05-09T08:04:50.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.091+0000 7ff19f800640 1 -- start start 2024-05-09T08:04:50.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.092+0000 7ff19f800640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7ff198072d90 0x7ff1980c5fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:04:50.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.092+0000 7ff19f800640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff1980c2e50 0x7ff198073170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:04:50.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.092+0000 7ff19f800640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff1980c1230 con 0x7ff198157d80 2024-05-09T08:04:50.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.092+0000 7ff19f800640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7ff1980c1510 con 0x7ff198072d90 2024-05-09T08:04:50.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.092+0000 7ff19f800640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7ff19806f830 con 0x7ff1980c2e50 2024-05-09T08:04:50.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.093+0000 7ff19ce00640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7ff198072d90 0x7ff1980c5fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:04:50.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.093+0000 7ff197e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff1980c2e50 0x7ff198073170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:04:50.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.093+0000 7ff19ce00640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7ff198072d90 0x7ff1980c5fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45236/0 (socket says 172.21.15.1:45236) 2024-05-09T08:04:50.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.093+0000 7ff19ce00640 1 -- 172.21.15.1:0/4014802576 learned_addr learned my addr 172.21.15.1:0/4014802576 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:04:50.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.093+0000 7ff19d800640 1 --1- 172.21.15.1:0/4014802576 >> v1:172.21.15.1:6789/0 conn(0x7ff198157d80 0x7ff198070030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:35616/0 (socket says 172.21.15.1:35616) 2024-05-09T08:04:50.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.093+0000 7ff19ce00640 1 -- 172.21.15.1:0/4014802576 >> v2:172.21.15.7:3300/0 conn(0x7ff1980c2e50 msgr2=0x7ff198073170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:04:50.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.093+0000 7ff19ce00640 1 --2- 172.21.15.1:0/4014802576 >> v2:172.21.15.7:3300/0 conn(0x7ff1980c2e50 0x7ff198073170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:04:50.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.093+0000 7ff19ce00640 1 -- 172.21.15.1:0/4014802576 >> v1:172.21.15.1:6789/0 conn(0x7ff198157d80 legacy=0x7ff198070030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:04:50.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.093+0000 7ff19ce00640 1 -- 172.21.15.1:0/4014802576 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff19806fb10 con 0x7ff198072d90 2024-05-09T08:04:50.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.094+0000 7ff19ce00640 1 --2- 172.21.15.1:0/4014802576 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff198072d90 0x7ff1980c5fb0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7ff1980cc720 tx=0x7ff18c02a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41a968202b41c333 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:04:50.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.095+0000 7ff197400640 1 -- 172.21.15.1:0/4014802576 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff18c037070 con 0x7ff198072d90 2024-05-09T08:04:50.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.095+0000 7ff197400640 1 -- 172.21.15.1:0/4014802576 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff18c02ae50 con 0x7ff198072d90 2024-05-09T08:04:50.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.095+0000 7ff197400640 1 -- 172.21.15.1:0/4014802576 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff18c032860 con 0x7ff198072d90 2024-05-09T08:04:50.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.095+0000 7ff19f800640 1 -- 172.21.15.1:0/4014802576 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff198072d90 msgr2=0x7ff1980c5fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:04:50.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.095+0000 7ff19f800640 1 --2- 172.21.15.1:0/4014802576 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff198072d90 0x7ff1980c5fb0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7ff1980cc720 tx=0x7ff18c02a300 comp rx=0 tx=0).stop 2024-05-09T08:04:50.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.095+0000 7ff19f800640 1 -- 172.21.15.1:0/4014802576 shutdown_connections 2024-05-09T08:04:50.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.095+0000 7ff19f800640 1 --2- 172.21.15.1:0/4014802576 >> v2:172.21.15.7:3300/0 conn(0x7ff1980c2e50 0x7ff198073170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:04:50.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.095+0000 7ff19f800640 1 --2- 172.21.15.1:0/4014802576 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff198072d90 0x7ff1980c5fb0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:04:50.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.095+0000 7ff19f800640 1 -- 172.21.15.1:0/4014802576 >> 172.21.15.1:0/4014802576 conn(0x7ff19806dad0 msgr2=0x7ff19806deb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:04:50.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.095+0000 7ff19f800640 1 -- 172.21.15.1:0/4014802576 shutdown_connections 2024-05-09T08:04:50.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.095+0000 7ff19f800640 1 -- 172.21.15.1:0/4014802576 wait complete. 2024-05-09T08:04:50.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.097+0000 7ff19f800640 1 Processor -- start 2024-05-09T08:04:50.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.097+0000 7ff19f800640 1 -- start start 2024-05-09T08:04:50.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.098+0000 7ff19f800640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff198072d90 0x7ff198155170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:04:50.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.098+0000 7ff19f800640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ff1980c2e50 0x7ff1981f43e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:04:50.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.098+0000 7ff19f800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff198157d80 0x7ff1981f0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:04:50.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.098+0000 7ff19f800640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ff198151a60 con 0x7ff198072d90 2024-05-09T08:04:50.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.098+0000 7ff19f800640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff1981518e0 con 0x7ff198157d80 2024-05-09T08:04:50.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.098+0000 7ff19f800640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7ff1980c6a00 con 0x7ff1980c2e50 2024-05-09T08:04:50.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.098+0000 7ff19ce00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff198072d90 0x7ff198155170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:04:50.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.098+0000 7ff19ce00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff198072d90 0x7ff198155170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45240/0 (socket says 172.21.15.1:45240) 2024-05-09T08:04:50.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.098+0000 7ff19ce00640 1 -- 172.21.15.1:0/3599764205 learned_addr learned my addr 172.21.15.1:0/3599764205 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:04:50.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.098+0000 7ff19d800640 1 --2- 172.21.15.1:0/3599764205 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff198157d80 0x7ff1981f0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:04:50.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.098+0000 7ff197e00640 1 --2- 172.21.15.1:0/3599764205 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ff1980c2e50 0x7ff1981f43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:04:50.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.099+0000 7ff19ce00640 1 -- 172.21.15.1:0/3599764205 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ff1980c2e50 msgr2=0x7ff1981f43e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:04:50.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.099+0000 7ff19ce00640 1 --2- 172.21.15.1:0/3599764205 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ff1980c2e50 0x7ff1981f43e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:04:50.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.099+0000 7ff19ce00640 1 -- 172.21.15.1:0/3599764205 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff198157d80 msgr2=0x7ff1981f0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:04:50.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.099+0000 7ff19ce00640 1 --2- 172.21.15.1:0/3599764205 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff198157d80 0x7ff1981f0ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:04:50.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.099+0000 7ff19ce00640 1 -- 172.21.15.1:0/3599764205 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1981f4920 con 0x7ff198072d90 2024-05-09T08:04:50.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.099+0000 7ff197e00640 1 --2- 172.21.15.1:0/3599764205 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ff1980c2e50 0x7ff1981f43e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T08:04:50.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.099+0000 7ff19ce00640 1 --2- 172.21.15.1:0/3599764205 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff198072d90 0x7ff198155170 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7ff18c02a340 tx=0x7ff18c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:04:50.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.099+0000 7ff194c00640 1 -- 172.21.15.1:0/3599764205 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff18c041020 con 0x7ff198072d90 2024-05-09T08:04:50.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.099+0000 7ff194c00640 1 -- 172.21.15.1:0/3599764205 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff18c03c890 con 0x7ff198072d90 2024-05-09T08:04:50.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.099+0000 7ff194c00640 1 -- 172.21.15.1:0/3599764205 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff18c037040 con 0x7ff198072d90 2024-05-09T08:04:50.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.099+0000 7ff19f800640 1 -- 172.21.15.1:0/3599764205 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1981f13e0 con 0x7ff198072d90 2024-05-09T08:04:50.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.100+0000 7ff19f800640 1 -- 172.21.15.1:0/3599764205 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1981571c0 con 0x7ff198072d90 2024-05-09T08:04:50.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.100+0000 7ff19f800640 1 --2- 172.21.15.1:0/3599764205 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff1981574a0 0x7ff19814deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:04:50.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.100+0000 7ff19f800640 1 -- 172.21.15.1:0/3599764205 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff1980c28a0 con 0x7ff1981574a0 2024-05-09T08:04:50.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.101+0000 7ff197e00640 1 --2- 172.21.15.1:0/3599764205 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff1981574a0 0x7ff19814deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:04:50.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.101+0000 7ff197e00640 1 -- 172.21.15.1:0/3599764205 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff198074d70 con 0x7ff1981574a0 2024-05-09T08:04:50.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.102+0000 7ff197e00640 1 --2- 172.21.15.1:0/3599764205 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff1981574a0 0x7ff19814deb0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff1980c57c0 tx=0x7ff184011c90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:04:50.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.102+0000 7ff194c00640 1 -- 172.21.15.1:0/3599764205 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff18400fa00 con 0x7ff1981574a0 2024-05-09T08:04:50.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.102+0000 7ff194c00640 1 -- 172.21.15.1:0/3599764205 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff18c044020 con 0x7ff198072d90 2024-05-09T08:04:50.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.103+0000 7ff194c00640 1 --2- 172.21.15.1:0/3599764205 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7ff17806bd30 0x7ff17806e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:04:50.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.103+0000 7ff194c00640 1 -- 172.21.15.1:0/3599764205 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7ff18c0ab970 con 0x7ff198072d90 2024-05-09T08:04:50.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.103+0000 7ff194c00640 1 -- 172.21.15.1:0/3599764205 <== mon.1 v2:172.21.15.7:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7ff18400fba0 con 0x7ff1981574a0 2024-05-09T08:04:50.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.103+0000 7ff194c00640 1 -- 172.21.15.1:0/3599764205 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff1981574a0 msgr2=0x7ff19814deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:04:50.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.103+0000 7ff194c00640 1 --2- 172.21.15.1:0/3599764205 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff1981574a0 0x7ff19814deb0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff1980c57c0 tx=0x7ff184011c90 comp rx=0 tx=0).stop 2024-05-09T08:04:50.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.103+0000 7ff194c00640 1 -- 172.21.15.1:0/3599764205 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff1981574a0 msgr2=0x7ff19814deb0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-09T08:04:50.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.103+0000 7ff194c00640 1 --2- 172.21.15.1:0/3599764205 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff1981574a0 0x7ff19814deb0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:04:50.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.103+0000 7ff19d800640 1 --2- 172.21.15.1:0/3599764205 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7ff17806bd30 0x7ff17806e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:04:50.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.104+0000 7ff19d800640 1 --2- 172.21.15.1:0/3599764205 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7ff17806bd30 0x7ff17806e1f0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ff1900012e0 tx=0x7ff1900015d0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:04:50.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.111+0000 7ff19f800640 1 --2- 172.21.15.1:0/3599764205 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff198156a30 0x7ff19814a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:04:50.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.111+0000 7ff19f800640 1 -- 172.21.15.1:0/3599764205 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff1980c5b80 con 0x7ff198156a30 2024-05-09T08:04:50.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.111+0000 7ff197e00640 1 --2- 172.21.15.1:0/3599764205 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff198156a30 0x7ff19814a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:04:50.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.111+0000 7ff197e00640 1 -- 172.21.15.1:0/3599764205 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7ff19806e100 con 0x7ff198156a30 2024-05-09T08:04:50.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.112+0000 7ff197e00640 1 --2- 172.21.15.1:0/3599764205 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff198156a30 0x7ff19814a600 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff18401c000 tx=0x7ff1840046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:04:50.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.112+0000 7ff194c00640 1 -- 172.21.15.1:0/3599764205 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff184006740 con 0x7ff198156a30 2024-05-09T08:04:50.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.112+0000 7ff194c00640 1 -- 172.21.15.1:0/3599764205 <== mon.1 v2:172.21.15.7:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4704 (secure 0 0 0) 0x7ff184026070 con 0x7ff198156a30 2024-05-09T08:04:50.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.112+0000 7ff194c00640 1 -- 172.21.15.1:0/3599764205 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff198156a30 msgr2=0x7ff19814a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:04:50.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.112+0000 7ff194c00640 1 --2- 172.21.15.1:0/3599764205 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff198156a30 0x7ff19814a600 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff18401c000 tx=0x7ff1840046c0 comp rx=0 tx=0).stop 2024-05-09T08:04:50.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.112+0000 7ff194c00640 1 -- 172.21.15.1:0/3599764205 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff198156a30 msgr2=0x7ff19814a600 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-05-09T08:04:50.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.112+0000 7ff194c00640 1 --2- 172.21.15.1:0/3599764205 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff198156a30 0x7ff19814a600 secure :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff18401c000 tx=0x7ff1840046c0 comp rx=0 tx=0).stop 2024-05-09T08:04:50.128 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-05-09T08:04:50.128 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "b", 2024-05-09T08:04:50.128 INFO:teuthology.orchestra.run.smithi001.stdout: "rank": 1, 2024-05-09T08:04:50.128 INFO:teuthology.orchestra.run.smithi001.stdout: "state": "peon", 2024-05-09T08:04:50.128 INFO:teuthology.orchestra.run.smithi001.stdout: "election_epoch": 20, 2024-05-09T08:04:50.128 INFO:teuthology.orchestra.run.smithi001.stdout: "uptime": 57774, 2024-05-09T08:04:50.128 INFO:teuthology.orchestra.run.smithi001.stdout: "quorum": [ 2024-05-09T08:04:50.128 INFO:teuthology.orchestra.run.smithi001.stdout: 0, 2024-05-09T08:04:50.128 INFO:teuthology.orchestra.run.smithi001.stdout: 1, 2024-05-09T08:04:50.128 INFO:teuthology.orchestra.run.smithi001.stdout: 2 2024-05-09T08:04:50.128 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:04:50.128 INFO:teuthology.orchestra.run.smithi001.stdout: "quorum_age": 2, 2024-05-09T08:04:50.129 INFO:teuthology.orchestra.run.smithi001.stdout: "features": { 2024-05-09T08:04:50.129 INFO:teuthology.orchestra.run.smithi001.stdout: "required_con": "2450521708007866372", 2024-05-09T08:04:50.129 INFO:teuthology.orchestra.run.smithi001.stdout: "required_mon": [ 2024-05-09T08:04:50.129 INFO:teuthology.orchestra.run.smithi001.stdout: "kraken", 2024-05-09T08:04:50.129 INFO:teuthology.orchestra.run.smithi001.stdout: "luminous", 2024-05-09T08:04:50.129 INFO:teuthology.orchestra.run.smithi001.stdout: "mimic", 2024-05-09T08:04:50.129 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap-prune", 2024-05-09T08:04:50.129 INFO:teuthology.orchestra.run.smithi001.stdout: "nautilus", 2024-05-09T08:04:50.129 INFO:teuthology.orchestra.run.smithi001.stdout: "octopus", 2024-05-09T08:04:50.129 INFO:teuthology.orchestra.run.smithi001.stdout: "pacific", 2024-05-09T08:04:50.129 INFO:teuthology.orchestra.run.smithi001.stdout: "elector-pinging", 2024-05-09T08:04:50.129 INFO:teuthology.orchestra.run.smithi001.stdout: "quincy", 2024-05-09T08:04:50.129 INFO:teuthology.orchestra.run.smithi001.stdout: "reef", 2024-05-09T08:04:50.129 INFO:teuthology.orchestra.run.smithi001.stdout: "squid" 2024-05-09T08:04:50.130 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:04:50.130 INFO:teuthology.orchestra.run.smithi001.stdout: "quorum_con": "4540701547738038271", 2024-05-09T08:04:50.130 INFO:teuthology.orchestra.run.smithi001.stdout: "quorum_mon": [ 2024-05-09T08:04:50.130 INFO:teuthology.orchestra.run.smithi001.stdout: "kraken", 2024-05-09T08:04:50.130 INFO:teuthology.orchestra.run.smithi001.stdout: "luminous", 2024-05-09T08:04:50.130 INFO:teuthology.orchestra.run.smithi001.stdout: "mimic", 2024-05-09T08:04:50.130 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap-prune", 2024-05-09T08:04:50.130 INFO:teuthology.orchestra.run.smithi001.stdout: "nautilus", 2024-05-09T08:04:50.130 INFO:teuthology.orchestra.run.smithi001.stdout: "octopus", 2024-05-09T08:04:50.130 INFO:teuthology.orchestra.run.smithi001.stdout: "pacific", 2024-05-09T08:04:50.130 INFO:teuthology.orchestra.run.smithi001.stdout: "elector-pinging", 2024-05-09T08:04:50.130 INFO:teuthology.orchestra.run.smithi001.stdout: "quincy", 2024-05-09T08:04:50.130 INFO:teuthology.orchestra.run.smithi001.stdout: "reef", 2024-05-09T08:04:50.130 INFO:teuthology.orchestra.run.smithi001.stdout: "squid" 2024-05-09T08:04:50.131 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:04:50.131 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:04:50.131 INFO:teuthology.orchestra.run.smithi001.stdout: "outside_quorum": [], 2024-05-09T08:04:50.131 INFO:teuthology.orchestra.run.smithi001.stdout: "extra_probe_peers": [], 2024-05-09T08:04:50.131 INFO:teuthology.orchestra.run.smithi001.stdout: "sync_provider": [], 2024-05-09T08:04:50.131 INFO:teuthology.orchestra.run.smithi001.stdout: "monmap": { 2024-05-09T08:04:50.131 INFO:teuthology.orchestra.run.smithi001.stdout: "epoch": 1, 2024-05-09T08:04:50.131 INFO:teuthology.orchestra.run.smithi001.stdout: "fsid": "54329ec8-3270-4043-9a82-c6cd7420aa7f", 2024-05-09T08:04:50.131 INFO:teuthology.orchestra.run.smithi001.stdout: "modified": "2024-05-09T08:01:21.819270Z", 2024-05-09T08:04:50.131 INFO:teuthology.orchestra.run.smithi001.stdout: "created": "2024-05-09T08:01:21.819270Z", 2024-05-09T08:04:50.131 INFO:teuthology.orchestra.run.smithi001.stdout: "min_mon_release": 19, 2024-05-09T08:04:50.131 INFO:teuthology.orchestra.run.smithi001.stdout: "min_mon_release_name": "squid", 2024-05-09T08:04:50.131 INFO:teuthology.orchestra.run.smithi001.stdout: "election_strategy": 1, 2024-05-09T08:04:50.131 INFO:teuthology.orchestra.run.smithi001.stdout: "disallowed_leaders: ": "", 2024-05-09T08:04:50.132 INFO:teuthology.orchestra.run.smithi001.stdout: "stretch_mode": false, 2024-05-09T08:04:50.132 INFO:teuthology.orchestra.run.smithi001.stdout: "tiebreaker_mon": "", 2024-05-09T08:04:50.132 INFO:teuthology.orchestra.run.smithi001.stdout: "removed_ranks: ": "", 2024-05-09T08:04:50.132 INFO:teuthology.orchestra.run.smithi001.stdout: "features": { 2024-05-09T08:04:50.132 INFO:teuthology.orchestra.run.smithi001.stdout: "persistent": [ 2024-05-09T08:04:50.132 INFO:teuthology.orchestra.run.smithi001.stdout: "kraken", 2024-05-09T08:04:50.132 INFO:teuthology.orchestra.run.smithi001.stdout: "luminous", 2024-05-09T08:04:50.132 INFO:teuthology.orchestra.run.smithi001.stdout: "mimic", 2024-05-09T08:04:50.132 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap-prune", 2024-05-09T08:04:50.132 INFO:teuthology.orchestra.run.smithi001.stdout: "nautilus", 2024-05-09T08:04:50.132 INFO:teuthology.orchestra.run.smithi001.stdout: "octopus", 2024-05-09T08:04:50.132 INFO:teuthology.orchestra.run.smithi001.stdout: "pacific", 2024-05-09T08:04:50.132 INFO:teuthology.orchestra.run.smithi001.stdout: "elector-pinging", 2024-05-09T08:04:50.132 INFO:teuthology.orchestra.run.smithi001.stdout: "quincy", 2024-05-09T08:04:50.133 INFO:teuthology.orchestra.run.smithi001.stdout: "reef", 2024-05-09T08:04:50.133 INFO:teuthology.orchestra.run.smithi001.stdout: "squid" 2024-05-09T08:04:50.133 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:04:50.133 INFO:teuthology.orchestra.run.smithi001.stdout: "optional": [] 2024-05-09T08:04:50.133 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:04:50.133 INFO:teuthology.orchestra.run.smithi001.stdout: "mons": [ 2024-05-09T08:04:50.133 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:04:50.133 INFO:teuthology.orchestra.run.smithi001.stdout: "rank": 0, 2024-05-09T08:04:50.133 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "a", 2024-05-09T08:04:50.133 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-05-09T08:04:50.133 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-05-09T08:04:50.133 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:04:50.133 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-05-09T08:04:50.134 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:3300", 2024-05-09T08:04:50.134 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:04:50.134 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:04:50.134 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:04:50.134 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-05-09T08:04:50.134 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6789", 2024-05-09T08:04:50.134 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:04:50.134 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:04:50.134 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:04:50.134 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:04:50.134 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6789/0", 2024-05-09T08:04:50.134 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.1:6789/0", 2024-05-09T08:04:50.134 INFO:teuthology.orchestra.run.smithi001.stdout: "priority": 0, 2024-05-09T08:04:50.135 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 0, 2024-05-09T08:04:50.135 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_location": "{}" 2024-05-09T08:04:50.135 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:04:50.135 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:04:50.135 INFO:teuthology.orchestra.run.smithi001.stdout: "rank": 1, 2024-05-09T08:04:50.135 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "b", 2024-05-09T08:04:50.135 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-05-09T08:04:50.135 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-05-09T08:04:50.135 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:04:50.135 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-05-09T08:04:50.135 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:3300", 2024-05-09T08:04:50.135 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:04:50.136 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:04:50.136 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:04:50.136 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-05-09T08:04:50.136 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:6789", 2024-05-09T08:04:50.136 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:04:50.136 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:04:50.136 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:04:50.136 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:04:50.136 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:6789/0", 2024-05-09T08:04:50.136 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.7:6789/0", 2024-05-09T08:04:50.136 INFO:teuthology.orchestra.run.smithi001.stdout: "priority": 0, 2024-05-09T08:04:50.136 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 0, 2024-05-09T08:04:50.137 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_location": "{}" 2024-05-09T08:04:50.137 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:04:50.137 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:04:50.137 INFO:teuthology.orchestra.run.smithi001.stdout: "rank": 2, 2024-05-09T08:04:50.137 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "c", 2024-05-09T08:04:50.137 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-05-09T08:04:50.137 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-05-09T08:04:50.137 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:04:50.137 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-05-09T08:04:50.137 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:3301", 2024-05-09T08:04:50.137 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:04:50.137 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:04:50.138 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:04:50.138 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-05-09T08:04:50.138 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:6790", 2024-05-09T08:04:50.138 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:04:50.138 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:04:50.138 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:04:50.138 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:04:50.138 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:6790/0", 2024-05-09T08:04:50.138 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.7:6790/0", 2024-05-09T08:04:50.138 INFO:teuthology.orchestra.run.smithi001.stdout: "priority": 0, 2024-05-09T08:04:50.138 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 0, 2024-05-09T08:04:50.139 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_location": "{}" 2024-05-09T08:04:50.139 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:04:50.139 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:04:50.139 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:04:50.139 INFO:teuthology.orchestra.run.smithi001.stdout: "feature_map": { 2024-05-09T08:04:50.139 INFO:teuthology.orchestra.run.smithi001.stdout: "mon": [ 2024-05-09T08:04:50.139 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:04:50.139 INFO:teuthology.orchestra.run.smithi001.stdout: "features": "0x3f03cffffffdffff", 2024-05-09T08:04:50.139 INFO:teuthology.orchestra.run.smithi001.stdout: "release": "luminous", 2024-05-09T08:04:50.139 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 1 2024-05-09T08:04:50.139 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:04:50.140 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:04:50.140 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": [ 2024-05-09T08:04:50.140 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:04:50.140 INFO:teuthology.orchestra.run.smithi001.stdout: "features": "0x3f03cffffffdffff", 2024-05-09T08:04:50.140 INFO:teuthology.orchestra.run.smithi001.stdout: "release": "luminous", 2024-05-09T08:04:50.140 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 1 2024-05-09T08:04:50.140 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:04:50.140 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:04:50.140 INFO:teuthology.orchestra.run.smithi001.stdout: "client": [ 2024-05-09T08:04:50.140 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:04:50.140 INFO:teuthology.orchestra.run.smithi001.stdout: "features": "0x3f03cffffffdffff", 2024-05-09T08:04:50.140 INFO:teuthology.orchestra.run.smithi001.stdout: "release": "luminous", 2024-05-09T08:04:50.141 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 1 2024-05-09T08:04:50.141 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:04:50.141 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:04:50.141 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:04:50.141 INFO:teuthology.orchestra.run.smithi001.stdout: "stretch_mode": false 2024-05-09T08:04:50.141 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-05-09T08:04:50.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.113+0000 7ff19f800640 1 -- 172.21.15.1:0/3599764205 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7ff17806bd30 msgr2=0x7ff17806e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:04:50.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.113+0000 7ff19f800640 1 --2- 172.21.15.1:0/3599764205 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7ff17806bd30 0x7ff17806e1f0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ff1900012e0 tx=0x7ff1900015d0 comp rx=0 tx=0).stop 2024-05-09T08:04:50.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.113+0000 7ff19f800640 1 -- 172.21.15.1:0/3599764205 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff198072d90 msgr2=0x7ff198155170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:04:50.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.113+0000 7ff19f800640 1 --2- 172.21.15.1:0/3599764205 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff198072d90 0x7ff198155170 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7ff18c02a340 tx=0x7ff18c0046c0 comp rx=0 tx=0).stop 2024-05-09T08:04:50.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.113+0000 7ff19d800640 1 -- 172.21.15.1:0/3599764205 reap_dead start 2024-05-09T08:04:50.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.113+0000 7ff19d800640 1 -- 172.21.15.1:0/3599764205 reap_dead start 2024-05-09T08:04:50.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.113+0000 7ff19f800640 1 -- 172.21.15.1:0/3599764205 shutdown_connections 2024-05-09T08:04:50.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.113+0000 7ff19f800640 1 -- 172.21.15.1:0/3599764205 >> 172.21.15.1:0/3599764205 conn(0x7ff19806dad0 msgr2=0x7ff1981e6bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:04:50.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.113+0000 7ff19f800640 1 -- 172.21.15.1:0/3599764205 shutdown_connections 2024-05-09T08:04:50.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.113+0000 7ff19f800640 1 -- 172.21.15.1:0/3599764205 wait complete. 2024-05-09T08:04:50.144 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-05-09T08:04:50.144 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-05-09T08:04:50.145 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-05-09T08:04:50.259 DEBUG:teuthology.orchestra.run.smithi001:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T08:04:50.266 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T08:04:50.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.324+0000 7fec07200640 1 Processor -- start 2024-05-09T08:04:50.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.324+0000 7fec07200640 1 -- start start 2024-05-09T08:04:50.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.326+0000 7fec07200640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fec000cc260 0x7fec0006ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:04:50.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.326+0000 7fec07200640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fec0006cdc0 0x7fec000cb090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:04:50.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.326+0000 7fec07200640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fec0015e630 con 0x7fec0006c9e0 2024-05-09T08:04:50.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.326+0000 7fec07200640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7fec000ca920 con 0x7fec000cc260 2024-05-09T08:04:50.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.326+0000 7fec07200640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7fec0015ebe0 con 0x7fec0006cdc0 2024-05-09T08:04:50.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.326+0000 7febffe00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fec0006c9e0 0x7fec001545b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.1:42686/0 (socket says 172.21.15.1:42686) 2024-05-09T08:04:50.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.326+0000 7febff400640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fec000cc260 0x7fec0006ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:04:50.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.326+0000 7febffe00640 1 -- 172.21.15.1:0/1567286999 learned_addr learned my addr 172.21.15.1:0/1567286999 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T08:04:50.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.326+0000 7fec05200640 1 --2- 172.21.15.1:0/1567286999 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fec0006cdc0 0x7fec000cb090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:04:50.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.327+0000 7febfea00640 1 -- 172.21.15.1:0/1567286999 <== mon.1 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1849452411 0 0) 0x7fec0015e630 con 0x7fec0006c9e0 2024-05-09T08:04:50.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.327+0000 7febfea00640 1 -- 172.21.15.1:0/1567286999 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7febe4003520 con 0x7fec0006c9e0 2024-05-09T08:04:50.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.327+0000 7fec05200640 1 -- 172.21.15.1:0/1567286999 >> v2:172.21.15.1:3300/0 conn(0x7fec000cc260 msgr2=0x7fec0006ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:04:50.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.327+0000 7fec05200640 1 --2- 172.21.15.1:0/1567286999 >> v2:172.21.15.1:3300/0 conn(0x7fec000cc260 0x7fec0006ddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:04:50.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.327+0000 7fec05200640 1 -- 172.21.15.1:0/1567286999 >> v1:172.21.15.7:6789/0 conn(0x7fec0006c9e0 legacy=0x7fec001545b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:04:50.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.327+0000 7fec05200640 1 -- 172.21.15.1:0/1567286999 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec0015e900 con 0x7fec0006cdc0 2024-05-09T08:04:50.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.327+0000 7febff400640 1 --2- 172.21.15.1:0/1567286999 >> v2:172.21.15.1:3300/0 conn(0x7fec000cc260 0x7fec0006ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:04:50.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.328+0000 7fec05200640 1 --2- 172.21.15.1:0/1567286999 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fec0006cdc0 0x7fec000cb090 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7febf8009d90 tx=0x7febf802a5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2092659f545261ab server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:04:50.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.328+0000 7febfea00640 1 -- 172.21.15.1:0/1567286999 <== mon.2 v2:172.21.15.7:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febf8031600 con 0x7fec0006cdc0 2024-05-09T08:04:50.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.328+0000 7febfea00640 1 -- 172.21.15.1:0/1567286999 <== mon.2 v2:172.21.15.7:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7febf80317a0 con 0x7fec0006cdc0 2024-05-09T08:04:50.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.328+0000 7febfea00640 1 -- 172.21.15.1:0/1567286999 <== mon.2 v2:172.21.15.7:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febf8031b70 con 0x7fec0006cdc0 2024-05-09T08:04:50.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.329+0000 7fec07200640 1 -- 172.21.15.1:0/1567286999 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fec0006cdc0 msgr2=0x7fec000cb090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:04:50.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.329+0000 7fec07200640 1 --2- 172.21.15.1:0/1567286999 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fec0006cdc0 0x7fec000cb090 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7febf8009d90 tx=0x7febf802a5d0 comp rx=0 tx=0).stop 2024-05-09T08:04:50.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.329+0000 7fec07200640 1 -- 172.21.15.1:0/1567286999 shutdown_connections 2024-05-09T08:04:50.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.329+0000 7fec07200640 1 --2- 172.21.15.1:0/1567286999 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fec0006cdc0 0x7fec000cb090 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:04:50.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.329+0000 7fec07200640 1 --2- 172.21.15.1:0/1567286999 >> v2:172.21.15.1:3300/0 conn(0x7fec000cc260 0x7fec0006ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:04:50.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.329+0000 7fec07200640 1 -- 172.21.15.1:0/1567286999 >> 172.21.15.1:0/1567286999 conn(0x7fec000cffa0 msgr2=0x7fec000d03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:04:50.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.329+0000 7fec07200640 1 -- 172.21.15.1:0/1567286999 shutdown_connections 2024-05-09T08:04:50.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.329+0000 7fec07200640 1 -- 172.21.15.1:0/1567286999 wait complete. 2024-05-09T08:04:50.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.330+0000 7fec07200640 1 Processor -- start 2024-05-09T08:04:50.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.331+0000 7fec07200640 1 -- start start 2024-05-09T08:04:50.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.331+0000 7fec07200640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fec0006c9e0 0x7fec000ceac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:04:50.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.331+0000 7fec07200640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fec0006cdc0 0x7fec0014f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:04:50.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.331+0000 7fec07200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fec000cc260 0x7fec0014fae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:04:50.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.331+0000 7fec07200640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fec000cab70 con 0x7fec0006cdc0 2024-05-09T08:04:50.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.331+0000 7fec07200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fec0006d980 con 0x7fec000cc260 2024-05-09T08:04:50.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.331+0000 7fec07200640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7fec00154cc0 con 0x7fec0006c9e0 2024-05-09T08:04:50.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.332+0000 7febff400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fec0006cdc0 0x7fec0014f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:04:50.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.332+0000 7febffe00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fec0006c9e0 0x7fec000ceac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:04:50.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.332+0000 7fec05200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fec000cc260 0x7fec0014fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:04:50.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.332+0000 7febff400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fec0006cdc0 0x7fec0014f5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45262/0 (socket says 172.21.15.1:45262) 2024-05-09T08:04:50.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.332+0000 7fec05200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fec000cc260 0x7fec0014fae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.1:45140/0 (socket says 172.21.15.1:45140) 2024-05-09T08:04:50.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.332+0000 7febff400640 1 -- 172.21.15.1:0/3645848850 learned_addr learned my addr 172.21.15.1:0/3645848850 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:04:50.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.332+0000 7febff400640 1 -- 172.21.15.1:0/3645848850 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fec0006c9e0 msgr2=0x7fec000ceac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:04:50.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.332+0000 7febff400640 1 --2- 172.21.15.1:0/3645848850 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fec0006c9e0 0x7fec000ceac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:04:50.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.332+0000 7febff400640 1 -- 172.21.15.1:0/3645848850 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fec000cc260 msgr2=0x7fec0014fae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:04:50.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.332+0000 7febff400640 1 --2- 172.21.15.1:0/3645848850 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fec000cc260 0x7fec0014fae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:04:50.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.332+0000 7febff400640 1 -- 172.21.15.1:0/3645848850 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec000cf000 con 0x7fec0006cdc0 2024-05-09T08:04:50.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.332+0000 7febff400640 1 --2- 172.21.15.1:0/3645848850 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fec0006cdc0 0x7fec0014f5a0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7febec00ec10 tx=0x7febec00c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:04:50.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.333+0000 7febf7e00640 1 -- 172.21.15.1:0/3645848850 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febec017070 con 0x7fec0006cdc0 2024-05-09T08:04:50.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.333+0000 7febf7e00640 1 -- 172.21.15.1:0/3645848850 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7febec00cc30 con 0x7fec0006cdc0 2024-05-09T08:04:50.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.333+0000 7febf7e00640 1 -- 172.21.15.1:0/3645848850 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febec013470 con 0x7fec0006cdc0 2024-05-09T08:04:50.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.333+0000 7fec07200640 1 -- 172.21.15.1:0/3645848850 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec000cf290 con 0x7fec0006cdc0 2024-05-09T08:04:50.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.333+0000 7fec07200640 1 -- 172.21.15.1:0/3645848850 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec000cf850 con 0x7fec0006cdc0 2024-05-09T08:04:50.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.334+0000 7fec07200640 1 --2- 172.21.15.1:0/3645848850 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fec0014eac0 0x7fec001f33c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:04:50.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.334+0000 7fec07200640 1 -- 172.21.15.1:0/3645848850 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7fec0014f110 con 0x7fec0014eac0 2024-05-09T08:04:50.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.335+0000 7febffe00640 1 --2- 172.21.15.1:0/3645848850 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fec0014eac0 0x7fec001f33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:04:50.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.335+0000 7febf7e00640 1 -- 172.21.15.1:0/3645848850 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7febec01a070 con 0x7fec0006cdc0 2024-05-09T08:04:50.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.335+0000 7febffe00640 1 -- 172.21.15.1:0/3645848850 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fec00074d70 con 0x7fec0014eac0 2024-05-09T08:04:50.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.335+0000 7febffe00640 1 --2- 172.21.15.1:0/3645848850 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fec0014eac0 0x7fec001f33c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7febf0000f60 tx=0x7febf000f9d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:04:50.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.336+0000 7febf7e00640 1 --2- 172.21.15.1:0/3645848850 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7febe006be30 0x7febe006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:04:50.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.336+0000 7febf7e00640 1 -- 172.21.15.1:0/3645848850 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7febec08b540 con 0x7fec0006cdc0 2024-05-09T08:04:50.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.336+0000 7fec05200640 1 --2- 172.21.15.1:0/3645848850 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7febe006be30 0x7febe006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:04:50.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.336+0000 7febf7e00640 1 -- 172.21.15.1:0/3645848850 <== mon.2 v2:172.21.15.7:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febf0005ac0 con 0x7fec0014eac0 2024-05-09T08:04:50.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.336+0000 7febf7e00640 1 -- 172.21.15.1:0/3645848850 <== mon.2 v2:172.21.15.7:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7febf0018070 con 0x7fec0014eac0 2024-05-09T08:04:50.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.336+0000 7febf7e00640 1 -- 172.21.15.1:0/3645848850 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fec0014eac0 msgr2=0x7fec001f33c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:04:50.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.336+0000 7febf7e00640 1 --2- 172.21.15.1:0/3645848850 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fec0014eac0 0x7fec001f33c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7febf0000f60 tx=0x7febf000f9d0 comp rx=0 tx=0).stop 2024-05-09T08:04:50.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.336+0000 7febf7e00640 1 -- 172.21.15.1:0/3645848850 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fec0014eac0 msgr2=0x7fec001f33c0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-05-09T08:04:50.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.336+0000 7febf7e00640 1 --2- 172.21.15.1:0/3645848850 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fec0014eac0 0x7fec001f33c0 secure :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7febf0000f60 tx=0x7febf000f9d0 comp rx=0 tx=0).stop 2024-05-09T08:04:50.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.337+0000 7fec05200640 1 --2- 172.21.15.1:0/3645848850 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7febe006be30 0x7febe006e2f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7febf8034000 tx=0x7febf802f000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:04:50.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.345+0000 7fec07200640 1 --2- 172.21.15.1:0/3645848850 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fec001f3900 0x7fec00146450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:04:50.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.345+0000 7fec07200640 1 -- 172.21.15.1:0/3645848850 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7fec00154590 con 0x7fec001f3900 2024-05-09T08:04:50.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.345+0000 7febffe00640 1 --2- 172.21.15.1:0/3645848850 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fec001f3900 0x7fec00146450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:04:50.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.346+0000 7febffe00640 1 -- 172.21.15.1:0/3645848850 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fec0014b220 con 0x7fec001f3900 2024-05-09T08:04:50.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.346+0000 7febffe00640 1 --2- 172.21.15.1:0/3645848850 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fec001f3900 0x7fec00146450 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7febf0000f60 tx=0x7febf00010a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:04:50.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.346+0000 7febf7e00640 1 -- 172.21.15.1:0/3645848850 <== mon.2 v2:172.21.15.7:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febf001d9d0 con 0x7fec001f3900 2024-05-09T08:04:50.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.346+0000 7febf7e00640 1 -- 172.21.15.1:0/3645848850 <== mon.2 v2:172.21.15.7:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5188 (secure 0 0 0) 0x7febf001d9d0 con 0x7fec001f3900 2024-05-09T08:04:50.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.346+0000 7febf7e00640 1 -- 172.21.15.1:0/3645848850 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fec001f3900 msgr2=0x7fec00146450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:04:50.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.346+0000 7febf7e00640 1 --2- 172.21.15.1:0/3645848850 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fec001f3900 0x7fec00146450 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7febf0000f60 tx=0x7febf00010a0 comp rx=0 tx=0).stop 2024-05-09T08:04:50.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.346+0000 7febf7e00640 1 -- 172.21.15.1:0/3645848850 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fec001f3900 msgr2=0x7fec00146450 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-05-09T08:04:50.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.346+0000 7febf7e00640 1 --2- 172.21.15.1:0/3645848850 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fec001f3900 0x7fec00146450 secure :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7febf0000f60 tx=0x7febf00010a0 comp rx=0 tx=0).stop 2024-05-09T08:04:50.361 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-05-09T08:04:50.362 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "c", 2024-05-09T08:04:50.362 INFO:teuthology.orchestra.run.smithi001.stdout: "rank": 2, 2024-05-09T08:04:50.362 INFO:teuthology.orchestra.run.smithi001.stdout: "state": "peon", 2024-05-09T08:04:50.362 INFO:teuthology.orchestra.run.smithi001.stdout: "election_epoch": 20, 2024-05-09T08:04:50.362 INFO:teuthology.orchestra.run.smithi001.stdout: "uptime": 173762, 2024-05-09T08:04:50.362 INFO:teuthology.orchestra.run.smithi001.stdout: "quorum": [ 2024-05-09T08:04:50.362 INFO:teuthology.orchestra.run.smithi001.stdout: 0, 2024-05-09T08:04:50.362 INFO:teuthology.orchestra.run.smithi001.stdout: 1, 2024-05-09T08:04:50.362 INFO:teuthology.orchestra.run.smithi001.stdout: 2 2024-05-09T08:04:50.362 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:04:50.362 INFO:teuthology.orchestra.run.smithi001.stdout: "quorum_age": 3, 2024-05-09T08:04:50.362 INFO:teuthology.orchestra.run.smithi001.stdout: "features": { 2024-05-09T08:04:50.363 INFO:teuthology.orchestra.run.smithi001.stdout: "required_con": "2450521708007866372", 2024-05-09T08:04:50.363 INFO:teuthology.orchestra.run.smithi001.stdout: "required_mon": [ 2024-05-09T08:04:50.363 INFO:teuthology.orchestra.run.smithi001.stdout: "kraken", 2024-05-09T08:04:50.363 INFO:teuthology.orchestra.run.smithi001.stdout: "luminous", 2024-05-09T08:04:50.363 INFO:teuthology.orchestra.run.smithi001.stdout: "mimic", 2024-05-09T08:04:50.363 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap-prune", 2024-05-09T08:04:50.363 INFO:teuthology.orchestra.run.smithi001.stdout: "nautilus", 2024-05-09T08:04:50.363 INFO:teuthology.orchestra.run.smithi001.stdout: "octopus", 2024-05-09T08:04:50.363 INFO:teuthology.orchestra.run.smithi001.stdout: "pacific", 2024-05-09T08:04:50.363 INFO:teuthology.orchestra.run.smithi001.stdout: "elector-pinging", 2024-05-09T08:04:50.363 INFO:teuthology.orchestra.run.smithi001.stdout: "quincy", 2024-05-09T08:04:50.363 INFO:teuthology.orchestra.run.smithi001.stdout: "reef", 2024-05-09T08:04:50.363 INFO:teuthology.orchestra.run.smithi001.stdout: "squid" 2024-05-09T08:04:50.363 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:04:50.364 INFO:teuthology.orchestra.run.smithi001.stdout: "quorum_con": "4540701547738038271", 2024-05-09T08:04:50.364 INFO:teuthology.orchestra.run.smithi001.stdout: "quorum_mon": [ 2024-05-09T08:04:50.364 INFO:teuthology.orchestra.run.smithi001.stdout: "kraken", 2024-05-09T08:04:50.364 INFO:teuthology.orchestra.run.smithi001.stdout: "luminous", 2024-05-09T08:04:50.364 INFO:teuthology.orchestra.run.smithi001.stdout: "mimic", 2024-05-09T08:04:50.364 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap-prune", 2024-05-09T08:04:50.364 INFO:teuthology.orchestra.run.smithi001.stdout: "nautilus", 2024-05-09T08:04:50.364 INFO:teuthology.orchestra.run.smithi001.stdout: "octopus", 2024-05-09T08:04:50.364 INFO:teuthology.orchestra.run.smithi001.stdout: "pacific", 2024-05-09T08:04:50.364 INFO:teuthology.orchestra.run.smithi001.stdout: "elector-pinging", 2024-05-09T08:04:50.364 INFO:teuthology.orchestra.run.smithi001.stdout: "quincy", 2024-05-09T08:04:50.364 INFO:teuthology.orchestra.run.smithi001.stdout: "reef", 2024-05-09T08:04:50.364 INFO:teuthology.orchestra.run.smithi001.stdout: "squid" 2024-05-09T08:04:50.364 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:04:50.365 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:04:50.365 INFO:teuthology.orchestra.run.smithi001.stdout: "outside_quorum": [], 2024-05-09T08:04:50.365 INFO:teuthology.orchestra.run.smithi001.stdout: "extra_probe_peers": [], 2024-05-09T08:04:50.365 INFO:teuthology.orchestra.run.smithi001.stdout: "sync_provider": [], 2024-05-09T08:04:50.365 INFO:teuthology.orchestra.run.smithi001.stdout: "monmap": { 2024-05-09T08:04:50.365 INFO:teuthology.orchestra.run.smithi001.stdout: "epoch": 1, 2024-05-09T08:04:50.365 INFO:teuthology.orchestra.run.smithi001.stdout: "fsid": "54329ec8-3270-4043-9a82-c6cd7420aa7f", 2024-05-09T08:04:50.365 INFO:teuthology.orchestra.run.smithi001.stdout: "modified": "2024-05-09T08:01:21.819270Z", 2024-05-09T08:04:50.365 INFO:teuthology.orchestra.run.smithi001.stdout: "created": "2024-05-09T08:01:21.819270Z", 2024-05-09T08:04:50.365 INFO:teuthology.orchestra.run.smithi001.stdout: "min_mon_release": 19, 2024-05-09T08:04:50.365 INFO:teuthology.orchestra.run.smithi001.stdout: "min_mon_release_name": "squid", 2024-05-09T08:04:50.365 INFO:teuthology.orchestra.run.smithi001.stdout: "election_strategy": 1, 2024-05-09T08:04:50.365 INFO:teuthology.orchestra.run.smithi001.stdout: "disallowed_leaders: ": "", 2024-05-09T08:04:50.365 INFO:teuthology.orchestra.run.smithi001.stdout: "stretch_mode": false, 2024-05-09T08:04:50.366 INFO:teuthology.orchestra.run.smithi001.stdout: "tiebreaker_mon": "", 2024-05-09T08:04:50.366 INFO:teuthology.orchestra.run.smithi001.stdout: "removed_ranks: ": "", 2024-05-09T08:04:50.366 INFO:teuthology.orchestra.run.smithi001.stdout: "features": { 2024-05-09T08:04:50.366 INFO:teuthology.orchestra.run.smithi001.stdout: "persistent": [ 2024-05-09T08:04:50.366 INFO:teuthology.orchestra.run.smithi001.stdout: "kraken", 2024-05-09T08:04:50.366 INFO:teuthology.orchestra.run.smithi001.stdout: "luminous", 2024-05-09T08:04:50.366 INFO:teuthology.orchestra.run.smithi001.stdout: "mimic", 2024-05-09T08:04:50.366 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap-prune", 2024-05-09T08:04:50.366 INFO:teuthology.orchestra.run.smithi001.stdout: "nautilus", 2024-05-09T08:04:50.366 INFO:teuthology.orchestra.run.smithi001.stdout: "octopus", 2024-05-09T08:04:50.366 INFO:teuthology.orchestra.run.smithi001.stdout: "pacific", 2024-05-09T08:04:50.366 INFO:teuthology.orchestra.run.smithi001.stdout: "elector-pinging", 2024-05-09T08:04:50.366 INFO:teuthology.orchestra.run.smithi001.stdout: "quincy", 2024-05-09T08:04:50.366 INFO:teuthology.orchestra.run.smithi001.stdout: "reef", 2024-05-09T08:04:50.367 INFO:teuthology.orchestra.run.smithi001.stdout: "squid" 2024-05-09T08:04:50.367 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:04:50.367 INFO:teuthology.orchestra.run.smithi001.stdout: "optional": [] 2024-05-09T08:04:50.367 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:04:50.367 INFO:teuthology.orchestra.run.smithi001.stdout: "mons": [ 2024-05-09T08:04:50.367 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:04:50.367 INFO:teuthology.orchestra.run.smithi001.stdout: "rank": 0, 2024-05-09T08:04:50.367 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "a", 2024-05-09T08:04:50.367 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-05-09T08:04:50.367 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-05-09T08:04:50.367 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:04:50.367 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-05-09T08:04:50.367 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:3300", 2024-05-09T08:04:50.368 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:04:50.368 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:04:50.368 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:04:50.368 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-05-09T08:04:50.368 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6789", 2024-05-09T08:04:50.368 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:04:50.368 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:04:50.368 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:04:50.368 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:04:50.368 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6789/0", 2024-05-09T08:04:50.368 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.1:6789/0", 2024-05-09T08:04:50.368 INFO:teuthology.orchestra.run.smithi001.stdout: "priority": 0, 2024-05-09T08:04:50.368 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 0, 2024-05-09T08:04:50.368 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_location": "{}" 2024-05-09T08:04:50.369 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:04:50.369 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:04:50.369 INFO:teuthology.orchestra.run.smithi001.stdout: "rank": 1, 2024-05-09T08:04:50.369 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "b", 2024-05-09T08:04:50.369 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-05-09T08:04:50.369 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-05-09T08:04:50.369 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:04:50.369 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-05-09T08:04:50.369 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:3300", 2024-05-09T08:04:50.369 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:04:50.369 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:04:50.369 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:04:50.369 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-05-09T08:04:50.369 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:6789", 2024-05-09T08:04:50.370 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:04:50.370 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:04:50.370 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:04:50.370 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:04:50.370 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:6789/0", 2024-05-09T08:04:50.370 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.7:6789/0", 2024-05-09T08:04:50.370 INFO:teuthology.orchestra.run.smithi001.stdout: "priority": 0, 2024-05-09T08:04:50.370 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 0, 2024-05-09T08:04:50.370 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_location": "{}" 2024-05-09T08:04:50.370 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:04:50.370 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:04:50.370 INFO:teuthology.orchestra.run.smithi001.stdout: "rank": 2, 2024-05-09T08:04:50.370 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "c", 2024-05-09T08:04:50.370 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-05-09T08:04:50.371 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-05-09T08:04:50.371 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:04:50.371 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-05-09T08:04:50.371 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:3301", 2024-05-09T08:04:50.371 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:04:50.371 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:04:50.371 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:04:50.371 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-05-09T08:04:50.371 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:6790", 2024-05-09T08:04:50.371 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:04:50.371 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:04:50.371 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:04:50.371 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:04:50.372 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:6790/0", 2024-05-09T08:04:50.372 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.7:6790/0", 2024-05-09T08:04:50.372 INFO:teuthology.orchestra.run.smithi001.stdout: "priority": 0, 2024-05-09T08:04:50.372 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 0, 2024-05-09T08:04:50.372 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_location": "{}" 2024-05-09T08:04:50.372 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:04:50.372 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:04:50.372 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:04:50.372 INFO:teuthology.orchestra.run.smithi001.stdout: "feature_map": { 2024-05-09T08:04:50.372 INFO:teuthology.orchestra.run.smithi001.stdout: "mon": [ 2024-05-09T08:04:50.372 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:04:50.372 INFO:teuthology.orchestra.run.smithi001.stdout: "features": "0x3f03cffffffdffff", 2024-05-09T08:04:50.372 INFO:teuthology.orchestra.run.smithi001.stdout: "release": "luminous", 2024-05-09T08:04:50.372 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 1 2024-05-09T08:04:50.373 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:04:50.373 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:04:50.373 INFO:teuthology.orchestra.run.smithi001.stdout: "mds": [ 2024-05-09T08:04:50.373 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:04:50.373 INFO:teuthology.orchestra.run.smithi001.stdout: "features": "0x3f03cffffffdffff", 2024-05-09T08:04:50.373 INFO:teuthology.orchestra.run.smithi001.stdout: "release": "luminous", 2024-05-09T08:04:50.373 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 6 2024-05-09T08:04:50.373 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:04:50.373 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:04:50.373 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": [ 2024-05-09T08:04:50.373 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:04:50.373 INFO:teuthology.orchestra.run.smithi001.stdout: "features": "0x3f03cffffffdffff", 2024-05-09T08:04:50.373 INFO:teuthology.orchestra.run.smithi001.stdout: "release": "luminous", 2024-05-09T08:04:50.373 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 7 2024-05-09T08:04:50.374 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:04:50.374 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:04:50.374 INFO:teuthology.orchestra.run.smithi001.stdout: "client": [ 2024-05-09T08:04:50.374 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:04:50.374 INFO:teuthology.orchestra.run.smithi001.stdout: "features": "0x2f018fb87aa4aafe", 2024-05-09T08:04:50.374 INFO:teuthology.orchestra.run.smithi001.stdout: "release": "luminous", 2024-05-09T08:04:50.374 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 1 2024-05-09T08:04:50.374 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:04:50.374 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:04:50.374 INFO:teuthology.orchestra.run.smithi001.stdout: "features": "0x3f03cffffffdffff", 2024-05-09T08:04:50.374 INFO:teuthology.orchestra.run.smithi001.stdout: "release": "luminous", 2024-05-09T08:04:50.374 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 5 2024-05-09T08:04:50.374 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:04:50.375 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:04:50.375 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr": [ 2024-05-09T08:04:50.375 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:04:50.375 INFO:teuthology.orchestra.run.smithi001.stdout: "features": "0x3f03cffffffdffff", 2024-05-09T08:04:50.375 INFO:teuthology.orchestra.run.smithi001.stdout: "release": "luminous", 2024-05-09T08:04:50.375 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 2 2024-05-09T08:04:50.375 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:04:50.375 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:04:50.375 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:04:50.375 INFO:teuthology.orchestra.run.smithi001.stdout: "stretch_mode": false 2024-05-09T08:04:50.375 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-05-09T08:04:50.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.347+0000 7fec07200640 1 -- 172.21.15.1:0/3645848850 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7febe006be30 msgr2=0x7febe006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:04:50.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.347+0000 7fec07200640 1 --2- 172.21.15.1:0/3645848850 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7febe006be30 0x7febe006e2f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7febf8034000 tx=0x7febf802f000 comp rx=0 tx=0).stop 2024-05-09T08:04:50.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.347+0000 7fec07200640 1 -- 172.21.15.1:0/3645848850 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fec0006cdc0 msgr2=0x7fec0014f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:04:50.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.347+0000 7fec07200640 1 --2- 172.21.15.1:0/3645848850 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fec0006cdc0 0x7fec0014f5a0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7febec00ec10 tx=0x7febec00c520 comp rx=0 tx=0).stop 2024-05-09T08:04:50.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.347+0000 7fec05200640 1 -- 172.21.15.1:0/3645848850 reap_dead start 2024-05-09T08:04:50.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.347+0000 7fec05200640 1 -- 172.21.15.1:0/3645848850 reap_dead start 2024-05-09T08:04:50.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.347+0000 7fec07200640 1 -- 172.21.15.1:0/3645848850 shutdown_connections 2024-05-09T08:04:50.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.347+0000 7fec07200640 1 -- 172.21.15.1:0/3645848850 >> 172.21.15.1:0/3645848850 conn(0x7fec000cffa0 msgr2=0x7fec000b2f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:04:50.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.347+0000 7fec07200640 1 -- 172.21.15.1:0/3645848850 shutdown_connections 2024-05-09T08:04:50.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.347+0000 7fec07200640 1 -- 172.21.15.1:0/3645848850 wait complete. 2024-05-09T08:04:50.378 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-05-09T08:04:50.378 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-05-09T08:04:50.378 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-05-09T08:04:50.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.551+0000 7f3e88e00640 1 Processor -- start 2024-05-09T08:04:50.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.551+0000 7f3e88e00640 1 -- start start 2024-05-09T08:04:50.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.553+0000 7f3e88e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f3e8415ba50 0x7f3e8406d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:04:50.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.553+0000 7f3e88e00640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3e8415b550 con 0x7f3e8415a5e0 2024-05-09T08:04:50.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.553+0000 7f3e88e00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3e8406c8b0 con 0x7f3e840cd040 2024-05-09T08:04:50.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.553+0000 7f3e88e00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f3e8406cec0 con 0x7f3e8415ba50 2024-05-09T08:04:50.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.553+0000 7f3e83e00640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f3e8415a5e0 0x7f3e840ca5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:35622/0 (socket says 172.21.15.1:35622) 2024-05-09T08:04:50.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.553+0000 7f3e83e00640 1 -- 172.21.15.1:0/4274224004 learned_addr learned my addr 172.21.15.1:0/4274224004 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T08:04:50.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.553+0000 7f3e7aa00640 1 --2- 172.21.15.1:0/4274224004 >> v2:172.21.15.7:3300/0 conn(0x7f3e8415ba50 0x7f3e8406d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:04:50.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.553+0000 7f3e82a00640 1 -- 172.21.15.1:0/4274224004 <== mon.1 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1274962863 0 0) 0x7f3e8406c8b0 con 0x7f3e840cd040 2024-05-09T08:04:50.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.553+0000 7f3e7aa00640 1 -- 172.21.15.1:0/4274224004 >> v1:172.21.15.7:6789/0 conn(0x7f3e840cd040 legacy=0x7f3e8406da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:04:50.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.553+0000 7f3e7aa00640 1 -- 172.21.15.1:0/4274224004 >> v1:172.21.15.1:6789/0 conn(0x7f3e8415a5e0 legacy=0x7f3e840ca5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:04:50.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.554+0000 7f3e7aa00640 1 -- 172.21.15.1:0/4274224004 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e8415af90 con 0x7f3e8415ba50 2024-05-09T08:04:50.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.555+0000 7f3e7aa00640 1 --2- 172.21.15.1:0/4274224004 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3e8415ba50 0x7f3e8406d350 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f3e6c0097a0 tx=0x7f3e6c02a000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=daac1d55c8c6eda7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:04:50.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.555+0000 7f3e82a00640 1 -- 172.21.15.1:0/4274224004 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e6c037070 con 0x7f3e8415ba50 2024-05-09T08:04:50.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.555+0000 7f3e82a00640 1 -- 172.21.15.1:0/4274224004 <== mon.1 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e6c02ac90 con 0x7f3e8415ba50 2024-05-09T08:04:50.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.555+0000 7f3e82a00640 1 -- 172.21.15.1:0/4274224004 <== mon.1 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e6c032660 con 0x7f3e8415ba50 2024-05-09T08:04:50.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.556+0000 7f3e88e00640 1 -- 172.21.15.1:0/4274224004 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3e8415ba50 msgr2=0x7f3e8406d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:04:50.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.556+0000 7f3e88e00640 1 --2- 172.21.15.1:0/4274224004 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3e8415ba50 0x7f3e8406d350 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f3e6c0097a0 tx=0x7f3e6c02a000 comp rx=0 tx=0).stop 2024-05-09T08:04:50.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.556+0000 7f3e88e00640 1 -- 172.21.15.1:0/4274224004 shutdown_connections 2024-05-09T08:04:50.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.556+0000 7f3e88e00640 1 --2- 172.21.15.1:0/4274224004 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3e8415ba50 0x7f3e8406d350 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:04:50.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.556+0000 7f3e88e00640 1 -- 172.21.15.1:0/4274224004 >> 172.21.15.1:0/4274224004 conn(0x7f3e840b1780 msgr2=0x7f3e840b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:04:50.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.556+0000 7f3e88e00640 1 -- 172.21.15.1:0/4274224004 shutdown_connections 2024-05-09T08:04:50.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.556+0000 7f3e88e00640 1 -- 172.21.15.1:0/4274224004 wait complete. 2024-05-09T08:04:50.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.557+0000 7f3e88e00640 1 Processor -- start 2024-05-09T08:04:50.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.558+0000 7f3e88e00640 1 -- start start 2024-05-09T08:04:50.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.558+0000 7f3e88e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3e840cd040 0x7f3e840c4b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:04:50.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.558+0000 7f3e88e00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f3e8415a5e0 0x7f3e84159b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:04:50.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.558+0000 7f3e88e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3e8415ba50 0x7f3e840c3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:04:50.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.558+0000 7f3e88e00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f3e840cad00 con 0x7f3e840cd040 2024-05-09T08:04:50.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.558+0000 7f3e88e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f3e8406cb50 con 0x7f3e8415ba50 2024-05-09T08:04:50.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.558+0000 7f3e88e00640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f3e840cae80 con 0x7f3e8415a5e0 2024-05-09T08:04:50.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.559+0000 7f3e83e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3e8415ba50 0x7f3e840c3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:04:50.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.559+0000 7f3e7aa00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f3e8415a5e0 0x7f3e84159b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:04:50.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.559+0000 7f3e83400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3e840cd040 0x7f3e840c4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:04:50.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.559+0000 7f3e7aa00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f3e8415a5e0 0x7f3e84159b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3301/0 says I am v2:172.21.15.1:60836/0 (socket says 172.21.15.1:60836) 2024-05-09T08:04:50.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.559+0000 7f3e7aa00640 1 -- 172.21.15.1:0/879070338 learned_addr learned my addr 172.21.15.1:0/879070338 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:04:50.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.559+0000 7f3e83e00640 1 -- 172.21.15.1:0/879070338 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f3e8415a5e0 msgr2=0x7f3e84159b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:04:50.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.559+0000 7f3e83e00640 1 --2- 172.21.15.1:0/879070338 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f3e8415a5e0 0x7f3e84159b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:04:50.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.559+0000 7f3e83e00640 1 -- 172.21.15.1:0/879070338 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3e840cd040 msgr2=0x7f3e840c4b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:04:50.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.559+0000 7f3e83e00640 1 --2- 172.21.15.1:0/879070338 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3e840cd040 0x7f3e840c4b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:04:50.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.559+0000 7f3e83e00640 1 -- 172.21.15.1:0/879070338 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e840cdeb0 con 0x7f3e8415ba50 2024-05-09T08:04:50.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.559+0000 7f3e7aa00640 1 --2- 172.21.15.1:0/879070338 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f3e8415a5e0 0x7f3e84159b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:04:50.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.559+0000 7f3e83400640 1 --2- 172.21.15.1:0/879070338 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3e840cd040 0x7f3e840c4b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:04:50.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.560+0000 7f3e83e00640 1 --2- 172.21.15.1:0/879070338 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3e8415ba50 0x7f3e840c3bd0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f3e7c00c450 tx=0x7f3e7c00c910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:04:50.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.560+0000 7f3e7be00640 1 -- 172.21.15.1:0/879070338 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e7c018070 con 0x7f3e8415ba50 2024-05-09T08:04:50.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.560+0000 7f3e7be00640 1 -- 172.21.15.1:0/879070338 <== mon.1 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e7c0074e0 con 0x7f3e8415ba50 2024-05-09T08:04:50.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.560+0000 7f3e88e00640 1 -- 172.21.15.1:0/879070338 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e841e4a60 con 0x7f3e8415ba50 2024-05-09T08:04:50.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.560+0000 7f3e7be00640 1 -- 172.21.15.1:0/879070338 <== mon.1 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e7c007690 con 0x7f3e8415ba50 2024-05-09T08:04:50.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.560+0000 7f3e88e00640 1 -- 172.21.15.1:0/879070338 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e840cda50 con 0x7f3e8415ba50 2024-05-09T08:04:50.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.561+0000 7f3e88e00640 1 -- 172.21.15.1:0/879070338 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e8404f660 con 0x7f3e8415ba50 2024-05-09T08:04:50.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.562+0000 7f3e7be00640 1 -- 172.21.15.1:0/879070338 <== mon.1 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3e7c01b070 con 0x7f3e8415ba50 2024-05-09T08:04:50.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.563+0000 7f3e7be00640 1 --2- 172.21.15.1:0/879070338 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f3e6006be30 0x7f3e6006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:04:50.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.563+0000 7f3e7be00640 1 -- 172.21.15.1:0/879070338 <== mon.1 v2:172.21.15.7:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7f3e7c08a510 con 0x7f3e8415ba50 2024-05-09T08:04:50.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.563+0000 7f3e83400640 1 --2- 172.21.15.1:0/879070338 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f3e6006be30 0x7f3e6006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:04:50.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.564+0000 7f3e83400640 1 --2- 172.21.15.1:0/879070338 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f3e6006be30 0x7f3e6006e2f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f3e740012e0 tx=0x7f3e74009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:04:50.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.568+0000 7f3e7be00640 1 -- 172.21.15.1:0/879070338 <== mon.1 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f3e7c0583e0 con 0x7f3e8415ba50 2024-05-09T08:04:50.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.842+0000 7f3e88e00640 1 -- 172.21.15.1:0/879070338 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "mon scrub"} v 0) -- 0x7f3e8406e260 con 0x7f3e8415ba50 2024-05-09T08:04:50.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.845+0000 7f3e7be00640 1 -- 172.21.15.1:0/879070338 <== mon.1 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) ==== 57+0+0 (secure 0 0 0) 0x7f3e7c05c6e0 con 0x7f3e8415ba50 2024-05-09T08:04:50.863 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.849+0000 7f3e88e00640 1 -- 172.21.15.1:0/879070338 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f3e6006be30 msgr2=0x7f3e6006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:04:50.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.849+0000 7f3e88e00640 1 --2- 172.21.15.1:0/879070338 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f3e6006be30 0x7f3e6006e2f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f3e740012e0 tx=0x7f3e74009040 comp rx=0 tx=0).stop 2024-05-09T08:04:50.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.849+0000 7f3e88e00640 1 -- 172.21.15.1:0/879070338 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3e8415ba50 msgr2=0x7f3e840c3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:04:50.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.849+0000 7f3e88e00640 1 --2- 172.21.15.1:0/879070338 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3e8415ba50 0x7f3e840c3bd0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f3e7c00c450 tx=0x7f3e7c00c910 comp rx=0 tx=0).stop 2024-05-09T08:04:50.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.849+0000 7f3e88e00640 1 -- 172.21.15.1:0/879070338 shutdown_connections 2024-05-09T08:04:50.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.849+0000 7f3e88e00640 1 --2- 172.21.15.1:0/879070338 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f3e6006be30 0x7f3e6006e2f0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:04:50.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.849+0000 7f3e88e00640 1 --2- 172.21.15.1:0/879070338 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3e8415ba50 0x7f3e840c3bd0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:04:50.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.849+0000 7f3e88e00640 1 --2- 172.21.15.1:0/879070338 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f3e8415a5e0 0x7f3e84159b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:04:50.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.849+0000 7f3e88e00640 1 --2- 172.21.15.1:0/879070338 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3e840cd040 0x7f3e840c4b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:04:50.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.849+0000 7f3e88e00640 1 -- 172.21.15.1:0/879070338 >> 172.21.15.1:0/879070338 conn(0x7f3e840b1780 msgr2=0x7f3e841541b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:04:50.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.849+0000 7f3e88e00640 1 -- 172.21.15.1:0/879070338 shutdown_connections 2024-05-09T08:04:50.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:04:50.849+0000 7f3e88e00640 1 -- 172.21.15.1:0/879070338 wait complete. 2024-05-09T08:04:50.882 INFO:tasks.mon_thrash.mon_thrasher:waiting for 10.0 secs before continuing thrashing 2024-05-09T08:04:52.999 INFO:tasks.workunit.client.0.smithi001.stdout: 2024-05-09T08:04:52.999 INFO:tasks.workunit.client.0.smithi001.stdout: Children see throughput for 1 readers = 3730355.00 kB/sec 2024-05-09T08:04:52.999 INFO:tasks.workunit.client.0.smithi001.stdout: Parent sees throughput for 1 readers = 3720572.00 kB/sec 2024-05-09T08:04:53.000 INFO:tasks.workunit.client.0.smithi001.stdout: Min throughput per process = 3730355.00 kB/sec 2024-05-09T08:04:53.000 INFO:tasks.workunit.client.0.smithi001.stdout: Max throughput per process = 3730355.00 kB/sec 2024-05-09T08:04:53.000 INFO:tasks.workunit.client.0.smithi001.stdout: Avg throughput per process = 3730355.00 kB/sec 2024-05-09T08:04:53.000 INFO:tasks.workunit.client.0.smithi001.stdout: Min xfer = 1048576.00 kB 2024-05-09T08:04:55.502 INFO:tasks.workunit.client.0.smithi001.stdout: 2024-05-09T08:04:55.502 INFO:tasks.workunit.client.0.smithi001.stdout: Children see throughput for 1 re-readers = 2624603.00 kB/sec 2024-05-09T08:04:55.502 INFO:tasks.workunit.client.0.smithi001.stdout: Parent sees throughput for 1 re-readers = 2619442.76 kB/sec 2024-05-09T08:04:55.502 INFO:tasks.workunit.client.0.smithi001.stdout: Min throughput per process = 2624603.00 kB/sec 2024-05-09T08:04:55.502 INFO:tasks.workunit.client.0.smithi001.stdout: Max throughput per process = 2624603.00 kB/sec 2024-05-09T08:04:55.503 INFO:tasks.workunit.client.0.smithi001.stdout: Avg throughput per process = 2624603.00 kB/sec 2024-05-09T08:04:55.503 INFO:tasks.workunit.client.0.smithi001.stdout: Min xfer = 1048576.00 kB 2024-05-09T08:05:00.883 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-05-09T08:05:00.883 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-05-09T08:05:01.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.090+0000 7f6a7bc00640 1 Processor -- start 2024-05-09T08:05:01.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.090+0000 7f6a7bc00640 1 -- start start 2024-05-09T08:05:01.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.092+0000 7f6a7bc00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f6a74151660 0x7f6a740c6c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:05:01.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.092+0000 7f6a7bc00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f6a740c1ba0 0x7f6a74072d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:05:01.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.092+0000 7f6a7bc00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6a740c72f0 con 0x7f6a740b1d30 2024-05-09T08:05:01.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.093+0000 7f6a7bc00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f6a74151a40 con 0x7f6a740c1ba0 2024-05-09T08:05:01.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.093+0000 7f6a7bc00640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f6a74157620 con 0x7f6a74151660 2024-05-09T08:05:01.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.093+0000 7f6a6be00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f6a740c1ba0 0x7f6a74072d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:05:01.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.093+0000 7f6a79200640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f6a74151660 0x7f6a740c6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:05:01.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.093+0000 7f6a79c00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f6a740b1d30 0x7f6a740c7a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.1:56180/0 (socket says 172.21.15.1:56180) 2024-05-09T08:05:01.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.093+0000 7f6a79c00640 1 -- 172.21.15.1:0/2179576946 learned_addr learned my addr 172.21.15.1:0/2179576946 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T08:05:01.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.093+0000 7f6a73e00640 1 -- 172.21.15.1:0/2179576946 <== mon.1 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3978660252 0 0) 0x7f6a740c72f0 con 0x7f6a740b1d30 2024-05-09T08:05:01.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.094+0000 7f6a79200640 1 -- 172.21.15.1:0/2179576946 >> v2:172.21.15.7:3300/0 conn(0x7f6a740c1ba0 msgr2=0x7f6a74072d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:05:01.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.094+0000 7f6a79200640 1 --2- 172.21.15.1:0/2179576946 >> v2:172.21.15.7:3300/0 conn(0x7f6a740c1ba0 0x7f6a74072d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:05:01.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.094+0000 7f6a79200640 1 -- 172.21.15.1:0/2179576946 >> v1:172.21.15.7:6789/0 conn(0x7f6a740b1d30 legacy=0x7f6a740c7a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:05:01.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.094+0000 7f6a79200640 1 -- 172.21.15.1:0/2179576946 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a74157900 con 0x7f6a74151660 2024-05-09T08:05:01.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.094+0000 7f6a6be00640 1 --2- 172.21.15.1:0/2179576946 >> v2:172.21.15.7:3300/0 conn(0x7f6a740c1ba0 0x7f6a74072d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:05:01.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.095+0000 7f6a79200640 1 --2- 172.21.15.1:0/2179576946 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f6a74151660 0x7f6a740c6c90 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f6a580097a0 tx=0x7f6a5802a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7256b2996a75ea66 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:05:01.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.095+0000 7f6a73e00640 1 -- 172.21.15.1:0/2179576946 <== mon.2 v2:172.21.15.7:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a58037070 con 0x7f6a74151660 2024-05-09T08:05:01.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.095+0000 7f6a73e00640 1 -- 172.21.15.1:0/2179576946 <== mon.2 v2:172.21.15.7:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a58032440 con 0x7f6a74151660 2024-05-09T08:05:01.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.095+0000 7f6a73e00640 1 -- 172.21.15.1:0/2179576946 <== mon.2 v2:172.21.15.7:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a58032810 con 0x7f6a74151660 2024-05-09T08:05:01.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.096+0000 7f6a7bc00640 1 -- 172.21.15.1:0/2179576946 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f6a74151660 msgr2=0x7f6a740c6c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:05:01.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.096+0000 7f6a7bc00640 1 --2- 172.21.15.1:0/2179576946 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f6a74151660 0x7f6a740c6c90 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f6a580097a0 tx=0x7f6a5802a180 comp rx=0 tx=0).stop 2024-05-09T08:05:01.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.096+0000 7f6a7bc00640 1 -- 172.21.15.1:0/2179576946 shutdown_connections 2024-05-09T08:05:01.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.096+0000 7f6a7bc00640 1 --2- 172.21.15.1:0/2179576946 >> v2:172.21.15.7:3300/0 conn(0x7f6a740c1ba0 0x7f6a74072d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:05:01.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.096+0000 7f6a7bc00640 1 --2- 172.21.15.1:0/2179576946 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f6a74151660 0x7f6a740c6c90 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:05:01.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.096+0000 7f6a7bc00640 1 -- 172.21.15.1:0/2179576946 >> 172.21.15.1:0/2179576946 conn(0x7f6a740cc070 msgr2=0x7f6a740cc480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:05:01.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.096+0000 7f6a7bc00640 1 -- 172.21.15.1:0/2179576946 shutdown_connections 2024-05-09T08:05:01.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.096+0000 7f6a7bc00640 1 -- 172.21.15.1:0/2179576946 wait complete. 2024-05-09T08:05:01.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.098+0000 7f6a7bc00640 1 Processor -- start 2024-05-09T08:05:01.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.098+0000 7f6a7bc00640 1 -- start start 2024-05-09T08:05:01.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.099+0000 7f6a7bc00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f6a740b1d30 0x7f6a740cad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:05:01.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.099+0000 7f6a7bc00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6a740c1ba0 0x7f6a741448a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:05:01.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.099+0000 7f6a7bc00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6a74151660 0x7f6a741edfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:05:01.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.099+0000 7f6a7bc00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f6a741581a0 con 0x7f6a740c1ba0 2024-05-09T08:05:01.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.099+0000 7f6a7bc00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f6a74157ec0 con 0x7f6a74151660 2024-05-09T08:05:01.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.099+0000 7f6a7bc00640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f6a741572d0 con 0x7f6a740b1d30 2024-05-09T08:05:01.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.099+0000 7f6a6be00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6a740c1ba0 0x7f6a741448a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:05:01.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.099+0000 7f6a79200640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f6a740b1d30 0x7f6a740cad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:05:01.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.099+0000 7f6a79c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6a74151660 0x7f6a741edfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:05:01.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.099+0000 7f6a6be00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6a740c1ba0 0x7f6a741448a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:36698/0 (socket says 172.21.15.1:36698) 2024-05-09T08:05:01.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.099+0000 7f6a6be00640 1 -- 172.21.15.1:0/2190403028 learned_addr learned my addr 172.21.15.1:0/2190403028 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:05:01.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.100+0000 7f6a6be00640 1 -- 172.21.15.1:0/2190403028 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f6a740b1d30 msgr2=0x7f6a740cad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:05:01.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.100+0000 7f6a6be00640 1 --2- 172.21.15.1:0/2190403028 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f6a740b1d30 0x7f6a740cad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:05:01.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.100+0000 7f6a6be00640 1 -- 172.21.15.1:0/2190403028 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6a74151660 msgr2=0x7f6a741edfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:05:01.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.100+0000 7f6a6be00640 1 --2- 172.21.15.1:0/2190403028 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6a74151660 0x7f6a741edfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:05:01.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.100+0000 7f6a6be00640 1 -- 172.21.15.1:0/2190403028 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a741f2440 con 0x7f6a740c1ba0 2024-05-09T08:05:01.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.100+0000 7f6a6be00640 1 --2- 172.21.15.1:0/2190403028 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6a740c1ba0 0x7f6a741448a0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f6a6000e9b0 tx=0x7f6a6000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:05:01.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.100+0000 7f6a71600640 1 -- 172.21.15.1:0/2190403028 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a60017070 con 0x7f6a740c1ba0 2024-05-09T08:05:01.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.100+0000 7f6a71600640 1 -- 172.21.15.1:0/2190403028 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a6000ca00 con 0x7f6a740c1ba0 2024-05-09T08:05:01.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.100+0000 7f6a7bc00640 1 -- 172.21.15.1:0/2190403028 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a740cb720 con 0x7f6a740c1ba0 2024-05-09T08:05:01.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.100+0000 7f6a71600640 1 -- 172.21.15.1:0/2190403028 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a6000cdf0 con 0x7f6a740c1ba0 2024-05-09T08:05:01.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.100+0000 7f6a7bc00640 1 -- 172.21.15.1:0/2190403028 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a741f1680 con 0x7f6a740c1ba0 2024-05-09T08:05:01.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.102+0000 7f6a7bc00640 1 -- 172.21.15.1:0/2190403028 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a74074d70 con 0x7f6a740c1ba0 2024-05-09T08:05:01.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.103+0000 7f6a71600640 1 -- 172.21.15.1:0/2190403028 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6a6001a070 con 0x7f6a740c1ba0 2024-05-09T08:05:01.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.104+0000 7f6a71600640 1 --2- 172.21.15.1:0/2190403028 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f6a4806be30 0x7f6a4806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:05:01.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.104+0000 7f6a71600640 1 -- 172.21.15.1:0/2190403028 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7f6a6008adf0 con 0x7f6a740c1ba0 2024-05-09T08:05:01.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.104+0000 7f6a79200640 1 --2- 172.21.15.1:0/2190403028 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f6a4806be30 0x7f6a4806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:05:01.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.105+0000 7f6a79200640 1 --2- 172.21.15.1:0/2190403028 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f6a4806be30 0x7f6a4806e2f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f6a58034000 tx=0x7f6a5802e000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:05:01.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.110+0000 7f6a71600640 1 -- 172.21.15.1:0/2190403028 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f6a60058cc0 con 0x7f6a740c1ba0 2024-05-09T08:05:01.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.397+0000 7f6a7bc00640 1 -- 172.21.15.1:0/2190403028 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f6a740b1900 con 0x7f6a740c1ba0 2024-05-09T08:05:01.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.398+0000 7f6a71600640 1 -- 172.21.15.1:0/2190403028 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1545 (secure 0 0 0) 0x7f6a6005cfc0 con 0x7f6a740c1ba0 2024-05-09T08:05:01.412 INFO:teuthology.orchestra.run.smithi001.stdout:{"election_epoch":20,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":14,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"54329ec8-3270-4043-9a82-c6cd7420aa7f","modified":"2024-05-09T08:01:21.819270Z","created":"2024-05-09T08:01:21.819270Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:3300","nonce":0},{"type":"v1","addr":"172.21.15.1:6789","nonce":0}]},"addr":"172.21.15.1:6789/0","public_addr":"172.21.15.1:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:3300","nonce":0},{"type":"v1","addr":"172.21.15.7:6789","nonce":0}]},"addr":"172.21.15.7:6789/0","public_addr":"172.21.15.7:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:3301","nonce":0},{"type":"v1","addr":"172.21.15.7:6790","nonce":0}]},"addr":"172.21.15.7:6790/0","public_addr":"172.21.15.7:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-05-09T08:05:01.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.401+0000 7f6a7bc00640 1 -- 172.21.15.1:0/2190403028 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f6a4806be30 msgr2=0x7f6a4806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:05:01.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.401+0000 7f6a7bc00640 1 --2- 172.21.15.1:0/2190403028 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f6a4806be30 0x7f6a4806e2f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f6a58034000 tx=0x7f6a5802e000 comp rx=0 tx=0).stop 2024-05-09T08:05:01.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.401+0000 7f6a7bc00640 1 -- 172.21.15.1:0/2190403028 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6a740c1ba0 msgr2=0x7f6a741448a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:05:01.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.401+0000 7f6a7bc00640 1 --2- 172.21.15.1:0/2190403028 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6a740c1ba0 0x7f6a741448a0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f6a6000e9b0 tx=0x7f6a6000ee70 comp rx=0 tx=0).stop 2024-05-09T08:05:01.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.401+0000 7f6a7bc00640 1 -- 172.21.15.1:0/2190403028 shutdown_connections 2024-05-09T08:05:01.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.401+0000 7f6a7bc00640 1 --2- 172.21.15.1:0/2190403028 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f6a4806be30 0x7f6a4806e2f0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:05:01.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.401+0000 7f6a7bc00640 1 --2- 172.21.15.1:0/2190403028 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6a74151660 0x7f6a741edfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:05:01.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.401+0000 7f6a7bc00640 1 --2- 172.21.15.1:0/2190403028 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6a740c1ba0 0x7f6a741448a0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:05:01.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.401+0000 7f6a7bc00640 1 --2- 172.21.15.1:0/2190403028 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f6a740b1d30 0x7f6a740cad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:05:01.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.401+0000 7f6a7bc00640 1 -- 172.21.15.1:0/2190403028 >> 172.21.15.1:0/2190403028 conn(0x7f6a740cc070 msgr2=0x7f6a74155890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:05:01.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.401+0000 7f6a7bc00640 1 -- 172.21.15.1:0/2190403028 shutdown_connections 2024-05-09T08:05:01.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.401+0000 7f6a7bc00640 1 -- 172.21.15.1:0/2190403028 wait complete. 2024-05-09T08:05:01.440 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-05-09T08:05:01.440 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-05-09T08:05:01.440 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-05-09T08:05:01.440 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-05-09T08:05:01.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.609+0000 7f3353a00640 1 Processor -- start 2024-05-09T08:05:01.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.609+0000 7f3353a00640 1 -- start start 2024-05-09T08:05:01.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.610+0000 7f3353a00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f334c06c900 0x7f334c0c7280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:05:01.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.610+0000 7f3353a00640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f334c15b860 con 0x7f334c0c77c0 2024-05-09T08:05:01.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.610+0000 7f3353a00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f334c0b5de0 con 0x7f334c06cce0 2024-05-09T08:05:01.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.610+0000 7f3353a00640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f334c0c7f30 con 0x7f334c06c900 2024-05-09T08:05:01.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.610+0000 7f3343e00640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f334c0c77c0 0x7f334c0c81f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:58514/0 (socket says 172.21.15.1:58514) 2024-05-09T08:05:01.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.610+0000 7f3343e00640 1 -- 172.21.15.1:0/975197445 learned_addr learned my addr 172.21.15.1:0/975197445 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T08:05:01.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.611+0000 7f3351000640 1 --2- 172.21.15.1:0/975197445 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f334c06c900 0x7f334c0c7280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:05:01.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.611+0000 7f334be00640 1 -- 172.21.15.1:0/975197445 <== mon.1 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1650766818 0 0) 0x7f334c0b5de0 con 0x7f334c06cce0 2024-05-09T08:05:01.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.611+0000 7f334be00640 1 -- 172.21.15.1:0/975197445 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3330003520 con 0x7f334c06cce0 2024-05-09T08:05:01.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.611+0000 7f334be00640 1 -- 172.21.15.1:0/975197445 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1248829398 0 0) 0x7f334c15b860 con 0x7f334c0c77c0 2024-05-09T08:05:01.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.611+0000 7f334be00640 1 -- 172.21.15.1:0/975197445 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f334c0b5de0 con 0x7f334c0c77c0 2024-05-09T08:05:01.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.611+0000 7f3351000640 1 -- 172.21.15.1:0/975197445 >> v1:172.21.15.7:6789/0 conn(0x7f334c06cce0 legacy=0x7f334c1547c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:05:01.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.611+0000 7f3351000640 1 -- 172.21.15.1:0/975197445 >> v1:172.21.15.1:6789/0 conn(0x7f334c0c77c0 legacy=0x7f334c0c81f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:05:01.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.611+0000 7f3351000640 1 -- 172.21.15.1:0/975197445 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f334c0c7c50 con 0x7f334c06c900 2024-05-09T08:05:01.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.612+0000 7f3351000640 1 --2- 172.21.15.1:0/975197445 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f334c06c900 0x7f334c0c7280 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f333c009dc0 tx=0x7f333c02a6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d5727d91495d2bfe server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:05:01.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.613+0000 7f334be00640 1 -- 172.21.15.1:0/975197445 <== mon.2 v2:172.21.15.7:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f333c007740 con 0x7f334c06c900 2024-05-09T08:05:01.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.613+0000 7f334be00640 1 -- 172.21.15.1:0/975197445 <== mon.2 v2:172.21.15.7:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f333c0078e0 con 0x7f334c06c900 2024-05-09T08:05:01.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.613+0000 7f334be00640 1 -- 172.21.15.1:0/975197445 <== mon.2 v2:172.21.15.7:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f333c007cb0 con 0x7f334c06c900 2024-05-09T08:05:01.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.613+0000 7f3353a00640 1 -- 172.21.15.1:0/975197445 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f334c06c900 msgr2=0x7f334c0c7280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:05:01.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.613+0000 7f3353a00640 1 --2- 172.21.15.1:0/975197445 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f334c06c900 0x7f334c0c7280 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f333c009dc0 tx=0x7f333c02a6d0 comp rx=0 tx=0).stop 2024-05-09T08:05:01.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.613+0000 7f3353a00640 1 -- 172.21.15.1:0/975197445 shutdown_connections 2024-05-09T08:05:01.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.613+0000 7f3353a00640 1 --2- 172.21.15.1:0/975197445 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f334c06c900 0x7f334c0c7280 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:05:01.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.613+0000 7f3353a00640 1 -- 172.21.15.1:0/975197445 >> 172.21.15.1:0/975197445 conn(0x7f334c0cfe80 msgr2=0x7f334c0d0290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:05:01.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.613+0000 7f3353a00640 1 -- 172.21.15.1:0/975197445 shutdown_connections 2024-05-09T08:05:01.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.613+0000 7f3353a00640 1 -- 172.21.15.1:0/975197445 wait complete. 2024-05-09T08:05:01.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.615+0000 7f3353a00640 1 Processor -- start 2024-05-09T08:05:01.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.615+0000 7f3353a00640 1 -- start start 2024-05-09T08:05:01.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.616+0000 7f3353a00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f334c06c900 0x7f334c157970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:05:01.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.616+0000 7f3353a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f334c06cce0 0x7f334c1533f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:05:01.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.616+0000 7f3353a00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f334c0c4ee0 0x7f334c1f3440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:05:01.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.616+0000 7f3353a00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f334c15b060 con 0x7f334c06c900 2024-05-09T08:05:01.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.616+0000 7f3353a00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f334c15ad80 con 0x7f334c06cce0 2024-05-09T08:05:01.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.616+0000 7f3353a00640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f334c0c8900 con 0x7f334c0c4ee0 2024-05-09T08:05:01.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.617+0000 7f3351000640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f334c06c900 0x7f334c157970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:05:01.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.617+0000 7f3351000640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f334c06c900 0x7f334c157970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:36708/0 (socket says 172.21.15.1:36708) 2024-05-09T08:05:01.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.617+0000 7f3351000640 1 -- 172.21.15.1:0/165238076 learned_addr learned my addr 172.21.15.1:0/165238076 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:05:01.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.617+0000 7f3343e00640 1 --2- 172.21.15.1:0/165238076 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f334c06cce0 0x7f334c1533f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:05:01.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.617+0000 7f3351a00640 1 --2- 172.21.15.1:0/165238076 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f334c0c4ee0 0x7f334c1f3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:05:01.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.617+0000 7f3351000640 1 -- 172.21.15.1:0/165238076 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f334c0c4ee0 msgr2=0x7f334c1f3440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:05:01.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.617+0000 7f3351000640 1 --2- 172.21.15.1:0/165238076 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f334c0c4ee0 0x7f334c1f3440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:05:01.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.617+0000 7f3351000640 1 -- 172.21.15.1:0/165238076 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f334c06cce0 msgr2=0x7f334c1533f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:05:01.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.617+0000 7f3351000640 1 --2- 172.21.15.1:0/165238076 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f334c06cce0 0x7f334c1533f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:05:01.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.617+0000 7f3351000640 1 -- 172.21.15.1:0/165238076 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f334c153cf0 con 0x7f334c06c900 2024-05-09T08:05:01.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.618+0000 7f3351000640 1 --2- 172.21.15.1:0/165238076 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f334c06c900 0x7f334c157970 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f333c000f80 tx=0x7f333c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:05:01.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.618+0000 7f3349600640 1 -- 172.21.15.1:0/165238076 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f333c040070 con 0x7f334c06c900 2024-05-09T08:05:01.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.618+0000 7f3349600640 1 -- 172.21.15.1:0/165238076 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f333c031c40 con 0x7f334c06c900 2024-05-09T08:05:01.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.618+0000 7f3349600640 1 -- 172.21.15.1:0/165238076 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f333c009450 con 0x7f334c06c900 2024-05-09T08:05:01.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.618+0000 7f3353a00640 1 -- 172.21.15.1:0/165238076 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f334c155380 con 0x7f334c06c900 2024-05-09T08:05:01.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.618+0000 7f3353a00640 1 -- 172.21.15.1:0/165238076 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f334c155940 con 0x7f334c06c900 2024-05-09T08:05:01.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.619+0000 7f3353a00640 1 --2- 172.21.15.1:0/165238076 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f334c0c77c0 0x7f334c14c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:05:01.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.619+0000 7f3353a00640 1 -- 172.21.15.1:0/165238076 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f334c15a570 con 0x7f334c0c77c0 2024-05-09T08:05:01.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.619+0000 7f3343e00640 1 --2- 172.21.15.1:0/165238076 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f334c0c77c0 0x7f334c14c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:05:01.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.620+0000 7f3343e00640 1 -- 172.21.15.1:0/165238076 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f334c074d70 con 0x7f334c0c77c0 2024-05-09T08:05:01.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.620+0000 7f3343e00640 1 --2- 172.21.15.1:0/165238076 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f334c0c77c0 0x7f334c14c6a0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f3334005880 tx=0x7f333400f9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:05:01.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.620+0000 7f3349600640 1 -- 172.21.15.1:0/165238076 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3334005ac0 con 0x7f334c0c77c0 2024-05-09T08:05:01.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.620+0000 7f3349600640 1 -- 172.21.15.1:0/165238076 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f333c039040 con 0x7f334c06c900 2024-05-09T08:05:01.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.621+0000 7f3349600640 1 --2- 172.21.15.1:0/165238076 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f332806bd60 0x7f332806e220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:05:01.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.621+0000 7f3349600640 1 -- 172.21.15.1:0/165238076 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7f333c0ac270 con 0x7f334c06c900 2024-05-09T08:05:01.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.622+0000 7f3351a00640 1 --2- 172.21.15.1:0/165238076 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f332806bd60 0x7f332806e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:05:01.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.622+0000 7f3349600640 1 -- 172.21.15.1:0/165238076 <== mon.0 v2:172.21.15.1:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f3334018070 con 0x7f334c0c77c0 2024-05-09T08:05:01.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.622+0000 7f3349600640 1 -- 172.21.15.1:0/165238076 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f334c0c77c0 msgr2=0x7f334c14c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:05:01.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.622+0000 7f3349600640 1 --2- 172.21.15.1:0/165238076 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f334c0c77c0 0x7f334c14c6a0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f3334005880 tx=0x7f333400f9d0 comp rx=0 tx=0).stop 2024-05-09T08:05:01.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.622+0000 7f3349600640 1 -- 172.21.15.1:0/165238076 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f334c0c77c0 msgr2=0x7f334c14c6a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-05-09T08:05:01.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.622+0000 7f3349600640 1 --2- 172.21.15.1:0/165238076 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f334c0c77c0 0x7f334c14c6a0 secure :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f3334005880 tx=0x7f333400f9d0 comp rx=0 tx=0).stop 2024-05-09T08:05:01.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.622+0000 7f3351a00640 1 --2- 172.21.15.1:0/165238076 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f332806bd60 0x7f332806e220 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f334c0c5350 tx=0x7f3344009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:05:01.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.629+0000 7f3353a00640 1 --2- 172.21.15.1:0/165238076 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f334c155b80 0x7f334c14cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:05:01.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.629+0000 7f3353a00640 1 -- 172.21.15.1:0/165238076 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f334c152a60 con 0x7f334c155b80 2024-05-09T08:05:01.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.629+0000 7f3343e00640 1 --2- 172.21.15.1:0/165238076 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f334c155b80 0x7f334c14cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:05:01.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.630+0000 7f3343e00640 1 -- 172.21.15.1:0/165238076 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f334c06b570 con 0x7f334c155b80 2024-05-09T08:05:01.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.630+0000 7f3343e00640 1 --2- 172.21.15.1:0/165238076 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f334c155b80 0x7f334c14cbe0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f3334005880 tx=0x7f33340010a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:05:01.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.630+0000 7f3349600640 1 -- 172.21.15.1:0/165238076 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f333401db10 con 0x7f334c155b80 2024-05-09T08:05:01.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.631+0000 7f3349600640 1 -- 172.21.15.1:0/165238076 <== mon.0 v2:172.21.15.1:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4537 (secure 0 0 0) 0x7f33340133f0 con 0x7f334c155b80 2024-05-09T08:05:01.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.631+0000 7f3349600640 1 -- 172.21.15.1:0/165238076 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f334c155b80 msgr2=0x7f334c14cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:05:01.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.631+0000 7f3349600640 1 --2- 172.21.15.1:0/165238076 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f334c155b80 0x7f334c14cbe0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f3334005880 tx=0x7f33340010a0 comp rx=0 tx=0).stop 2024-05-09T08:05:01.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.631+0000 7f3349600640 1 -- 172.21.15.1:0/165238076 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f334c155b80 msgr2=0x7f334c14cbe0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-05-09T08:05:01.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.631+0000 7f3349600640 1 --2- 172.21.15.1:0/165238076 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f334c155b80 0x7f334c14cbe0 secure :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f3334005880 tx=0x7f33340010a0 comp rx=0 tx=0).stop 2024-05-09T08:05:01.646 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-05-09T08:05:01.647 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "a", 2024-05-09T08:05:01.647 INFO:teuthology.orchestra.run.smithi001.stdout: "rank": 0, 2024-05-09T08:05:01.647 INFO:teuthology.orchestra.run.smithi001.stdout: "state": "leader", 2024-05-09T08:05:01.647 INFO:teuthology.orchestra.run.smithi001.stdout: "election_epoch": 20, 2024-05-09T08:05:01.647 INFO:teuthology.orchestra.run.smithi001.stdout: "uptime": 20663, 2024-05-09T08:05:01.647 INFO:teuthology.orchestra.run.smithi001.stdout: "quorum": [ 2024-05-09T08:05:01.647 INFO:teuthology.orchestra.run.smithi001.stdout: 0, 2024-05-09T08:05:01.648 INFO:teuthology.orchestra.run.smithi001.stdout: 1, 2024-05-09T08:05:01.648 INFO:teuthology.orchestra.run.smithi001.stdout: 2 2024-05-09T08:05:01.648 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:05:01.648 INFO:teuthology.orchestra.run.smithi001.stdout: "quorum_age": 14, 2024-05-09T08:05:01.648 INFO:teuthology.orchestra.run.smithi001.stdout: "features": { 2024-05-09T08:05:01.648 INFO:teuthology.orchestra.run.smithi001.stdout: "required_con": "2450521708007866372", 2024-05-09T08:05:01.648 INFO:teuthology.orchestra.run.smithi001.stdout: "required_mon": [ 2024-05-09T08:05:01.648 INFO:teuthology.orchestra.run.smithi001.stdout: "kraken", 2024-05-09T08:05:01.648 INFO:teuthology.orchestra.run.smithi001.stdout: "luminous", 2024-05-09T08:05:01.648 INFO:teuthology.orchestra.run.smithi001.stdout: "mimic", 2024-05-09T08:05:01.648 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap-prune", 2024-05-09T08:05:01.649 INFO:teuthology.orchestra.run.smithi001.stdout: "nautilus", 2024-05-09T08:05:01.649 INFO:teuthology.orchestra.run.smithi001.stdout: "octopus", 2024-05-09T08:05:01.649 INFO:teuthology.orchestra.run.smithi001.stdout: "pacific", 2024-05-09T08:05:01.649 INFO:teuthology.orchestra.run.smithi001.stdout: "elector-pinging", 2024-05-09T08:05:01.649 INFO:teuthology.orchestra.run.smithi001.stdout: "quincy", 2024-05-09T08:05:01.649 INFO:teuthology.orchestra.run.smithi001.stdout: "reef", 2024-05-09T08:05:01.649 INFO:teuthology.orchestra.run.smithi001.stdout: "squid" 2024-05-09T08:05:01.649 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:05:01.650 INFO:teuthology.orchestra.run.smithi001.stdout: "quorum_con": "4540701547738038271", 2024-05-09T08:05:01.650 INFO:teuthology.orchestra.run.smithi001.stdout: "quorum_mon": [ 2024-05-09T08:05:01.650 INFO:teuthology.orchestra.run.smithi001.stdout: "kraken", 2024-05-09T08:05:01.650 INFO:teuthology.orchestra.run.smithi001.stdout: "luminous", 2024-05-09T08:05:01.650 INFO:teuthology.orchestra.run.smithi001.stdout: "mimic", 2024-05-09T08:05:01.650 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap-prune", 2024-05-09T08:05:01.650 INFO:teuthology.orchestra.run.smithi001.stdout: "nautilus", 2024-05-09T08:05:01.650 INFO:teuthology.orchestra.run.smithi001.stdout: "octopus", 2024-05-09T08:05:01.650 INFO:teuthology.orchestra.run.smithi001.stdout: "pacific", 2024-05-09T08:05:01.650 INFO:teuthology.orchestra.run.smithi001.stdout: "elector-pinging", 2024-05-09T08:05:01.650 INFO:teuthology.orchestra.run.smithi001.stdout: "quincy", 2024-05-09T08:05:01.650 INFO:teuthology.orchestra.run.smithi001.stdout: "reef", 2024-05-09T08:05:01.651 INFO:teuthology.orchestra.run.smithi001.stdout: "squid" 2024-05-09T08:05:01.651 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:05:01.651 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:05:01.651 INFO:teuthology.orchestra.run.smithi001.stdout: "outside_quorum": [], 2024-05-09T08:05:01.651 INFO:teuthology.orchestra.run.smithi001.stdout: "extra_probe_peers": [], 2024-05-09T08:05:01.651 INFO:teuthology.orchestra.run.smithi001.stdout: "sync_provider": [], 2024-05-09T08:05:01.651 INFO:teuthology.orchestra.run.smithi001.stdout: "monmap": { 2024-05-09T08:05:01.651 INFO:teuthology.orchestra.run.smithi001.stdout: "epoch": 1, 2024-05-09T08:05:01.651 INFO:teuthology.orchestra.run.smithi001.stdout: "fsid": "54329ec8-3270-4043-9a82-c6cd7420aa7f", 2024-05-09T08:05:01.651 INFO:teuthology.orchestra.run.smithi001.stdout: "modified": "2024-05-09T08:01:21.819270Z", 2024-05-09T08:05:01.651 INFO:teuthology.orchestra.run.smithi001.stdout: "created": "2024-05-09T08:01:21.819270Z", 2024-05-09T08:05:01.651 INFO:teuthology.orchestra.run.smithi001.stdout: "min_mon_release": 19, 2024-05-09T08:05:01.651 INFO:teuthology.orchestra.run.smithi001.stdout: "min_mon_release_name": "squid", 2024-05-09T08:05:01.652 INFO:teuthology.orchestra.run.smithi001.stdout: "election_strategy": 1, 2024-05-09T08:05:01.652 INFO:teuthology.orchestra.run.smithi001.stdout: "disallowed_leaders: ": "", 2024-05-09T08:05:01.652 INFO:teuthology.orchestra.run.smithi001.stdout: "stretch_mode": false, 2024-05-09T08:05:01.652 INFO:teuthology.orchestra.run.smithi001.stdout: "tiebreaker_mon": "", 2024-05-09T08:05:01.652 INFO:teuthology.orchestra.run.smithi001.stdout: "removed_ranks: ": "", 2024-05-09T08:05:01.652 INFO:teuthology.orchestra.run.smithi001.stdout: "features": { 2024-05-09T08:05:01.652 INFO:teuthology.orchestra.run.smithi001.stdout: "persistent": [ 2024-05-09T08:05:01.652 INFO:teuthology.orchestra.run.smithi001.stdout: "kraken", 2024-05-09T08:05:01.652 INFO:teuthology.orchestra.run.smithi001.stdout: "luminous", 2024-05-09T08:05:01.652 INFO:teuthology.orchestra.run.smithi001.stdout: "mimic", 2024-05-09T08:05:01.652 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap-prune", 2024-05-09T08:05:01.652 INFO:teuthology.orchestra.run.smithi001.stdout: "nautilus", 2024-05-09T08:05:01.653 INFO:teuthology.orchestra.run.smithi001.stdout: "octopus", 2024-05-09T08:05:01.653 INFO:teuthology.orchestra.run.smithi001.stdout: "pacific", 2024-05-09T08:05:01.653 INFO:teuthology.orchestra.run.smithi001.stdout: "elector-pinging", 2024-05-09T08:05:01.653 INFO:teuthology.orchestra.run.smithi001.stdout: "quincy", 2024-05-09T08:05:01.653 INFO:teuthology.orchestra.run.smithi001.stdout: "reef", 2024-05-09T08:05:01.653 INFO:teuthology.orchestra.run.smithi001.stdout: "squid" 2024-05-09T08:05:01.653 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:05:01.653 INFO:teuthology.orchestra.run.smithi001.stdout: "optional": [] 2024-05-09T08:05:01.653 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:05:01.653 INFO:teuthology.orchestra.run.smithi001.stdout: "mons": [ 2024-05-09T08:05:01.653 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:05:01.653 INFO:teuthology.orchestra.run.smithi001.stdout: "rank": 0, 2024-05-09T08:05:01.653 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "a", 2024-05-09T08:05:01.654 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-05-09T08:05:01.654 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-05-09T08:05:01.654 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:05:01.654 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-05-09T08:05:01.654 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:3300", 2024-05-09T08:05:01.654 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:05:01.654 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:05:01.654 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:05:01.654 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-05-09T08:05:01.654 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6789", 2024-05-09T08:05:01.654 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:05:01.654 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:05:01.655 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:05:01.655 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:05:01.655 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6789/0", 2024-05-09T08:05:01.655 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.1:6789/0", 2024-05-09T08:05:01.655 INFO:teuthology.orchestra.run.smithi001.stdout: "priority": 0, 2024-05-09T08:05:01.655 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 0, 2024-05-09T08:05:01.655 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_location": "{}" 2024-05-09T08:05:01.655 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:05:01.655 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:05:01.655 INFO:teuthology.orchestra.run.smithi001.stdout: "rank": 1, 2024-05-09T08:05:01.656 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "b", 2024-05-09T08:05:01.656 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-05-09T08:05:01.656 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-05-09T08:05:01.656 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:05:01.656 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-05-09T08:05:01.656 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:3300", 2024-05-09T08:05:01.656 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:05:01.656 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:05:01.656 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:05:01.656 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-05-09T08:05:01.656 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:6789", 2024-05-09T08:05:01.656 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:05:01.656 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:05:01.657 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:05:01.657 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:05:01.657 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:6789/0", 2024-05-09T08:05:01.657 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.7:6789/0", 2024-05-09T08:05:01.657 INFO:teuthology.orchestra.run.smithi001.stdout: "priority": 0, 2024-05-09T08:05:01.657 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 0, 2024-05-09T08:05:01.657 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_location": "{}" 2024-05-09T08:05:01.657 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:05:01.657 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:05:01.657 INFO:teuthology.orchestra.run.smithi001.stdout: "rank": 2, 2024-05-09T08:05:01.657 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "c", 2024-05-09T08:05:01.657 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-05-09T08:05:01.658 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-05-09T08:05:01.658 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:05:01.658 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-05-09T08:05:01.658 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:3301", 2024-05-09T08:05:01.658 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:05:01.658 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:05:01.658 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:05:01.658 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-05-09T08:05:01.658 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:6790", 2024-05-09T08:05:01.658 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:05:01.659 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:05:01.659 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:05:01.659 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:05:01.659 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:6790/0", 2024-05-09T08:05:01.659 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.7:6790/0", 2024-05-09T08:05:01.659 INFO:teuthology.orchestra.run.smithi001.stdout: "priority": 0, 2024-05-09T08:05:01.659 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 0, 2024-05-09T08:05:01.659 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_location": "{}" 2024-05-09T08:05:01.659 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:05:01.659 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:05:01.659 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:05:01.659 INFO:teuthology.orchestra.run.smithi001.stdout: "feature_map": { 2024-05-09T08:05:01.660 INFO:teuthology.orchestra.run.smithi001.stdout: "mon": [ 2024-05-09T08:05:01.660 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:05:01.660 INFO:teuthology.orchestra.run.smithi001.stdout: "features": "0x3f03cffffffdffff", 2024-05-09T08:05:01.660 INFO:teuthology.orchestra.run.smithi001.stdout: "release": "luminous", 2024-05-09T08:05:01.660 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 1 2024-05-09T08:05:01.660 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:05:01.660 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:05:01.660 INFO:teuthology.orchestra.run.smithi001.stdout: "client": [ 2024-05-09T08:05:01.660 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:05:01.660 INFO:teuthology.orchestra.run.smithi001.stdout: "features": "0x3f03cffffffdffff", 2024-05-09T08:05:01.660 INFO:teuthology.orchestra.run.smithi001.stdout: "release": "luminous", 2024-05-09T08:05:01.660 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 2 2024-05-09T08:05:01.660 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:05:01.661 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:05:01.661 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:05:01.661 INFO:teuthology.orchestra.run.smithi001.stdout: "stretch_mode": false 2024-05-09T08:05:01.661 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-05-09T08:05:01.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.631+0000 7f3353a00640 1 -- 172.21.15.1:0/165238076 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f332806bd60 msgr2=0x7f332806e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:05:01.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.631+0000 7f3353a00640 1 --2- 172.21.15.1:0/165238076 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f332806bd60 0x7f332806e220 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f334c0c5350 tx=0x7f3344009040 comp rx=0 tx=0).stop 2024-05-09T08:05:01.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.631+0000 7f3353a00640 1 -- 172.21.15.1:0/165238076 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f334c06c900 msgr2=0x7f334c157970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:05:01.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.631+0000 7f3353a00640 1 --2- 172.21.15.1:0/165238076 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f334c06c900 0x7f334c157970 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f333c000f80 tx=0x7f333c004a40 comp rx=0 tx=0).stop 2024-05-09T08:05:01.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.631+0000 7f3351a00640 1 -- 172.21.15.1:0/165238076 reap_dead start 2024-05-09T08:05:01.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.631+0000 7f3351a00640 1 -- 172.21.15.1:0/165238076 reap_dead start 2024-05-09T08:05:01.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.631+0000 7f3353a00640 1 -- 172.21.15.1:0/165238076 shutdown_connections 2024-05-09T08:05:01.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.631+0000 7f3353a00640 1 -- 172.21.15.1:0/165238076 >> 172.21.15.1:0/165238076 conn(0x7f334c0cfe80 msgr2=0x7f334c0d0260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:05:01.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.631+0000 7f3353a00640 1 -- 172.21.15.1:0/165238076 shutdown_connections 2024-05-09T08:05:01.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.632+0000 7f3353a00640 1 -- 172.21.15.1:0/165238076 wait complete. 2024-05-09T08:05:01.664 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-05-09T08:05:01.664 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-05-09T08:05:01.665 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-05-09T08:05:01.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.855+0000 7f944be00640 1 Processor -- start 2024-05-09T08:05:01.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.855+0000 7f944be00640 1 -- start start 2024-05-09T08:05:01.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.857+0000 7f944be00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f944c0cac60 0x7f944c0cda00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:05:01.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.857+0000 7f944be00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f944c0cc5f0 0x7f944c0cdf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:05:01.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.857+0000 7f944be00640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f944c0ce4e0 0x7f944c15a570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:05:01.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.857+0000 7f944be00640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f944c0ccb20 con 0x7f944c0ce4e0 2024-05-09T08:05:01.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.857+0000 7f944be00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f944c0cce00 con 0x7f944c0cac60 2024-05-09T08:05:01.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.857+0000 7f944be00640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f944c0cf120 con 0x7f944c0cc5f0 2024-05-09T08:05:01.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.857+0000 7f944b400640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f944c0ce4e0 0x7f944c15a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:05:01.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.857+0000 7f944aa00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f944c0cac60 0x7f944c0cda00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:05:01.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.857+0000 7f9442000640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f944c0cc5f0 0x7f944c0cdf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:05:01.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.857+0000 7f944b400640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f944c0ce4e0 0x7f944c15a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:36740/0 (socket says 172.21.15.1:36740) 2024-05-09T08:05:01.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.857+0000 7f944b400640 1 -- 172.21.15.1:0/890927237 learned_addr learned my addr 172.21.15.1:0/890927237 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:05:01.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.858+0000 7f9442000640 1 -- 172.21.15.1:0/890927237 >> v2:172.21.15.7:3300/0 conn(0x7f944c0cac60 msgr2=0x7f944c0cda00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:05:01.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.858+0000 7f9442000640 1 --2- 172.21.15.1:0/890927237 >> v2:172.21.15.7:3300/0 conn(0x7f944c0cac60 0x7f944c0cda00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:05:01.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.858+0000 7f9442000640 1 -- 172.21.15.1:0/890927237 >> v2:172.21.15.1:3300/0 conn(0x7f944c0ce4e0 msgr2=0x7f944c15a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:05:01.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.858+0000 7f9442000640 1 --2- 172.21.15.1:0/890927237 >> v2:172.21.15.1:3300/0 conn(0x7f944c0ce4e0 0x7f944c15a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:05:01.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.858+0000 7f9442000640 1 -- 172.21.15.1:0/890927237 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f944c0cf400 con 0x7f944c0cc5f0 2024-05-09T08:05:01.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.859+0000 7f9442000640 1 --2- 172.21.15.1:0/890927237 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f944c0cc5f0 0x7f944c0cdf40 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f944c0735d0 tx=0x7f943402a1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=89b021b2ee2aaf85 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:05:01.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.859+0000 7f944a000640 1 -- 172.21.15.1:0/890927237 <== mon.2 v2:172.21.15.7:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9434037070 con 0x7f944c0cc5f0 2024-05-09T08:05:01.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.859+0000 7f944a000640 1 -- 172.21.15.1:0/890927237 <== mon.2 v2:172.21.15.7:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9434032440 con 0x7f944c0cc5f0 2024-05-09T08:05:01.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.859+0000 7f944a000640 1 -- 172.21.15.1:0/890927237 <== mon.2 v2:172.21.15.7:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9434032810 con 0x7f944c0cc5f0 2024-05-09T08:05:01.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.860+0000 7f944be00640 1 -- 172.21.15.1:0/890927237 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f944c0cc5f0 msgr2=0x7f944c0cdf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:05:01.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.860+0000 7f944be00640 1 --2- 172.21.15.1:0/890927237 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f944c0cc5f0 0x7f944c0cdf40 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f944c0735d0 tx=0x7f943402a1a0 comp rx=0 tx=0).stop 2024-05-09T08:05:01.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.860+0000 7f944be00640 1 -- 172.21.15.1:0/890927237 shutdown_connections 2024-05-09T08:05:01.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.860+0000 7f944be00640 1 --2- 172.21.15.1:0/890927237 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f944c0cc5f0 0x7f944c0cdf40 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:05:01.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.860+0000 7f944be00640 1 --2- 172.21.15.1:0/890927237 >> v2:172.21.15.1:3300/0 conn(0x7f944c0ce4e0 0x7f944c15a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:05:01.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.860+0000 7f944be00640 1 --2- 172.21.15.1:0/890927237 >> v2:172.21.15.7:3300/0 conn(0x7f944c0cac60 0x7f944c0cda00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:05:01.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.860+0000 7f944be00640 1 -- 172.21.15.1:0/890927237 >> 172.21.15.1:0/890927237 conn(0x7f944c0cfc20 msgr2=0x7f944c0d0030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:05:01.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.860+0000 7f944be00640 1 -- 172.21.15.1:0/890927237 shutdown_connections 2024-05-09T08:05:01.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.860+0000 7f944be00640 1 -- 172.21.15.1:0/890927237 wait complete. 2024-05-09T08:05:01.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.862+0000 7f944be00640 1 Processor -- start 2024-05-09T08:05:01.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.862+0000 7f944be00640 1 -- start start 2024-05-09T08:05:01.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.863+0000 7f944be00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f944c0cac60 0x7f944c1541c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:05:01.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.863+0000 7f944be00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f944c0cc5f0 0x7f944c1f0910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:05:01.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.863+0000 7f944be00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f944c0ce4e0 0x7f944c1f0e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:05:01.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.863+0000 7f944be00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f944c0cea30 con 0x7f944c0cc5f0 2024-05-09T08:05:01.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.863+0000 7f944be00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f944c0cd550 con 0x7f944c0cac60 2024-05-09T08:05:01.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.863+0000 7f944be00640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f944c0cebb0 con 0x7f944c0ce4e0 2024-05-09T08:05:01.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.863+0000 7f944aa00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f944c0cac60 0x7f944c1541c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:05:01.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.864+0000 7f944b400640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f944c0ce4e0 0x7f944c1f0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:05:01.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.864+0000 7f944aa00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f944c0cac60 0x7f944c1541c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.1:35612/0 (socket says 172.21.15.1:35612) 2024-05-09T08:05:01.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.864+0000 7f944b400640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f944c0ce4e0 0x7f944c1f0e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3301/0 says I am v2:172.21.15.1:38204/0 (socket says 172.21.15.1:38204) 2024-05-09T08:05:01.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.864+0000 7f944aa00640 1 -- 172.21.15.1:0/2562904156 learned_addr learned my addr 172.21.15.1:0/2562904156 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:05:01.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.864+0000 7f9442000640 1 --2- 172.21.15.1:0/2562904156 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f944c0cc5f0 0x7f944c1f0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:05:01.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.864+0000 7f944b400640 1 -- 172.21.15.1:0/2562904156 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f944c0cac60 msgr2=0x7f944c1541c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:05:01.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.864+0000 7f944b400640 1 --2- 172.21.15.1:0/2562904156 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f944c0cac60 0x7f944c1541c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:05:01.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.864+0000 7f944b400640 1 -- 172.21.15.1:0/2562904156 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f944c0cc5f0 msgr2=0x7f944c1f0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:05:01.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.864+0000 7f944b400640 1 --2- 172.21.15.1:0/2562904156 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f944c0cc5f0 0x7f944c1f0910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:05:01.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.864+0000 7f944b400640 1 -- 172.21.15.1:0/2562904156 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f944c15a490 con 0x7f944c0ce4e0 2024-05-09T08:05:01.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.864+0000 7f9442000640 1 --2- 172.21.15.1:0/2562904156 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f944c0cc5f0 0x7f944c1f0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:05:01.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.864+0000 7f944aa00640 1 --2- 172.21.15.1:0/2562904156 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f944c0cac60 0x7f944c1541c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:05:01.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.864+0000 7f944b400640 1 --2- 172.21.15.1:0/2562904156 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f944c0ce4e0 0x7f944c1f0e50 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f944400eb80 tx=0x7f944400c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:05:01.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.865+0000 7f9443400640 1 -- 172.21.15.1:0/2562904156 <== mon.2 v2:172.21.15.7:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f944400cb20 con 0x7f944c0ce4e0 2024-05-09T08:05:01.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.865+0000 7f9443400640 1 -- 172.21.15.1:0/2562904156 <== mon.2 v2:172.21.15.7:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f944400ccc0 con 0x7f944c0ce4e0 2024-05-09T08:05:01.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.865+0000 7f9443400640 1 -- 172.21.15.1:0/2562904156 <== mon.2 v2:172.21.15.7:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9444003bb0 con 0x7f944c0ce4e0 2024-05-09T08:05:01.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.865+0000 7f944be00640 1 -- 172.21.15.1:0/2562904156 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f944c15a730 con 0x7f944c0ce4e0 2024-05-09T08:05:01.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.865+0000 7f944be00640 1 -- 172.21.15.1:0/2562904156 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f944c15acf0 con 0x7f944c0ce4e0 2024-05-09T08:05:01.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.866+0000 7f944be00640 1 --2- 172.21.15.1:0/2562904156 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f944c0c7fc0 0x7f944c15bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:05:01.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.866+0000 7f944be00640 1 -- 172.21.15.1:0/2562904156 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f944c04f660 con 0x7f944c0c7fc0 2024-05-09T08:05:01.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.866+0000 7f944aa00640 1 --2- 172.21.15.1:0/2562904156 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f944c0c7fc0 0x7f944c15bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:05:01.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.867+0000 7f944aa00640 1 -- 172.21.15.1:0/2562904156 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f944c0cc3d0 con 0x7f944c0c7fc0 2024-05-09T08:05:01.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.867+0000 7f944aa00640 1 --2- 172.21.15.1:0/2562904156 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f944c0c7fc0 0x7f944c15bd30 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f943c011db0 tx=0x7f943c00f610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:05:01.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.867+0000 7f9443400640 1 -- 172.21.15.1:0/2562904156 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f943c004030 con 0x7f944c0c7fc0 2024-05-09T08:05:01.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.867+0000 7f9443400640 1 -- 172.21.15.1:0/2562904156 <== mon.2 v2:172.21.15.7:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9444012050 con 0x7f944c0ce4e0 2024-05-09T08:05:01.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.868+0000 7f9443400640 1 --2- 172.21.15.1:0/2562904156 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f942806bd30 0x7f942806e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:05:01.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.868+0000 7f9443400640 1 -- 172.21.15.1:0/2562904156 <== mon.2 v2:172.21.15.7:3301/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7f944408b460 con 0x7f944c0ce4e0 2024-05-09T08:05:01.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.869+0000 7f9442000640 1 --2- 172.21.15.1:0/2562904156 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f942806bd30 0x7f942806e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:05:01.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.869+0000 7f9443400640 1 -- 172.21.15.1:0/2562904156 <== mon.1 v2:172.21.15.7:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f943c0041d0 con 0x7f944c0c7fc0 2024-05-09T08:05:01.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.869+0000 7f9443400640 1 -- 172.21.15.1:0/2562904156 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f944c0c7fc0 msgr2=0x7f944c15bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:05:01.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.869+0000 7f9443400640 1 --2- 172.21.15.1:0/2562904156 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f944c0c7fc0 0x7f944c15bd30 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f943c011db0 tx=0x7f943c00f610 comp rx=0 tx=0).stop 2024-05-09T08:05:01.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.869+0000 7f9443400640 1 -- 172.21.15.1:0/2562904156 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f944c0c7fc0 msgr2=0x7f944c15bd30 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-05-09T08:05:01.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.869+0000 7f9443400640 1 --2- 172.21.15.1:0/2562904156 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f944c0c7fc0 0x7f944c15bd30 secure :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f943c011db0 tx=0x7f943c00f610 comp rx=0 tx=0).stop 2024-05-09T08:05:01.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.869+0000 7f9442000640 1 --2- 172.21.15.1:0/2562904156 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f942806bd30 0x7f942806e1f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f9434034000 tx=0x7f943402e000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:05:01.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.876+0000 7f944be00640 1 --2- 172.21.15.1:0/2562904156 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f944c15c270 0x7f944c0c6f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:05:01.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.876+0000 7f944be00640 1 -- 172.21.15.1:0/2562904156 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f944c0c9b60 con 0x7f944c15c270 2024-05-09T08:05:01.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.876+0000 7f944aa00640 1 --2- 172.21.15.1:0/2562904156 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f944c15c270 0x7f944c0c6f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:05:01.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.877+0000 7f944aa00640 1 -- 172.21.15.1:0/2562904156 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f944c06e0a0 con 0x7f944c15c270 2024-05-09T08:05:01.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.877+0000 7f944aa00640 1 --2- 172.21.15.1:0/2562904156 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f944c15c270 0x7f944c0c6f20 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f943c01b000 tx=0x7f943c0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:05:01.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.877+0000 7f9443400640 1 -- 172.21.15.1:0/2562904156 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f943c014910 con 0x7f944c15c270 2024-05-09T08:05:01.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.878+0000 7f9443400640 1 -- 172.21.15.1:0/2562904156 <== mon.1 v2:172.21.15.7:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4705 (secure 0 0 0) 0x7f943c01c3a0 con 0x7f944c15c270 2024-05-09T08:05:01.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.878+0000 7f9443400640 1 -- 172.21.15.1:0/2562904156 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f944c15c270 msgr2=0x7f944c0c6f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:05:01.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.878+0000 7f9443400640 1 --2- 172.21.15.1:0/2562904156 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f944c15c270 0x7f944c0c6f20 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f943c01b000 tx=0x7f943c0046c0 comp rx=0 tx=0).stop 2024-05-09T08:05:01.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.878+0000 7f9443400640 1 -- 172.21.15.1:0/2562904156 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f944c15c270 msgr2=0x7f944c0c6f20 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-05-09T08:05:01.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.878+0000 7f9443400640 1 --2- 172.21.15.1:0/2562904156 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f944c15c270 0x7f944c0c6f20 secure :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f943c01b000 tx=0x7f943c0046c0 comp rx=0 tx=0).stop 2024-05-09T08:05:01.893 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-05-09T08:05:01.893 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "b", 2024-05-09T08:05:01.893 INFO:teuthology.orchestra.run.smithi001.stdout: "rank": 1, 2024-05-09T08:05:01.893 INFO:teuthology.orchestra.run.smithi001.stdout: "state": "peon", 2024-05-09T08:05:01.893 INFO:teuthology.orchestra.run.smithi001.stdout: "election_epoch": 20, 2024-05-09T08:05:01.894 INFO:teuthology.orchestra.run.smithi001.stdout: "uptime": 69539, 2024-05-09T08:05:01.894 INFO:teuthology.orchestra.run.smithi001.stdout: "quorum": [ 2024-05-09T08:05:01.894 INFO:teuthology.orchestra.run.smithi001.stdout: 0, 2024-05-09T08:05:01.894 INFO:teuthology.orchestra.run.smithi001.stdout: 1, 2024-05-09T08:05:01.894 INFO:teuthology.orchestra.run.smithi001.stdout: 2 2024-05-09T08:05:01.894 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:05:01.895 INFO:teuthology.orchestra.run.smithi001.stdout: "quorum_age": 14, 2024-05-09T08:05:01.895 INFO:teuthology.orchestra.run.smithi001.stdout: "features": { 2024-05-09T08:05:01.895 INFO:teuthology.orchestra.run.smithi001.stdout: "required_con": "2450521708007866372", 2024-05-09T08:05:01.895 INFO:teuthology.orchestra.run.smithi001.stdout: "required_mon": [ 2024-05-09T08:05:01.895 INFO:teuthology.orchestra.run.smithi001.stdout: "kraken", 2024-05-09T08:05:01.895 INFO:teuthology.orchestra.run.smithi001.stdout: "luminous", 2024-05-09T08:05:01.895 INFO:teuthology.orchestra.run.smithi001.stdout: "mimic", 2024-05-09T08:05:01.895 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap-prune", 2024-05-09T08:05:01.895 INFO:teuthology.orchestra.run.smithi001.stdout: "nautilus", 2024-05-09T08:05:01.895 INFO:teuthology.orchestra.run.smithi001.stdout: "octopus", 2024-05-09T08:05:01.895 INFO:teuthology.orchestra.run.smithi001.stdout: "pacific", 2024-05-09T08:05:01.896 INFO:teuthology.orchestra.run.smithi001.stdout: "elector-pinging", 2024-05-09T08:05:01.896 INFO:teuthology.orchestra.run.smithi001.stdout: "quincy", 2024-05-09T08:05:01.896 INFO:teuthology.orchestra.run.smithi001.stdout: "reef", 2024-05-09T08:05:01.896 INFO:teuthology.orchestra.run.smithi001.stdout: "squid" 2024-05-09T08:05:01.896 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:05:01.896 INFO:teuthology.orchestra.run.smithi001.stdout: "quorum_con": "4540701547738038271", 2024-05-09T08:05:01.896 INFO:teuthology.orchestra.run.smithi001.stdout: "quorum_mon": [ 2024-05-09T08:05:01.896 INFO:teuthology.orchestra.run.smithi001.stdout: "kraken", 2024-05-09T08:05:01.896 INFO:teuthology.orchestra.run.smithi001.stdout: "luminous", 2024-05-09T08:05:01.897 INFO:teuthology.orchestra.run.smithi001.stdout: "mimic", 2024-05-09T08:05:01.897 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap-prune", 2024-05-09T08:05:01.897 INFO:teuthology.orchestra.run.smithi001.stdout: "nautilus", 2024-05-09T08:05:01.897 INFO:teuthology.orchestra.run.smithi001.stdout: "octopus", 2024-05-09T08:05:01.897 INFO:teuthology.orchestra.run.smithi001.stdout: "pacific", 2024-05-09T08:05:01.897 INFO:teuthology.orchestra.run.smithi001.stdout: "elector-pinging", 2024-05-09T08:05:01.897 INFO:teuthology.orchestra.run.smithi001.stdout: "quincy", 2024-05-09T08:05:01.897 INFO:teuthology.orchestra.run.smithi001.stdout: "reef", 2024-05-09T08:05:01.898 INFO:teuthology.orchestra.run.smithi001.stdout: "squid" 2024-05-09T08:05:01.898 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:05:01.898 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:05:01.898 INFO:teuthology.orchestra.run.smithi001.stdout: "outside_quorum": [], 2024-05-09T08:05:01.898 INFO:teuthology.orchestra.run.smithi001.stdout: "extra_probe_peers": [], 2024-05-09T08:05:01.898 INFO:teuthology.orchestra.run.smithi001.stdout: "sync_provider": [], 2024-05-09T08:05:01.898 INFO:teuthology.orchestra.run.smithi001.stdout: "monmap": { 2024-05-09T08:05:01.898 INFO:teuthology.orchestra.run.smithi001.stdout: "epoch": 1, 2024-05-09T08:05:01.898 INFO:teuthology.orchestra.run.smithi001.stdout: "fsid": "54329ec8-3270-4043-9a82-c6cd7420aa7f", 2024-05-09T08:05:01.898 INFO:teuthology.orchestra.run.smithi001.stdout: "modified": "2024-05-09T08:01:21.819270Z", 2024-05-09T08:05:01.898 INFO:teuthology.orchestra.run.smithi001.stdout: "created": "2024-05-09T08:01:21.819270Z", 2024-05-09T08:05:01.899 INFO:teuthology.orchestra.run.smithi001.stdout: "min_mon_release": 19, 2024-05-09T08:05:01.899 INFO:teuthology.orchestra.run.smithi001.stdout: "min_mon_release_name": "squid", 2024-05-09T08:05:01.899 INFO:teuthology.orchestra.run.smithi001.stdout: "election_strategy": 1, 2024-05-09T08:05:01.899 INFO:teuthology.orchestra.run.smithi001.stdout: "disallowed_leaders: ": "", 2024-05-09T08:05:01.899 INFO:teuthology.orchestra.run.smithi001.stdout: "stretch_mode": false, 2024-05-09T08:05:01.899 INFO:teuthology.orchestra.run.smithi001.stdout: "tiebreaker_mon": "", 2024-05-09T08:05:01.899 INFO:teuthology.orchestra.run.smithi001.stdout: "removed_ranks: ": "", 2024-05-09T08:05:01.899 INFO:teuthology.orchestra.run.smithi001.stdout: "features": { 2024-05-09T08:05:01.900 INFO:teuthology.orchestra.run.smithi001.stdout: "persistent": [ 2024-05-09T08:05:01.900 INFO:teuthology.orchestra.run.smithi001.stdout: "kraken", 2024-05-09T08:05:01.900 INFO:teuthology.orchestra.run.smithi001.stdout: "luminous", 2024-05-09T08:05:01.900 INFO:teuthology.orchestra.run.smithi001.stdout: "mimic", 2024-05-09T08:05:01.900 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap-prune", 2024-05-09T08:05:01.900 INFO:teuthology.orchestra.run.smithi001.stdout: "nautilus", 2024-05-09T08:05:01.900 INFO:teuthology.orchestra.run.smithi001.stdout: "octopus", 2024-05-09T08:05:01.900 INFO:teuthology.orchestra.run.smithi001.stdout: "pacific", 2024-05-09T08:05:01.900 INFO:teuthology.orchestra.run.smithi001.stdout: "elector-pinging", 2024-05-09T08:05:01.901 INFO:teuthology.orchestra.run.smithi001.stdout: "quincy", 2024-05-09T08:05:01.901 INFO:teuthology.orchestra.run.smithi001.stdout: "reef", 2024-05-09T08:05:01.901 INFO:teuthology.orchestra.run.smithi001.stdout: "squid" 2024-05-09T08:05:01.901 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:05:01.901 INFO:teuthology.orchestra.run.smithi001.stdout: "optional": [] 2024-05-09T08:05:01.902 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:05:01.902 INFO:teuthology.orchestra.run.smithi001.stdout: "mons": [ 2024-05-09T08:05:01.902 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:05:01.902 INFO:teuthology.orchestra.run.smithi001.stdout: "rank": 0, 2024-05-09T08:05:01.902 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "a", 2024-05-09T08:05:01.902 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-05-09T08:05:01.902 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-05-09T08:05:01.903 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:05:01.903 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-05-09T08:05:01.903 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:3300", 2024-05-09T08:05:01.903 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:05:01.903 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:05:01.903 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:05:01.903 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-05-09T08:05:01.903 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6789", 2024-05-09T08:05:01.904 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:05:01.904 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:05:01.904 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:05:01.904 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:05:01.904 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6789/0", 2024-05-09T08:05:01.904 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.1:6789/0", 2024-05-09T08:05:01.904 INFO:teuthology.orchestra.run.smithi001.stdout: "priority": 0, 2024-05-09T08:05:01.904 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 0, 2024-05-09T08:05:01.904 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_location": "{}" 2024-05-09T08:05:01.904 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:05:01.905 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:05:01.905 INFO:teuthology.orchestra.run.smithi001.stdout: "rank": 1, 2024-05-09T08:05:01.905 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "b", 2024-05-09T08:05:01.905 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-05-09T08:05:01.905 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-05-09T08:05:01.905 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:05:01.905 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-05-09T08:05:01.905 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:3300", 2024-05-09T08:05:01.905 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:05:01.905 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:05:01.905 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:05:01.905 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-05-09T08:05:01.905 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:6789", 2024-05-09T08:05:01.905 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:05:01.906 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:05:01.906 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:05:01.906 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:05:01.906 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:6789/0", 2024-05-09T08:05:01.906 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.7:6789/0", 2024-05-09T08:05:01.906 INFO:teuthology.orchestra.run.smithi001.stdout: "priority": 0, 2024-05-09T08:05:01.906 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 0, 2024-05-09T08:05:01.906 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_location": "{}" 2024-05-09T08:05:01.906 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:05:01.906 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:05:01.906 INFO:teuthology.orchestra.run.smithi001.stdout: "rank": 2, 2024-05-09T08:05:01.906 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "c", 2024-05-09T08:05:01.907 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-05-09T08:05:01.907 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-05-09T08:05:01.907 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:05:01.907 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-05-09T08:05:01.907 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:3301", 2024-05-09T08:05:01.907 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:05:01.907 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:05:01.907 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:05:01.907 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-05-09T08:05:01.907 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:6790", 2024-05-09T08:05:01.907 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:05:01.907 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:05:01.908 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:05:01.908 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:05:01.908 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:6790/0", 2024-05-09T08:05:01.908 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.7:6790/0", 2024-05-09T08:05:01.908 INFO:teuthology.orchestra.run.smithi001.stdout: "priority": 0, 2024-05-09T08:05:01.908 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 0, 2024-05-09T08:05:01.908 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_location": "{}" 2024-05-09T08:05:01.908 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:05:01.908 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:05:01.908 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:05:01.908 INFO:teuthology.orchestra.run.smithi001.stdout: "feature_map": { 2024-05-09T08:05:01.908 INFO:teuthology.orchestra.run.smithi001.stdout: "mon": [ 2024-05-09T08:05:01.909 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:05:01.909 INFO:teuthology.orchestra.run.smithi001.stdout: "features": "0x3f03cffffffdffff", 2024-05-09T08:05:01.909 INFO:teuthology.orchestra.run.smithi001.stdout: "release": "luminous", 2024-05-09T08:05:01.909 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 1 2024-05-09T08:05:01.909 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:05:01.909 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:05:01.909 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": [ 2024-05-09T08:05:01.909 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:05:01.909 INFO:teuthology.orchestra.run.smithi001.stdout: "features": "0x3f03cffffffdffff", 2024-05-09T08:05:01.909 INFO:teuthology.orchestra.run.smithi001.stdout: "release": "luminous", 2024-05-09T08:05:01.909 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 1 2024-05-09T08:05:01.909 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:05:01.910 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:05:01.910 INFO:teuthology.orchestra.run.smithi001.stdout: "client": [ 2024-05-09T08:05:01.910 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:05:01.910 INFO:teuthology.orchestra.run.smithi001.stdout: "features": "0x3f03cffffffdffff", 2024-05-09T08:05:01.910 INFO:teuthology.orchestra.run.smithi001.stdout: "release": "luminous", 2024-05-09T08:05:01.910 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 1 2024-05-09T08:05:01.910 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:05:01.910 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:05:01.910 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:05:01.910 INFO:teuthology.orchestra.run.smithi001.stdout: "stretch_mode": false 2024-05-09T08:05:01.910 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-05-09T08:05:01.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.878+0000 7f944be00640 1 -- 172.21.15.1:0/2562904156 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f942806bd30 msgr2=0x7f942806e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:05:01.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.878+0000 7f944be00640 1 --2- 172.21.15.1:0/2562904156 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f942806bd30 0x7f942806e1f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f9434034000 tx=0x7f943402e000 comp rx=0 tx=0).stop 2024-05-09T08:05:01.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.878+0000 7f944be00640 1 -- 172.21.15.1:0/2562904156 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f944c0ce4e0 msgr2=0x7f944c1f0e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:05:01.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.878+0000 7f944be00640 1 --2- 172.21.15.1:0/2562904156 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f944c0ce4e0 0x7f944c1f0e50 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f944400eb80 tx=0x7f944400c490 comp rx=0 tx=0).stop 2024-05-09T08:05:01.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.878+0000 7f944b400640 1 -- 172.21.15.1:0/2562904156 reap_dead start 2024-05-09T08:05:01.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.878+0000 7f944b400640 1 -- 172.21.15.1:0/2562904156 reap_dead start 2024-05-09T08:05:01.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.878+0000 7f944be00640 1 -- 172.21.15.1:0/2562904156 shutdown_connections 2024-05-09T08:05:01.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.878+0000 7f944be00640 1 -- 172.21.15.1:0/2562904156 >> 172.21.15.1:0/2562904156 conn(0x7f944c0cfc20 msgr2=0x7f944c0c9030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:05:01.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.878+0000 7f944be00640 1 -- 172.21.15.1:0/2562904156 shutdown_connections 2024-05-09T08:05:01.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:01.878+0000 7f944be00640 1 -- 172.21.15.1:0/2562904156 wait complete. 2024-05-09T08:05:01.913 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-05-09T08:05:01.913 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-05-09T08:05:01.914 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-05-09T08:05:02.003 INFO:tasks.workunit.client.0.smithi001.stdout: 2024-05-09T08:05:02.003 INFO:tasks.workunit.client.0.smithi001.stdout: 2024-05-09T08:05:02.003 INFO:tasks.workunit.client.0.smithi001.stdout: 2024-05-09T08:05:02.003 INFO:tasks.workunit.client.0.smithi001.stdout:iozone test complete. 2024-05-09T08:05:02.006 INFO:tasks.workunit.client.0.smithi001.stderr:+ iozone -c -e -s 1024M -r 1M -t 1 -F f2 -i 0 -i 1 2024-05-09T08:05:02.096 INFO:tasks.workunit.client.0.smithi001.stdout: Iozone: Performance Test of File I/O 2024-05-09T08:05:02.096 INFO:tasks.workunit.client.0.smithi001.stdout: Version $Revision: 3.494 $ 2024-05-09T08:05:02.096 INFO:tasks.workunit.client.0.smithi001.stdout: Compiled for 64 bit mode. 2024-05-09T08:05:02.096 INFO:tasks.workunit.client.0.smithi001.stdout: Build: linux-AMD64 2024-05-09T08:05:02.096 INFO:tasks.workunit.client.0.smithi001.stdout: 2024-05-09T08:05:02.096 INFO:tasks.workunit.client.0.smithi001.stdout: Contributors:William Norcott, Don Capps, Isom Crawford, Kirby Collins 2024-05-09T08:05:02.096 INFO:tasks.workunit.client.0.smithi001.stdout: Al Slater, Scott Rhine, Mike Wisner, Ken Goss 2024-05-09T08:05:02.097 INFO:tasks.workunit.client.0.smithi001.stdout: Steve Landherr, Brad Smith, Mark Kelly, Dr. Alain CYR, 2024-05-09T08:05:02.097 INFO:tasks.workunit.client.0.smithi001.stdout: Randy Dunlap, Mark Montague, Dan Million, Gavin Brebner, 2024-05-09T08:05:02.097 INFO:tasks.workunit.client.0.smithi001.stdout: Jean-Marc Zucconi, Jeff Blomberg, Benny Halevy, Dave Boone, 2024-05-09T08:05:02.097 INFO:tasks.workunit.client.0.smithi001.stdout: Erik Habbinga, Kris Strecker, Walter Wong, Joshua Root, 2024-05-09T08:05:02.097 INFO:tasks.workunit.client.0.smithi001.stdout: Fabrice Bacchella, Zhenghua Xue, Qin Li, Darren Sawyer, 2024-05-09T08:05:02.097 INFO:tasks.workunit.client.0.smithi001.stdout: Vangel Bojaxhi, Ben England, Vikentsi Lapa, 2024-05-09T08:05:02.097 INFO:tasks.workunit.client.0.smithi001.stdout: Alexey Skidanov, Sudhir Kumar. 2024-05-09T08:05:02.097 INFO:tasks.workunit.client.0.smithi001.stdout: 2024-05-09T08:05:02.097 INFO:tasks.workunit.client.0.smithi001.stdout: Run began: Thu May 9 08:05:01 2024 2024-05-09T08:05:02.097 INFO:tasks.workunit.client.0.smithi001.stdout: 2024-05-09T08:05:02.097 INFO:tasks.workunit.client.0.smithi001.stdout: Include close in write timing 2024-05-09T08:05:02.097 INFO:tasks.workunit.client.0.smithi001.stdout: Include fsync in write timing 2024-05-09T08:05:02.098 INFO:tasks.workunit.client.0.smithi001.stdout: File size set to 1048576 kB 2024-05-09T08:05:02.098 INFO:tasks.workunit.client.0.smithi001.stdout: Record Size 1024 kB 2024-05-09T08:05:02.098 INFO:tasks.workunit.client.0.smithi001.stdout: Command line used: iozone -c -e -s 1024M -r 1M -t 1 -F f2 -i 0 -i 1 2024-05-09T08:05:02.098 INFO:tasks.workunit.client.0.smithi001.stdout: Output is in kBytes/sec 2024-05-09T08:05:02.098 INFO:tasks.workunit.client.0.smithi001.stdout: Time Resolution = 0.000001 seconds. 2024-05-09T08:05:02.098 INFO:tasks.workunit.client.0.smithi001.stdout: Processor cache size set to 1024 kBytes. 2024-05-09T08:05:02.098 INFO:tasks.workunit.client.0.smithi001.stdout: Processor cache line size set to 32 bytes. 2024-05-09T08:05:02.098 INFO:tasks.workunit.client.0.smithi001.stdout: File stride size set to 17 * record size. 2024-05-09T08:05:02.098 INFO:tasks.workunit.client.0.smithi001.stdout: Throughput test with 1 process 2024-05-09T08:05:02.099 INFO:tasks.workunit.client.0.smithi001.stdout: Each process writes a 1048576 kByte file in 1024 kByte records 2024-05-09T08:05:02.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:02.101+0000 7fe2f3e00640 1 Processor -- start 2024-05-09T08:05:02.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:02.101+0000 7fe2f3e00640 1 -- start start 2024-05-09T08:05:02.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:02.103+0000 7fe2f3e00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fe2f40cac60 0x7fe2f40cda00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:05:02.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:02.103+0000 7fe2f3e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe2f40cc5f0 0x7fe2f40cdf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:05:02.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:02.103+0000 7fe2f3e00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe2f40ccb20 con 0x7fe2f40ce4e0 2024-05-09T08:05:02.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:02.103+0000 7fe2f3e00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fe2f40cce00 con 0x7fe2f40cc5f0 2024-05-09T08:05:02.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:02.103+0000 7fe2f3e00640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7fe2f40cf120 con 0x7fe2f40cac60 2024-05-09T08:05:02.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:02.103+0000 7fe2f2a00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fe2f40cac60 0x7fe2f40cda00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:05:02.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:02.103+0000 7fe2f2a00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fe2f40cac60 0x7fe2f40cda00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3301/0 says I am v2:172.21.15.1:38210/0 (socket says 172.21.15.1:38210) 2024-05-09T08:05:02.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:02.103+0000 7fe2f2a00640 1 -- 172.21.15.1:0/989273799 learned_addr learned my addr 172.21.15.1:0/989273799 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:05:02.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:02.103+0000 7fe2f3400640 1 --1- 172.21.15.1:0/989273799 >> v1:172.21.15.7:6789/0 conn(0x7fe2f40ce4e0 0x7fe2f415a570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.1:56204/0 (socket says 172.21.15.1:56204) 2024-05-09T08:05:02.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:02.103+0000 7fe2ebe00640 1 --2- 172.21.15.1:0/989273799 >> v2:172.21.15.7:3300/0 conn(0x7fe2f40cc5f0 0x7fe2f40cdf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:05:02.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:02.104+0000 7fe2f2a00640 1 -- 172.21.15.1:0/989273799 >> v2:172.21.15.7:3300/0 conn(0x7fe2f40cc5f0 msgr2=0x7fe2f40cdf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:05:02.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:02.104+0000 7fe2f2a00640 1 --2- 172.21.15.1:0/989273799 >> v2:172.21.15.7:3300/0 conn(0x7fe2f40cc5f0 0x7fe2f40cdf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:05:02.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:02.104+0000 7fe2f2a00640 1 -- 172.21.15.1:0/989273799 >> v1:172.21.15.7:6789/0 conn(0x7fe2f40ce4e0 legacy=0x7fe2f415a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:05:02.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:02.104+0000 7fe2f2a00640 1 -- 172.21.15.1:0/989273799 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2f40cf400 con 0x7fe2f40cac60 2024-05-09T08:05:02.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:02.104+0000 7fe2f2000640 1 -- 172.21.15.1:0/989273799 <== mon.1 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 287427309 0 0) 0x7fe2f40ccb20 con 0x7fe2f40ce4e0 2024-05-09T08:05:02.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:02.105+0000 7fe2f2a00640 1 --2- 172.21.15.1:0/989273799 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fe2f40cac60 0x7fe2f40cda00 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fe2e4009870 tx=0x7fe2e402a300 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e3bff925120281a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:05:02.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:02.105+0000 7fe2f2000640 1 -- 172.21.15.1:0/989273799 <== mon.2 v2:172.21.15.7:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2e4037070 con 0x7fe2f40cac60 2024-05-09T08:05:02.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:02.105+0000 7fe2f2000640 1 -- 172.21.15.1:0/989273799 <== mon.2 v2:172.21.15.7:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2e402ae50 con 0x7fe2f40cac60 2024-05-09T08:05:02.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:02.105+0000 7fe2f2000640 1 -- 172.21.15.1:0/989273799 <== mon.2 v2:172.21.15.7:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2e4032860 con 0x7fe2f40cac60 2024-05-09T08:05:02.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:02.106+0000 7fe2f3e00640 1 -- 172.21.15.1:0/989273799 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fe2f40cac60 msgr2=0x7fe2f40cda00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:05:02.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:02.106+0000 7fe2f3e00640 1 --2- 172.21.15.1:0/989273799 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fe2f40cac60 0x7fe2f40cda00 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fe2e4009870 tx=0x7fe2e402a300 comp rx=0 tx=0).stop 2024-05-09T08:05:02.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:02.106+0000 7fe2f3e00640 1 -- 172.21.15.1:0/989273799 shutdown_connections 2024-05-09T08:05:02.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:02.106+0000 7fe2f3e00640 1 --2- 172.21.15.1:0/989273799 >> v2:172.21.15.7:3300/0 conn(0x7fe2f40cc5f0 0x7fe2f40cdf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:05:02.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:02.106+0000 7fe2f3e00640 1 --2- 172.21.15.1:0/989273799 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fe2f40cac60 0x7fe2f40cda00 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:05:02.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:02.106+0000 7fe2f3e00640 1 -- 172.21.15.1:0/989273799 >> 172.21.15.1:0/989273799 conn(0x7fe2f40cfc20 msgr2=0x7fe2f40d0030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:05:02.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:02.106+0000 7fe2f3e00640 1 -- 172.21.15.1:0/989273799 shutdown_connections 2024-05-09T08:05:02.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:02.106+0000 7fe2f3e00640 1 -- 172.21.15.1:0/989273799 wait complete. 2024-05-09T08:05:02.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:02.108+0000 7fe2f3e00640 1 Processor -- start 2024-05-09T08:05:02.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:02.108+0000 7fe2f3e00640 1 -- start start 2024-05-09T08:05:02.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:02.109+0000 7fe2f3e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe2f40cac60 0x7fe2f4152380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:05:02.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:02.109+0000 7fe2f3e00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fe2f40cc5f0 0x7fe2f40c8b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:05:02.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:02.109+0000 7fe2f3e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe2f40ce4e0 0x7fe2f41f39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:05:02.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:02.109+0000 7fe2f3e00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fe2f415ac80 con 0x7fe2f40cac60 2024-05-09T08:05:02.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:02.109+0000 7fe2f3e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe2f40cd550 con 0x7fe2f40ce4e0 2024-05-09T08:05:02.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:02.109+0000 7fe2f3e00640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7fe2f415ae00 con 0x7fe2f40cc5f0 2024-05-09T08:05:02.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:02.109+0000 7fe2f2a00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe2f40cac60 0x7fe2f4152380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:05:02.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:02.109+0000 7fe2ebe00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fe2f40cc5f0 0x7fe2f40c8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:05:02.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:02.110+0000 7fe2f2a00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe2f40cac60 0x7fe2f4152380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:36756/0 (socket says 172.21.15.1:36756) 2024-05-09T08:05:02.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:02.110+0000 7fe2f3400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe2f40ce4e0 0x7fe2f41f39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:05:02.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:02.110+0000 7fe2f2a00640 1 -- 172.21.15.1:0/164167231 learned_addr learned my addr 172.21.15.1:0/164167231 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:05:02.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:02.110+0000 7fe2ebe00640 1 -- 172.21.15.1:0/164167231 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe2f40ce4e0 msgr2=0x7fe2f41f39b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:05:02.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:02.110+0000 7fe2ebe00640 1 --2- 172.21.15.1:0/164167231 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe2f40ce4e0 0x7fe2f41f39b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:05:02.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:02.110+0000 7fe2ebe00640 1 -- 172.21.15.1:0/164167231 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe2f40cac60 msgr2=0x7fe2f4152380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:05:02.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:02.110+0000 7fe2ebe00640 1 --2- 172.21.15.1:0/164167231 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe2f40cac60 0x7fe2f4152380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:05:02.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:02.110+0000 7fe2ebe00640 1 -- 172.21.15.1:0/164167231 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2f40c92b0 con 0x7fe2f40cc5f0 2024-05-09T08:05:02.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:02.110+0000 7fe2f3400640 1 --2- 172.21.15.1:0/164167231 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe2f40ce4e0 0x7fe2f41f39b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:05:02.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:02.110+0000 7fe2f2a00640 1 --2- 172.21.15.1:0/164167231 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe2f40cac60 0x7fe2f4152380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:05:02.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:02.110+0000 7fe2ebe00640 1 --2- 172.21.15.1:0/164167231 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fe2f40cc5f0 0x7fe2f40c8b70 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fe2dc00e850 tx=0x7fe2dc00ed10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:05:02.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:02.111+0000 7fe2eaa00640 1 -- 172.21.15.1:0/164167231 <== mon.2 v2:172.21.15.7:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2dc017070 con 0x7fe2f40cc5f0 2024-05-09T08:05:02.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:02.111+0000 7fe2eaa00640 1 -- 172.21.15.1:0/164167231 <== mon.2 v2:172.21.15.7:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2dc00c8e0 con 0x7fe2f40cc5f0 2024-05-09T08:05:02.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:02.111+0000 7fe2eaa00640 1 -- 172.21.15.1:0/164167231 <== mon.2 v2:172.21.15.7:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2dc00ccb0 con 0x7fe2f40cc5f0 2024-05-09T08:05:02.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:02.111+0000 7fe2f3e00640 1 -- 172.21.15.1:0/164167231 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2f41f0400 con 0x7fe2f40cc5f0 2024-05-09T08:05:02.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:02.111+0000 7fe2f3e00640 1 -- 172.21.15.1:0/164167231 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2f41f06c0 con 0x7fe2f40cc5f0 2024-05-09T08:05:02.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:02.112+0000 7fe2f3e00640 1 --2- 172.21.15.1:0/164167231 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fe2f415bc00 0x7fe2f40b2810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:05:02.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:02.112+0000 7fe2f3e00640 1 -- 172.21.15.1:0/164167231 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7fe2f415b810 con 0x7fe2f415bc00 2024-05-09T08:05:02.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:02.112+0000 7fe2f2a00640 1 --2- 172.21.15.1:0/164167231 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fe2f415bc00 0x7fe2f40b2810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:05:02.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:02.112+0000 7fe2f2a00640 1 -- 172.21.15.1:0/164167231 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe2f404f660 con 0x7fe2f415bc00 2024-05-09T08:05:02.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:02.113+0000 7fe2f2a00640 1 --2- 172.21.15.1:0/164167231 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fe2f415bc00 0x7fe2f40b2810 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fe2e4035040 tx=0x7fe2e40046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:05:02.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:02.113+0000 7fe2eaa00640 1 -- 172.21.15.1:0/164167231 <== mon.2 v2:172.21.15.7:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2e4037040 con 0x7fe2f415bc00 2024-05-09T08:05:02.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:02.113+0000 7fe2eaa00640 1 -- 172.21.15.1:0/164167231 <== mon.2 v2:172.21.15.7:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe2dc01a070 con 0x7fe2f40cc5f0 2024-05-09T08:05:02.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:02.114+0000 7fe2eaa00640 1 --2- 172.21.15.1:0/164167231 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fe2d006bd60 0x7fe2d006e220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:05:02.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:02.114+0000 7fe2eaa00640 1 -- 172.21.15.1:0/164167231 <== mon.2 v2:172.21.15.7:3301/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7fe2dc08b350 con 0x7fe2f40cc5f0 2024-05-09T08:05:02.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:02.114+0000 7fe2f3400640 1 --2- 172.21.15.1:0/164167231 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fe2d006bd60 0x7fe2d006e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:05:02.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:02.114+0000 7fe2eaa00640 1 -- 172.21.15.1:0/164167231 <== mon.2 v2:172.21.15.7:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fe2e402ae50 con 0x7fe2f415bc00 2024-05-09T08:05:02.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:02.114+0000 7fe2eaa00640 1 -- 172.21.15.1:0/164167231 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fe2f415bc00 msgr2=0x7fe2f40b2810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:05:02.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:02.114+0000 7fe2eaa00640 1 --2- 172.21.15.1:0/164167231 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fe2f415bc00 0x7fe2f40b2810 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fe2e4035040 tx=0x7fe2e40046c0 comp rx=0 tx=0).stop 2024-05-09T08:05:02.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:02.114+0000 7fe2eaa00640 1 -- 172.21.15.1:0/164167231 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fe2f415bc00 msgr2=0x7fe2f40b2810 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-05-09T08:05:02.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:02.114+0000 7fe2eaa00640 1 --2- 172.21.15.1:0/164167231 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fe2f415bc00 0x7fe2f40b2810 secure :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fe2e4035040 tx=0x7fe2e40046c0 comp rx=0 tx=0).stop 2024-05-09T08:05:02.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:02.115+0000 7fe2f3400640 1 --2- 172.21.15.1:0/164167231 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fe2d006bd60 0x7fe2d006e220 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fe2ec007f30 tx=0x7fe2ec007e80 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:05:02.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:02.122+0000 7fe2f3e00640 1 --2- 172.21.15.1:0/164167231 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fe2f40b5380 0x7fe2f40b2d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:05:02.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:02.122+0000 7fe2f3e00640 1 -- 172.21.15.1:0/164167231 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7fe2f415e830 con 0x7fe2f40b5380 2024-05-09T08:05:02.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:02.122+0000 7fe2f2a00640 1 --2- 172.21.15.1:0/164167231 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fe2f40b5380 0x7fe2f40b2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:05:02.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:02.122+0000 7fe2f2a00640 1 -- 172.21.15.1:0/164167231 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fe2f40c9b60 con 0x7fe2f40b5380 2024-05-09T08:05:02.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:02.123+0000 7fe2f2a00640 1 --2- 172.21.15.1:0/164167231 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fe2f40b5380 0x7fe2f40b2d50 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fe2e402a340 tx=0x7fe2e4008030 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:05:02.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:02.123+0000 7fe2eaa00640 1 -- 172.21.15.1:0/164167231 <== mon.2 v2:172.21.15.7:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2e404c2b0 con 0x7fe2f40b5380 2024-05-09T08:05:02.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:02.123+0000 7fe2eaa00640 1 -- 172.21.15.1:0/164167231 <== mon.2 v2:172.21.15.7:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5189 (secure 0 0 0) 0x7fe2e404c2b0 con 0x7fe2f40b5380 2024-05-09T08:05:02.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:02.123+0000 7fe2eaa00640 1 -- 172.21.15.1:0/164167231 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fe2f40b5380 msgr2=0x7fe2f40b2d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:05:02.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:02.123+0000 7fe2eaa00640 1 --2- 172.21.15.1:0/164167231 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fe2f40b5380 0x7fe2f40b2d50 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fe2e402a340 tx=0x7fe2e4008030 comp rx=0 tx=0).stop 2024-05-09T08:05:02.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:02.123+0000 7fe2eaa00640 1 -- 172.21.15.1:0/164167231 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fe2f40b5380 msgr2=0x7fe2f40b2d50 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-09T08:05:02.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:02.123+0000 7fe2eaa00640 1 --2- 172.21.15.1:0/164167231 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fe2f40b5380 0x7fe2f40b2d50 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:05:02.138 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-05-09T08:05:02.139 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "c", 2024-05-09T08:05:02.139 INFO:teuthology.orchestra.run.smithi001.stdout: "rank": 2, 2024-05-09T08:05:02.139 INFO:teuthology.orchestra.run.smithi001.stdout: "state": "peon", 2024-05-09T08:05:02.139 INFO:teuthology.orchestra.run.smithi001.stdout: "election_epoch": 20, 2024-05-09T08:05:02.139 INFO:teuthology.orchestra.run.smithi001.stdout: "uptime": 185539, 2024-05-09T08:05:02.139 INFO:teuthology.orchestra.run.smithi001.stdout: "quorum": [ 2024-05-09T08:05:02.139 INFO:teuthology.orchestra.run.smithi001.stdout: 0, 2024-05-09T08:05:02.139 INFO:teuthology.orchestra.run.smithi001.stdout: 1, 2024-05-09T08:05:02.139 INFO:teuthology.orchestra.run.smithi001.stdout: 2 2024-05-09T08:05:02.139 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:05:02.139 INFO:teuthology.orchestra.run.smithi001.stdout: "quorum_age": 14, 2024-05-09T08:05:02.139 INFO:teuthology.orchestra.run.smithi001.stdout: "features": { 2024-05-09T08:05:02.139 INFO:teuthology.orchestra.run.smithi001.stdout: "required_con": "2450521708007866372", 2024-05-09T08:05:02.139 INFO:teuthology.orchestra.run.smithi001.stdout: "required_mon": [ 2024-05-09T08:05:02.140 INFO:teuthology.orchestra.run.smithi001.stdout: "kraken", 2024-05-09T08:05:02.140 INFO:teuthology.orchestra.run.smithi001.stdout: "luminous", 2024-05-09T08:05:02.140 INFO:teuthology.orchestra.run.smithi001.stdout: "mimic", 2024-05-09T08:05:02.140 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap-prune", 2024-05-09T08:05:02.140 INFO:teuthology.orchestra.run.smithi001.stdout: "nautilus", 2024-05-09T08:05:02.140 INFO:teuthology.orchestra.run.smithi001.stdout: "octopus", 2024-05-09T08:05:02.140 INFO:teuthology.orchestra.run.smithi001.stdout: "pacific", 2024-05-09T08:05:02.140 INFO:teuthology.orchestra.run.smithi001.stdout: "elector-pinging", 2024-05-09T08:05:02.140 INFO:teuthology.orchestra.run.smithi001.stdout: "quincy", 2024-05-09T08:05:02.140 INFO:teuthology.orchestra.run.smithi001.stdout: "reef", 2024-05-09T08:05:02.140 INFO:teuthology.orchestra.run.smithi001.stdout: "squid" 2024-05-09T08:05:02.140 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:05:02.140 INFO:teuthology.orchestra.run.smithi001.stdout: "quorum_con": "4540701547738038271", 2024-05-09T08:05:02.141 INFO:teuthology.orchestra.run.smithi001.stdout: "quorum_mon": [ 2024-05-09T08:05:02.141 INFO:teuthology.orchestra.run.smithi001.stdout: "kraken", 2024-05-09T08:05:02.141 INFO:teuthology.orchestra.run.smithi001.stdout: "luminous", 2024-05-09T08:05:02.141 INFO:teuthology.orchestra.run.smithi001.stdout: "mimic", 2024-05-09T08:05:02.141 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap-prune", 2024-05-09T08:05:02.141 INFO:teuthology.orchestra.run.smithi001.stdout: "nautilus", 2024-05-09T08:05:02.141 INFO:teuthology.orchestra.run.smithi001.stdout: "octopus", 2024-05-09T08:05:02.141 INFO:teuthology.orchestra.run.smithi001.stdout: "pacific", 2024-05-09T08:05:02.141 INFO:teuthology.orchestra.run.smithi001.stdout: "elector-pinging", 2024-05-09T08:05:02.141 INFO:teuthology.orchestra.run.smithi001.stdout: "quincy", 2024-05-09T08:05:02.141 INFO:teuthology.orchestra.run.smithi001.stdout: "reef", 2024-05-09T08:05:02.141 INFO:teuthology.orchestra.run.smithi001.stdout: "squid" 2024-05-09T08:05:02.141 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:05:02.141 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:05:02.141 INFO:teuthology.orchestra.run.smithi001.stdout: "outside_quorum": [], 2024-05-09T08:05:02.142 INFO:teuthology.orchestra.run.smithi001.stdout: "extra_probe_peers": [], 2024-05-09T08:05:02.142 INFO:teuthology.orchestra.run.smithi001.stdout: "sync_provider": [], 2024-05-09T08:05:02.142 INFO:teuthology.orchestra.run.smithi001.stdout: "monmap": { 2024-05-09T08:05:02.142 INFO:teuthology.orchestra.run.smithi001.stdout: "epoch": 1, 2024-05-09T08:05:02.142 INFO:teuthology.orchestra.run.smithi001.stdout: "fsid": "54329ec8-3270-4043-9a82-c6cd7420aa7f", 2024-05-09T08:05:02.142 INFO:teuthology.orchestra.run.smithi001.stdout: "modified": "2024-05-09T08:01:21.819270Z", 2024-05-09T08:05:02.142 INFO:teuthology.orchestra.run.smithi001.stdout: "created": "2024-05-09T08:01:21.819270Z", 2024-05-09T08:05:02.142 INFO:teuthology.orchestra.run.smithi001.stdout: "min_mon_release": 19, 2024-05-09T08:05:02.142 INFO:teuthology.orchestra.run.smithi001.stdout: "min_mon_release_name": "squid", 2024-05-09T08:05:02.142 INFO:teuthology.orchestra.run.smithi001.stdout: "election_strategy": 1, 2024-05-09T08:05:02.142 INFO:teuthology.orchestra.run.smithi001.stdout: "disallowed_leaders: ": "", 2024-05-09T08:05:02.143 INFO:teuthology.orchestra.run.smithi001.stdout: "stretch_mode": false, 2024-05-09T08:05:02.143 INFO:teuthology.orchestra.run.smithi001.stdout: "tiebreaker_mon": "", 2024-05-09T08:05:02.143 INFO:teuthology.orchestra.run.smithi001.stdout: "removed_ranks: ": "", 2024-05-09T08:05:02.143 INFO:teuthology.orchestra.run.smithi001.stdout: "features": { 2024-05-09T08:05:02.143 INFO:teuthology.orchestra.run.smithi001.stdout: "persistent": [ 2024-05-09T08:05:02.143 INFO:teuthology.orchestra.run.smithi001.stdout: "kraken", 2024-05-09T08:05:02.143 INFO:teuthology.orchestra.run.smithi001.stdout: "luminous", 2024-05-09T08:05:02.143 INFO:teuthology.orchestra.run.smithi001.stdout: "mimic", 2024-05-09T08:05:02.143 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap-prune", 2024-05-09T08:05:02.143 INFO:teuthology.orchestra.run.smithi001.stdout: "nautilus", 2024-05-09T08:05:02.143 INFO:teuthology.orchestra.run.smithi001.stdout: "octopus", 2024-05-09T08:05:02.143 INFO:teuthology.orchestra.run.smithi001.stdout: "pacific", 2024-05-09T08:05:02.143 INFO:teuthology.orchestra.run.smithi001.stdout: "elector-pinging", 2024-05-09T08:05:02.144 INFO:teuthology.orchestra.run.smithi001.stdout: "quincy", 2024-05-09T08:05:02.144 INFO:teuthology.orchestra.run.smithi001.stdout: "reef", 2024-05-09T08:05:02.144 INFO:teuthology.orchestra.run.smithi001.stdout: "squid" 2024-05-09T08:05:02.144 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:05:02.144 INFO:teuthology.orchestra.run.smithi001.stdout: "optional": [] 2024-05-09T08:05:02.144 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:05:02.144 INFO:teuthology.orchestra.run.smithi001.stdout: "mons": [ 2024-05-09T08:05:02.144 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:05:02.144 INFO:teuthology.orchestra.run.smithi001.stdout: "rank": 0, 2024-05-09T08:05:02.144 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "a", 2024-05-09T08:05:02.144 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-05-09T08:05:02.144 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-05-09T08:05:02.144 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:05:02.144 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-05-09T08:05:02.145 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:3300", 2024-05-09T08:05:02.145 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:05:02.145 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:05:02.146 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:05:02.146 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-05-09T08:05:02.146 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6789", 2024-05-09T08:05:02.146 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:05:02.146 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:05:02.146 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:05:02.146 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:05:02.146 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6789/0", 2024-05-09T08:05:02.146 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.1:6789/0", 2024-05-09T08:05:02.146 INFO:teuthology.orchestra.run.smithi001.stdout: "priority": 0, 2024-05-09T08:05:02.146 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 0, 2024-05-09T08:05:02.146 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_location": "{}" 2024-05-09T08:05:02.146 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:05:02.146 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:05:02.147 INFO:teuthology.orchestra.run.smithi001.stdout: "rank": 1, 2024-05-09T08:05:02.147 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "b", 2024-05-09T08:05:02.147 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-05-09T08:05:02.147 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-05-09T08:05:02.147 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:05:02.147 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-05-09T08:05:02.147 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:3300", 2024-05-09T08:05:02.147 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:05:02.147 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:05:02.147 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:05:02.147 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-05-09T08:05:02.147 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:6789", 2024-05-09T08:05:02.147 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:05:02.147 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:05:02.148 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:05:02.148 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:05:02.148 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:6789/0", 2024-05-09T08:05:02.148 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.7:6789/0", 2024-05-09T08:05:02.148 INFO:teuthology.orchestra.run.smithi001.stdout: "priority": 0, 2024-05-09T08:05:02.148 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 0, 2024-05-09T08:05:02.148 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_location": "{}" 2024-05-09T08:05:02.148 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:05:02.148 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:05:02.148 INFO:teuthology.orchestra.run.smithi001.stdout: "rank": 2, 2024-05-09T08:05:02.148 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "c", 2024-05-09T08:05:02.148 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-05-09T08:05:02.148 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-05-09T08:05:02.149 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:05:02.149 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-05-09T08:05:02.149 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:3301", 2024-05-09T08:05:02.149 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:05:02.149 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:05:02.149 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:05:02.149 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-05-09T08:05:02.149 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:6790", 2024-05-09T08:05:02.149 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:05:02.149 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:05:02.149 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:05:02.149 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:05:02.149 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:6790/0", 2024-05-09T08:05:02.149 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.7:6790/0", 2024-05-09T08:05:02.150 INFO:teuthology.orchestra.run.smithi001.stdout: "priority": 0, 2024-05-09T08:05:02.150 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 0, 2024-05-09T08:05:02.150 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_location": "{}" 2024-05-09T08:05:02.150 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:05:02.150 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:05:02.150 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:05:02.150 INFO:teuthology.orchestra.run.smithi001.stdout: "feature_map": { 2024-05-09T08:05:02.150 INFO:teuthology.orchestra.run.smithi001.stdout: "mon": [ 2024-05-09T08:05:02.150 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:05:02.150 INFO:teuthology.orchestra.run.smithi001.stdout: "features": "0x3f03cffffffdffff", 2024-05-09T08:05:02.150 INFO:teuthology.orchestra.run.smithi001.stdout: "release": "luminous", 2024-05-09T08:05:02.150 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 1 2024-05-09T08:05:02.150 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:05:02.150 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:05:02.151 INFO:teuthology.orchestra.run.smithi001.stdout: "mds": [ 2024-05-09T08:05:02.151 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:05:02.151 INFO:teuthology.orchestra.run.smithi001.stdout: "features": "0x3f03cffffffdffff", 2024-05-09T08:05:02.151 INFO:teuthology.orchestra.run.smithi001.stdout: "release": "luminous", 2024-05-09T08:05:02.151 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 6 2024-05-09T08:05:02.151 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:05:02.151 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:05:02.151 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": [ 2024-05-09T08:05:02.151 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:05:02.151 INFO:teuthology.orchestra.run.smithi001.stdout: "features": "0x3f03cffffffdffff", 2024-05-09T08:05:02.151 INFO:teuthology.orchestra.run.smithi001.stdout: "release": "luminous", 2024-05-09T08:05:02.151 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 7 2024-05-09T08:05:02.151 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:05:02.151 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:05:02.152 INFO:teuthology.orchestra.run.smithi001.stdout: "client": [ 2024-05-09T08:05:02.152 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:05:02.152 INFO:teuthology.orchestra.run.smithi001.stdout: "features": "0x2f018fb87aa4aafe", 2024-05-09T08:05:02.152 INFO:teuthology.orchestra.run.smithi001.stdout: "release": "luminous", 2024-05-09T08:05:02.152 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 1 2024-05-09T08:05:02.152 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:05:02.152 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:05:02.152 INFO:teuthology.orchestra.run.smithi001.stdout: "features": "0x3f03cffffffdffff", 2024-05-09T08:05:02.152 INFO:teuthology.orchestra.run.smithi001.stdout: "release": "luminous", 2024-05-09T08:05:02.152 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 6 2024-05-09T08:05:02.152 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:05:02.152 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:05:02.152 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr": [ 2024-05-09T08:05:02.152 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:05:02.153 INFO:teuthology.orchestra.run.smithi001.stdout: "features": "0x3f03cffffffdffff", 2024-05-09T08:05:02.153 INFO:teuthology.orchestra.run.smithi001.stdout: "release": "luminous", 2024-05-09T08:05:02.153 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 2 2024-05-09T08:05:02.153 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:05:02.153 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:05:02.153 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:05:02.153 INFO:teuthology.orchestra.run.smithi001.stdout: "stretch_mode": false 2024-05-09T08:05:02.153 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-05-09T08:05:02.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:02.124+0000 7fe2f3e00640 1 -- 172.21.15.1:0/164167231 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fe2d006bd60 msgr2=0x7fe2d006e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:05:02.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:02.124+0000 7fe2f3e00640 1 --2- 172.21.15.1:0/164167231 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fe2d006bd60 0x7fe2d006e220 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fe2ec007f30 tx=0x7fe2ec007e80 comp rx=0 tx=0).stop 2024-05-09T08:05:02.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:02.124+0000 7fe2f3e00640 1 -- 172.21.15.1:0/164167231 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fe2f40cc5f0 msgr2=0x7fe2f40c8b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:05:02.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:02.124+0000 7fe2f3e00640 1 --2- 172.21.15.1:0/164167231 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fe2f40cc5f0 0x7fe2f40c8b70 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fe2dc00e850 tx=0x7fe2dc00ed10 comp rx=0 tx=0).stop 2024-05-09T08:05:02.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:02.124+0000 7fe2f3400640 1 -- 172.21.15.1:0/164167231 reap_dead start 2024-05-09T08:05:02.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:02.124+0000 7fe2f3e00640 1 -- 172.21.15.1:0/164167231 shutdown_connections 2024-05-09T08:05:02.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:02.124+0000 7fe2f3e00640 1 --2- 172.21.15.1:0/164167231 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fe2f40cc5f0 0x7fe2f40c8b70 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:05:02.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:02.124+0000 7fe2f3e00640 1 -- 172.21.15.1:0/164167231 >> 172.21.15.1:0/164167231 conn(0x7fe2f40cfc20 msgr2=0x7fe2f4154080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:05:02.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:02.124+0000 7fe2f3e00640 1 -- 172.21.15.1:0/164167231 shutdown_connections 2024-05-09T08:05:02.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:02.124+0000 7fe2f3e00640 1 -- 172.21.15.1:0/164167231 wait complete. 2024-05-09T08:05:02.157 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-05-09T08:05:02.157 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['a'] 2024-05-09T08:05:02.157 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2024-05-09T08:05:02.158 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.a 2024-05-09T08:05:02.158 INFO:tasks.mon_thrash.mon_thrasher:killing mon.a 2024-05-09T08:05:02.158 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-05-09T08:05:02.158 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T08:05:02.187 INFO:tasks.ceph.mon.a:Stopped 2024-05-09T08:05:02.187 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2024-05-09T08:05:02.188 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-05-09T08:05:02.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:02.410+0000 7f5b8a600640 1 Processor -- start 2024-05-09T08:05:02.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:02.410+0000 7f5b8a600640 1 -- start start 2024-05-09T08:05:02.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:02.413+0000 7f5b8a600640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f5b84151540 0x7f5b840caef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:05:02.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:02.413+0000 7f5b8a600640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f5b8415b090 0x7f5b840c4f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:05:02.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:02.413+0000 7f5b8a600640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b840cc270 con 0x7f5b840cfe20 2024-05-09T08:05:02.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:02.413+0000 7f5b8a600640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f5b840cc550 con 0x7f5b8415b090 2024-05-09T08:05:02.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:02.413+0000 7f5b8a600640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f5b840cc830 con 0x7f5b84151540 2024-05-09T08:05:02.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:02.413+0000 7f5b83e00640 1 -- >> v1:172.21.15.1:6789/0 conn(0x7f5b840cfe20 legacy=0x7f5b8415dc00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.1:6789/0 2024-05-09T08:05:02.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:02.413+0000 7f5b7aa00640 1 -- >> v2:172.21.15.1:3300/0 conn(0x7f5b8415b090 msgr2=0x7f5b840c4f50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.1:3300/0 2024-05-09T08:05:02.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:02.413+0000 7f5b7aa00640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f5b8415b090 0x7f5b840c4f50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T08:05:02.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:02.413+0000 7f5b83400640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f5b84151540 0x7f5b840caef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:05:02.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:02.413+0000 7f5b83400640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f5b84151540 0x7f5b840caef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3301/0 says I am v2:172.21.15.1:38234/0 (socket says 172.21.15.1:38234) 2024-05-09T08:05:02.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:02.413+0000 7f5b83400640 1 -- 172.21.15.1:0/1019683729 learned_addr learned my addr 172.21.15.1:0/1019683729 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:05:02.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:02.413+0000 7f5b83400640 1 -- 172.21.15.1:0/1019683729 >> v2:172.21.15.1:3300/0 conn(0x7f5b8415b090 msgr2=0x7f5b840c4f50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T08:05:02.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:02.413+0000 7f5b83400640 1 --2- 172.21.15.1:0/1019683729 >> v2:172.21.15.1:3300/0 conn(0x7f5b8415b090 0x7f5b840c4f50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:05:02.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:02.413+0000 7f5b83400640 1 -- 172.21.15.1:0/1019683729 >> v1:172.21.15.1:6789/0 conn(0x7f5b840cfe20 legacy=0x7f5b8415dc00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T08:05:02.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:02.414+0000 7f5b83400640 1 -- 172.21.15.1:0/1019683729 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b840ccb10 con 0x7f5b84151540 2024-05-09T08:05:02.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:02.415+0000 7f5b83400640 1 --2- 172.21.15.1:0/1019683729 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f5b84151540 0x7f5b840caef0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f5b74009e50 tx=0x7f5b7402a780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=253c10e88007dbb server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:05:02.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:02.415+0000 7f5b82a00640 1 -- 172.21.15.1:0/1019683729 <== mon.2 v2:172.21.15.7:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b74036070 con 0x7f5b84151540 2024-05-09T08:05:02.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:02.415+0000 7f5b82a00640 1 -- 172.21.15.1:0/1019683729 <== mon.2 v2:172.21.15.7:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b74031600 con 0x7f5b84151540 2024-05-09T08:05:02.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:02.415+0000 7f5b82a00640 1 -- 172.21.15.1:0/1019683729 <== mon.2 v2:172.21.15.7:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b740319d0 con 0x7f5b84151540 2024-05-09T08:05:02.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:02.416+0000 7f5b8a600640 1 -- 172.21.15.1:0/1019683729 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f5b84151540 msgr2=0x7f5b840caef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:05:02.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:02.416+0000 7f5b8a600640 1 --2- 172.21.15.1:0/1019683729 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f5b84151540 0x7f5b840caef0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f5b74009e50 tx=0x7f5b7402a780 comp rx=0 tx=0).stop 2024-05-09T08:05:02.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:02.416+0000 7f5b8a600640 1 -- 172.21.15.1:0/1019683729 shutdown_connections 2024-05-09T08:05:02.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:02.416+0000 7f5b8a600640 1 --2- 172.21.15.1:0/1019683729 >> v2:172.21.15.1:3300/0 conn(0x7f5b8415b090 0x7f5b840c4f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:05:02.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:02.416+0000 7f5b8a600640 1 --2- 172.21.15.1:0/1019683729 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f5b84151540 0x7f5b840caef0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:05:02.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:02.416+0000 7f5b8a600640 1 -- 172.21.15.1:0/1019683729 >> 172.21.15.1:0/1019683729 conn(0x7f5b8408bff0 msgr2=0x7f5b8408c400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:05:02.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:02.416+0000 7f5b8a600640 1 -- 172.21.15.1:0/1019683729 shutdown_connections 2024-05-09T08:05:02.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:02.416+0000 7f5b8a600640 1 -- 172.21.15.1:0/1019683729 wait complete. 2024-05-09T08:05:02.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:02.418+0000 7f5b8a600640 1 Processor -- start 2024-05-09T08:05:02.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:02.418+0000 7f5b8a600640 1 -- start start 2024-05-09T08:05:02.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:02.419+0000 7f5b8a600640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f5b840cfe20 0x7f5b841f1420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:05:02.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:02.419+0000 7f5b8a600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b84151540 0x7f5b840b4270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:05:02.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:02.419+0000 7f5b8a600640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5b8415b090 0x7f5b8415cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:05:02.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:02.419+0000 7f5b8a600640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f5b840cb560 con 0x7f5b8415b090 2024-05-09T08:05:02.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:02.419+0000 7f5b8a600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5b840cd0d0 con 0x7f5b84151540 2024-05-09T08:05:02.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:02.419+0000 7f5b8a600640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f5b8415b490 con 0x7f5b840cfe20 2024-05-09T08:05:02.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:02.419+0000 7f5b83e00640 1 -- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5b8415b090 msgr2=0x7f5b8415cba0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.1:3300/0 2024-05-09T08:05:02.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:02.419+0000 7f5b83e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5b8415b090 0x7f5b8415cba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T08:05:02.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:02.419+0000 7f5b83400640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f5b840cfe20 0x7f5b841f1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:05:02.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:02.419+0000 7f5b7aa00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b84151540 0x7f5b840b4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:05:02.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:02.419+0000 7f5b83400640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f5b840cfe20 0x7f5b841f1420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3301/0 says I am v2:172.21.15.1:38240/0 (socket says 172.21.15.1:38240) 2024-05-09T08:05:02.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:02.419+0000 7f5b83400640 1 -- 172.21.15.1:0/2741089800 learned_addr learned my addr 172.21.15.1:0/2741089800 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:05:02.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:02.419+0000 7f5b83400640 1 -- 172.21.15.1:0/2741089800 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b84151540 msgr2=0x7f5b840b4270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:05:02.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:02.419+0000 7f5b83400640 1 --2- 172.21.15.1:0/2741089800 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b84151540 0x7f5b840b4270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:05:02.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:02.419+0000 7f5b83400640 1 -- 172.21.15.1:0/2741089800 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5b8415b090 msgr2=0x7f5b8415cba0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T08:05:02.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:02.419+0000 7f5b83400640 1 --2- 172.21.15.1:0/2741089800 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5b8415b090 0x7f5b8415cba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:05:02.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:02.419+0000 7f5b83400640 1 -- 172.21.15.1:0/2741089800 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b840c72a0 con 0x7f5b840cfe20 2024-05-09T08:05:02.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:02.420+0000 7f5b83400640 1 --2- 172.21.15.1:0/2741089800 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f5b840cfe20 0x7f5b841f1420 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f5b74009e50 tx=0x7f5b74004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:05:02.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:02.420+0000 7f5b7be00640 1 -- 172.21.15.1:0/2741089800 <== mon.2 v2:172.21.15.7:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b74041020 con 0x7f5b840cfe20 2024-05-09T08:05:02.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:02.420+0000 7f5b7be00640 1 -- 172.21.15.1:0/2741089800 <== mon.2 v2:172.21.15.7:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b74007800 con 0x7f5b840cfe20 2024-05-09T08:05:02.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:02.420+0000 7f5b7be00640 1 -- 172.21.15.1:0/2741089800 <== mon.2 v2:172.21.15.7:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b74036040 con 0x7f5b840cfe20 2024-05-09T08:05:02.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:02.420+0000 7f5b8a600640 1 -- 172.21.15.1:0/2741089800 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b840c7540 con 0x7f5b840cfe20 2024-05-09T08:05:02.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:02.421+0000 7f5b8a600640 1 -- 172.21.15.1:0/2741089800 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b840c7b00 con 0x7f5b840cfe20 2024-05-09T08:05:02.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:02.422+0000 7f5b8a600640 1 -- 172.21.15.1:0/2741089800 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b840b3520 con 0x7f5b840cfe20 2024-05-09T08:05:02.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:02.423+0000 7f5b7be00640 1 -- 172.21.15.1:0/2741089800 <== mon.2 v2:172.21.15.7:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5b74044020 con 0x7f5b840cfe20 2024-05-09T08:05:02.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:02.423+0000 7f5b7be00640 1 --2- 172.21.15.1:0/2741089800 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f5b6006bde0 0x7f5b6006e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:05:02.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:02.424+0000 7f5b7be00640 1 -- 172.21.15.1:0/2741089800 <== mon.2 v2:172.21.15.7:3301/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7f5b740abfe0 con 0x7f5b840cfe20 2024-05-09T08:05:02.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:02.424+0000 7f5b7aa00640 1 --2- 172.21.15.1:0/2741089800 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f5b6006bde0 0x7f5b6006e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:05:02.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:02.425+0000 7f5b7aa00640 1 --2- 172.21.15.1:0/2741089800 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f5b6006bde0 0x7f5b6006e2a0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f5b840b2620 tx=0x7f5b6c009110 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:05:02.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:02.428+0000 7f5b7be00640 1 -- 172.21.15.1:0/2741089800 <== mon.2 v2:172.21.15.7:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f5b840b3520 con 0x7f5b840cfe20 2024-05-09T08:05:02.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:02.713+0000 7f5b8a600640 1 -- 172.21.15.1:0/2741089800 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f5b840ca710 con 0x7f5b840cfe20 2024-05-09T08:05:02.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:02.714+0000 7f5b7be00640 1 -- 172.21.15.1:0/2741089800 <== mon.2 v2:172.21.15.7:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1545 (secure 0 0 0) 0x7f5b840ca710 con 0x7f5b840cfe20 2024-05-09T08:05:02.728 INFO:teuthology.orchestra.run.smithi001.stdout:{"election_epoch":20,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":15,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"54329ec8-3270-4043-9a82-c6cd7420aa7f","modified":"2024-05-09T08:01:21.819270Z","created":"2024-05-09T08:01:21.819270Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:3300","nonce":0},{"type":"v1","addr":"172.21.15.1:6789","nonce":0}]},"addr":"172.21.15.1:6789/0","public_addr":"172.21.15.1:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:3300","nonce":0},{"type":"v1","addr":"172.21.15.7:6789","nonce":0}]},"addr":"172.21.15.7:6789/0","public_addr":"172.21.15.7:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:3301","nonce":0},{"type":"v1","addr":"172.21.15.7:6790","nonce":0}]},"addr":"172.21.15.7:6790/0","public_addr":"172.21.15.7:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-05-09T08:05:02.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:02.716+0000 7f5b8a600640 1 -- 172.21.15.1:0/2741089800 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f5b6006bde0 msgr2=0x7f5b6006e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:05:02.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:02.716+0000 7f5b8a600640 1 --2- 172.21.15.1:0/2741089800 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f5b6006bde0 0x7f5b6006e2a0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f5b840b2620 tx=0x7f5b6c009110 comp rx=0 tx=0).stop 2024-05-09T08:05:02.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:02.716+0000 7f5b8a600640 1 -- 172.21.15.1:0/2741089800 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f5b840cfe20 msgr2=0x7f5b841f1420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:05:02.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:02.716+0000 7f5b8a600640 1 --2- 172.21.15.1:0/2741089800 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f5b840cfe20 0x7f5b841f1420 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f5b74009e50 tx=0x7f5b74004a40 comp rx=0 tx=0).stop 2024-05-09T08:05:02.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:02.716+0000 7f5b8a600640 1 -- 172.21.15.1:0/2741089800 shutdown_connections 2024-05-09T08:05:02.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:02.716+0000 7f5b8a600640 1 --2- 172.21.15.1:0/2741089800 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f5b6006bde0 0x7f5b6006e2a0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:05:02.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:02.716+0000 7f5b8a600640 1 --2- 172.21.15.1:0/2741089800 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b84151540 0x7f5b840b4270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:05:02.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:02.716+0000 7f5b8a600640 1 --2- 172.21.15.1:0/2741089800 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5b8415b090 0x7f5b8415cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:05:02.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:02.716+0000 7f5b8a600640 1 --2- 172.21.15.1:0/2741089800 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f5b840cfe20 0x7f5b841f1420 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:05:02.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:02.716+0000 7f5b8a600640 1 -- 172.21.15.1:0/2741089800 >> 172.21.15.1:0/2741089800 conn(0x7f5b8408bff0 msgr2=0x7f5b840c6460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:05:02.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:02.717+0000 7f5b8a600640 1 -- 172.21.15.1:0/2741089800 shutdown_connections 2024-05-09T08:05:02.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:02.717+0000 7f5b8a600640 1 -- 172.21.15.1:0/2741089800 wait complete. 2024-05-09T08:05:05.750 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-05-09T08:05:06.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:06.053+0000 7ff63f800640 1 Processor -- start 2024-05-09T08:05:06.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:06.053+0000 7ff63f800640 1 -- start start 2024-05-09T08:05:06.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:06.054+0000 7ff63f800640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff63806dca0 0x7ff638072900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:05:06.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:06.054+0000 7ff63f800640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7ff63806ece0 0x7ff638159010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:05:06.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:06.054+0000 7ff63f800640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff638073200 con 0x7ff6380c6ea0 2024-05-09T08:05:06.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:06.054+0000 7ff63f800640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7ff638072e40 con 0x7ff63806ece0 2024-05-09T08:05:06.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:06.054+0000 7ff63f800640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7ff6380737c0 con 0x7ff63806dca0 2024-05-09T08:05:06.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:06.055+0000 7ff637e00640 1 -- >> v1:172.21.15.1:6789/0 conn(0x7ff6380c6ea0 legacy=0x7ff63806e080 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.1:6789/0 2024-05-09T08:05:06.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:06.055+0000 7ff63d800640 1 -- >> v2:172.21.15.1:3300/0 conn(0x7ff63806ece0 msgr2=0x7ff638159010 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.1:3300/0 2024-05-09T08:05:06.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:06.055+0000 7ff63d800640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7ff63806ece0 0x7ff638159010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T08:05:06.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:06.055+0000 7ff63ce00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff63806dca0 0x7ff638072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:05:06.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:06.055+0000 7ff63ce00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff63806dca0 0x7ff638072900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.1:45444/0 (socket says 172.21.15.1:45444) 2024-05-09T08:05:06.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:06.055+0000 7ff63ce00640 1 -- 172.21.15.1:0/777675094 learned_addr learned my addr 172.21.15.1:0/777675094 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:05:06.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:06.055+0000 7ff63ce00640 1 -- 172.21.15.1:0/777675094 >> v2:172.21.15.1:3300/0 conn(0x7ff63806ece0 msgr2=0x7ff638159010 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T08:05:06.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:06.055+0000 7ff63ce00640 1 --2- 172.21.15.1:0/777675094 >> v2:172.21.15.1:3300/0 conn(0x7ff63806ece0 0x7ff638159010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:05:06.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:06.055+0000 7ff63ce00640 1 -- 172.21.15.1:0/777675094 >> v1:172.21.15.1:6789/0 conn(0x7ff6380c6ea0 legacy=0x7ff63806e080 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T08:05:06.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:06.055+0000 7ff63ce00640 1 -- 172.21.15.1:0/777675094 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6380734e0 con 0x7ff63806dca0 2024-05-09T08:05:06.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:06.056+0000 7ff63ce00640 1 --2- 172.21.15.1:0/777675094 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff63806dca0 0x7ff638072900 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff6380cccb0 tx=0x7ff62c02a2e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e214817958321203 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:05:06.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:06.057+0000 7ff637400640 1 -- 172.21.15.1:0/777675094 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff62c037070 con 0x7ff63806dca0 2024-05-09T08:05:06.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:06.057+0000 7ff637400640 1 -- 172.21.15.1:0/777675094 <== mon.1 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff62c032440 con 0x7ff63806dca0 2024-05-09T08:05:06.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:06.057+0000 7ff637400640 1 -- 172.21.15.1:0/777675094 <== mon.1 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff62c032810 con 0x7ff63806dca0 2024-05-09T08:05:06.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:06.057+0000 7ff63f800640 1 -- 172.21.15.1:0/777675094 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff63806dca0 msgr2=0x7ff638072900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:05:06.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:06.057+0000 7ff63f800640 1 --2- 172.21.15.1:0/777675094 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff63806dca0 0x7ff638072900 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff6380cccb0 tx=0x7ff62c02a2e0 comp rx=0 tx=0).stop 2024-05-09T08:05:06.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:06.057+0000 7ff63f800640 1 -- 172.21.15.1:0/777675094 shutdown_connections 2024-05-09T08:05:06.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:06.057+0000 7ff63f800640 1 --2- 172.21.15.1:0/777675094 >> v2:172.21.15.1:3300/0 conn(0x7ff63806ece0 0x7ff638159010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:05:06.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:06.057+0000 7ff63f800640 1 --2- 172.21.15.1:0/777675094 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff63806dca0 0x7ff638072900 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:05:06.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:06.057+0000 7ff63f800640 1 -- 172.21.15.1:0/777675094 >> 172.21.15.1:0/777675094 conn(0x7ff6380cc070 msgr2=0x7ff6380cc480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:05:06.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:06.058+0000 7ff63f800640 1 -- 172.21.15.1:0/777675094 shutdown_connections 2024-05-09T08:05:06.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:06.058+0000 7ff63f800640 1 -- 172.21.15.1:0/777675094 wait complete. 2024-05-09T08:05:06.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:06.059+0000 7ff63f800640 1 Processor -- start 2024-05-09T08:05:06.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:06.059+0000 7ff63f800640 1 -- start start 2024-05-09T08:05:06.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:06.060+0000 7ff63f800640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ff63806dca0 0x7ff638157710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:05:06.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:06.060+0000 7ff63f800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff63806ece0 0x7ff6381386e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:05:06.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:06.060+0000 7ff63f800640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff6380c6ea0 0x7ff6380cafa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:05:06.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:06.060+0000 7ff63f800640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ff638072470 con 0x7ff6380c6ea0 2024-05-09T08:05:06.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:06.060+0000 7ff63f800640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff638159550 con 0x7ff63806ece0 2024-05-09T08:05:06.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:06.060+0000 7ff63f800640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7ff638072100 con 0x7ff63806dca0 2024-05-09T08:05:06.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:06.061+0000 7ff63d800640 1 -- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff6380c6ea0 msgr2=0x7ff6380cafa0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.1:3300/0 2024-05-09T08:05:06.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:06.061+0000 7ff63d800640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff6380c6ea0 0x7ff6380cafa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T08:05:06.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:06.061+0000 7ff637e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff63806ece0 0x7ff6381386e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:05:06.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:06.061+0000 7ff63ce00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ff63806dca0 0x7ff638157710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:05:06.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:06.061+0000 7ff637e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff63806ece0 0x7ff6381386e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.1:45454/0 (socket says 172.21.15.1:45454) 2024-05-09T08:05:06.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:06.061+0000 7ff63ce00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ff63806dca0 0x7ff638157710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3301/0 says I am v2:172.21.15.1:37094/0 (socket says 172.21.15.1:37094) 2024-05-09T08:05:06.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:06.061+0000 7ff637e00640 1 -- 172.21.15.1:0/1092328144 learned_addr learned my addr 172.21.15.1:0/1092328144 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:05:06.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:06.061+0000 7ff637e00640 1 -- 172.21.15.1:0/1092328144 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ff63806dca0 msgr2=0x7ff638157710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:05:06.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:06.061+0000 7ff637e00640 1 --2- 172.21.15.1:0/1092328144 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ff63806dca0 0x7ff638157710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:05:06.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:06.061+0000 7ff637e00640 1 -- 172.21.15.1:0/1092328144 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff6380c6ea0 msgr2=0x7ff6380cafa0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T08:05:06.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:06.061+0000 7ff637e00640 1 --2- 172.21.15.1:0/1092328144 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff6380c6ea0 0x7ff6380cafa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:05:06.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:06.061+0000 7ff637e00640 1 -- 172.21.15.1:0/1092328144 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff638157c50 con 0x7ff63806ece0 2024-05-09T08:05:06.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:06.062+0000 7ff637e00640 1 --2- 172.21.15.1:0/1092328144 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff63806ece0 0x7ff6381386e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff62400b580 tx=0x7ff62400ba40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:05:06.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:06.062+0000 7ff634c00640 1 -- 172.21.15.1:0/1092328144 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff624013020 con 0x7ff63806ece0 2024-05-09T08:05:06.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:06.062+0000 7ff634c00640 1 -- 172.21.15.1:0/1092328144 <== mon.1 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff624003de0 con 0x7ff63806ece0 2024-05-09T08:05:06.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:06.062+0000 7ff634c00640 1 -- 172.21.15.1:0/1092328144 <== mon.1 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6240052b0 con 0x7ff63806ece0 2024-05-09T08:05:06.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:06.062+0000 7ff63f800640 1 -- 172.21.15.1:0/1092328144 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff638156af0 con 0x7ff63806ece0 2024-05-09T08:05:06.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:06.062+0000 7ff63f800640 1 -- 172.21.15.1:0/1092328144 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff638156530 con 0x7ff63806ece0 2024-05-09T08:05:06.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:06.064+0000 7ff63f800640 1 -- 172.21.15.1:0/1092328144 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff638074d70 con 0x7ff63806ece0 2024-05-09T08:05:06.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:06.066+0000 7ff634c00640 1 -- 172.21.15.1:0/1092328144 <== mon.1 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff624016020 con 0x7ff63806ece0 2024-05-09T08:05:06.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:06.068+0000 7ff634c00640 1 --2- 172.21.15.1:0/1092328144 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7ff61806bde0 0x7ff61806e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:05:06.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:06.068+0000 7ff634c00640 1 -- 172.21.15.1:0/1092328144 <== mon.1 v2:172.21.15.7:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7ff62408af70 con 0x7ff63806ece0 2024-05-09T08:05:06.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:06.071+0000 7ff63ce00640 1 --2- 172.21.15.1:0/1092328144 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7ff61806bde0 0x7ff61806e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:05:06.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:06.072+0000 7ff634c00640 1 -- 172.21.15.1:0/1092328144 <== mon.1 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ff62401d020 con 0x7ff63806ece0 2024-05-09T08:05:06.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:06.073+0000 7ff63ce00640 1 --2- 172.21.15.1:0/1092328144 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7ff61806bde0 0x7ff61806e2a0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ff62c034000 tx=0x7ff62c02e000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:05:06.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:06.621+0000 7ff63f800640 1 -- 172.21.15.1:0/1092328144 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7ff6380ccd20 con 0x7ff63806ece0 2024-05-09T08:05:06.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:06.622+0000 7ff634c00640 1 -- 172.21.15.1:0/1092328144 <== mon.1 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1545 (secure 0 0 0) 0x7ff624059520 con 0x7ff63806ece0 2024-05-09T08:05:06.636 INFO:teuthology.orchestra.run.smithi001.stdout:{"election_epoch":20,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":19,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"54329ec8-3270-4043-9a82-c6cd7420aa7f","modified":"2024-05-09T08:01:21.819270Z","created":"2024-05-09T08:01:21.819270Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:3300","nonce":0},{"type":"v1","addr":"172.21.15.1:6789","nonce":0}]},"addr":"172.21.15.1:6789/0","public_addr":"172.21.15.1:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:3300","nonce":0},{"type":"v1","addr":"172.21.15.7:6789","nonce":0}]},"addr":"172.21.15.7:6789/0","public_addr":"172.21.15.7:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:3301","nonce":0},{"type":"v1","addr":"172.21.15.7:6790","nonce":0}]},"addr":"172.21.15.7:6790/0","public_addr":"172.21.15.7:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-05-09T08:05:06.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:06.626+0000 7ff63f800640 1 -- 172.21.15.1:0/1092328144 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7ff61806bde0 msgr2=0x7ff61806e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:05:06.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:06.626+0000 7ff63f800640 1 --2- 172.21.15.1:0/1092328144 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7ff61806bde0 0x7ff61806e2a0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ff62c034000 tx=0x7ff62c02e000 comp rx=0 tx=0).stop 2024-05-09T08:05:06.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:06.626+0000 7ff63f800640 1 -- 172.21.15.1:0/1092328144 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff63806ece0 msgr2=0x7ff6381386e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:05:06.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:06.626+0000 7ff63f800640 1 --2- 172.21.15.1:0/1092328144 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff63806ece0 0x7ff6381386e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff62400b580 tx=0x7ff62400ba40 comp rx=0 tx=0).stop 2024-05-09T08:05:06.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:06.627+0000 7ff63f800640 1 -- 172.21.15.1:0/1092328144 shutdown_connections 2024-05-09T08:05:06.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:06.627+0000 7ff63f800640 1 --2- 172.21.15.1:0/1092328144 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7ff61806bde0 0x7ff61806e2a0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:05:06.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:06.627+0000 7ff63f800640 1 --2- 172.21.15.1:0/1092328144 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff63806ece0 0x7ff6381386e0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:05:06.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:06.627+0000 7ff63f800640 1 --2- 172.21.15.1:0/1092328144 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff6380c6ea0 0x7ff6380cafa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:05:06.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:06.627+0000 7ff63f800640 1 --2- 172.21.15.1:0/1092328144 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ff63806dca0 0x7ff638157710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:05:06.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:06.627+0000 7ff63f800640 1 -- 172.21.15.1:0/1092328144 >> 172.21.15.1:0/1092328144 conn(0x7ff6380cc070 msgr2=0x7ff6380c7bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:05:06.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:06.628+0000 7ff63f800640 1 -- 172.21.15.1:0/1092328144 shutdown_connections 2024-05-09T08:05:06.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:06.628+0000 7ff63f800640 1 -- 172.21.15.1:0/1092328144 wait complete. 2024-05-09T08:05:07.253 INFO:tasks.workunit.client.0.smithi001.stdout: 2024-05-09T08:05:07.253 INFO:tasks.workunit.client.0.smithi001.stdout: Children see throughput for 1 initial writers = 204877.14 kB/sec 2024-05-09T08:05:07.253 INFO:tasks.workunit.client.0.smithi001.stdout: Parent sees throughput for 1 initial writers = 204849.41 kB/sec 2024-05-09T08:05:07.253 INFO:tasks.workunit.client.0.smithi001.stdout: Min throughput per process = 204877.14 kB/sec 2024-05-09T08:05:07.254 INFO:tasks.workunit.client.0.smithi001.stdout: Max throughput per process = 204877.14 kB/sec 2024-05-09T08:05:07.254 INFO:tasks.workunit.client.0.smithi001.stdout: Avg throughput per process = 204877.14 kB/sec 2024-05-09T08:05:07.254 INFO:tasks.workunit.client.0.smithi001.stdout: Min xfer = 1048576.00 kB 2024-05-09T08:05:09.674 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-05-09T08:05:09.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:09.884+0000 7f9010e00640 1 Processor -- start 2024-05-09T08:05:09.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:09.885+0000 7f9010e00640 1 -- start start 2024-05-09T08:05:09.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:09.886+0000 7f9010e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f900c0c8180 0x7f900c0cc730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:05:09.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:09.886+0000 7f9010e00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f900c0c7660 0x7f900c0c7ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:05:09.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:09.886+0000 7f9010e00640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f900c15c790 con 0x7f900c152500 2024-05-09T08:05:09.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:09.886+0000 7f9010e00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f900c0c5e50 con 0x7f900c0c8180 2024-05-09T08:05:09.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:09.886+0000 7f9010e00640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f900c0c22d0 con 0x7f900c0c7660 2024-05-09T08:05:09.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:09.886+0000 7f900b400640 1 -- >> v1:172.21.15.1:6789/0 conn(0x7f900c152500 legacy=0x7f900c15c080 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.1:6789/0 2024-05-09T08:05:09.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:09.887+0000 7f900be00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f900c0c7660 0x7f900c0c7ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:05:09.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:09.887+0000 7f9003e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f900c0c8180 0x7f900c0cc730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:05:09.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:09.887+0000 7f900be00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f900c0c7660 0x7f900c0c7ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3301/0 says I am v2:172.21.15.1:37106/0 (socket says 172.21.15.1:37106) 2024-05-09T08:05:09.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:09.887+0000 7f900be00640 1 -- 172.21.15.1:0/2106947808 learned_addr learned my addr 172.21.15.1:0/2106947808 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:05:09.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:09.887+0000 7f9003e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f900c0c8180 0x7f900c0cc730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.1:45470/0 (socket says 172.21.15.1:45470) 2024-05-09T08:05:09.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:09.887+0000 7f9003e00640 1 -- 172.21.15.1:0/2106947808 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f900c0c7660 msgr2=0x7f900c0c7ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:05:09.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:09.887+0000 7f9003e00640 1 --2- 172.21.15.1:0/2106947808 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f900c0c7660 0x7f900c0c7ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:05:09.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:09.887+0000 7f9003e00640 1 -- 172.21.15.1:0/2106947808 >> v1:172.21.15.1:6789/0 conn(0x7f900c152500 legacy=0x7f900c15c080 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T08:05:09.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:09.887+0000 7f9003e00640 1 -- 172.21.15.1:0/2106947808 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f900c0c6a30 con 0x7f900c0c8180 2024-05-09T08:05:09.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:09.887+0000 7f900be00640 1 --2- 172.21.15.1:0/2106947808 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f900c0c7660 0x7f900c0c7ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:05:09.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:09.888+0000 7f9003e00640 1 --2- 172.21.15.1:0/2106947808 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f900c0c8180 0x7f900c0cc730 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f8ff40097a0 tx=0x7f8ff402a130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=165a6794b92388a7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:05:09.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:09.889+0000 7f900aa00640 1 -- 172.21.15.1:0/2106947808 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ff4037070 con 0x7f900c0c8180 2024-05-09T08:05:09.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:09.889+0000 7f900aa00640 1 -- 172.21.15.1:0/2106947808 <== mon.1 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ff402ae40 con 0x7f900c0c8180 2024-05-09T08:05:09.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:09.889+0000 7f900aa00640 1 -- 172.21.15.1:0/2106947808 <== mon.1 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ff4032660 con 0x7f900c0c8180 2024-05-09T08:05:09.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:09.889+0000 7f9010e00640 1 -- 172.21.15.1:0/2106947808 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f900c0c8180 msgr2=0x7f900c0cc730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:05:09.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:09.889+0000 7f9010e00640 1 --2- 172.21.15.1:0/2106947808 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f900c0c8180 0x7f900c0cc730 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f8ff40097a0 tx=0x7f8ff402a130 comp rx=0 tx=0).stop 2024-05-09T08:05:09.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:09.890+0000 7f9010e00640 1 -- 172.21.15.1:0/2106947808 shutdown_connections 2024-05-09T08:05:09.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:09.890+0000 7f9010e00640 1 --2- 172.21.15.1:0/2106947808 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f900c0c7660 0x7f900c0c7ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:05:09.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:09.890+0000 7f9010e00640 1 --2- 172.21.15.1:0/2106947808 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f900c0c8180 0x7f900c0cc730 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:05:09.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:09.890+0000 7f9010e00640 1 -- 172.21.15.1:0/2106947808 >> 172.21.15.1:0/2106947808 conn(0x7f900c0cbc10 msgr2=0x7f900c0cc020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:05:09.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:09.890+0000 7f9010e00640 1 -- 172.21.15.1:0/2106947808 shutdown_connections 2024-05-09T08:05:09.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:09.890+0000 7f9010e00640 1 -- 172.21.15.1:0/2106947808 wait complete. 2024-05-09T08:05:09.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:09.892+0000 7f9010e00640 1 Processor -- start 2024-05-09T08:05:09.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:09.892+0000 7f9010e00640 1 -- start start 2024-05-09T08:05:09.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:09.893+0000 7f9010e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f900c0c7660 0x7f900c0ca980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:05:09.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:09.893+0000 7f9010e00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f900c0c8180 0x7f900c1561b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:05:09.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:09.893+0000 7f9010e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f900c152500 0x7f900c143560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:05:09.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:09.893+0000 7f9010e00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f900c0c72b0 con 0x7f900c152500 2024-05-09T08:05:09.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:09.893+0000 7f9010e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f900c0c6fd0 con 0x7f900c0c7660 2024-05-09T08:05:09.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:09.893+0000 7f9010e00640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f900c15b880 con 0x7f900c0c8180 2024-05-09T08:05:09.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:09.893+0000 7f900be00640 1 -- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f900c152500 msgr2=0x7f900c143560 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.1:3300/0 2024-05-09T08:05:09.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:09.893+0000 7f900be00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f900c152500 0x7f900c143560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T08:05:09.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:09.894+0000 7f900b400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f900c0c7660 0x7f900c0ca980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:05:09.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:09.894+0000 7f9003e00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f900c0c8180 0x7f900c1561b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:05:09.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:09.894+0000 7f900b400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f900c0c7660 0x7f900c0ca980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.1:45478/0 (socket says 172.21.15.1:45478) 2024-05-09T08:05:09.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:09.894+0000 7f900b400640 1 -- 172.21.15.1:0/1810719797 learned_addr learned my addr 172.21.15.1:0/1810719797 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:05:09.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:09.894+0000 7f900b400640 1 -- 172.21.15.1:0/1810719797 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f900c0c8180 msgr2=0x7f900c1561b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:05:09.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:09.894+0000 7f900b400640 1 --2- 172.21.15.1:0/1810719797 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f900c0c8180 0x7f900c1561b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:05:09.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:09.894+0000 7f900b400640 1 -- 172.21.15.1:0/1810719797 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f900c152500 msgr2=0x7f900c143560 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T08:05:09.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:09.894+0000 7f900b400640 1 --2- 172.21.15.1:0/1810719797 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f900c152500 0x7f900c143560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:05:09.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:09.894+0000 7f900b400640 1 -- 172.21.15.1:0/1810719797 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f900c1499c0 con 0x7f900c0c7660 2024-05-09T08:05:09.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:09.894+0000 7f900b400640 1 --2- 172.21.15.1:0/1810719797 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f900c0c7660 0x7f900c0ca980 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f8ffc00b550 tx=0x7f8ffc00ba10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:05:09.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:09.895+0000 7f9003400640 1 -- 172.21.15.1:0/1810719797 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ffc003dc0 con 0x7f900c0c7660 2024-05-09T08:05:09.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:09.895+0000 7f9003400640 1 -- 172.21.15.1:0/1810719797 <== mon.1 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ffc015070 con 0x7f900c0c7660 2024-05-09T08:05:09.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:09.895+0000 7f9003400640 1 -- 172.21.15.1:0/1810719797 <== mon.1 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ffc0052b0 con 0x7f900c0c7660 2024-05-09T08:05:09.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:09.895+0000 7f9010e00640 1 -- 172.21.15.1:0/1810719797 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f900c0cb0e0 con 0x7f900c0c7660 2024-05-09T08:05:09.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:09.895+0000 7f9010e00640 1 -- 172.21.15.1:0/1810719797 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f900c14a210 con 0x7f900c0c7660 2024-05-09T08:05:09.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:09.896+0000 7f9010e00640 1 -- 172.21.15.1:0/1810719797 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f900c072dd0 con 0x7f900c0c7660 2024-05-09T08:05:09.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:09.898+0000 7f9003400640 1 -- 172.21.15.1:0/1810719797 <== mon.1 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8ffc005450 con 0x7f900c0c7660 2024-05-09T08:05:09.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:09.898+0000 7f9003400640 1 --2- 172.21.15.1:0/1810719797 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f8fe806be60 0x7f8fe806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:05:09.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:09.898+0000 7f9003400640 1 -- 172.21.15.1:0/1810719797 <== mon.1 v2:172.21.15.7:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7f8ffc08b090 con 0x7f900c0c7660 2024-05-09T08:05:09.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:09.899+0000 7f9003e00640 1 --2- 172.21.15.1:0/1810719797 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f8fe806be60 0x7f8fe806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:05:09.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:09.899+0000 7f9003e00640 1 --2- 172.21.15.1:0/1810719797 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f8fe806be60 0x7f8fe806e320 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f8ff4034000 tx=0x7f8ff402e000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:05:09.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:09.901+0000 7f9003400640 1 -- 172.21.15.1:0/1810719797 <== mon.1 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f900c072dd0 con 0x7f900c0c7660 2024-05-09T08:05:10.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:10.176+0000 7f9010e00640 1 -- 172.21.15.1:0/1810719797 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f900c04f660 con 0x7f900c0c7660 2024-05-09T08:05:10.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:10.177+0000 7f9003400640 1 -- 172.21.15.1:0/1810719797 <== mon.1 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1545 (secure 0 0 0) 0x7f8ffc0596c0 con 0x7f900c0c7660 2024-05-09T08:05:10.191 INFO:teuthology.orchestra.run.smithi001.stdout:{"election_epoch":20,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":22,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"54329ec8-3270-4043-9a82-c6cd7420aa7f","modified":"2024-05-09T08:01:21.819270Z","created":"2024-05-09T08:01:21.819270Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:3300","nonce":0},{"type":"v1","addr":"172.21.15.1:6789","nonce":0}]},"addr":"172.21.15.1:6789/0","public_addr":"172.21.15.1:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:3300","nonce":0},{"type":"v1","addr":"172.21.15.7:6789","nonce":0}]},"addr":"172.21.15.7:6789/0","public_addr":"172.21.15.7:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:3301","nonce":0},{"type":"v1","addr":"172.21.15.7:6790","nonce":0}]},"addr":"172.21.15.7:6790/0","public_addr":"172.21.15.7:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-05-09T08:05:10.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:10.180+0000 7f9010e00640 1 -- 172.21.15.1:0/1810719797 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f8fe806be60 msgr2=0x7f8fe806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:05:10.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:10.180+0000 7f9010e00640 1 --2- 172.21.15.1:0/1810719797 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f8fe806be60 0x7f8fe806e320 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f8ff4034000 tx=0x7f8ff402e000 comp rx=0 tx=0).stop 2024-05-09T08:05:10.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:10.180+0000 7f9010e00640 1 -- 172.21.15.1:0/1810719797 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f900c0c7660 msgr2=0x7f900c0ca980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:05:10.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:10.180+0000 7f9010e00640 1 --2- 172.21.15.1:0/1810719797 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f900c0c7660 0x7f900c0ca980 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f8ffc00b550 tx=0x7f8ffc00ba10 comp rx=0 tx=0).stop 2024-05-09T08:05:10.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:10.180+0000 7f9010e00640 1 -- 172.21.15.1:0/1810719797 shutdown_connections 2024-05-09T08:05:10.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:10.180+0000 7f9010e00640 1 --2- 172.21.15.1:0/1810719797 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f8fe806be60 0x7f8fe806e320 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:05:10.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:10.180+0000 7f9010e00640 1 --2- 172.21.15.1:0/1810719797 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f900c152500 0x7f900c143560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:05:10.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:10.180+0000 7f9010e00640 1 --2- 172.21.15.1:0/1810719797 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f900c0c8180 0x7f900c1561b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:05:10.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:10.180+0000 7f9010e00640 1 --2- 172.21.15.1:0/1810719797 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f900c0c7660 0x7f900c0ca980 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:05:10.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:10.180+0000 7f9010e00640 1 -- 172.21.15.1:0/1810719797 >> 172.21.15.1:0/1810719797 conn(0x7f900c0cbc10 msgr2=0x7f900c15c080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:05:10.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:10.180+0000 7f9010e00640 1 -- 172.21.15.1:0/1810719797 shutdown_connections 2024-05-09T08:05:10.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:10.180+0000 7f9010e00640 1 -- 172.21.15.1:0/1810719797 wait complete. 2024-05-09T08:05:13.215 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-05-09T08:05:13.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:13.505+0000 7ff4fc200640 1 Processor -- start 2024-05-09T08:05:13.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:13.505+0000 7ff4fc200640 1 -- start start 2024-05-09T08:05:13.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:13.507+0000 7ff4fc200640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ff4f4150da0 0x7ff4f406c610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:05:13.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:13.508+0000 7ff4fc200640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff4f40c67d0 0x7ff4f40c8660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:05:13.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:13.508+0000 7ff4fc200640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4f40c58a0 con 0x7ff4f40c8ba0 2024-05-09T08:05:13.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:13.508+0000 7ff4fc200640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7ff4f406cb50 con 0x7ff4f40c67d0 2024-05-09T08:05:13.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:13.508+0000 7ff4fc200640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7ff4f40c3e50 con 0x7ff4f4150da0 2024-05-09T08:05:13.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:13.508+0000 7ff4f9800640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ff4f4150da0 0x7ff4f406c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:05:13.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:13.508+0000 7ff4f9800640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ff4f4150da0 0x7ff4f406c610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3301/0 says I am v2:172.21.15.1:37114/0 (socket says 172.21.15.1:37114) 2024-05-09T08:05:13.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:13.508+0000 7ff4f9800640 1 -- 172.21.15.1:0/2172183075 learned_addr learned my addr 172.21.15.1:0/2172183075 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:05:13.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:13.509+0000 7ff4f9800640 1 -- 172.21.15.1:0/2172183075 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ff4f4150da0 msgr2=0x7ff4f406c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-05-09T08:05:13.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:13.509+0000 7ff4f9800640 1 -- 172.21.15.1:0/2172183075 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ff4f4150da0 msgr2=0x7ff4f406c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-09T08:05:13.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:13.509+0000 7ff4f9800640 1 --2- 172.21.15.1:0/2172183075 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ff4f4150da0 0x7ff4f406c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-09T08:05:13.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:13.509+0000 7ff4f9800640 1 --2- 172.21.15.1:0/2172183075 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ff4f4150da0 0x7ff4f406c610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T08:05:13.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:13.509+0000 7ff4f8e00640 1 --2- 172.21.15.1:0/2172183075 >> v2:172.21.15.7:3300/0 conn(0x7ff4f40c67d0 0x7ff4f40c8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:05:13.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:13.510+0000 7ff4f8e00640 1 -- 172.21.15.1:0/2172183075 >> v2:172.21.15.7:3300/0 conn(0x7ff4f40c67d0 msgr2=0x7ff4f40c8660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-09T08:05:13.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:13.510+0000 7ff4f8e00640 1 -- 172.21.15.1:0/2172183075 >> v2:172.21.15.7:3300/0 conn(0x7ff4f40c67d0 msgr2=0x7ff4f40c8660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-09T08:05:13.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:13.510+0000 7ff4f8e00640 1 --2- 172.21.15.1:0/2172183075 >> v2:172.21.15.7:3300/0 conn(0x7ff4f40c67d0 0x7ff4f40c8660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-09T08:05:13.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:13.510+0000 7ff4f8e00640 1 --2- 172.21.15.1:0/2172183075 >> v2:172.21.15.7:3300/0 conn(0x7ff4f40c67d0 0x7ff4f40c8660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T08:05:13.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:13.710+0000 7ff4f9800640 1 --2- 172.21.15.1:0/2172183075 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ff4f4150da0 0x7ff4f406c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:05:13.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:13.710+0000 7ff4f9800640 1 -- 172.21.15.1:0/2172183075 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ff4f4150da0 msgr2=0x7ff4f406c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-09T08:05:13.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:13.710+0000 7ff4f9800640 1 -- 172.21.15.1:0/2172183075 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ff4f4150da0 msgr2=0x7ff4f406c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-09T08:05:13.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:13.710+0000 7ff4f9800640 1 --2- 172.21.15.1:0/2172183075 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ff4f4150da0 0x7ff4f406c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-09T08:05:13.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:13.710+0000 7ff4f9800640 1 --2- 172.21.15.1:0/2172183075 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ff4f4150da0 0x7ff4f406c610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-09T08:05:13.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:13.711+0000 7ff4f8e00640 1 --2- 172.21.15.1:0/2172183075 >> v2:172.21.15.7:3300/0 conn(0x7ff4f40c67d0 0x7ff4f40c8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:05:13.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:13.711+0000 7ff4f8e00640 1 -- 172.21.15.1:0/2172183075 >> v2:172.21.15.7:3300/0 conn(0x7ff4f40c67d0 msgr2=0x7ff4f40c8660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-09T08:05:13.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:13.711+0000 7ff4f8e00640 1 -- 172.21.15.1:0/2172183075 >> v2:172.21.15.7:3300/0 conn(0x7ff4f40c67d0 msgr2=0x7ff4f40c8660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-09T08:05:13.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:13.711+0000 7ff4f8e00640 1 --2- 172.21.15.1:0/2172183075 >> v2:172.21.15.7:3300/0 conn(0x7ff4f40c67d0 0x7ff4f40c8660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-09T08:05:13.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:13.711+0000 7ff4f8e00640 1 --2- 172.21.15.1:0/2172183075 >> v2:172.21.15.7:3300/0 conn(0x7ff4f40c67d0 0x7ff4f40c8660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-09T08:05:14.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:14.111+0000 7ff4f9800640 1 --2- 172.21.15.1:0/2172183075 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ff4f4150da0 0x7ff4f406c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:05:14.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:14.111+0000 7ff4f9800640 1 -- 172.21.15.1:0/2172183075 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ff4f4150da0 msgr2=0x7ff4f406c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-09T08:05:14.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:14.111+0000 7ff4f9800640 1 -- 172.21.15.1:0/2172183075 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ff4f4150da0 msgr2=0x7ff4f406c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-09T08:05:14.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:14.111+0000 7ff4f9800640 1 --2- 172.21.15.1:0/2172183075 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ff4f4150da0 0x7ff4f406c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-09T08:05:14.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:14.111+0000 7ff4f9800640 1 --2- 172.21.15.1:0/2172183075 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ff4f4150da0 0x7ff4f406c610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-09T08:05:14.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:14.112+0000 7ff4f8e00640 1 --2- 172.21.15.1:0/2172183075 >> v2:172.21.15.7:3300/0 conn(0x7ff4f40c67d0 0x7ff4f40c8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:05:14.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:14.112+0000 7ff4f8e00640 1 -- 172.21.15.1:0/2172183075 >> v2:172.21.15.7:3300/0 conn(0x7ff4f40c67d0 msgr2=0x7ff4f40c8660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-09T08:05:14.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:14.112+0000 7ff4f8e00640 1 -- 172.21.15.1:0/2172183075 >> v2:172.21.15.7:3300/0 conn(0x7ff4f40c67d0 msgr2=0x7ff4f40c8660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-09T08:05:14.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:14.112+0000 7ff4f8e00640 1 --2- 172.21.15.1:0/2172183075 >> v2:172.21.15.7:3300/0 conn(0x7ff4f40c67d0 0x7ff4f40c8660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-09T08:05:14.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:14.112+0000 7ff4f8e00640 1 --2- 172.21.15.1:0/2172183075 >> v2:172.21.15.7:3300/0 conn(0x7ff4f40c67d0 0x7ff4f40c8660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-09T08:05:14.366 INFO:tasks.workunit.client.0.smithi001.stdout: 2024-05-09T08:05:14.366 INFO:tasks.workunit.client.0.smithi001.stdout: Children see throughput for 1 rewriters = 254929.31 kB/sec 2024-05-09T08:05:14.366 INFO:tasks.workunit.client.0.smithi001.stdout: Parent sees throughput for 1 rewriters = 254870.01 kB/sec 2024-05-09T08:05:14.366 INFO:tasks.workunit.client.0.smithi001.stdout: Min throughput per process = 254929.31 kB/sec 2024-05-09T08:05:14.366 INFO:tasks.workunit.client.0.smithi001.stdout: Max throughput per process = 254929.31 kB/sec 2024-05-09T08:05:14.367 INFO:tasks.workunit.client.0.smithi001.stdout: Avg throughput per process = 254929.31 kB/sec 2024-05-09T08:05:14.367 INFO:tasks.workunit.client.0.smithi001.stdout: Min xfer = 1048576.00 kB 2024-05-09T08:05:14.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:14.913+0000 7ff4f8e00640 1 --2- 172.21.15.1:0/2172183075 >> v2:172.21.15.7:3300/0 conn(0x7ff4f40c67d0 0x7ff4f40c8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:05:14.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:14.913+0000 7ff4f9800640 1 --2- 172.21.15.1:0/2172183075 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ff4f4150da0 0x7ff4f406c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:05:14.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:14.914+0000 7ff4f8e00640 1 -- 172.21.15.1:0/2172183075 >> v2:172.21.15.7:3300/0 conn(0x7ff4f40c67d0 msgr2=0x7ff4f40c8660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-05-09T08:05:14.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:14.914+0000 7ff4f8e00640 1 -- 172.21.15.1:0/2172183075 >> v2:172.21.15.7:3300/0 conn(0x7ff4f40c67d0 msgr2=0x7ff4f40c8660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-09T08:05:14.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:14.914+0000 7ff4f8e00640 1 --2- 172.21.15.1:0/2172183075 >> v2:172.21.15.7:3300/0 conn(0x7ff4f40c67d0 0x7ff4f40c8660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-09T08:05:14.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:14.914+0000 7ff4f9800640 1 -- 172.21.15.1:0/2172183075 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ff4f4150da0 msgr2=0x7ff4f406c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-09T08:05:14.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:14.914+0000 7ff4f9800640 1 -- 172.21.15.1:0/2172183075 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ff4f4150da0 msgr2=0x7ff4f406c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-09T08:05:14.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:14.914+0000 7ff4f9800640 1 --2- 172.21.15.1:0/2172183075 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ff4f4150da0 0x7ff4f406c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-09T08:05:14.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:14.914+0000 7ff4f8e00640 1 --2- 172.21.15.1:0/2172183075 >> v2:172.21.15.7:3300/0 conn(0x7ff4f40c67d0 0x7ff4f40c8660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-09T08:05:14.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:14.914+0000 7ff4f9800640 1 --2- 172.21.15.1:0/2172183075 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ff4f4150da0 0x7ff4f406c610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-09T08:05:16.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:16.265+0000 7ff4f3e00640 1 -- 172.21.15.1:0/2172183075 <== mon.1 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 76447797 0 0) 0x7ff4f40c58a0 con 0x7ff4f40c8ba0 2024-05-09T08:05:16.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:16.265+0000 7ff4f3e00640 1 -- 172.21.15.1:0/2172183075 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff4dc005730 con 0x7ff4f40c8ba0 2024-05-09T08:05:16.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:16.271+0000 7ff4f3e00640 1 -- 172.21.15.1:0/2172183075 <== mon.1 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3013240763 0 0) 0x7ff4dc005730 con 0x7ff4f40c8ba0 2024-05-09T08:05:16.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:16.271+0000 7ff4f3e00640 1 -- 172.21.15.1:0/2172183075 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ff4f4150da0 msgr2=0x7ff4f406c610 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T08:05:16.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:16.271+0000 7ff4f3e00640 1 --2- 172.21.15.1:0/2172183075 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ff4f4150da0 0x7ff4f406c610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:05:16.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:16.271+0000 7ff4f3e00640 1 -- 172.21.15.1:0/2172183075 >> v2:172.21.15.7:3300/0 conn(0x7ff4f40c67d0 msgr2=0x7ff4f40c8660 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T08:05:16.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:16.271+0000 7ff4f3e00640 1 --2- 172.21.15.1:0/2172183075 >> v2:172.21.15.7:3300/0 conn(0x7ff4f40c67d0 0x7ff4f40c8660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:05:16.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:16.271+0000 7ff4f3e00640 1 -- 172.21.15.1:0/2172183075 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4f40c5b10 con 0x7ff4f40c8ba0 2024-05-09T08:05:16.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:16.271+0000 7ff4f3e00640 1 -- 172.21.15.1:0/2172183075 <== mon.1 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4179846320 0 0) 0x7ff4ec0019f0 con 0x7ff4f40c8ba0 2024-05-09T08:05:16.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:16.271+0000 7ff4f3e00640 1 -- 172.21.15.1:0/2172183075 >> v1:172.21.15.7:6789/0 conn(0x7ff4f40c8ba0 legacy=0x7ff4f4072570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:05:16.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:16.271+0000 7ff4f3e00640 1 --2- 172.21.15.1:0/2172183075 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ff4f4150da0 0x7ff4dc005bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:05:16.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:16.271+0000 7ff4f3e00640 1 --2- 172.21.15.1:0/2172183075 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff4dc006240 0x7ff4dc006a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:05:16.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:16.272+0000 7ff4f3e00640 1 --2- 172.21.15.1:0/2172183075 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4dc006f50 0x7ff4dc0073d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:05:16.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:16.272+0000 7ff4f9800640 1 --2- 172.21.15.1:0/2172183075 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ff4f4150da0 0x7ff4dc005bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:05:16.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:16.272+0000 7ff4f8e00640 1 -- 172.21.15.1:0/2172183075 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff4dc006240 msgr2=0x7ff4dc006a10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.1:3300/0 2024-05-09T08:05:16.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:16.272+0000 7ff4f3e00640 1 -- 172.21.15.1:0/2172183075 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ff4f40c3e50 con 0x7ff4dc006240 2024-05-09T08:05:16.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:16.272+0000 7ff4f8e00640 1 --2- 172.21.15.1:0/2172183075 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff4dc006240 0x7ff4dc006a10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T08:05:16.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:16.272+0000 7ff4f3e00640 1 -- 172.21.15.1:0/2172183075 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff4dc00dcb0 con 0x7ff4dc006f50 2024-05-09T08:05:16.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:16.272+0000 7ff4f3e00640 1 -- 172.21.15.1:0/2172183075 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7ff4dc00deb0 con 0x7ff4f4150da0 2024-05-09T08:05:16.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:16.272+0000 7ff4fa200640 1 --2- 172.21.15.1:0/2172183075 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4dc006f50 0x7ff4dc0073d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:05:16.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:16.273+0000 7ff4f9800640 1 -- 172.21.15.1:0/2172183075 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4dc006f50 msgr2=0x7ff4dc0073d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:05:16.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:16.273+0000 7ff4f9800640 1 --2- 172.21.15.1:0/2172183075 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4dc006f50 0x7ff4dc0073d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:05:16.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:16.273+0000 7ff4f9800640 1 -- 172.21.15.1:0/2172183075 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff4dc006240 msgr2=0x7ff4dc006a10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T08:05:16.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:16.273+0000 7ff4f9800640 1 --2- 172.21.15.1:0/2172183075 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff4dc006240 0x7ff4dc006a10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:05:16.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:16.273+0000 7ff4f9800640 1 -- 172.21.15.1:0/2172183075 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4dc005730 con 0x7ff4f4150da0 2024-05-09T08:05:16.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:16.275+0000 7ff4f9800640 1 --2- 172.21.15.1:0/2172183075 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ff4f4150da0 0x7ff4dc005bc0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7ff4e8012ca0 tx=0x7ff4dc007a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2256c6b7d4c5fe10 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:05:16.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:16.275+0000 7ff4f3e00640 1 -- 172.21.15.1:0/2172183075 <== mon.2 v2:172.21.15.7:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4e8031070 con 0x7ff4f4150da0 2024-05-09T08:05:16.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:16.275+0000 7ff4f3e00640 1 -- 172.21.15.1:0/2172183075 <== mon.2 v2:172.21.15.7:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4e802bd10 con 0x7ff4f4150da0 2024-05-09T08:05:16.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:16.275+0000 7ff4f3e00640 1 -- 172.21.15.1:0/2172183075 <== mon.2 v2:172.21.15.7:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4e80128a0 con 0x7ff4f4150da0 2024-05-09T08:05:16.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:16.277+0000 7ff4fc200640 1 -- 172.21.15.1:0/2172183075 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ff4f4150da0 msgr2=0x7ff4dc005bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:05:16.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:16.277+0000 7ff4fc200640 1 --2- 172.21.15.1:0/2172183075 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ff4f4150da0 0x7ff4dc005bc0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7ff4e8012ca0 tx=0x7ff4dc007a40 comp rx=0 tx=0).stop 2024-05-09T08:05:16.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:16.277+0000 7ff4fa200640 1 -- 172.21.15.1:0/2172183075 reap_dead start 2024-05-09T08:05:16.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:16.277+0000 7ff4fc200640 1 -- 172.21.15.1:0/2172183075 shutdown_connections 2024-05-09T08:05:16.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:16.277+0000 7ff4fc200640 1 -- 172.21.15.1:0/2172183075 >> 172.21.15.1:0/2172183075 conn(0x7ff4f40cc5f0 msgr2=0x7ff4f40cca00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:05:16.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:16.277+0000 7ff4fc200640 1 -- 172.21.15.1:0/2172183075 shutdown_connections 2024-05-09T08:05:16.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:16.277+0000 7ff4fc200640 1 -- 172.21.15.1:0/2172183075 wait complete. 2024-05-09T08:05:16.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:16.281+0000 7ff4fc200640 1 Processor -- start 2024-05-09T08:05:16.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:16.281+0000 7ff4fc200640 1 -- start start 2024-05-09T08:05:16.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:16.282+0000 7ff4fc200640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff4f4150da0 0x7ff4f4145940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:05:16.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:16.283+0000 7ff4fc200640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ff4f4140d80 0x7ff4f4146410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:05:16.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:16.283+0000 7ff4fc200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4f4147d60 0x7ff4f41ef5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:05:16.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:16.283+0000 7ff4fc200640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ff4f40737c0 con 0x7ff4f4150da0 2024-05-09T08:05:16.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:16.283+0000 7ff4fc200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff4f40c60d0 con 0x7ff4f4147d60 2024-05-09T08:05:16.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:16.283+0000 7ff4fc200640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7ff4f40c3af0 con 0x7ff4f4140d80 2024-05-09T08:05:16.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:16.283+0000 7ff4f9800640 1 -- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff4f4150da0 msgr2=0x7ff4f4145940 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.1:3300/0 2024-05-09T08:05:16.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:16.283+0000 7ff4f9800640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff4f4150da0 0x7ff4f4145940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T08:05:16.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:16.283+0000 7ff4f8e00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ff4f4140d80 0x7ff4f4146410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:05:16.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:16.283+0000 7ff4fa200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4f4147d60 0x7ff4f41ef5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:05:16.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:16.283+0000 7ff4f8e00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ff4f4140d80 0x7ff4f4146410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3301/0 says I am v2:172.21.15.1:39692/0 (socket says 172.21.15.1:39692) 2024-05-09T08:05:16.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:16.283+0000 7ff4f8e00640 1 -- 172.21.15.1:0/3593810704 learned_addr learned my addr 172.21.15.1:0/3593810704 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:05:16.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:16.284+0000 7ff4fa200640 1 -- 172.21.15.1:0/3593810704 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ff4f4140d80 msgr2=0x7ff4f4146410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:05:16.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:16.284+0000 7ff4fa200640 1 --2- 172.21.15.1:0/3593810704 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ff4f4140d80 0x7ff4f4146410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:05:16.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:16.284+0000 7ff4fa200640 1 -- 172.21.15.1:0/3593810704 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff4f4150da0 msgr2=0x7ff4f4145940 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T08:05:16.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:16.284+0000 7ff4fa200640 1 --2- 172.21.15.1:0/3593810704 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff4f4150da0 0x7ff4f4145940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:05:16.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:16.284+0000 7ff4fa200640 1 -- 172.21.15.1:0/3593810704 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4f40ca430 con 0x7ff4f4147d60 2024-05-09T08:05:16.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:16.284+0000 7ff4f8e00640 1 --2- 172.21.15.1:0/3593810704 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ff4f4140d80 0x7ff4f4146410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:05:16.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:16.284+0000 7ff4fa200640 1 --2- 172.21.15.1:0/3593810704 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4f4147d60 0x7ff4f41ef5b0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff4f40c8f80 tx=0x7ff4ec00ff00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:05:16.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:16.284+0000 7ff4f1600640 1 -- 172.21.15.1:0/3593810704 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4ec019070 con 0x7ff4f4147d60 2024-05-09T08:05:16.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:16.285+0000 7ff4f1600640 1 -- 172.21.15.1:0/3593810704 <== mon.1 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4ec006d60 con 0x7ff4f4147d60 2024-05-09T08:05:16.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:16.285+0000 7ff4f1600640 1 -- 172.21.15.1:0/3593810704 <== mon.1 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4ec003bb0 con 0x7ff4f4147d60 2024-05-09T08:05:16.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:16.285+0000 7ff4fc200640 1 -- 172.21.15.1:0/3593810704 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4f41482c0 con 0x7ff4f4147d60 2024-05-09T08:05:16.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:16.285+0000 7ff4fc200640 1 -- 172.21.15.1:0/3593810704 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4f40ca9d0 con 0x7ff4f4147d60 2024-05-09T08:05:16.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:16.287+0000 7ff4fc200640 1 -- 172.21.15.1:0/3593810704 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4f4074d70 con 0x7ff4f4147d60 2024-05-09T08:05:16.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:16.288+0000 7ff4f1600640 1 -- 172.21.15.1:0/3593810704 <== mon.1 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff4ec00d800 con 0x7ff4f4147d60 2024-05-09T08:05:16.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:16.289+0000 7ff4f1600640 1 --2- 172.21.15.1:0/3593810704 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7ff4d806be10 0x7ff4d806e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:05:16.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:16.289+0000 7ff4f1600640 1 -- 172.21.15.1:0/3593810704 <== mon.1 v2:172.21.15.7:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7ff4ec08a710 con 0x7ff4f4147d60 2024-05-09T08:05:16.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:16.289+0000 7ff4f9800640 1 --2- 172.21.15.1:0/3593810704 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7ff4d806be10 0x7ff4d806e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:05:16.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:16.290+0000 7ff4f9800640 1 --2- 172.21.15.1:0/3593810704 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7ff4d806be10 0x7ff4d806e2d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7ff4e8036000 tx=0x7ff4e8031000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:05:16.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:16.294+0000 7ff4f1600640 1 -- 172.21.15.1:0/3593810704 <== mon.1 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ff4ec058550 con 0x7ff4f4147d60 2024-05-09T08:05:16.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:16.584+0000 7ff4fc200640 1 -- 172.21.15.1:0/3593810704 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7ff4f40c80f0 con 0x7ff4f4147d60 2024-05-09T08:05:16.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:16.585+0000 7ff4f1600640 1 -- 172.21.15.1:0/3593810704 <== mon.1 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1538 (secure 0 0 0) 0x7ff4ec05c850 con 0x7ff4f4147d60 2024-05-09T08:05:16.599 INFO:teuthology.orchestra.run.smithi001.stdout:{"election_epoch":22,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"54329ec8-3270-4043-9a82-c6cd7420aa7f","modified":"2024-05-09T08:01:21.819270Z","created":"2024-05-09T08:01:21.819270Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:3300","nonce":0},{"type":"v1","addr":"172.21.15.1:6789","nonce":0}]},"addr":"172.21.15.1:6789/0","public_addr":"172.21.15.1:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:3300","nonce":0},{"type":"v1","addr":"172.21.15.7:6789","nonce":0}]},"addr":"172.21.15.7:6789/0","public_addr":"172.21.15.7:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:3301","nonce":0},{"type":"v1","addr":"172.21.15.7:6790","nonce":0}]},"addr":"172.21.15.7:6790/0","public_addr":"172.21.15.7:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-05-09T08:05:16.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:16.587+0000 7ff4fc200640 1 -- 172.21.15.1:0/3593810704 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7ff4d806be10 msgr2=0x7ff4d806e2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:05:16.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:16.587+0000 7ff4fc200640 1 --2- 172.21.15.1:0/3593810704 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7ff4d806be10 0x7ff4d806e2d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7ff4e8036000 tx=0x7ff4e8031000 comp rx=0 tx=0).stop 2024-05-09T08:05:16.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:16.587+0000 7ff4fc200640 1 -- 172.21.15.1:0/3593810704 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4f4147d60 msgr2=0x7ff4f41ef5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:05:16.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:16.587+0000 7ff4fc200640 1 --2- 172.21.15.1:0/3593810704 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4f4147d60 0x7ff4f41ef5b0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff4f40c8f80 tx=0x7ff4ec00ff00 comp rx=0 tx=0).stop 2024-05-09T08:05:16.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:16.587+0000 7ff4fc200640 1 -- 172.21.15.1:0/3593810704 shutdown_connections 2024-05-09T08:05:16.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:16.587+0000 7ff4fc200640 1 --2- 172.21.15.1:0/3593810704 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7ff4d806be10 0x7ff4d806e2d0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:05:16.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:16.587+0000 7ff4fc200640 1 --2- 172.21.15.1:0/3593810704 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4f4147d60 0x7ff4f41ef5b0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:05:16.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:16.587+0000 7ff4fc200640 1 --2- 172.21.15.1:0/3593810704 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ff4f4140d80 0x7ff4f4146410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:05:16.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:16.587+0000 7ff4fc200640 1 --2- 172.21.15.1:0/3593810704 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff4f4150da0 0x7ff4f4145940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:05:16.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:16.587+0000 7ff4fc200640 1 -- 172.21.15.1:0/3593810704 >> 172.21.15.1:0/3593810704 conn(0x7ff4f40cc5f0 msgr2=0x7ff4f40c7900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:05:16.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:16.588+0000 7ff4fc200640 1 -- 172.21.15.1:0/3593810704 shutdown_connections 2024-05-09T08:05:16.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:16.588+0000 7ff4fc200640 1 -- 172.21.15.1:0/3593810704 wait complete. 2024-05-09T08:05:16.620 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2024-05-09T08:05:16.620 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon b 2024-05-09T08:05:16.621 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-05-09T08:05:16.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:16.818+0000 7f9022800640 1 Processor -- start 2024-05-09T08:05:16.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:16.819+0000 7f9022800640 1 -- start start 2024-05-09T08:05:16.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:16.821+0000 7f9022800640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f901c0ca5c0 0x7f901c0caa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:05:16.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:16.821+0000 7f9022800640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f901c15a7e0 con 0x7f901c06cc80 2024-05-09T08:05:16.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:16.821+0000 7f9022800640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f901c15aaa0 con 0x7f901c0ccd70 2024-05-09T08:05:16.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:16.821+0000 7f9022800640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f901c15a5b0 con 0x7f901c0ca5c0 2024-05-09T08:05:16.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:16.821+0000 7f9021e00640 1 -- >> v1:172.21.15.1:6789/0 conn(0x7f901c06cc80 legacy=0x7f901c06d110 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.1:6789/0 2024-05-09T08:05:16.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:16.821+0000 7f9020a00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f901c0ca5c0 0x7f901c0caa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:05:16.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:16.821+0000 7f9021400640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f901c0ccd70 0x7f901c0cf200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.1:39092/0 (socket says 172.21.15.1:39092) 2024-05-09T08:05:16.836 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:16.821+0000 7f9021400640 1 -- 172.21.15.1:0/3386178992 learned_addr learned my addr 172.21.15.1:0/3386178992 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T08:05:16.836 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:16.822+0000 7f9020a00640 1 -- 172.21.15.1:0/3386178992 >> v1:172.21.15.7:6789/0 conn(0x7f901c0ccd70 legacy=0x7f901c0cf200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:05:16.836 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:16.822+0000 7f9020a00640 1 -- 172.21.15.1:0/3386178992 >> v1:172.21.15.1:6789/0 conn(0x7f901c06cc80 legacy=0x7f901c06d110 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T08:05:16.836 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:16.822+0000 7f9020a00640 1 -- 172.21.15.1:0/3386178992 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f901c15ad80 con 0x7f901c0ca5c0 2024-05-09T08:05:16.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:16.822+0000 7f901be00640 1 -- 172.21.15.1:0/3386178992 <== mon.1 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1207763033 0 0) 0x7f901c15aaa0 con 0x7f901c0ccd70 2024-05-09T08:05:16.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:16.823+0000 7f9020a00640 1 --2- 172.21.15.1:0/3386178992 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f901c0ca5c0 0x7f901c0caa00 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f90080097a0 tx=0x7f900802a1e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=72daf0d03c860bd4 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:05:16.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:16.823+0000 7f901be00640 1 -- 172.21.15.1:0/3386178992 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9008037070 con 0x7f901c0ca5c0 2024-05-09T08:05:16.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:16.823+0000 7f901be00640 1 -- 172.21.15.1:0/3386178992 <== mon.1 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9008032440 con 0x7f901c0ca5c0 2024-05-09T08:05:16.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:16.823+0000 7f901be00640 1 -- 172.21.15.1:0/3386178992 <== mon.1 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9008032810 con 0x7f901c0ca5c0 2024-05-09T08:05:16.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:16.824+0000 7f9022800640 1 -- 172.21.15.1:0/3386178992 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f901c0ca5c0 msgr2=0x7f901c0caa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:05:16.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:16.824+0000 7f9022800640 1 --2- 172.21.15.1:0/3386178992 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f901c0ca5c0 0x7f901c0caa00 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f90080097a0 tx=0x7f900802a1e0 comp rx=0 tx=0).stop 2024-05-09T08:05:16.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:16.824+0000 7f9022800640 1 -- 172.21.15.1:0/3386178992 shutdown_connections 2024-05-09T08:05:16.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:16.824+0000 7f9022800640 1 --2- 172.21.15.1:0/3386178992 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f901c0ca5c0 0x7f901c0caa00 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:05:16.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:16.824+0000 7f9022800640 1 -- 172.21.15.1:0/3386178992 >> 172.21.15.1:0/3386178992 conn(0x7f901c0b1790 msgr2=0x7f901c0b1ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:05:16.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:16.824+0000 7f9022800640 1 -- 172.21.15.1:0/3386178992 shutdown_connections 2024-05-09T08:05:16.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:16.824+0000 7f9022800640 1 -- 172.21.15.1:0/3386178992 wait complete. 2024-05-09T08:05:16.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:16.826+0000 7f9022800640 1 Processor -- start 2024-05-09T08:05:16.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:16.826+0000 7f9022800640 1 -- start start 2024-05-09T08:05:16.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:16.827+0000 7f9022800640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f901c06cc80 0x7f901c1f42e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:05:16.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:16.827+0000 7f9022800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f901c0ca5c0 0x7f901c1f0c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:05:16.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:16.827+0000 7f9022800640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f901c0ccd70 0x7f901c1518b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:05:16.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:16.827+0000 7f9022800640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f901c15b340 con 0x7f901c06cc80 2024-05-09T08:05:16.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:16.827+0000 7f9022800640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f901c06c7f0 con 0x7f901c0ca5c0 2024-05-09T08:05:16.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:16.827+0000 7f9022800640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f901c0cf910 con 0x7f901c0ccd70 2024-05-09T08:05:16.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:16.827+0000 7f9021400640 1 -- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f901c06cc80 msgr2=0x7f901c1f42e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.1:3300/0 2024-05-09T08:05:16.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:16.827+0000 7f9021400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f901c06cc80 0x7f901c1f42e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T08:05:16.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:16.827+0000 7f9021e00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f901c0ccd70 0x7f901c1518b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:05:16.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:16.827+0000 7f9020a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f901c0ca5c0 0x7f901c1f0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:05:16.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:16.827+0000 7f9021e00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f901c0ccd70 0x7f901c1518b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3301/0 says I am v2:172.21.15.1:39708/0 (socket says 172.21.15.1:39708) 2024-05-09T08:05:16.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:16.827+0000 7f9021e00640 1 -- 172.21.15.1:0/1988855491 learned_addr learned my addr 172.21.15.1:0/1988855491 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:05:16.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:16.828+0000 7f9021e00640 1 -- 172.21.15.1:0/1988855491 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f901c0ca5c0 msgr2=0x7f901c1f0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:05:16.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:16.828+0000 7f9021e00640 1 --2- 172.21.15.1:0/1988855491 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f901c0ca5c0 0x7f901c1f0c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:05:16.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:16.828+0000 7f9021e00640 1 -- 172.21.15.1:0/1988855491 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f901c06cc80 msgr2=0x7f901c1f42e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T08:05:16.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:16.828+0000 7f9021e00640 1 --2- 172.21.15.1:0/1988855491 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f901c06cc80 0x7f901c1f42e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:05:16.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:16.828+0000 7f9021e00640 1 -- 172.21.15.1:0/1988855491 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f901c0c8f30 con 0x7f901c0ccd70 2024-05-09T08:05:16.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:16.828+0000 7f9021e00640 1 --2- 172.21.15.1:0/1988855491 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f901c0ccd70 0x7f901c1518b0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f901400be90 tx=0x7f901400bf80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:05:16.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:16.828+0000 7f9019600640 1 -- 172.21.15.1:0/1988855491 <== mon.2 v2:172.21.15.7:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9014016430 con 0x7f901c0ccd70 2024-05-09T08:05:16.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:16.828+0000 7f9019600640 1 -- 172.21.15.1:0/1988855491 <== mon.2 v2:172.21.15.7:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9014010030 con 0x7f901c0ccd70 2024-05-09T08:05:16.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:16.828+0000 7f9019600640 1 -- 172.21.15.1:0/1988855491 <== mon.2 v2:172.21.15.7:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9014016810 con 0x7f901c0ccd70 2024-05-09T08:05:16.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:16.828+0000 7f9022800640 1 -- 172.21.15.1:0/1988855491 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f901c0c8690 con 0x7f901c0ccd70 2024-05-09T08:05:16.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:16.828+0000 7f9022800640 1 -- 172.21.15.1:0/1988855491 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f901c0c8c50 con 0x7f901c0ccd70 2024-05-09T08:05:16.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:16.829+0000 7f9022800640 1 --2- 172.21.15.1:0/1988855491 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f901c151df0 0x7f901c0b4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:05:16.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:16.829+0000 7f9022800640 1 -- 172.21.15.1:0/1988855491 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f901c155880 con 0x7f901c151df0 2024-05-09T08:05:16.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:16.830+0000 7f9021400640 1 --2- 172.21.15.1:0/1988855491 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f901c151df0 0x7f901c0b4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:05:16.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:16.830+0000 7f9021400640 1 -- 172.21.15.1:0/1988855491 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f901c1521d0 con 0x7f901c151df0 2024-05-09T08:05:16.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:16.830+0000 7f9021400640 1 --2- 172.21.15.1:0/1988855491 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f901c151df0 0x7f901c0b4770 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f90100010e0 tx=0x7f901000cb10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:05:16.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:16.831+0000 7f9019600640 1 -- 172.21.15.1:0/1988855491 <== mon.2 v2:172.21.15.7:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f90140153e0 con 0x7f901c0ccd70 2024-05-09T08:05:16.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:16.831+0000 7f9019600640 1 --2- 172.21.15.1:0/1988855491 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f8ffc06bde0 0x7f8ffc06e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:05:16.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:16.832+0000 7f9019600640 1 -- 172.21.15.1:0/1988855491 <== mon.2 v2:172.21.15.7:3301/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7f901408b6d0 con 0x7f901c0ccd70 2024-05-09T08:05:16.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:16.832+0000 7f9020a00640 1 --2- 172.21.15.1:0/1988855491 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f8ffc06bde0 0x7f8ffc06e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:05:16.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:16.832+0000 7f9019600640 1 -- 172.21.15.1:0/1988855491 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9010017070 con 0x7f901c151df0 2024-05-09T08:05:16.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:16.832+0000 7f9019600640 1 -- 172.21.15.1:0/1988855491 <== mon.1 v2:172.21.15.7:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f901000ce50 con 0x7f901c151df0 2024-05-09T08:05:16.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:16.832+0000 7f9019600640 1 -- 172.21.15.1:0/1988855491 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f901c151df0 msgr2=0x7f901c0b4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:05:16.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:16.832+0000 7f9019600640 1 --2- 172.21.15.1:0/1988855491 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f901c151df0 0x7f901c0b4770 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f90100010e0 tx=0x7f901000cb10 comp rx=0 tx=0).stop 2024-05-09T08:05:16.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:16.832+0000 7f9019600640 1 -- 172.21.15.1:0/1988855491 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f901c151df0 msgr2=0x7f901c0b4770 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-05-09T08:05:16.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:16.832+0000 7f9019600640 1 --2- 172.21.15.1:0/1988855491 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f901c151df0 0x7f901c0b4770 secure :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f90100010e0 tx=0x7f901000cb10 comp rx=0 tx=0).stop 2024-05-09T08:05:16.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:16.833+0000 7f9020a00640 1 --2- 172.21.15.1:0/1988855491 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f8ffc06bde0 0x7f8ffc06e2a0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f9008034000 tx=0x7f900802e000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:05:16.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:16.839+0000 7f9022800640 1 --2- 172.21.15.1:0/1988855491 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f901c0b4cb0 0x7f901c0c6f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:05:16.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:16.839+0000 7f9022800640 1 -- 172.21.15.1:0/1988855491 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f901c0c9100 con 0x7f901c0b4cb0 2024-05-09T08:05:16.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:16.840+0000 7f9021400640 1 --2- 172.21.15.1:0/1988855491 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f901c0b4cb0 0x7f901c0c6f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:05:16.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:16.840+0000 7f9021400640 1 -- 172.21.15.1:0/1988855491 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f901c04f660 con 0x7f901c0b4cb0 2024-05-09T08:05:16.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:16.840+0000 7f9021400640 1 --2- 172.21.15.1:0/1988855491 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f901c0b4cb0 0x7f901c0c6f20 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f9010011000 tx=0x7f901000f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:05:16.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:16.841+0000 7f9019600640 1 -- 172.21.15.1:0/1988855491 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90100075d0 con 0x7f901c0b4cb0 2024-05-09T08:05:16.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:16.841+0000 7f9019600640 1 -- 172.21.15.1:0/1988855491 <== mon.1 v2:172.21.15.7:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4695 (secure 0 0 0) 0x7f90100075d0 con 0x7f901c0b4cb0 2024-05-09T08:05:16.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:16.841+0000 7f9019600640 1 -- 172.21.15.1:0/1988855491 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f901c0b4cb0 msgr2=0x7f901c0c6f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:05:16.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:16.841+0000 7f9019600640 1 --2- 172.21.15.1:0/1988855491 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f901c0b4cb0 0x7f901c0c6f20 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f9010011000 tx=0x7f901000f000 comp rx=0 tx=0).stop 2024-05-09T08:05:16.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:16.841+0000 7f9019600640 1 -- 172.21.15.1:0/1988855491 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f901c0b4cb0 msgr2=0x7f901c0c6f20 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-05-09T08:05:16.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:16.841+0000 7f9019600640 1 --2- 172.21.15.1:0/1988855491 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f901c0b4cb0 0x7f901c0c6f20 secure :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f9010011000 tx=0x7f901000f000 comp rx=0 tx=0).stop 2024-05-09T08:05:16.856 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-05-09T08:05:16.857 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "b", 2024-05-09T08:05:16.857 INFO:teuthology.orchestra.run.smithi001.stdout: "rank": 1, 2024-05-09T08:05:16.857 INFO:teuthology.orchestra.run.smithi001.stdout: "state": "leader", 2024-05-09T08:05:16.857 INFO:teuthology.orchestra.run.smithi001.stdout: "election_epoch": 22, 2024-05-09T08:05:16.857 INFO:teuthology.orchestra.run.smithi001.stdout: "uptime": 84502, 2024-05-09T08:05:16.857 INFO:teuthology.orchestra.run.smithi001.stdout: "quorum": [ 2024-05-09T08:05:16.857 INFO:teuthology.orchestra.run.smithi001.stdout: 1, 2024-05-09T08:05:16.857 INFO:teuthology.orchestra.run.smithi001.stdout: 2 2024-05-09T08:05:16.857 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:05:16.858 INFO:teuthology.orchestra.run.smithi001.stdout: "quorum_age": 0, 2024-05-09T08:05:16.858 INFO:teuthology.orchestra.run.smithi001.stdout: "features": { 2024-05-09T08:05:16.858 INFO:teuthology.orchestra.run.smithi001.stdout: "required_con": "2450521708007866372", 2024-05-09T08:05:16.858 INFO:teuthology.orchestra.run.smithi001.stdout: "required_mon": [ 2024-05-09T08:05:16.858 INFO:teuthology.orchestra.run.smithi001.stdout: "kraken", 2024-05-09T08:05:16.858 INFO:teuthology.orchestra.run.smithi001.stdout: "luminous", 2024-05-09T08:05:16.859 INFO:teuthology.orchestra.run.smithi001.stdout: "mimic", 2024-05-09T08:05:16.859 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap-prune", 2024-05-09T08:05:16.859 INFO:teuthology.orchestra.run.smithi001.stdout: "nautilus", 2024-05-09T08:05:16.859 INFO:teuthology.orchestra.run.smithi001.stdout: "octopus", 2024-05-09T08:05:16.859 INFO:teuthology.orchestra.run.smithi001.stdout: "pacific", 2024-05-09T08:05:16.859 INFO:teuthology.orchestra.run.smithi001.stdout: "elector-pinging", 2024-05-09T08:05:16.859 INFO:teuthology.orchestra.run.smithi001.stdout: "quincy", 2024-05-09T08:05:16.860 INFO:teuthology.orchestra.run.smithi001.stdout: "reef", 2024-05-09T08:05:16.860 INFO:teuthology.orchestra.run.smithi001.stdout: "squid" 2024-05-09T08:05:16.860 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:05:16.860 INFO:teuthology.orchestra.run.smithi001.stdout: "quorum_con": "4540701547738038271", 2024-05-09T08:05:16.860 INFO:teuthology.orchestra.run.smithi001.stdout: "quorum_mon": [ 2024-05-09T08:05:16.860 INFO:teuthology.orchestra.run.smithi001.stdout: "kraken", 2024-05-09T08:05:16.861 INFO:teuthology.orchestra.run.smithi001.stdout: "luminous", 2024-05-09T08:05:16.861 INFO:teuthology.orchestra.run.smithi001.stdout: "mimic", 2024-05-09T08:05:16.861 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap-prune", 2024-05-09T08:05:16.861 INFO:teuthology.orchestra.run.smithi001.stdout: "nautilus", 2024-05-09T08:05:16.861 INFO:teuthology.orchestra.run.smithi001.stdout: "octopus", 2024-05-09T08:05:16.861 INFO:teuthology.orchestra.run.smithi001.stdout: "pacific", 2024-05-09T08:05:16.861 INFO:teuthology.orchestra.run.smithi001.stdout: "elector-pinging", 2024-05-09T08:05:16.862 INFO:teuthology.orchestra.run.smithi001.stdout: "quincy", 2024-05-09T08:05:16.862 INFO:teuthology.orchestra.run.smithi001.stdout: "reef", 2024-05-09T08:05:16.862 INFO:teuthology.orchestra.run.smithi001.stdout: "squid" 2024-05-09T08:05:16.862 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:05:16.862 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:05:16.862 INFO:teuthology.orchestra.run.smithi001.stdout: "outside_quorum": [], 2024-05-09T08:05:16.863 INFO:teuthology.orchestra.run.smithi001.stdout: "extra_probe_peers": [], 2024-05-09T08:05:16.863 INFO:teuthology.orchestra.run.smithi001.stdout: "sync_provider": [], 2024-05-09T08:05:16.863 INFO:teuthology.orchestra.run.smithi001.stdout: "monmap": { 2024-05-09T08:05:16.863 INFO:teuthology.orchestra.run.smithi001.stdout: "epoch": 1, 2024-05-09T08:05:16.863 INFO:teuthology.orchestra.run.smithi001.stdout: "fsid": "54329ec8-3270-4043-9a82-c6cd7420aa7f", 2024-05-09T08:05:16.863 INFO:teuthology.orchestra.run.smithi001.stdout: "modified": "2024-05-09T08:01:21.819270Z", 2024-05-09T08:05:16.863 INFO:teuthology.orchestra.run.smithi001.stdout: "created": "2024-05-09T08:01:21.819270Z", 2024-05-09T08:05:16.863 INFO:teuthology.orchestra.run.smithi001.stdout: "min_mon_release": 19, 2024-05-09T08:05:16.863 INFO:teuthology.orchestra.run.smithi001.stdout: "min_mon_release_name": "squid", 2024-05-09T08:05:16.864 INFO:teuthology.orchestra.run.smithi001.stdout: "election_strategy": 1, 2024-05-09T08:05:16.864 INFO:teuthology.orchestra.run.smithi001.stdout: "disallowed_leaders: ": "", 2024-05-09T08:05:16.864 INFO:teuthology.orchestra.run.smithi001.stdout: "stretch_mode": false, 2024-05-09T08:05:16.864 INFO:teuthology.orchestra.run.smithi001.stdout: "tiebreaker_mon": "", 2024-05-09T08:05:16.864 INFO:teuthology.orchestra.run.smithi001.stdout: "removed_ranks: ": "", 2024-05-09T08:05:16.864 INFO:teuthology.orchestra.run.smithi001.stdout: "features": { 2024-05-09T08:05:16.864 INFO:teuthology.orchestra.run.smithi001.stdout: "persistent": [ 2024-05-09T08:05:16.864 INFO:teuthology.orchestra.run.smithi001.stdout: "kraken", 2024-05-09T08:05:16.864 INFO:teuthology.orchestra.run.smithi001.stdout: "luminous", 2024-05-09T08:05:16.864 INFO:teuthology.orchestra.run.smithi001.stdout: "mimic", 2024-05-09T08:05:16.865 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap-prune", 2024-05-09T08:05:16.865 INFO:teuthology.orchestra.run.smithi001.stdout: "nautilus", 2024-05-09T08:05:16.865 INFO:teuthology.orchestra.run.smithi001.stdout: "octopus", 2024-05-09T08:05:16.865 INFO:teuthology.orchestra.run.smithi001.stdout: "pacific", 2024-05-09T08:05:16.865 INFO:teuthology.orchestra.run.smithi001.stdout: "elector-pinging", 2024-05-09T08:05:16.865 INFO:teuthology.orchestra.run.smithi001.stdout: "quincy", 2024-05-09T08:05:16.865 INFO:teuthology.orchestra.run.smithi001.stdout: "reef", 2024-05-09T08:05:16.865 INFO:teuthology.orchestra.run.smithi001.stdout: "squid" 2024-05-09T08:05:16.865 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:05:16.865 INFO:teuthology.orchestra.run.smithi001.stdout: "optional": [] 2024-05-09T08:05:16.866 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:05:16.866 INFO:teuthology.orchestra.run.smithi001.stdout: "mons": [ 2024-05-09T08:05:16.866 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:05:16.866 INFO:teuthology.orchestra.run.smithi001.stdout: "rank": 0, 2024-05-09T08:05:16.866 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "a", 2024-05-09T08:05:16.866 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-05-09T08:05:16.866 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-05-09T08:05:16.866 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:05:16.866 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-05-09T08:05:16.866 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:3300", 2024-05-09T08:05:16.867 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:05:16.867 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:05:16.867 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:05:16.867 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-05-09T08:05:16.867 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6789", 2024-05-09T08:05:16.867 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:05:16.867 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:05:16.867 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:05:16.867 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:05:16.867 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6789/0", 2024-05-09T08:05:16.867 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.1:6789/0", 2024-05-09T08:05:16.867 INFO:teuthology.orchestra.run.smithi001.stdout: "priority": 0, 2024-05-09T08:05:16.868 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 0, 2024-05-09T08:05:16.868 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_location": "{}" 2024-05-09T08:05:16.868 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:05:16.868 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:05:16.868 INFO:teuthology.orchestra.run.smithi001.stdout: "rank": 1, 2024-05-09T08:05:16.868 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "b", 2024-05-09T08:05:16.868 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-05-09T08:05:16.868 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-05-09T08:05:16.868 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:05:16.868 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-05-09T08:05:16.868 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:3300", 2024-05-09T08:05:16.868 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:05:16.869 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:05:16.869 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:05:16.869 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-05-09T08:05:16.869 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:6789", 2024-05-09T08:05:16.869 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:05:16.869 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:05:16.869 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:05:16.869 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:05:16.869 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:6789/0", 2024-05-09T08:05:16.869 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.7:6789/0", 2024-05-09T08:05:16.869 INFO:teuthology.orchestra.run.smithi001.stdout: "priority": 0, 2024-05-09T08:05:16.869 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 0, 2024-05-09T08:05:16.869 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_location": "{}" 2024-05-09T08:05:16.869 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:05:16.870 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:05:16.870 INFO:teuthology.orchestra.run.smithi001.stdout: "rank": 2, 2024-05-09T08:05:16.870 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "c", 2024-05-09T08:05:16.870 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-05-09T08:05:16.870 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-05-09T08:05:16.870 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:05:16.870 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-05-09T08:05:16.870 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:3301", 2024-05-09T08:05:16.870 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:05:16.870 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:05:16.870 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:05:16.870 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-05-09T08:05:16.871 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:6790", 2024-05-09T08:05:16.871 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:05:16.871 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:05:16.871 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:05:16.871 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:05:16.871 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:6790/0", 2024-05-09T08:05:16.871 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.7:6790/0", 2024-05-09T08:05:16.871 INFO:teuthology.orchestra.run.smithi001.stdout: "priority": 0, 2024-05-09T08:05:16.871 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 0, 2024-05-09T08:05:16.871 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_location": "{}" 2024-05-09T08:05:16.871 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:05:16.871 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:05:16.871 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:05:16.872 INFO:teuthology.orchestra.run.smithi001.stdout: "feature_map": { 2024-05-09T08:05:16.872 INFO:teuthology.orchestra.run.smithi001.stdout: "mon": [ 2024-05-09T08:05:16.872 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:05:16.872 INFO:teuthology.orchestra.run.smithi001.stdout: "features": "0x3f03cffffffdffff", 2024-05-09T08:05:16.872 INFO:teuthology.orchestra.run.smithi001.stdout: "release": "luminous", 2024-05-09T08:05:16.872 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 1 2024-05-09T08:05:16.872 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:05:16.872 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:05:16.872 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": [ 2024-05-09T08:05:16.872 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:05:16.872 INFO:teuthology.orchestra.run.smithi001.stdout: "features": "0x3f03cffffffdffff", 2024-05-09T08:05:16.872 INFO:teuthology.orchestra.run.smithi001.stdout: "release": "luminous", 2024-05-09T08:05:16.873 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 1 2024-05-09T08:05:16.873 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:05:16.873 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:05:16.873 INFO:teuthology.orchestra.run.smithi001.stdout: "client": [ 2024-05-09T08:05:16.873 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:05:16.873 INFO:teuthology.orchestra.run.smithi001.stdout: "features": "0x3f03cffffffdffff", 2024-05-09T08:05:16.873 INFO:teuthology.orchestra.run.smithi001.stdout: "release": "luminous", 2024-05-09T08:05:16.873 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 1 2024-05-09T08:05:16.873 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:05:16.873 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:05:16.873 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:05:16.873 INFO:teuthology.orchestra.run.smithi001.stdout: "stretch_mode": false 2024-05-09T08:05:16.873 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-05-09T08:05:16.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:16.841+0000 7f9022800640 1 -- 172.21.15.1:0/1988855491 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f8ffc06bde0 msgr2=0x7f8ffc06e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:05:16.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:16.841+0000 7f9022800640 1 --2- 172.21.15.1:0/1988855491 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f8ffc06bde0 0x7f8ffc06e2a0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f9008034000 tx=0x7f900802e000 comp rx=0 tx=0).stop 2024-05-09T08:05:16.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:16.841+0000 7f9022800640 1 -- 172.21.15.1:0/1988855491 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f901c0ccd70 msgr2=0x7f901c1518b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:05:16.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:16.841+0000 7f9022800640 1 --2- 172.21.15.1:0/1988855491 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f901c0ccd70 0x7f901c1518b0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f901400be90 tx=0x7f901400bf80 comp rx=0 tx=0).stop 2024-05-09T08:05:16.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:16.841+0000 7f9021e00640 1 -- 172.21.15.1:0/1988855491 reap_dead start 2024-05-09T08:05:16.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:16.841+0000 7f9021e00640 1 -- 172.21.15.1:0/1988855491 reap_dead start 2024-05-09T08:05:16.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:16.841+0000 7f9022800640 1 -- 172.21.15.1:0/1988855491 shutdown_connections 2024-05-09T08:05:16.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:16.841+0000 7f9022800640 1 -- 172.21.15.1:0/1988855491 >> 172.21.15.1:0/1988855491 conn(0x7f901c0b1790 msgr2=0x7f901c0cda20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:05:16.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:16.842+0000 7f9022800640 1 -- 172.21.15.1:0/1988855491 shutdown_connections 2024-05-09T08:05:16.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:16.842+0000 7f9022800640 1 -- 172.21.15.1:0/1988855491 wait complete. 2024-05-09T08:05:16.876 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-05-09T08:05:16.876 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon c 2024-05-09T08:05:16.877 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-05-09T08:05:17.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:17.071+0000 7f2a3f200640 1 Processor -- start 2024-05-09T08:05:17.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:17.071+0000 7f2a3f200640 1 -- start start 2024-05-09T08:05:17.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:17.072+0000 7f2a3f200640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f2a400cac40 0x7f2a4006da40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:05:17.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:17.072+0000 7f2a3f200640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f2a400cd2a0 0x7f2a4015ba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:05:17.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:17.072+0000 7f2a3f200640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a400cb020 con 0x7f2a4015a610 2024-05-09T08:05:17.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:17.072+0000 7f2a3f200640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f2a40154dd0 con 0x7f2a400cd2a0 2024-05-09T08:05:17.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:17.072+0000 7f2a3f200640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f2a400cce10 con 0x7f2a400cac40 2024-05-09T08:05:17.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:17.072+0000 7f2a35400640 1 -- >> v2:172.21.15.1:3300/0 conn(0x7f2a400cd2a0 msgr2=0x7f2a4015ba80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.1:3300/0 2024-05-09T08:05:17.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:17.073+0000 7f2a35400640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f2a400cd2a0 0x7f2a4015ba80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T08:05:17.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:17.073+0000 7f2a3de00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f2a400cac40 0x7f2a4006da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:05:17.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:17.073+0000 7f2a3e800640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f2a4015a610 0x7f2a400cff10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.1:39100/0 (socket says 172.21.15.1:39100) 2024-05-09T08:05:17.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:17.073+0000 7f2a3e800640 1 -- 172.21.15.1:0/3284694831 learned_addr learned my addr 172.21.15.1:0/3284694831 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T08:05:17.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:17.073+0000 7f2a3d400640 1 -- 172.21.15.1:0/3284694831 <== mon.1 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1493996813 0 0) 0x7f2a400cb020 con 0x7f2a4015a610 2024-05-09T08:05:17.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:17.073+0000 7f2a3d400640 1 -- 172.21.15.1:0/3284694831 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2a24003520 con 0x7f2a4015a610 2024-05-09T08:05:17.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:17.073+0000 7f2a3de00640 1 -- 172.21.15.1:0/3284694831 >> v2:172.21.15.1:3300/0 conn(0x7f2a400cd2a0 msgr2=0x7f2a4015ba80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T08:05:17.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:17.073+0000 7f2a3de00640 1 --2- 172.21.15.1:0/3284694831 >> v2:172.21.15.1:3300/0 conn(0x7f2a400cd2a0 0x7f2a4015ba80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:05:17.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:17.073+0000 7f2a3de00640 1 -- 172.21.15.1:0/3284694831 >> v1:172.21.15.7:6789/0 conn(0x7f2a4015a610 legacy=0x7f2a400cff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:05:17.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:17.073+0000 7f2a3de00640 1 -- 172.21.15.1:0/3284694831 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a400cc550 con 0x7f2a400cac40 2024-05-09T08:05:17.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:17.075+0000 7f2a3de00640 1 --2- 172.21.15.1:0/3284694831 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2a400cac40 0x7f2a4006da40 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f2a400d0c90 tx=0x7f2a3002a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=741fe444e3090a4e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:05:17.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:17.075+0000 7f2a3d400640 1 -- 172.21.15.1:0/3284694831 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a30037070 con 0x7f2a400cac40 2024-05-09T08:05:17.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:17.075+0000 7f2a3d400640 1 -- 172.21.15.1:0/3284694831 <== mon.1 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a30032440 con 0x7f2a400cac40 2024-05-09T08:05:17.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:17.075+0000 7f2a3d400640 1 -- 172.21.15.1:0/3284694831 <== mon.1 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a300327f0 con 0x7f2a400cac40 2024-05-09T08:05:17.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:17.076+0000 7f2a3f200640 1 -- 172.21.15.1:0/3284694831 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2a400cac40 msgr2=0x7f2a4006da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:05:17.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:17.076+0000 7f2a3f200640 1 --2- 172.21.15.1:0/3284694831 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2a400cac40 0x7f2a4006da40 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f2a400d0c90 tx=0x7f2a3002a1c0 comp rx=0 tx=0).stop 2024-05-09T08:05:17.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:17.076+0000 7f2a3f200640 1 -- 172.21.15.1:0/3284694831 shutdown_connections 2024-05-09T08:05:17.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:17.076+0000 7f2a3f200640 1 --2- 172.21.15.1:0/3284694831 >> v2:172.21.15.1:3300/0 conn(0x7f2a400cd2a0 0x7f2a4015ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:05:17.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:17.076+0000 7f2a3f200640 1 --2- 172.21.15.1:0/3284694831 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2a400cac40 0x7f2a4006da40 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:05:17.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:17.076+0000 7f2a3f200640 1 -- 172.21.15.1:0/3284694831 >> 172.21.15.1:0/3284694831 conn(0x7f2a400670b0 msgr2=0x7f2a400674c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:05:17.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:17.076+0000 7f2a3f200640 1 -- 172.21.15.1:0/3284694831 shutdown_connections 2024-05-09T08:05:17.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:17.076+0000 7f2a3f200640 1 -- 172.21.15.1:0/3284694831 wait complete. 2024-05-09T08:05:17.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:17.078+0000 7f2a3f200640 1 Processor -- start 2024-05-09T08:05:17.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:17.078+0000 7f2a3f200640 1 -- start start 2024-05-09T08:05:17.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:17.079+0000 7f2a3f200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2a400cac40 0x7f2a400c4520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:05:17.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:17.079+0000 7f2a3f200640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f2a4015a610 0x7f2a401f35f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:05:17.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:17.079+0000 7f2a3f200640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2a40152b20 0x7f2a40157200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:05:17.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:17.079+0000 7f2a3f200640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f2a400cc7c0 con 0x7f2a40152b20 2024-05-09T08:05:17.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:17.079+0000 7f2a3f200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2a4015b410 con 0x7f2a400cac40 2024-05-09T08:05:17.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:17.079+0000 7f2a3f200640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f2a400ccaa0 con 0x7f2a4015a610 2024-05-09T08:05:17.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:17.079+0000 7f2a3e800640 1 -- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2a40152b20 msgr2=0x7f2a40157200 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.1:3300/0 2024-05-09T08:05:17.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:17.079+0000 7f2a3e800640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2a40152b20 0x7f2a40157200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T08:05:17.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:17.079+0000 7f2a35400640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f2a4015a610 0x7f2a401f35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:05:17.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:17.079+0000 7f2a3de00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2a400cac40 0x7f2a400c4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:05:17.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:17.079+0000 7f2a35400640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f2a4015a610 0x7f2a401f35f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3301/0 says I am v2:172.21.15.1:39724/0 (socket says 172.21.15.1:39724) 2024-05-09T08:05:17.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:17.079+0000 7f2a35400640 1 -- 172.21.15.1:0/3840490138 learned_addr learned my addr 172.21.15.1:0/3840490138 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:05:17.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:17.080+0000 7f2a35400640 1 -- 172.21.15.1:0/3840490138 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2a400cac40 msgr2=0x7f2a400c4520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:05:17.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:17.080+0000 7f2a35400640 1 --2- 172.21.15.1:0/3840490138 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2a400cac40 0x7f2a400c4520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:05:17.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:17.080+0000 7f2a35400640 1 -- 172.21.15.1:0/3840490138 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2a40152b20 msgr2=0x7f2a40157200 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T08:05:17.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:17.080+0000 7f2a35400640 1 --2- 172.21.15.1:0/3840490138 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2a40152b20 0x7f2a40157200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:05:17.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:17.080+0000 7f2a35400640 1 -- 172.21.15.1:0/3840490138 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a400c9400 con 0x7f2a4015a610 2024-05-09T08:05:17.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:17.080+0000 7f2a35400640 1 --2- 172.21.15.1:0/3840490138 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f2a4015a610 0x7f2a401f35f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f2a2800b550 tx=0x7f2a2800ba10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:05:17.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:17.080+0000 7f2a36a00640 1 -- 172.21.15.1:0/3840490138 <== mon.2 v2:172.21.15.7:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a28013020 con 0x7f2a4015a610 2024-05-09T08:05:17.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:17.080+0000 7f2a36a00640 1 -- 172.21.15.1:0/3840490138 <== mon.2 v2:172.21.15.7:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a28003de0 con 0x7f2a4015a610 2024-05-09T08:05:17.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:17.080+0000 7f2a36a00640 1 -- 172.21.15.1:0/3840490138 <== mon.2 v2:172.21.15.7:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a280052b0 con 0x7f2a4015a610 2024-05-09T08:05:17.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:17.080+0000 7f2a3f200640 1 -- 172.21.15.1:0/3840490138 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a40150330 con 0x7f2a4015a610 2024-05-09T08:05:17.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:17.080+0000 7f2a3f200640 1 -- 172.21.15.1:0/3840490138 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a401508f0 con 0x7f2a4015a610 2024-05-09T08:05:17.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:17.081+0000 7f2a3f200640 1 --2- 172.21.15.1:0/3840490138 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f2a400cd2a0 0x7f2a40152020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:05:17.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:17.081+0000 7f2a3f200640 1 -- 172.21.15.1:0/3840490138 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f2a4004f660 con 0x7f2a400cd2a0 2024-05-09T08:05:17.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:17.081+0000 7f2a3de00640 1 --2- 172.21.15.1:0/3840490138 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f2a400cd2a0 0x7f2a40152020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:05:17.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:17.082+0000 7f2a3de00640 1 -- 172.21.15.1:0/3840490138 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2a4006e0d0 con 0x7f2a400cd2a0 2024-05-09T08:05:17.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:17.082+0000 7f2a36a00640 1 -- 172.21.15.1:0/3840490138 <== mon.2 v2:172.21.15.7:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2a28016020 con 0x7f2a4015a610 2024-05-09T08:05:17.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:17.082+0000 7f2a3de00640 1 --2- 172.21.15.1:0/3840490138 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f2a400cd2a0 0x7f2a40152020 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f2a400ccd80 tx=0x7f2a300046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:05:17.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:17.083+0000 7f2a36a00640 1 --2- 172.21.15.1:0/3840490138 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f2a1c06be30 0x7f2a1c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:05:17.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:17.083+0000 7f2a36a00640 1 -- 172.21.15.1:0/3840490138 <== mon.2 v2:172.21.15.7:3301/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7f2a2808afb0 con 0x7f2a4015a610 2024-05-09T08:05:17.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:17.084+0000 7f2a3e800640 1 --2- 172.21.15.1:0/3840490138 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f2a1c06be30 0x7f2a1c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:05:17.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:17.084+0000 7f2a36a00640 1 -- 172.21.15.1:0/3840490138 <== mon.2 v2:172.21.15.7:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a30042030 con 0x7f2a400cd2a0 2024-05-09T08:05:17.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:17.084+0000 7f2a36a00640 1 -- 172.21.15.1:0/3840490138 <== mon.2 v2:172.21.15.7:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f2a300093c0 con 0x7f2a400cd2a0 2024-05-09T08:05:17.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:17.084+0000 7f2a36a00640 1 -- 172.21.15.1:0/3840490138 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f2a400cd2a0 msgr2=0x7f2a40152020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:05:17.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:17.084+0000 7f2a36a00640 1 --2- 172.21.15.1:0/3840490138 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f2a400cd2a0 0x7f2a40152020 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f2a400ccd80 tx=0x7f2a300046c0 comp rx=0 tx=0).stop 2024-05-09T08:05:17.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:17.084+0000 7f2a36a00640 1 -- 172.21.15.1:0/3840490138 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f2a400cd2a0 msgr2=0x7f2a40152020 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-05-09T08:05:17.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:17.084+0000 7f2a36a00640 1 --2- 172.21.15.1:0/3840490138 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f2a400cd2a0 0x7f2a40152020 secure :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f2a400ccd80 tx=0x7f2a300046c0 comp rx=0 tx=0).stop 2024-05-09T08:05:17.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:17.084+0000 7f2a3e800640 1 --2- 172.21.15.1:0/3840490138 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f2a1c06be30 0x7f2a1c06e2f0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f2a38004b90 tx=0x7f2a38001670 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:05:17.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:17.091+0000 7f2a3f200640 1 --2- 172.21.15.1:0/3840490138 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f2a40152560 0x7f2a4014ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:05:17.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:17.091+0000 7f2a3f200640 1 -- 172.21.15.1:0/3840490138 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f2a40073840 con 0x7f2a40152560 2024-05-09T08:05:17.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:17.092+0000 7f2a3de00640 1 --2- 172.21.15.1:0/3840490138 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f2a40152560 0x7f2a4014ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:05:17.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:17.092+0000 7f2a3de00640 1 -- 172.21.15.1:0/3840490138 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f2a40158ba0 con 0x7f2a40152560 2024-05-09T08:05:17.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:17.092+0000 7f2a3de00640 1 --2- 172.21.15.1:0/3840490138 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f2a40152560 0x7f2a4014ed70 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f2a30000c00 tx=0x7f2a30008030 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:05:17.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:17.093+0000 7f2a36a00640 1 -- 172.21.15.1:0/3840490138 <== mon.2 v2:172.21.15.7:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a30042070 con 0x7f2a40152560 2024-05-09T08:05:17.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:17.093+0000 7f2a36a00640 1 -- 172.21.15.1:0/3840490138 <== mon.2 v2:172.21.15.7:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5177 (secure 0 0 0) 0x7f2a3003e660 con 0x7f2a40152560 2024-05-09T08:05:17.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:17.093+0000 7f2a36a00640 1 -- 172.21.15.1:0/3840490138 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f2a40152560 msgr2=0x7f2a4014ed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:05:17.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:17.093+0000 7f2a36a00640 1 --2- 172.21.15.1:0/3840490138 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f2a40152560 0x7f2a4014ed70 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f2a30000c00 tx=0x7f2a30008030 comp rx=0 tx=0).stop 2024-05-09T08:05:17.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:17.093+0000 7f2a36a00640 1 -- 172.21.15.1:0/3840490138 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f2a40152560 msgr2=0x7f2a4014ed70 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-05-09T08:05:17.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:17.093+0000 7f2a36a00640 1 --2- 172.21.15.1:0/3840490138 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f2a40152560 0x7f2a4014ed70 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:05:17.108 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-05-09T08:05:17.108 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "c", 2024-05-09T08:05:17.108 INFO:teuthology.orchestra.run.smithi001.stdout: "rank": 2, 2024-05-09T08:05:17.108 INFO:teuthology.orchestra.run.smithi001.stdout: "state": "peon", 2024-05-09T08:05:17.108 INFO:teuthology.orchestra.run.smithi001.stdout: "election_epoch": 22, 2024-05-09T08:05:17.108 INFO:teuthology.orchestra.run.smithi001.stdout: "uptime": 200509, 2024-05-09T08:05:17.108 INFO:teuthology.orchestra.run.smithi001.stdout: "quorum": [ 2024-05-09T08:05:17.108 INFO:teuthology.orchestra.run.smithi001.stdout: 1, 2024-05-09T08:05:17.108 INFO:teuthology.orchestra.run.smithi001.stdout: 2 2024-05-09T08:05:17.109 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:05:17.109 INFO:teuthology.orchestra.run.smithi001.stdout: "quorum_age": 0, 2024-05-09T08:05:17.109 INFO:teuthology.orchestra.run.smithi001.stdout: "features": { 2024-05-09T08:05:17.109 INFO:teuthology.orchestra.run.smithi001.stdout: "required_con": "2450521708007866372", 2024-05-09T08:05:17.109 INFO:teuthology.orchestra.run.smithi001.stdout: "required_mon": [ 2024-05-09T08:05:17.109 INFO:teuthology.orchestra.run.smithi001.stdout: "kraken", 2024-05-09T08:05:17.109 INFO:teuthology.orchestra.run.smithi001.stdout: "luminous", 2024-05-09T08:05:17.109 INFO:teuthology.orchestra.run.smithi001.stdout: "mimic", 2024-05-09T08:05:17.109 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap-prune", 2024-05-09T08:05:17.109 INFO:teuthology.orchestra.run.smithi001.stdout: "nautilus", 2024-05-09T08:05:17.109 INFO:teuthology.orchestra.run.smithi001.stdout: "octopus", 2024-05-09T08:05:17.109 INFO:teuthology.orchestra.run.smithi001.stdout: "pacific", 2024-05-09T08:05:17.110 INFO:teuthology.orchestra.run.smithi001.stdout: "elector-pinging", 2024-05-09T08:05:17.110 INFO:teuthology.orchestra.run.smithi001.stdout: "quincy", 2024-05-09T08:05:17.110 INFO:teuthology.orchestra.run.smithi001.stdout: "reef", 2024-05-09T08:05:17.110 INFO:teuthology.orchestra.run.smithi001.stdout: "squid" 2024-05-09T08:05:17.110 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:05:17.110 INFO:teuthology.orchestra.run.smithi001.stdout: "quorum_con": "4540701547738038271", 2024-05-09T08:05:17.110 INFO:teuthology.orchestra.run.smithi001.stdout: "quorum_mon": [ 2024-05-09T08:05:17.110 INFO:teuthology.orchestra.run.smithi001.stdout: "kraken", 2024-05-09T08:05:17.110 INFO:teuthology.orchestra.run.smithi001.stdout: "luminous", 2024-05-09T08:05:17.110 INFO:teuthology.orchestra.run.smithi001.stdout: "mimic", 2024-05-09T08:05:17.110 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap-prune", 2024-05-09T08:05:17.110 INFO:teuthology.orchestra.run.smithi001.stdout: "nautilus", 2024-05-09T08:05:17.111 INFO:teuthology.orchestra.run.smithi001.stdout: "octopus", 2024-05-09T08:05:17.111 INFO:teuthology.orchestra.run.smithi001.stdout: "pacific", 2024-05-09T08:05:17.111 INFO:teuthology.orchestra.run.smithi001.stdout: "elector-pinging", 2024-05-09T08:05:17.111 INFO:teuthology.orchestra.run.smithi001.stdout: "quincy", 2024-05-09T08:05:17.111 INFO:teuthology.orchestra.run.smithi001.stdout: "reef", 2024-05-09T08:05:17.111 INFO:teuthology.orchestra.run.smithi001.stdout: "squid" 2024-05-09T08:05:17.111 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:05:17.111 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:05:17.111 INFO:teuthology.orchestra.run.smithi001.stdout: "outside_quorum": [], 2024-05-09T08:05:17.111 INFO:teuthology.orchestra.run.smithi001.stdout: "extra_probe_peers": [], 2024-05-09T08:05:17.111 INFO:teuthology.orchestra.run.smithi001.stdout: "sync_provider": [], 2024-05-09T08:05:17.111 INFO:teuthology.orchestra.run.smithi001.stdout: "monmap": { 2024-05-09T08:05:17.111 INFO:teuthology.orchestra.run.smithi001.stdout: "epoch": 1, 2024-05-09T08:05:17.112 INFO:teuthology.orchestra.run.smithi001.stdout: "fsid": "54329ec8-3270-4043-9a82-c6cd7420aa7f", 2024-05-09T08:05:17.112 INFO:teuthology.orchestra.run.smithi001.stdout: "modified": "2024-05-09T08:01:21.819270Z", 2024-05-09T08:05:17.112 INFO:teuthology.orchestra.run.smithi001.stdout: "created": "2024-05-09T08:01:21.819270Z", 2024-05-09T08:05:17.112 INFO:teuthology.orchestra.run.smithi001.stdout: "min_mon_release": 19, 2024-05-09T08:05:17.112 INFO:teuthology.orchestra.run.smithi001.stdout: "min_mon_release_name": "squid", 2024-05-09T08:05:17.112 INFO:teuthology.orchestra.run.smithi001.stdout: "election_strategy": 1, 2024-05-09T08:05:17.112 INFO:teuthology.orchestra.run.smithi001.stdout: "disallowed_leaders: ": "", 2024-05-09T08:05:17.112 INFO:teuthology.orchestra.run.smithi001.stdout: "stretch_mode": false, 2024-05-09T08:05:17.112 INFO:teuthology.orchestra.run.smithi001.stdout: "tiebreaker_mon": "", 2024-05-09T08:05:17.112 INFO:teuthology.orchestra.run.smithi001.stdout: "removed_ranks: ": "", 2024-05-09T08:05:17.112 INFO:teuthology.orchestra.run.smithi001.stdout: "features": { 2024-05-09T08:05:17.112 INFO:teuthology.orchestra.run.smithi001.stdout: "persistent": [ 2024-05-09T08:05:17.113 INFO:teuthology.orchestra.run.smithi001.stdout: "kraken", 2024-05-09T08:05:17.113 INFO:teuthology.orchestra.run.smithi001.stdout: "luminous", 2024-05-09T08:05:17.113 INFO:teuthology.orchestra.run.smithi001.stdout: "mimic", 2024-05-09T08:05:17.113 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap-prune", 2024-05-09T08:05:17.113 INFO:teuthology.orchestra.run.smithi001.stdout: "nautilus", 2024-05-09T08:05:17.113 INFO:teuthology.orchestra.run.smithi001.stdout: "octopus", 2024-05-09T08:05:17.113 INFO:teuthology.orchestra.run.smithi001.stdout: "pacific", 2024-05-09T08:05:17.113 INFO:teuthology.orchestra.run.smithi001.stdout: "elector-pinging", 2024-05-09T08:05:17.113 INFO:teuthology.orchestra.run.smithi001.stdout: "quincy", 2024-05-09T08:05:17.113 INFO:teuthology.orchestra.run.smithi001.stdout: "reef", 2024-05-09T08:05:17.113 INFO:teuthology.orchestra.run.smithi001.stdout: "squid" 2024-05-09T08:05:17.113 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:05:17.114 INFO:teuthology.orchestra.run.smithi001.stdout: "optional": [] 2024-05-09T08:05:17.114 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:05:17.114 INFO:teuthology.orchestra.run.smithi001.stdout: "mons": [ 2024-05-09T08:05:17.114 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:05:17.114 INFO:teuthology.orchestra.run.smithi001.stdout: "rank": 0, 2024-05-09T08:05:17.114 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "a", 2024-05-09T08:05:17.114 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-05-09T08:05:17.114 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-05-09T08:05:17.114 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:05:17.114 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-05-09T08:05:17.114 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:3300", 2024-05-09T08:05:17.114 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:05:17.115 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:05:17.115 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:05:17.115 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-05-09T08:05:17.115 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6789", 2024-05-09T08:05:17.115 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:05:17.115 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:05:17.115 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:05:17.115 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:05:17.115 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6789/0", 2024-05-09T08:05:17.115 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.1:6789/0", 2024-05-09T08:05:17.115 INFO:teuthology.orchestra.run.smithi001.stdout: "priority": 0, 2024-05-09T08:05:17.115 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 0, 2024-05-09T08:05:17.115 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_location": "{}" 2024-05-09T08:05:17.116 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:05:17.116 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:05:17.116 INFO:teuthology.orchestra.run.smithi001.stdout: "rank": 1, 2024-05-09T08:05:17.116 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "b", 2024-05-09T08:05:17.116 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-05-09T08:05:17.116 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-05-09T08:05:17.116 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:05:17.116 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-05-09T08:05:17.116 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:3300", 2024-05-09T08:05:17.116 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:05:17.116 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:05:17.116 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:05:17.117 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-05-09T08:05:17.117 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:6789", 2024-05-09T08:05:17.117 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:05:17.117 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:05:17.117 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:05:17.117 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:05:17.117 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:6789/0", 2024-05-09T08:05:17.117 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.7:6789/0", 2024-05-09T08:05:17.117 INFO:teuthology.orchestra.run.smithi001.stdout: "priority": 0, 2024-05-09T08:05:17.117 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 0, 2024-05-09T08:05:17.117 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_location": "{}" 2024-05-09T08:05:17.117 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:05:17.118 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:05:17.118 INFO:teuthology.orchestra.run.smithi001.stdout: "rank": 2, 2024-05-09T08:05:17.118 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "c", 2024-05-09T08:05:17.118 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-05-09T08:05:17.118 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-05-09T08:05:17.118 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:05:17.118 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-05-09T08:05:17.118 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:3301", 2024-05-09T08:05:17.118 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:05:17.118 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:05:17.118 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:05:17.119 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-05-09T08:05:17.119 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:6790", 2024-05-09T08:05:17.119 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:05:17.119 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:05:17.119 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:05:17.119 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:05:17.119 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:6790/0", 2024-05-09T08:05:17.119 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.7:6790/0", 2024-05-09T08:05:17.119 INFO:teuthology.orchestra.run.smithi001.stdout: "priority": 0, 2024-05-09T08:05:17.119 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 0, 2024-05-09T08:05:17.119 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_location": "{}" 2024-05-09T08:05:17.120 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:05:17.120 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:05:17.120 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:05:17.120 INFO:teuthology.orchestra.run.smithi001.stdout: "feature_map": { 2024-05-09T08:05:17.120 INFO:teuthology.orchestra.run.smithi001.stdout: "mon": [ 2024-05-09T08:05:17.120 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:05:17.120 INFO:teuthology.orchestra.run.smithi001.stdout: "features": "0x3f03cffffffdffff", 2024-05-09T08:05:17.120 INFO:teuthology.orchestra.run.smithi001.stdout: "release": "luminous", 2024-05-09T08:05:17.121 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 1 2024-05-09T08:05:17.121 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:05:17.121 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:05:17.121 INFO:teuthology.orchestra.run.smithi001.stdout: "mds": [ 2024-05-09T08:05:17.121 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:05:17.121 INFO:teuthology.orchestra.run.smithi001.stdout: "features": "0x3f03cffffffdffff", 2024-05-09T08:05:17.121 INFO:teuthology.orchestra.run.smithi001.stdout: "release": "luminous", 2024-05-09T08:05:17.121 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 6 2024-05-09T08:05:17.121 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:05:17.121 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:05:17.121 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": [ 2024-05-09T08:05:17.122 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:05:17.122 INFO:teuthology.orchestra.run.smithi001.stdout: "features": "0x3f03cffffffdffff", 2024-05-09T08:05:17.122 INFO:teuthology.orchestra.run.smithi001.stdout: "release": "luminous", 2024-05-09T08:05:17.122 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 7 2024-05-09T08:05:17.122 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:05:17.122 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:05:17.122 INFO:teuthology.orchestra.run.smithi001.stdout: "client": [ 2024-05-09T08:05:17.122 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:05:17.122 INFO:teuthology.orchestra.run.smithi001.stdout: "features": "0x2f018fb87aa4aafe", 2024-05-09T08:05:17.122 INFO:teuthology.orchestra.run.smithi001.stdout: "release": "luminous", 2024-05-09T08:05:17.123 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 1 2024-05-09T08:05:17.123 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:05:17.123 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:05:17.123 INFO:teuthology.orchestra.run.smithi001.stdout: "features": "0x3f03cffffffdffff", 2024-05-09T08:05:17.123 INFO:teuthology.orchestra.run.smithi001.stdout: "release": "luminous", 2024-05-09T08:05:17.123 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 6 2024-05-09T08:05:17.123 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:05:17.123 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:05:17.123 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr": [ 2024-05-09T08:05:17.123 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:05:17.124 INFO:teuthology.orchestra.run.smithi001.stdout: "features": "0x3f03cffffffdffff", 2024-05-09T08:05:17.124 INFO:teuthology.orchestra.run.smithi001.stdout: "release": "luminous", 2024-05-09T08:05:17.124 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 2 2024-05-09T08:05:17.124 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:05:17.124 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:05:17.124 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:05:17.124 INFO:teuthology.orchestra.run.smithi001.stdout: "stretch_mode": false 2024-05-09T08:05:17.124 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-05-09T08:05:17.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:17.094+0000 7f2a3f200640 1 -- 172.21.15.1:0/3840490138 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f2a1c06be30 msgr2=0x7f2a1c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:05:17.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:17.094+0000 7f2a3f200640 1 --2- 172.21.15.1:0/3840490138 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f2a1c06be30 0x7f2a1c06e2f0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f2a38004b90 tx=0x7f2a38001670 comp rx=0 tx=0).stop 2024-05-09T08:05:17.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:17.094+0000 7f2a3f200640 1 -- 172.21.15.1:0/3840490138 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f2a4015a610 msgr2=0x7f2a401f35f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:05:17.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:17.094+0000 7f2a3f200640 1 --2- 172.21.15.1:0/3840490138 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f2a4015a610 0x7f2a401f35f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f2a2800b550 tx=0x7f2a2800ba10 comp rx=0 tx=0).stop 2024-05-09T08:05:17.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:17.094+0000 7f2a3e800640 1 -- 172.21.15.1:0/3840490138 reap_dead start 2024-05-09T08:05:17.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:17.094+0000 7f2a3e800640 1 -- 172.21.15.1:0/3840490138 reap_dead start 2024-05-09T08:05:17.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:17.094+0000 7f2a3f200640 1 -- 172.21.15.1:0/3840490138 shutdown_connections 2024-05-09T08:05:17.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:17.094+0000 7f2a3f200640 1 -- 172.21.15.1:0/3840490138 >> 172.21.15.1:0/3840490138 conn(0x7f2a400670b0 msgr2=0x7f2a400ce530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:05:17.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:17.094+0000 7f2a3f200640 1 -- 172.21.15.1:0/3840490138 shutdown_connections 2024-05-09T08:05:17.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:17.094+0000 7f2a3f200640 1 -- 172.21.15.1:0/3840490138 wait complete. 2024-05-09T08:05:17.127 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-05-09T08:05:17.127 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2024-05-09T08:05:17.607 INFO:tasks.workunit.client.0.smithi001.stdout: 2024-05-09T08:05:17.607 INFO:tasks.workunit.client.0.smithi001.stdout: Children see throughput for 1 readers = 4927839.50 kB/sec 2024-05-09T08:05:17.607 INFO:tasks.workunit.client.0.smithi001.stdout: Parent sees throughput for 1 readers = 4908239.31 kB/sec 2024-05-09T08:05:17.607 INFO:tasks.workunit.client.0.smithi001.stdout: Min throughput per process = 4927839.50 kB/sec 2024-05-09T08:05:17.607 INFO:tasks.workunit.client.0.smithi001.stdout: Max throughput per process = 4927839.50 kB/sec 2024-05-09T08:05:17.607 INFO:tasks.workunit.client.0.smithi001.stdout: Avg throughput per process = 4927839.50 kB/sec 2024-05-09T08:05:17.607 INFO:tasks.workunit.client.0.smithi001.stdout: Min xfer = 1048576.00 kB 2024-05-09T08:05:20.008 INFO:tasks.workunit.client.0.smithi001.stdout: 2024-05-09T08:05:20.009 INFO:tasks.workunit.client.0.smithi001.stdout: Children see throughput for 1 re-readers = 3343960.75 kB/sec 2024-05-09T08:05:20.009 INFO:tasks.workunit.client.0.smithi001.stdout: Parent sees throughput for 1 re-readers = 3334665.64 kB/sec 2024-05-09T08:05:20.009 INFO:tasks.workunit.client.0.smithi001.stdout: Min throughput per process = 3343960.75 kB/sec 2024-05-09T08:05:20.009 INFO:tasks.workunit.client.0.smithi001.stdout: Max throughput per process = 3343960.75 kB/sec 2024-05-09T08:05:20.009 INFO:tasks.workunit.client.0.smithi001.stdout: Avg throughput per process = 3343960.75 kB/sec 2024-05-09T08:05:20.009 INFO:tasks.workunit.client.0.smithi001.stdout: Min xfer = 1048576.00 kB 2024-05-09T08:05:20.716 DEBUG:teuthology.orchestra.run.smithi001:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T08:05:20.725 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T08:05:26.487 INFO:tasks.workunit.client.0.smithi001.stdout: 2024-05-09T08:05:26.487 INFO:tasks.workunit.client.0.smithi001.stdout: 2024-05-09T08:05:26.487 INFO:tasks.workunit.client.0.smithi001.stdout: 2024-05-09T08:05:26.487 INFO:tasks.workunit.client.0.smithi001.stdout:iozone test complete. 2024-05-09T08:05:26.490 INFO:tasks.workunit.client.0.smithi001.stderr:+ iozone -c -e -s 10240M -r 1M -t 1 -F f3 -i 0 -i 1 2024-05-09T08:05:26.587 INFO:tasks.workunit.client.0.smithi001.stdout: Iozone: Performance Test of File I/O 2024-05-09T08:05:26.587 INFO:tasks.workunit.client.0.smithi001.stdout: Version $Revision: 3.494 $ 2024-05-09T08:05:26.587 INFO:tasks.workunit.client.0.smithi001.stdout: Compiled for 64 bit mode. 2024-05-09T08:05:26.587 INFO:tasks.workunit.client.0.smithi001.stdout: Build: linux-AMD64 2024-05-09T08:05:26.587 INFO:tasks.workunit.client.0.smithi001.stdout: 2024-05-09T08:05:26.587 INFO:tasks.workunit.client.0.smithi001.stdout: Contributors:William Norcott, Don Capps, Isom Crawford, Kirby Collins 2024-05-09T08:05:26.587 INFO:tasks.workunit.client.0.smithi001.stdout: Al Slater, Scott Rhine, Mike Wisner, Ken Goss 2024-05-09T08:05:26.587 INFO:tasks.workunit.client.0.smithi001.stdout: Steve Landherr, Brad Smith, Mark Kelly, Dr. Alain CYR, 2024-05-09T08:05:26.587 INFO:tasks.workunit.client.0.smithi001.stdout: Randy Dunlap, Mark Montague, Dan Million, Gavin Brebner, 2024-05-09T08:05:26.587 INFO:tasks.workunit.client.0.smithi001.stdout: Jean-Marc Zucconi, Jeff Blomberg, Benny Halevy, Dave Boone, 2024-05-09T08:05:26.588 INFO:tasks.workunit.client.0.smithi001.stdout: Erik Habbinga, Kris Strecker, Walter Wong, Joshua Root, 2024-05-09T08:05:26.588 INFO:tasks.workunit.client.0.smithi001.stdout: Fabrice Bacchella, Zhenghua Xue, Qin Li, Darren Sawyer, 2024-05-09T08:05:26.588 INFO:tasks.workunit.client.0.smithi001.stdout: Vangel Bojaxhi, Ben England, Vikentsi Lapa, 2024-05-09T08:05:26.588 INFO:tasks.workunit.client.0.smithi001.stdout: Alexey Skidanov, Sudhir Kumar. 2024-05-09T08:05:26.588 INFO:tasks.workunit.client.0.smithi001.stdout: 2024-05-09T08:05:26.588 INFO:tasks.workunit.client.0.smithi001.stdout: Run began: Thu May 9 08:05:26 2024 2024-05-09T08:05:26.588 INFO:tasks.workunit.client.0.smithi001.stdout: 2024-05-09T08:05:26.588 INFO:tasks.workunit.client.0.smithi001.stdout: Include close in write timing 2024-05-09T08:05:26.588 INFO:tasks.workunit.client.0.smithi001.stdout: Include fsync in write timing 2024-05-09T08:05:26.588 INFO:tasks.workunit.client.0.smithi001.stdout: File size set to 10485760 kB 2024-05-09T08:05:26.588 INFO:tasks.workunit.client.0.smithi001.stdout: Record Size 1024 kB 2024-05-09T08:05:26.588 INFO:tasks.workunit.client.0.smithi001.stdout: Command line used: iozone -c -e -s 10240M -r 1M -t 1 -F f3 -i 0 -i 1 2024-05-09T08:05:26.589 INFO:tasks.workunit.client.0.smithi001.stdout: Output is in kBytes/sec 2024-05-09T08:05:26.589 INFO:tasks.workunit.client.0.smithi001.stdout: Time Resolution = 0.000001 seconds. 2024-05-09T08:05:26.589 INFO:tasks.workunit.client.0.smithi001.stdout: Processor cache size set to 1024 kBytes. 2024-05-09T08:05:26.589 INFO:tasks.workunit.client.0.smithi001.stdout: Processor cache line size set to 32 bytes. 2024-05-09T08:05:26.589 INFO:tasks.workunit.client.0.smithi001.stdout: File stride size set to 17 * record size. 2024-05-09T08:05:26.589 INFO:tasks.workunit.client.0.smithi001.stdout: Throughput test with 1 process 2024-05-09T08:05:26.589 INFO:tasks.workunit.client.0.smithi001.stdout: Each process writes a 10485760 kByte file in 1024 kByte records 2024-05-09T08:05:37.128 INFO:tasks.mon_thrash.mon_thrasher:killing mon.a 2024-05-09T08:05:37.128 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.a 2024-05-09T08:05:37.129 INFO:tasks.ceph.mon.a:Restarting daemon 2024-05-09T08:05:37.129 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-05-09T08:05:37.136 INFO:tasks.ceph.mon.a:Started 2024-05-09T08:05:37.136 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-05-09T08:05:37.137 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-05-09T08:05:37.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:37.440+0000 7f9aa5400640 1 Processor -- start 2024-05-09T08:05:37.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:37.440+0000 7f9aa5400640 1 -- start start 2024-05-09T08:05:37.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:37.443+0000 7f9aa5400640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f9aa00ccb90 0x7f9aa01566e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:05:37.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:37.443+0000 7f9aa5400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f9aa00c7310 0x7f9aa0072e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:05:37.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:37.443+0000 7f9aa5400640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9aa0073360 con 0x7f9aa0150640 2024-05-09T08:05:37.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:37.443+0000 7f9aa5400640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f9aa00c7770 con 0x7f9aa00c7310 2024-05-09T08:05:37.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:37.443+0000 7f9aa5400640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f9aa00c2930 con 0x7f9aa00ccb90 2024-05-09T08:05:37.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:37.444+0000 7f9a9f200640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f9aa0150640 0x7f9aa006e710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.1:34870/0 (socket says 172.21.15.1:34870) 2024-05-09T08:05:37.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:37.444+0000 7f9a9f200640 1 -- 172.21.15.1:0/3926441536 learned_addr learned my addr 172.21.15.1:0/3926441536 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T08:05:37.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:37.446+0000 7f9a9e800640 1 --2- 172.21.15.1:0/3926441536 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f9aa00ccb90 0x7f9aa01566e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:05:37.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:37.446+0000 7f9a9de00640 1 --2- 172.21.15.1:0/3926441536 >> v2:172.21.15.7:3300/0 conn(0x7f9aa00c7310 0x7f9aa0072e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:05:37.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:37.446+0000 7f9a9e800640 1 -- 172.21.15.1:0/3926441536 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f9aa00ccb90 msgr2=0x7f9aa01566e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-05-09T08:05:37.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:37.446+0000 7f9a9e800640 1 -- 172.21.15.1:0/3926441536 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f9aa00ccb90 msgr2=0x7f9aa01566e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-09T08:05:37.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:37.446+0000 7f9a9e800640 1 --2- 172.21.15.1:0/3926441536 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f9aa00ccb90 0x7f9aa01566e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-09T08:05:37.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:37.446+0000 7f9a9e800640 1 --2- 172.21.15.1:0/3926441536 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f9aa00ccb90 0x7f9aa01566e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T08:05:37.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:37.447+0000 7f9a9de00640 1 -- 172.21.15.1:0/3926441536 >> v2:172.21.15.7:3300/0 conn(0x7f9aa00c7310 msgr2=0x7f9aa0072e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-05-09T08:05:37.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:37.447+0000 7f9a9de00640 1 -- 172.21.15.1:0/3926441536 >> v2:172.21.15.7:3300/0 conn(0x7f9aa00c7310 msgr2=0x7f9aa0072e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-09T08:05:37.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:37.447+0000 7f9a9de00640 1 --2- 172.21.15.1:0/3926441536 >> v2:172.21.15.7:3300/0 conn(0x7f9aa00c7310 0x7f9aa0072e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-09T08:05:37.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:37.447+0000 7f9a9de00640 1 --2- 172.21.15.1:0/3926441536 >> v2:172.21.15.7:3300/0 conn(0x7f9aa00c7310 0x7f9aa0072e20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T08:05:37.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:37.648+0000 7f9a9e800640 1 --2- 172.21.15.1:0/3926441536 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f9aa00ccb90 0x7f9aa01566e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:05:37.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:37.648+0000 7f9a9de00640 1 --2- 172.21.15.1:0/3926441536 >> v2:172.21.15.7:3300/0 conn(0x7f9aa00c7310 0x7f9aa0072e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:05:37.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:37.648+0000 7f9a9e800640 1 -- 172.21.15.1:0/3926441536 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f9aa00ccb90 msgr2=0x7f9aa01566e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-05-09T08:05:37.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:37.648+0000 7f9a9e800640 1 -- 172.21.15.1:0/3926441536 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f9aa00ccb90 msgr2=0x7f9aa01566e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-09T08:05:37.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:37.648+0000 7f9a9e800640 1 --2- 172.21.15.1:0/3926441536 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f9aa00ccb90 0x7f9aa01566e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-09T08:05:37.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:37.648+0000 7f9a9e800640 1 --2- 172.21.15.1:0/3926441536 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f9aa00ccb90 0x7f9aa01566e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-09T08:05:37.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:37.648+0000 7f9a9de00640 1 -- 172.21.15.1:0/3926441536 >> v2:172.21.15.7:3300/0 conn(0x7f9aa00c7310 msgr2=0x7f9aa0072e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-05-09T08:05:37.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:37.648+0000 7f9a9de00640 1 -- 172.21.15.1:0/3926441536 >> v2:172.21.15.7:3300/0 conn(0x7f9aa00c7310 msgr2=0x7f9aa0072e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-09T08:05:37.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:37.648+0000 7f9a9de00640 1 --2- 172.21.15.1:0/3926441536 >> v2:172.21.15.7:3300/0 conn(0x7f9aa00c7310 0x7f9aa0072e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-09T08:05:37.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:37.648+0000 7f9a9de00640 1 --2- 172.21.15.1:0/3926441536 >> v2:172.21.15.7:3300/0 conn(0x7f9aa00c7310 0x7f9aa0072e20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-09T08:05:38.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:38.049+0000 7f9a9de00640 1 --2- 172.21.15.1:0/3926441536 >> v2:172.21.15.7:3300/0 conn(0x7f9aa00c7310 0x7f9aa0072e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:05:38.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:38.049+0000 7f9a9e800640 1 --2- 172.21.15.1:0/3926441536 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f9aa00ccb90 0x7f9aa01566e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:05:38.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:38.049+0000 7f9a9de00640 1 -- 172.21.15.1:0/3926441536 >> v2:172.21.15.7:3300/0 conn(0x7f9aa00c7310 msgr2=0x7f9aa0072e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-05-09T08:05:38.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:38.049+0000 7f9a9de00640 1 -- 172.21.15.1:0/3926441536 >> v2:172.21.15.7:3300/0 conn(0x7f9aa00c7310 msgr2=0x7f9aa0072e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-09T08:05:38.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:38.049+0000 7f9a9de00640 1 --2- 172.21.15.1:0/3926441536 >> v2:172.21.15.7:3300/0 conn(0x7f9aa00c7310 0x7f9aa0072e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-09T08:05:38.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:38.049+0000 7f9a9de00640 1 --2- 172.21.15.1:0/3926441536 >> v2:172.21.15.7:3300/0 conn(0x7f9aa00c7310 0x7f9aa0072e20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-09T08:05:38.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:38.049+0000 7f9a9e800640 1 -- 172.21.15.1:0/3926441536 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f9aa00ccb90 msgr2=0x7f9aa01566e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-05-09T08:05:38.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:38.049+0000 7f9a9e800640 1 -- 172.21.15.1:0/3926441536 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f9aa00ccb90 msgr2=0x7f9aa01566e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-09T08:05:38.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:38.049+0000 7f9a9e800640 1 --2- 172.21.15.1:0/3926441536 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f9aa00ccb90 0x7f9aa01566e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-09T08:05:38.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:38.049+0000 7f9a9e800640 1 --2- 172.21.15.1:0/3926441536 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f9aa00ccb90 0x7f9aa01566e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-09T08:05:38.863 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:38.850+0000 7f9a9e800640 1 --2- 172.21.15.1:0/3926441536 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f9aa00ccb90 0x7f9aa01566e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:05:38.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:38.850+0000 7f9a9e800640 1 -- 172.21.15.1:0/3926441536 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f9aa00ccb90 msgr2=0x7f9aa01566e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-05-09T08:05:38.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:38.850+0000 7f9a9e800640 1 -- 172.21.15.1:0/3926441536 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f9aa00ccb90 msgr2=0x7f9aa01566e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-09T08:05:38.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:38.850+0000 7f9a9e800640 1 --2- 172.21.15.1:0/3926441536 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f9aa00ccb90 0x7f9aa01566e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-09T08:05:38.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:38.850+0000 7f9a9e800640 1 --2- 172.21.15.1:0/3926441536 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f9aa00ccb90 0x7f9aa01566e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-09T08:05:38.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:38.850+0000 7f9a9de00640 1 --2- 172.21.15.1:0/3926441536 >> v2:172.21.15.7:3300/0 conn(0x7f9aa00c7310 0x7f9aa0072e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:05:38.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:38.851+0000 7f9a9de00640 1 -- 172.21.15.1:0/3926441536 >> v2:172.21.15.7:3300/0 conn(0x7f9aa00c7310 msgr2=0x7f9aa0072e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-05-09T08:05:38.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:38.851+0000 7f9a9de00640 1 -- 172.21.15.1:0/3926441536 >> v2:172.21.15.7:3300/0 conn(0x7f9aa00c7310 msgr2=0x7f9aa0072e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-09T08:05:38.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:38.851+0000 7f9a9de00640 1 --2- 172.21.15.1:0/3926441536 >> v2:172.21.15.7:3300/0 conn(0x7f9aa00c7310 0x7f9aa0072e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-09T08:05:38.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:38.851+0000 7f9a9de00640 1 --2- 172.21.15.1:0/3926441536 >> v2:172.21.15.7:3300/0 conn(0x7f9aa00c7310 0x7f9aa0072e20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-09T08:05:40.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:40.443+0000 7f9a97e00640 1 -- 172.21.15.1:0/3926441536 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f9aa00ccb90 msgr2=0x7f9aa01566e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T08:05:40.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:40.443+0000 7f9a97e00640 1 --2- 172.21.15.1:0/3926441536 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f9aa00ccb90 0x7f9aa01566e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:05:40.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:40.443+0000 7f9a97e00640 1 -- 172.21.15.1:0/3926441536 >> v2:172.21.15.7:3300/0 conn(0x7f9aa00c7310 msgr2=0x7f9aa0072e20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T08:05:40.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:40.443+0000 7f9a97e00640 1 --2- 172.21.15.1:0/3926441536 >> v2:172.21.15.7:3300/0 conn(0x7f9aa00c7310 0x7f9aa0072e20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:05:40.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:40.443+0000 7f9a97e00640 1 -- 172.21.15.1:0/3926441536 >> v1:172.21.15.7:6789/0 conn(0x7f9aa0150640 legacy=0x7f9aa006e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:05:40.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:40.443+0000 7f9a97e00640 1 --2- 172.21.15.1:0/3926441536 >> v2:172.21.15.1:3300/0 conn(0x7f9a800073e0 0x7f9a800098a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:05:40.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:40.443+0000 7f9a97e00640 1 -- 172.21.15.1:0/3926441536 --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9aa00c2c10 con 0x7f9a800036a0 2024-05-09T08:05:40.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:40.443+0000 7f9a97e00640 1 -- 172.21.15.1:0/3926441536 --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f9aa00c7770 con 0x7f9a800073e0 2024-05-09T08:05:40.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:40.443+0000 7f9a9de00640 1 --2- 172.21.15.1:0/3926441536 >> v2:172.21.15.1:3300/0 conn(0x7f9a800073e0 0x7f9a800098a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:05:40.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:40.444+0000 7f9a9de00640 1 -- 172.21.15.1:0/3926441536 >> v2:172.21.15.1:3300/0 conn(0x7f9a800073e0 msgr2=0x7f9a800098a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-05-09T08:05:40.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:40.444+0000 7f9a9de00640 1 -- 172.21.15.1:0/3926441536 >> v2:172.21.15.1:3300/0 conn(0x7f9a800073e0 msgr2=0x7f9a800098a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-09T08:05:40.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:40.444+0000 7f9a9de00640 1 --2- 172.21.15.1:0/3926441536 >> v2:172.21.15.1:3300/0 conn(0x7f9a800073e0 0x7f9a800098a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-09T08:05:40.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:40.444+0000 7f9a9de00640 1 --2- 172.21.15.1:0/3926441536 >> v2:172.21.15.1:3300/0 conn(0x7f9a800073e0 0x7f9a800098a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T08:05:40.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:40.646+0000 7f9a9de00640 1 --2- 172.21.15.1:0/3926441536 >> v2:172.21.15.1:3300/0 conn(0x7f9a800073e0 0x7f9a800098a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:05:40.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:40.646+0000 7f9a9de00640 1 -- 172.21.15.1:0/3926441536 >> v2:172.21.15.1:3300/0 conn(0x7f9a800073e0 msgr2=0x7f9a800098a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-05-09T08:05:40.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:40.646+0000 7f9a9de00640 1 -- 172.21.15.1:0/3926441536 >> v2:172.21.15.1:3300/0 conn(0x7f9a800073e0 msgr2=0x7f9a800098a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-09T08:05:40.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:40.646+0000 7f9a9de00640 1 --2- 172.21.15.1:0/3926441536 >> v2:172.21.15.1:3300/0 conn(0x7f9a800073e0 0x7f9a800098a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-09T08:05:40.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:40.646+0000 7f9a9de00640 1 --2- 172.21.15.1:0/3926441536 >> v2:172.21.15.1:3300/0 conn(0x7f9a800073e0 0x7f9a800098a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-09T08:05:41.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:41.047+0000 7f9a9de00640 1 --2- 172.21.15.1:0/3926441536 >> v2:172.21.15.1:3300/0 conn(0x7f9a800073e0 0x7f9a800098a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:05:41.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:41.047+0000 7f9a9de00640 1 -- 172.21.15.1:0/3926441536 >> v2:172.21.15.1:3300/0 conn(0x7f9a800073e0 msgr2=0x7f9a800098a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-05-09T08:05:41.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:41.047+0000 7f9a9de00640 1 -- 172.21.15.1:0/3926441536 >> v2:172.21.15.1:3300/0 conn(0x7f9a800073e0 msgr2=0x7f9a800098a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-09T08:05:41.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:41.047+0000 7f9a9de00640 1 --2- 172.21.15.1:0/3926441536 >> v2:172.21.15.1:3300/0 conn(0x7f9a800073e0 0x7f9a800098a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-09T08:05:41.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:41.047+0000 7f9a9de00640 1 --2- 172.21.15.1:0/3926441536 >> v2:172.21.15.1:3300/0 conn(0x7f9a800073e0 0x7f9a800098a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-09T08:05:41.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:41.848+0000 7f9a9de00640 1 --2- 172.21.15.1:0/3926441536 >> v2:172.21.15.1:3300/0 conn(0x7f9a800073e0 0x7f9a800098a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:05:41.862 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:41.848+0000 7f9a9de00640 1 -- 172.21.15.1:0/3926441536 >> v2:172.21.15.1:3300/0 conn(0x7f9a800073e0 msgr2=0x7f9a800098a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-05-09T08:05:41.862 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:41.848+0000 7f9a9de00640 1 -- 172.21.15.1:0/3926441536 >> v2:172.21.15.1:3300/0 conn(0x7f9a800073e0 msgr2=0x7f9a800098a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-09T08:05:41.862 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:41.848+0000 7f9a9de00640 1 --2- 172.21.15.1:0/3926441536 >> v2:172.21.15.1:3300/0 conn(0x7f9a800073e0 0x7f9a800098a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-09T08:05:41.862 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:41.848+0000 7f9a9de00640 1 --2- 172.21.15.1:0/3926441536 >> v2:172.21.15.1:3300/0 conn(0x7f9a800073e0 0x7f9a800098a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-09T08:05:43.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:43.443+0000 7f9a97e00640 1 -- 172.21.15.1:0/3926441536 >> v2:172.21.15.1:3300/0 conn(0x7f9a800073e0 msgr2=0x7f9a800098a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T08:05:43.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:43.443+0000 7f9a97e00640 1 --2- 172.21.15.1:0/3926441536 >> v2:172.21.15.1:3300/0 conn(0x7f9a800073e0 0x7f9a800098a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:05:43.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:43.444+0000 7f9a97e00640 1 -- 172.21.15.1:0/3926441536 >> v1:172.21.15.1:6789/0 conn(0x7f9a800036a0 legacy=0x7f9a80005b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:05:43.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:43.444+0000 7f9a97e00640 1 --2- 172.21.15.1:0/3926441536 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f9aa00ccb90 0x7f9a8000a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:05:43.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:43.444+0000 7f9a97e00640 1 -- 172.21.15.1:0/3926441536 --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9a8000b040 con 0x7f9a800103a0 2024-05-09T08:05:43.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:43.444+0000 7f9a97e00640 1 -- 172.21.15.1:0/3926441536 --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9a80014f50 con 0x7f9aa0150640 2024-05-09T08:05:43.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:43.444+0000 7f9a97e00640 1 -- 172.21.15.1:0/3926441536 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f9aa00c7770 con 0x7f9aa00ccb90 2024-05-09T08:05:43.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:43.446+0000 7f9a9f200640 1 -- 172.21.15.1:0/3926441536 reap_dead start 2024-05-09T08:05:43.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:43.447+0000 7f9a9e800640 1 --2- 172.21.15.1:0/3926441536 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f9aa00ccb90 0x7f9a8000a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:05:43.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:43.447+0000 7f9a9e800640 1 -- 172.21.15.1:0/3926441536 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f9aa00ccb90 msgr2=0x7f9a8000a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-09T08:05:43.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:43.447+0000 7f9a9e800640 1 -- 172.21.15.1:0/3926441536 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f9aa00ccb90 msgr2=0x7f9a8000a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-09T08:05:43.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:43.447+0000 7f9a9e800640 1 --2- 172.21.15.1:0/3926441536 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f9aa00ccb90 0x7f9a8000a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-09T08:05:43.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:43.447+0000 7f9a9e800640 1 --2- 172.21.15.1:0/3926441536 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f9aa00ccb90 0x7f9a8000a060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T08:05:43.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:43.627+0000 7f9a9d400640 1 -- 172.21.15.1:0/3926441536 <== mon.1 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4230448395 0 0) 0x7f9a80014f50 con 0x7f9aa0150640 2024-05-09T08:05:43.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:43.627+0000 7f9a9d400640 1 -- 172.21.15.1:0/3926441536 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9a84003520 con 0x7f9aa0150640 2024-05-09T08:05:43.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:43.628+0000 7f9a9d400640 1 -- 172.21.15.1:0/3926441536 <== mon.1 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1918910375 0 0) 0x7f9a84003520 con 0x7f9aa0150640 2024-05-09T08:05:43.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:43.628+0000 7f9a9d400640 1 -- 172.21.15.1:0/3926441536 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f9aa00ccb90 msgr2=0x7f9a8000a060 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T08:05:43.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:43.628+0000 7f9a9d400640 1 --2- 172.21.15.1:0/3926441536 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f9aa00ccb90 0x7f9a8000a060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:05:43.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:43.628+0000 7f9a9d400640 1 -- 172.21.15.1:0/3926441536 >> v1:172.21.15.1:6789/0 conn(0x7f9a800103a0 legacy=0x7f9a80012790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:05:43.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:43.628+0000 7f9a9d400640 1 -- 172.21.15.1:0/3926441536 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a800161d0 con 0x7f9aa0150640 2024-05-09T08:05:43.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:43.628+0000 7f9a9d400640 1 -- 172.21.15.1:0/3926441536 <== mon.1 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4179846320 0 0) 0x7f9a90015580 con 0x7f9aa0150640 2024-05-09T08:05:43.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:43.628+0000 7f9a9d400640 1 -- 172.21.15.1:0/3926441536 >> v1:172.21.15.7:6789/0 conn(0x7f9aa0150640 legacy=0x7f9a8000a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:05:43.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:43.628+0000 7f9a9d400640 1 --2- 172.21.15.1:0/3926441536 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a840039b0 0x7f9a84003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:05:43.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:43.628+0000 7f9a9d400640 1 --2- 172.21.15.1:0/3926441536 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f9aa00ccb90 0x7f9a84004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:05:43.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:43.628+0000 7f9a9d400640 1 --2- 172.21.15.1:0/3926441536 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9a84004d40 0x7f9a840051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:05:43.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:43.628+0000 7f9a9d400640 1 -- 172.21.15.1:0/3926441536 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f9aa00c7770 con 0x7f9a84004d40 2024-05-09T08:05:43.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:43.628+0000 7f9a9d400640 1 -- 172.21.15.1:0/3926441536 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9a8400bbd0 con 0x7f9a840039b0 2024-05-09T08:05:43.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:43.628+0000 7f9a9d400640 1 -- 172.21.15.1:0/3926441536 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f9a8400bdd0 con 0x7f9aa00ccb90 2024-05-09T08:05:43.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:43.628+0000 7f9a9e800640 1 --2- 172.21.15.1:0/3926441536 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a840039b0 0x7f9a84003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:05:43.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:43.628+0000 7f9a9de00640 1 --2- 172.21.15.1:0/3926441536 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f9aa00ccb90 0x7f9a84004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:05:43.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:43.628+0000 7f9a9f200640 1 --2- 172.21.15.1:0/3926441536 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9a84004d40 0x7f9a840051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:05:43.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:43.629+0000 7f9a9e800640 1 -- 172.21.15.1:0/3926441536 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f9aa00ccb90 msgr2=0x7f9a84004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:05:43.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:43.629+0000 7f9a9e800640 1 --2- 172.21.15.1:0/3926441536 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f9aa00ccb90 0x7f9a84004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:05:43.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:43.629+0000 7f9a9e800640 1 -- 172.21.15.1:0/3926441536 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9a84004d40 msgr2=0x7f9a840051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:05:43.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:43.629+0000 7f9a9e800640 1 --2- 172.21.15.1:0/3926441536 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9a84004d40 0x7f9a840051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:05:43.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:43.629+0000 7f9a9e800640 1 -- 172.21.15.1:0/3926441536 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a84003520 con 0x7f9a840039b0 2024-05-09T08:05:43.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:43.630+0000 7f9a9de00640 1 --2- 172.21.15.1:0/3926441536 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f9aa00ccb90 0x7f9a84004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:05:43.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:43.630+0000 7f9a9e800640 1 --2- 172.21.15.1:0/3926441536 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a840039b0 0x7f9a84003dd0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9a80007260 tx=0x7f9a84005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7d3f83c6b4ccb53e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:05:43.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:43.630+0000 7f9a9d400640 1 -- 172.21.15.1:0/3926441536 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a8c013880 con 0x7f9a840039b0 2024-05-09T08:05:43.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:43.630+0000 7f9a9d400640 1 -- 172.21.15.1:0/3926441536 <== mon.1 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a8c004030 con 0x7f9a840039b0 2024-05-09T08:05:43.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:43.630+0000 7f9a9d400640 1 -- 172.21.15.1:0/3926441536 <== mon.1 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a8c004400 con 0x7f9a840039b0 2024-05-09T08:05:43.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:43.631+0000 7f9aa5400640 1 -- 172.21.15.1:0/3926441536 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a840039b0 msgr2=0x7f9a84003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:05:43.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:43.631+0000 7f9aa5400640 1 --2- 172.21.15.1:0/3926441536 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a840039b0 0x7f9a84003dd0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9a80007260 tx=0x7f9a84005830 comp rx=0 tx=0).stop 2024-05-09T08:05:43.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:43.631+0000 7f9a9f200640 1 -- 172.21.15.1:0/3926441536 reap_dead start 2024-05-09T08:05:43.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:43.631+0000 7f9aa5400640 1 -- 172.21.15.1:0/3926441536 shutdown_connections 2024-05-09T08:05:43.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:43.631+0000 7f9aa5400640 1 -- 172.21.15.1:0/3926441536 >> 172.21.15.1:0/3926441536 conn(0x7f9aa00cc070 msgr2=0x7f9aa00cc480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:05:43.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:43.631+0000 7f9aa5400640 1 -- 172.21.15.1:0/3926441536 shutdown_connections 2024-05-09T08:05:43.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:43.631+0000 7f9aa5400640 1 -- 172.21.15.1:0/3926441536 wait complete. 2024-05-09T08:05:43.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:43.633+0000 7f9aa5400640 1 Processor -- start 2024-05-09T08:05:43.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:43.634+0000 7f9aa5400640 1 -- start start 2024-05-09T08:05:43.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:43.634+0000 7f9aa5400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9a840039b0 0x7f9aa0154340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:05:43.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:43.634+0000 7f9aa5400640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f9aa0157840 0x7f9aa013d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:05:43.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:43.634+0000 7f9aa5400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9aa013dc70 0x7f9aa01f82a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:05:43.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:43.635+0000 7f9aa5400640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f9aa00c6df0 con 0x7f9a840039b0 2024-05-09T08:05:43.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:43.635+0000 7f9aa5400640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9aa00c6b10 con 0x7f9aa013dc70 2024-05-09T08:05:43.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:43.635+0000 7f9aa5400640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f9aa00c31d0 con 0x7f9aa0157840 2024-05-09T08:05:43.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:43.635+0000 7f9a9f200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9aa013dc70 0x7f9aa01f82a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:05:43.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:43.635+0000 7f9a9f200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9aa013dc70 0x7f9aa01f82a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.1:57496/0 (socket says 172.21.15.1:57496) 2024-05-09T08:05:43.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:43.635+0000 7f9a9f200640 1 -- 172.21.15.1:0/1855523442 learned_addr learned my addr 172.21.15.1:0/1855523442 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:05:43.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:43.635+0000 7f9a9de00640 1 --2- 172.21.15.1:0/1855523442 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f9aa0157840 0x7f9aa013d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:05:43.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:43.635+0000 7f9a9e800640 1 --2- 172.21.15.1:0/1855523442 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9a840039b0 0x7f9aa0154340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:05:43.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:43.635+0000 7f9a9f200640 1 -- 172.21.15.1:0/1855523442 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f9aa0157840 msgr2=0x7f9aa013d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:05:43.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:43.635+0000 7f9a9f200640 1 --2- 172.21.15.1:0/1855523442 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f9aa0157840 0x7f9aa013d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:05:43.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:43.635+0000 7f9a9f200640 1 -- 172.21.15.1:0/1855523442 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9a840039b0 msgr2=0x7f9aa0154340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:05:43.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:43.635+0000 7f9a9f200640 1 --2- 172.21.15.1:0/1855523442 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9a840039b0 0x7f9aa0154340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:05:43.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:43.635+0000 7f9a9f200640 1 -- 172.21.15.1:0/1855523442 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9aa01596a0 con 0x7f9aa013dc70 2024-05-09T08:05:43.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:43.636+0000 7f9a9f200640 1 --2- 172.21.15.1:0/1855523442 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9aa013dc70 0x7f9aa01f82a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f9aa00c6730 tx=0x7f9a98007a80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:05:43.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:43.636+0000 7f9a96a00640 1 -- 172.21.15.1:0/1855523442 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a90015d70 con 0x7f9aa013dc70 2024-05-09T08:05:43.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:43.636+0000 7f9a96a00640 1 -- 172.21.15.1:0/1855523442 <== mon.1 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a98018070 con 0x7f9aa013dc70 2024-05-09T08:05:43.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:43.636+0000 7f9a96a00640 1 -- 172.21.15.1:0/1855523442 <== mon.1 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a980020c0 con 0x7f9aa013dc70 2024-05-09T08:05:43.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:43.636+0000 7f9aa5400640 1 -- 172.21.15.1:0/1855523442 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9aa0155860 con 0x7f9aa013dc70 2024-05-09T08:05:43.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:43.636+0000 7f9aa5400640 1 -- 172.21.15.1:0/1855523442 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9aa015a730 con 0x7f9aa013dc70 2024-05-09T08:05:43.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:43.638+0000 7f9aa5400640 1 -- 172.21.15.1:0/1855523442 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9aa00c1330 con 0x7f9aa013dc70 2024-05-09T08:05:43.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:43.638+0000 7f9a96a00640 1 -- 172.21.15.1:0/1855523442 <== mon.1 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9a9800eb20 con 0x7f9aa013dc70 2024-05-09T08:05:43.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:43.639+0000 7f9a96a00640 1 --2- 172.21.15.1:0/1855523442 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f9a8006d230 0x7f9a8006f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:05:43.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:43.639+0000 7f9a96a00640 1 -- 172.21.15.1:0/1855523442 <== mon.1 v2:172.21.15.7:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7f9a9808b1b0 con 0x7f9aa013dc70 2024-05-09T08:05:43.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:43.639+0000 7f9a9e800640 1 --2- 172.21.15.1:0/1855523442 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f9a8006d230 0x7f9a8006f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:05:43.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:43.640+0000 7f9a9e800640 1 --2- 172.21.15.1:0/1855523442 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f9a8006d230 0x7f9a8006f6f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f9a8c034000 tx=0x7f9a8c02f000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:05:43.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:43.643+0000 7f9a96a00640 1 -- 172.21.15.1:0/1855523442 <== mon.1 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f9a98059640 con 0x7f9aa013dc70 2024-05-09T08:05:44.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.093+0000 7f9aa5400640 1 -- 172.21.15.1:0/1855523442 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f9aa00b1b00 con 0x7f9aa013dc70 2024-05-09T08:05:44.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.093+0000 7f9a96a00640 1 -- 172.21.15.1:0/1855523442 <== mon.1 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1544 (secure 0 0 0) 0x7f9a98058f40 con 0x7f9aa013dc70 2024-05-09T08:05:44.107 INFO:teuthology.orchestra.run.smithi001.stdout:{"election_epoch":30,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"54329ec8-3270-4043-9a82-c6cd7420aa7f","modified":"2024-05-09T08:01:21.819270Z","created":"2024-05-09T08:01:21.819270Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:3300","nonce":0},{"type":"v1","addr":"172.21.15.1:6789","nonce":0}]},"addr":"172.21.15.1:6789/0","public_addr":"172.21.15.1:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:3300","nonce":0},{"type":"v1","addr":"172.21.15.7:6789","nonce":0}]},"addr":"172.21.15.7:6789/0","public_addr":"172.21.15.7:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:3301","nonce":0},{"type":"v1","addr":"172.21.15.7:6790","nonce":0}]},"addr":"172.21.15.7:6790/0","public_addr":"172.21.15.7:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-05-09T08:05:44.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.096+0000 7f9aa5400640 1 -- 172.21.15.1:0/1855523442 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f9a8006d230 msgr2=0x7f9a8006f6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:05:44.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.096+0000 7f9aa5400640 1 --2- 172.21.15.1:0/1855523442 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f9a8006d230 0x7f9a8006f6f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f9a8c034000 tx=0x7f9a8c02f000 comp rx=0 tx=0).stop 2024-05-09T08:05:44.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.096+0000 7f9aa5400640 1 -- 172.21.15.1:0/1855523442 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9aa013dc70 msgr2=0x7f9aa01f82a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:05:44.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.096+0000 7f9aa5400640 1 --2- 172.21.15.1:0/1855523442 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9aa013dc70 0x7f9aa01f82a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f9aa00c6730 tx=0x7f9a98007a80 comp rx=0 tx=0).stop 2024-05-09T08:05:44.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.097+0000 7f9aa5400640 1 -- 172.21.15.1:0/1855523442 shutdown_connections 2024-05-09T08:05:44.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.097+0000 7f9aa5400640 1 --2- 172.21.15.1:0/1855523442 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f9a8006d230 0x7f9a8006f6f0 secure :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f9a8c034000 tx=0x7f9a8c02f000 comp rx=0 tx=0).stop 2024-05-09T08:05:44.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.097+0000 7f9aa5400640 1 --2- 172.21.15.1:0/1855523442 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9aa013dc70 0x7f9aa01f82a0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:05:44.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.097+0000 7f9aa5400640 1 --2- 172.21.15.1:0/1855523442 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f9aa0157840 0x7f9aa013d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:05:44.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.097+0000 7f9aa5400640 1 --2- 172.21.15.1:0/1855523442 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9a840039b0 0x7f9aa0154340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:05:44.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.097+0000 7f9aa5400640 1 -- 172.21.15.1:0/1855523442 >> 172.21.15.1:0/1855523442 conn(0x7f9aa00cc070 msgr2=0x7f9aa014e210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:05:44.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.098+0000 7f9aa5400640 1 -- 172.21.15.1:0/1855523442 shutdown_connections 2024-05-09T08:05:44.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.098+0000 7f9aa5400640 1 -- 172.21.15.1:0/1855523442 wait complete. 2024-05-09T08:05:44.128 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-05-09T08:05:44.128 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-05-09T08:05:44.128 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-05-09T08:05:44.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.416+0000 7fcc55000640 1 Processor -- start 2024-05-09T08:05:44.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.416+0000 7fcc55000640 1 -- start start 2024-05-09T08:05:44.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.418+0000 7fcc55000640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fcc50159b40 0x7fcc50156d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:05:44.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.419+0000 7fcc55000640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fcc501572c0 0x7fcc50160f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:05:44.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.419+0000 7fcc55000640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcc50070500 con 0x7fcc501507b0 2024-05-09T08:05:44.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.419+0000 7fcc55000640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fcc501568f0 con 0x7fcc501572c0 2024-05-09T08:05:44.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.419+0000 7fcc55000640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7fcc501565a0 con 0x7fcc50159b40 2024-05-09T08:05:44.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.419+0000 7fcc4de00640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7fcc501507b0 0x7fcc500728e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:57464/0 (socket says 172.21.15.1:57464) 2024-05-09T08:05:44.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.419+0000 7fcc4de00640 1 -- 172.21.15.1:0/791638374 learned_addr learned my addr 172.21.15.1:0/791638374 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T08:05:44.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.419+0000 7fcc4e800640 1 --2- 172.21.15.1:0/791638374 >> v2:172.21.15.7:3300/0 conn(0x7fcc501572c0 0x7fcc50160f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:05:44.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.420+0000 7fcc4e800640 1 -- 172.21.15.1:0/791638374 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fcc50159b40 msgr2=0x7fcc50156d80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T08:05:44.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.420+0000 7fcc4e800640 1 --2- 172.21.15.1:0/791638374 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fcc50159b40 0x7fcc50156d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:05:44.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.420+0000 7fcc4e800640 1 -- 172.21.15.1:0/791638374 >> v1:172.21.15.1:6789/0 conn(0x7fcc501507b0 legacy=0x7fcc500728e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:05:44.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.420+0000 7fcc4e800640 1 -- 172.21.15.1:0/791638374 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc50073520 con 0x7fcc501572c0 2024-05-09T08:05:44.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.421+0000 7fcc4e800640 1 --2- 172.21.15.1:0/791638374 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc501572c0 0x7fcc50160f20 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fcc50156a70 tx=0x7fcc4802c780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ab4b65e118f052ef server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:05:44.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.421+0000 7fcc4ca00640 1 -- 172.21.15.1:0/791638374 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc48032070 con 0x7fcc501572c0 2024-05-09T08:05:44.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.422+0000 7fcc4ca00640 1 -- 172.21.15.1:0/791638374 <== mon.1 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc48032620 con 0x7fcc501572c0 2024-05-09T08:05:44.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.422+0000 7fcc4ca00640 1 -- 172.21.15.1:0/791638374 <== mon.1 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc48032a30 con 0x7fcc501572c0 2024-05-09T08:05:44.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.422+0000 7fcc55000640 1 -- 172.21.15.1:0/791638374 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc501572c0 msgr2=0x7fcc50160f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:05:44.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.422+0000 7fcc55000640 1 --2- 172.21.15.1:0/791638374 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc501572c0 0x7fcc50160f20 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fcc50156a70 tx=0x7fcc4802c780 comp rx=0 tx=0).stop 2024-05-09T08:05:44.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.425+0000 7fcc55000640 1 -- 172.21.15.1:0/791638374 shutdown_connections 2024-05-09T08:05:44.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.425+0000 7fcc55000640 1 --2- 172.21.15.1:0/791638374 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fcc50159b40 0x7fcc50156d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:05:44.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.425+0000 7fcc55000640 1 --2- 172.21.15.1:0/791638374 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc501572c0 0x7fcc50160f20 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:05:44.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.425+0000 7fcc55000640 1 -- 172.21.15.1:0/791638374 >> 172.21.15.1:0/791638374 conn(0x7fcc5006dad0 msgr2=0x7fcc5006deb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:05:44.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.426+0000 7fcc55000640 1 -- 172.21.15.1:0/791638374 shutdown_connections 2024-05-09T08:05:44.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.426+0000 7fcc55000640 1 -- 172.21.15.1:0/791638374 wait complete. 2024-05-09T08:05:44.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.431+0000 7fcc55000640 1 Processor -- start 2024-05-09T08:05:44.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.432+0000 7fcc55000640 1 -- start start 2024-05-09T08:05:44.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.435+0000 7fcc55000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc501507b0 0x7fcc501f0240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:05:44.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.436+0000 7fcc55000640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fcc501572c0 0x7fcc5006fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:05:44.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.436+0000 7fcc55000640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcc50159b40 0x7fcc501f4050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:05:44.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.437+0000 7fcc55000640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fcc5015a3b0 con 0x7fcc50159b40 2024-05-09T08:05:44.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.437+0000 7fcc4e800640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcc50159b40 0x7fcc501f4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:05:44.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.437+0000 7fcc4e800640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcc50159b40 0x7fcc501f4050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:35502/0 (socket says 172.21.15.1:35502) 2024-05-09T08:05:44.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.437+0000 7fcc4e800640 1 -- 172.21.15.1:0/246080766 learned_addr learned my addr 172.21.15.1:0/246080766 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:05:44.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.437+0000 7fcc55000640 1 -- 172.21.15.1:0/246080766 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fcc5015a0d0 con 0x7fcc501507b0 2024-05-09T08:05:44.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.437+0000 7fcc4de00640 1 --2- 172.21.15.1:0/246080766 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc501507b0 0x7fcc501f0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:05:44.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.438+0000 7fcc55000640 1 -- 172.21.15.1:0/246080766 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7fcc5015a690 con 0x7fcc501572c0 2024-05-09T08:05:44.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.438+0000 7fcc4d400640 1 --2- 172.21.15.1:0/246080766 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fcc501572c0 0x7fcc5006fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:05:44.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.439+0000 7fcc4e800640 1 -- 172.21.15.1:0/246080766 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fcc501572c0 msgr2=0x7fcc5006fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:05:44.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.439+0000 7fcc4e800640 1 --2- 172.21.15.1:0/246080766 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fcc501572c0 0x7fcc5006fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:05:44.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.439+0000 7fcc4e800640 1 -- 172.21.15.1:0/246080766 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc501507b0 msgr2=0x7fcc501f0240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:05:44.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.439+0000 7fcc4e800640 1 --2- 172.21.15.1:0/246080766 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc501507b0 0x7fcc501f0240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:05:44.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.440+0000 7fcc4e800640 1 -- 172.21.15.1:0/246080766 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc501f08d0 con 0x7fcc50159b40 2024-05-09T08:05:44.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.441+0000 7fcc4e800640 1 --2- 172.21.15.1:0/246080766 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcc50159b40 0x7fcc501f4050 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fcc4802cd70 tx=0x7fcc48004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:05:44.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.442+0000 7fcc46000640 1 -- 172.21.15.1:0/246080766 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc48044020 con 0x7fcc50159b40 2024-05-09T08:05:44.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.442+0000 7fcc46000640 1 -- 172.21.15.1:0/246080766 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc480369a0 con 0x7fcc50159b40 2024-05-09T08:05:44.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.442+0000 7fcc55000640 1 -- 172.21.15.1:0/246080766 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc501f1150 con 0x7fcc50159b40 2024-05-09T08:05:44.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.442+0000 7fcc46000640 1 -- 172.21.15.1:0/246080766 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc48032070 con 0x7fcc50159b40 2024-05-09T08:05:44.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.442+0000 7fcc55000640 1 -- 172.21.15.1:0/246080766 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc501f0e70 con 0x7fcc50159b40 2024-05-09T08:05:44.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.447+0000 7fcc55000640 1 --2- 172.21.15.1:0/246080766 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcc28002a70 0x7fcc28004f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:05:44.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.447+0000 7fcc55000640 1 -- 172.21.15.1:0/246080766 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fcc28007ec0 con 0x7fcc28002a70 2024-05-09T08:05:44.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.451+0000 7fcc4de00640 1 --2- 172.21.15.1:0/246080766 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcc28002a70 0x7fcc28004f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:05:44.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.452+0000 7fcc4de00640 1 -- 172.21.15.1:0/246080766 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fcc280080c0 con 0x7fcc28002a70 2024-05-09T08:05:44.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.453+0000 7fcc46000640 1 -- 172.21.15.1:0/246080766 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fcc48005d10 con 0x7fcc50159b40 2024-05-09T08:05:44.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.454+0000 7fcc4de00640 1 --2- 172.21.15.1:0/246080766 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcc28002a70 0x7fcc28004f30 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fcc4000e390 tx=0x7fcc4000e850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:05:44.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.456+0000 7fcc46000640 1 --2- 172.21.15.1:0/246080766 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fcc3006bf00 0x7fcc3006e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:05:44.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.457+0000 7fcc46000640 1 -- 172.21.15.1:0/246080766 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7fcc480acb30 con 0x7fcc50159b40 2024-05-09T08:05:44.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.457+0000 7fcc46000640 1 -- 172.21.15.1:0/246080766 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc40019070 con 0x7fcc28002a70 2024-05-09T08:05:44.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.457+0000 7fcc46000640 1 -- 172.21.15.1:0/246080766 <== mon.0 v2:172.21.15.1:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fcc40005ac0 con 0x7fcc28002a70 2024-05-09T08:05:44.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.457+0000 7fcc46000640 1 -- 172.21.15.1:0/246080766 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcc28002a70 msgr2=0x7fcc28004f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:05:44.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.457+0000 7fcc46000640 1 --2- 172.21.15.1:0/246080766 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcc28002a70 0x7fcc28004f30 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fcc4000e390 tx=0x7fcc4000e850 comp rx=0 tx=0).stop 2024-05-09T08:05:44.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.457+0000 7fcc46000640 1 -- 172.21.15.1:0/246080766 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcc28002a70 msgr2=0x7fcc28004f30 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-05-09T08:05:44.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.457+0000 7fcc46000640 1 --2- 172.21.15.1:0/246080766 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcc28002a70 0x7fcc28004f30 secure :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fcc4000e390 tx=0x7fcc4000e850 comp rx=0 tx=0).stop 2024-05-09T08:05:44.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.458+0000 7fcc4d400640 1 --2- 172.21.15.1:0/246080766 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fcc3006bf00 0x7fcc3006e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:05:44.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.459+0000 7fcc4d400640 1 --2- 172.21.15.1:0/246080766 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fcc3006bf00 0x7fcc3006e3c0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fcc3c006fd0 tx=0x7fcc3c008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:05:44.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.476+0000 7fcc55000640 1 --2- 172.21.15.1:0/246080766 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcc28006f00 0x7fcc28007390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:05:44.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.477+0000 7fcc55000640 1 -- 172.21.15.1:0/246080766 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fcc2800a4a0 con 0x7fcc28006f00 2024-05-09T08:05:44.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.477+0000 7fcc4de00640 1 --2- 172.21.15.1:0/246080766 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcc28006f00 0x7fcc28007390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:05:44.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.478+0000 7fcc4de00640 1 -- 172.21.15.1:0/246080766 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fcc280078d0 con 0x7fcc28006f00 2024-05-09T08:05:44.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.479+0000 7fcc4de00640 1 --2- 172.21.15.1:0/246080766 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcc28006f00 0x7fcc28007390 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fcc4000e390 tx=0x7fcc40000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:05:44.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.479+0000 7fcc46000640 1 -- 172.21.15.1:0/246080766 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc40019040 con 0x7fcc28006f00 2024-05-09T08:05:44.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.480+0000 7fcc46000640 1 -- 172.21.15.1:0/246080766 <== mon.0 v2:172.21.15.1:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4535 (secure 0 0 0) 0x7fcc40007690 con 0x7fcc28006f00 2024-05-09T08:05:44.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.480+0000 7fcc46000640 1 -- 172.21.15.1:0/246080766 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcc28006f00 msgr2=0x7fcc28007390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:05:44.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.480+0000 7fcc46000640 1 --2- 172.21.15.1:0/246080766 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcc28006f00 0x7fcc28007390 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fcc4000e390 tx=0x7fcc40000ed0 comp rx=0 tx=0).stop 2024-05-09T08:05:44.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.480+0000 7fcc46000640 1 -- 172.21.15.1:0/246080766 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcc28006f00 msgr2=0x7fcc28007390 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-05-09T08:05:44.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.480+0000 7fcc46000640 1 --2- 172.21.15.1:0/246080766 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcc28006f00 0x7fcc28007390 secure :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fcc4000e390 tx=0x7fcc40000ed0 comp rx=0 tx=0).stop 2024-05-09T08:05:44.494 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-05-09T08:05:44.494 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "a", 2024-05-09T08:05:44.494 INFO:teuthology.orchestra.run.smithi001.stdout: "rank": 0, 2024-05-09T08:05:44.494 INFO:teuthology.orchestra.run.smithi001.stdout: "state": "leader", 2024-05-09T08:05:44.495 INFO:teuthology.orchestra.run.smithi001.stdout: "election_epoch": 30, 2024-05-09T08:05:44.495 INFO:teuthology.orchestra.run.smithi001.stdout: "uptime": 7095, 2024-05-09T08:05:44.495 INFO:teuthology.orchestra.run.smithi001.stdout: "quorum": [ 2024-05-09T08:05:44.495 INFO:teuthology.orchestra.run.smithi001.stdout: 0, 2024-05-09T08:05:44.495 INFO:teuthology.orchestra.run.smithi001.stdout: 1, 2024-05-09T08:05:44.495 INFO:teuthology.orchestra.run.smithi001.stdout: 2 2024-05-09T08:05:44.495 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:05:44.495 INFO:teuthology.orchestra.run.smithi001.stdout: "quorum_age": 0, 2024-05-09T08:05:44.495 INFO:teuthology.orchestra.run.smithi001.stdout: "features": { 2024-05-09T08:05:44.495 INFO:teuthology.orchestra.run.smithi001.stdout: "required_con": "2450521708007866372", 2024-05-09T08:05:44.495 INFO:teuthology.orchestra.run.smithi001.stdout: "required_mon": [ 2024-05-09T08:05:44.495 INFO:teuthology.orchestra.run.smithi001.stdout: "kraken", 2024-05-09T08:05:44.496 INFO:teuthology.orchestra.run.smithi001.stdout: "luminous", 2024-05-09T08:05:44.496 INFO:teuthology.orchestra.run.smithi001.stdout: "mimic", 2024-05-09T08:05:44.496 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap-prune", 2024-05-09T08:05:44.496 INFO:teuthology.orchestra.run.smithi001.stdout: "nautilus", 2024-05-09T08:05:44.496 INFO:teuthology.orchestra.run.smithi001.stdout: "octopus", 2024-05-09T08:05:44.496 INFO:teuthology.orchestra.run.smithi001.stdout: "pacific", 2024-05-09T08:05:44.496 INFO:teuthology.orchestra.run.smithi001.stdout: "elector-pinging", 2024-05-09T08:05:44.496 INFO:teuthology.orchestra.run.smithi001.stdout: "quincy", 2024-05-09T08:05:44.496 INFO:teuthology.orchestra.run.smithi001.stdout: "reef", 2024-05-09T08:05:44.496 INFO:teuthology.orchestra.run.smithi001.stdout: "squid" 2024-05-09T08:05:44.496 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:05:44.496 INFO:teuthology.orchestra.run.smithi001.stdout: "quorum_con": "4540701547738038271", 2024-05-09T08:05:44.496 INFO:teuthology.orchestra.run.smithi001.stdout: "quorum_mon": [ 2024-05-09T08:05:44.497 INFO:teuthology.orchestra.run.smithi001.stdout: "kraken", 2024-05-09T08:05:44.497 INFO:teuthology.orchestra.run.smithi001.stdout: "luminous", 2024-05-09T08:05:44.497 INFO:teuthology.orchestra.run.smithi001.stdout: "mimic", 2024-05-09T08:05:44.497 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap-prune", 2024-05-09T08:05:44.497 INFO:teuthology.orchestra.run.smithi001.stdout: "nautilus", 2024-05-09T08:05:44.497 INFO:teuthology.orchestra.run.smithi001.stdout: "octopus", 2024-05-09T08:05:44.497 INFO:teuthology.orchestra.run.smithi001.stdout: "pacific", 2024-05-09T08:05:44.497 INFO:teuthology.orchestra.run.smithi001.stdout: "elector-pinging", 2024-05-09T08:05:44.497 INFO:teuthology.orchestra.run.smithi001.stdout: "quincy", 2024-05-09T08:05:44.497 INFO:teuthology.orchestra.run.smithi001.stdout: "reef", 2024-05-09T08:05:44.497 INFO:teuthology.orchestra.run.smithi001.stdout: "squid" 2024-05-09T08:05:44.497 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:05:44.497 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:05:44.497 INFO:teuthology.orchestra.run.smithi001.stdout: "outside_quorum": [], 2024-05-09T08:05:44.498 INFO:teuthology.orchestra.run.smithi001.stdout: "extra_probe_peers": [], 2024-05-09T08:05:44.498 INFO:teuthology.orchestra.run.smithi001.stdout: "sync_provider": [], 2024-05-09T08:05:44.498 INFO:teuthology.orchestra.run.smithi001.stdout: "monmap": { 2024-05-09T08:05:44.498 INFO:teuthology.orchestra.run.smithi001.stdout: "epoch": 1, 2024-05-09T08:05:44.498 INFO:teuthology.orchestra.run.smithi001.stdout: "fsid": "54329ec8-3270-4043-9a82-c6cd7420aa7f", 2024-05-09T08:05:44.498 INFO:teuthology.orchestra.run.smithi001.stdout: "modified": "2024-05-09T08:01:21.819270Z", 2024-05-09T08:05:44.498 INFO:teuthology.orchestra.run.smithi001.stdout: "created": "2024-05-09T08:01:21.819270Z", 2024-05-09T08:05:44.498 INFO:teuthology.orchestra.run.smithi001.stdout: "min_mon_release": 19, 2024-05-09T08:05:44.498 INFO:teuthology.orchestra.run.smithi001.stdout: "min_mon_release_name": "squid", 2024-05-09T08:05:44.499 INFO:teuthology.orchestra.run.smithi001.stdout: "election_strategy": 1, 2024-05-09T08:05:44.499 INFO:teuthology.orchestra.run.smithi001.stdout: "disallowed_leaders: ": "", 2024-05-09T08:05:44.499 INFO:teuthology.orchestra.run.smithi001.stdout: "stretch_mode": false, 2024-05-09T08:05:44.499 INFO:teuthology.orchestra.run.smithi001.stdout: "tiebreaker_mon": "", 2024-05-09T08:05:44.499 INFO:teuthology.orchestra.run.smithi001.stdout: "removed_ranks: ": "", 2024-05-09T08:05:44.499 INFO:teuthology.orchestra.run.smithi001.stdout: "features": { 2024-05-09T08:05:44.500 INFO:teuthology.orchestra.run.smithi001.stdout: "persistent": [ 2024-05-09T08:05:44.500 INFO:teuthology.orchestra.run.smithi001.stdout: "kraken", 2024-05-09T08:05:44.500 INFO:teuthology.orchestra.run.smithi001.stdout: "luminous", 2024-05-09T08:05:44.500 INFO:teuthology.orchestra.run.smithi001.stdout: "mimic", 2024-05-09T08:05:44.500 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap-prune", 2024-05-09T08:05:44.500 INFO:teuthology.orchestra.run.smithi001.stdout: "nautilus", 2024-05-09T08:05:44.500 INFO:teuthology.orchestra.run.smithi001.stdout: "octopus", 2024-05-09T08:05:44.500 INFO:teuthology.orchestra.run.smithi001.stdout: "pacific", 2024-05-09T08:05:44.500 INFO:teuthology.orchestra.run.smithi001.stdout: "elector-pinging", 2024-05-09T08:05:44.500 INFO:teuthology.orchestra.run.smithi001.stdout: "quincy", 2024-05-09T08:05:44.500 INFO:teuthology.orchestra.run.smithi001.stdout: "reef", 2024-05-09T08:05:44.500 INFO:teuthology.orchestra.run.smithi001.stdout: "squid" 2024-05-09T08:05:44.500 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:05:44.501 INFO:teuthology.orchestra.run.smithi001.stdout: "optional": [] 2024-05-09T08:05:44.501 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:05:44.501 INFO:teuthology.orchestra.run.smithi001.stdout: "mons": [ 2024-05-09T08:05:44.501 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:05:44.501 INFO:teuthology.orchestra.run.smithi001.stdout: "rank": 0, 2024-05-09T08:05:44.501 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "a", 2024-05-09T08:05:44.501 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-05-09T08:05:44.501 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-05-09T08:05:44.501 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:05:44.501 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-05-09T08:05:44.501 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:3300", 2024-05-09T08:05:44.501 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:05:44.501 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:05:44.501 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:05:44.502 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-05-09T08:05:44.502 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6789", 2024-05-09T08:05:44.502 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:05:44.502 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:05:44.502 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:05:44.502 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:05:44.502 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6789/0", 2024-05-09T08:05:44.502 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.1:6789/0", 2024-05-09T08:05:44.502 INFO:teuthology.orchestra.run.smithi001.stdout: "priority": 0, 2024-05-09T08:05:44.502 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 0, 2024-05-09T08:05:44.503 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_location": "{}" 2024-05-09T08:05:44.503 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:05:44.503 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:05:44.503 INFO:teuthology.orchestra.run.smithi001.stdout: "rank": 1, 2024-05-09T08:05:44.503 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "b", 2024-05-09T08:05:44.503 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-05-09T08:05:44.503 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-05-09T08:05:44.503 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:05:44.503 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-05-09T08:05:44.503 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:3300", 2024-05-09T08:05:44.503 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:05:44.503 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:05:44.503 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:05:44.504 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-05-09T08:05:44.504 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:6789", 2024-05-09T08:05:44.504 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:05:44.504 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:05:44.504 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:05:44.504 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:05:44.504 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:6789/0", 2024-05-09T08:05:44.504 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.7:6789/0", 2024-05-09T08:05:44.504 INFO:teuthology.orchestra.run.smithi001.stdout: "priority": 0, 2024-05-09T08:05:44.504 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 0, 2024-05-09T08:05:44.504 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_location": "{}" 2024-05-09T08:05:44.504 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:05:44.504 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:05:44.504 INFO:teuthology.orchestra.run.smithi001.stdout: "rank": 2, 2024-05-09T08:05:44.504 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "c", 2024-05-09T08:05:44.505 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-05-09T08:05:44.505 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-05-09T08:05:44.505 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:05:44.505 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-05-09T08:05:44.505 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:3301", 2024-05-09T08:05:44.505 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:05:44.505 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:05:44.505 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:05:44.505 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-05-09T08:05:44.505 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:6790", 2024-05-09T08:05:44.505 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:05:44.505 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:05:44.505 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:05:44.505 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:05:44.505 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:6790/0", 2024-05-09T08:05:44.506 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.7:6790/0", 2024-05-09T08:05:44.506 INFO:teuthology.orchestra.run.smithi001.stdout: "priority": 0, 2024-05-09T08:05:44.506 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 0, 2024-05-09T08:05:44.506 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_location": "{}" 2024-05-09T08:05:44.506 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:05:44.506 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:05:44.506 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:05:44.506 INFO:teuthology.orchestra.run.smithi001.stdout: "feature_map": { 2024-05-09T08:05:44.506 INFO:teuthology.orchestra.run.smithi001.stdout: "mon": [ 2024-05-09T08:05:44.506 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:05:44.506 INFO:teuthology.orchestra.run.smithi001.stdout: "features": "0x3f03cffffffdffff", 2024-05-09T08:05:44.506 INFO:teuthology.orchestra.run.smithi001.stdout: "release": "luminous", 2024-05-09T08:05:44.506 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 1 2024-05-09T08:05:44.507 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:05:44.507 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:05:44.507 INFO:teuthology.orchestra.run.smithi001.stdout: "client": [ 2024-05-09T08:05:44.507 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:05:44.507 INFO:teuthology.orchestra.run.smithi001.stdout: "features": "0x3f03cffffffdffff", 2024-05-09T08:05:44.507 INFO:teuthology.orchestra.run.smithi001.stdout: "release": "luminous", 2024-05-09T08:05:44.507 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 2 2024-05-09T08:05:44.507 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:05:44.507 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:05:44.507 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:05:44.507 INFO:teuthology.orchestra.run.smithi001.stdout: "stretch_mode": false 2024-05-09T08:05:44.507 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-05-09T08:05:44.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.481+0000 7fcc55000640 1 -- 172.21.15.1:0/246080766 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fcc3006bf00 msgr2=0x7fcc3006e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:05:44.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.481+0000 7fcc55000640 1 --2- 172.21.15.1:0/246080766 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fcc3006bf00 0x7fcc3006e3c0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fcc3c006fd0 tx=0x7fcc3c008040 comp rx=0 tx=0).stop 2024-05-09T08:05:44.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.481+0000 7fcc55000640 1 -- 172.21.15.1:0/246080766 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcc50159b40 msgr2=0x7fcc501f4050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:05:44.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.481+0000 7fcc55000640 1 --2- 172.21.15.1:0/246080766 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcc50159b40 0x7fcc501f4050 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fcc4802cd70 tx=0x7fcc48004a40 comp rx=0 tx=0).stop 2024-05-09T08:05:44.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.481+0000 7fcc4e800640 1 -- 172.21.15.1:0/246080766 reap_dead start 2024-05-09T08:05:44.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.481+0000 7fcc4e800640 1 -- 172.21.15.1:0/246080766 reap_dead start 2024-05-09T08:05:44.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.481+0000 7fcc55000640 1 -- 172.21.15.1:0/246080766 shutdown_connections 2024-05-09T08:05:44.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.481+0000 7fcc55000640 1 -- 172.21.15.1:0/246080766 >> 172.21.15.1:0/246080766 conn(0x7fcc5006dad0 msgr2=0x7fcc50155400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:05:44.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.486+0000 7fcc55000640 1 -- 172.21.15.1:0/246080766 shutdown_connections 2024-05-09T08:05:44.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.488+0000 7fcc55000640 1 -- 172.21.15.1:0/246080766 wait complete. 2024-05-09T08:05:44.528 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-05-09T08:05:44.528 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-05-09T08:05:44.528 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-05-09T08:05:44.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.765+0000 7f2bfae00640 1 Processor -- start 2024-05-09T08:05:44.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.765+0000 7f2bfae00640 1 -- start start 2024-05-09T08:05:44.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.766+0000 7f2bfae00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f2bf41547e0 0x7f2bf40c9ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:05:44.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.766+0000 7f2bfae00640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f2bf415e5a0 0x7f2bf415afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:05:44.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.766+0000 7f2bfae00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2bf406bff0 con 0x7f2bf406e400 2024-05-09T08:05:44.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.766+0000 7f2bfae00640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f2bf415b4f0 con 0x7f2bf415e5a0 2024-05-09T08:05:44.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.766+0000 7f2bfae00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f2bf415ea10 con 0x7f2bf41547e0 2024-05-09T08:05:44.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.767+0000 7f2bf3e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f2bf41547e0 0x7f2bf40c9ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:05:44.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.767+0000 7f2bf8e00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f2bf406e400 0x7f2bf406b250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.1:54032/0 (socket says 172.21.15.1:54032) 2024-05-09T08:05:44.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.767+0000 7f2bf8e00640 1 -- 172.21.15.1:0/5849941 learned_addr learned my addr 172.21.15.1:0/5849941 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T08:05:44.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.767+0000 7f2bf3e00640 1 --2- 172.21.15.1:0/5849941 >> v2:172.21.15.7:3300/0 conn(0x7f2bf41547e0 0x7f2bf40c9ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.1:60200/0 (socket says 172.21.15.1:60200) 2024-05-09T08:05:44.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.767+0000 7f2bf2a00640 1 -- 172.21.15.1:0/5849941 <== mon.1 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 356475839 0 0) 0x7f2bf406bff0 con 0x7f2bf406e400 2024-05-09T08:05:44.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.767+0000 7f2bf3e00640 1 -- 172.21.15.1:0/5849941 >> v2:172.21.15.1:3300/0 conn(0x7f2bf415e5a0 msgr2=0x7f2bf415afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:05:44.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.767+0000 7f2bf3e00640 1 --2- 172.21.15.1:0/5849941 >> v2:172.21.15.1:3300/0 conn(0x7f2bf415e5a0 0x7f2bf415afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:05:44.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.767+0000 7f2bf3e00640 1 -- 172.21.15.1:0/5849941 >> v1:172.21.15.7:6789/0 conn(0x7f2bf406e400 legacy=0x7f2bf406b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:05:44.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.768+0000 7f2bf3e00640 1 -- 172.21.15.1:0/5849941 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bf406ec40 con 0x7f2bf41547e0 2024-05-09T08:05:44.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.769+0000 7f2bf3e00640 1 --2- 172.21.15.1:0/5849941 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2bf41547e0 0x7f2bf40c9ba0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f2bf40ce160 tx=0x7f2be402a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8bc4d15b1e087037 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:05:44.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.770+0000 7f2bf2a00640 1 -- 172.21.15.1:0/5849941 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2be4037070 con 0x7f2bf41547e0 2024-05-09T08:05:44.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.770+0000 7f2bf2a00640 1 -- 172.21.15.1:0/5849941 <== mon.1 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2be4032440 con 0x7f2bf41547e0 2024-05-09T08:05:44.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.770+0000 7f2bf2a00640 1 -- 172.21.15.1:0/5849941 <== mon.1 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2be4032810 con 0x7f2bf41547e0 2024-05-09T08:05:44.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.771+0000 7f2bfae00640 1 -- 172.21.15.1:0/5849941 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2bf41547e0 msgr2=0x7f2bf40c9ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:05:44.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.771+0000 7f2bfae00640 1 --2- 172.21.15.1:0/5849941 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2bf41547e0 0x7f2bf40c9ba0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f2bf40ce160 tx=0x7f2be402a270 comp rx=0 tx=0).stop 2024-05-09T08:05:44.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.772+0000 7f2bfae00640 1 -- 172.21.15.1:0/5849941 shutdown_connections 2024-05-09T08:05:44.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.772+0000 7f2bfae00640 1 --2- 172.21.15.1:0/5849941 >> v2:172.21.15.1:3300/0 conn(0x7f2bf415e5a0 0x7f2bf415afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:05:44.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.772+0000 7f2bfae00640 1 --2- 172.21.15.1:0/5849941 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2bf41547e0 0x7f2bf40c9ba0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:05:44.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.772+0000 7f2bfae00640 1 -- 172.21.15.1:0/5849941 >> 172.21.15.1:0/5849941 conn(0x7f2bf40708a0 msgr2=0x7f2bf4070cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:05:44.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.772+0000 7f2bfae00640 1 -- 172.21.15.1:0/5849941 shutdown_connections 2024-05-09T08:05:44.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.772+0000 7f2bfae00640 1 -- 172.21.15.1:0/5849941 wait complete. 2024-05-09T08:05:44.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.775+0000 7f2bfae00640 1 Processor -- start 2024-05-09T08:05:44.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.775+0000 7f2bfae00640 1 -- start start 2024-05-09T08:05:44.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.777+0000 7f2bfae00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2bf406e400 0x7f2bf41f2110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:05:44.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.777+0000 7f2bfae00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2bf41547e0 0x7f2bf415c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:05:44.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.777+0000 7f2bfae00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f2bf415e5a0 0x7f2bf41f3df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:05:44.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.777+0000 7f2bfae00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f2bf415dac0 con 0x7f2bf41547e0 2024-05-09T08:05:44.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.777+0000 7f2bfae00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2bf406f200 con 0x7f2bf406e400 2024-05-09T08:05:44.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.777+0000 7f2bfae00640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f2bf415dda0 con 0x7f2bf415e5a0 2024-05-09T08:05:44.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.777+0000 7f2bf8e00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f2bf415e5a0 0x7f2bf41f3df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:05:44.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.777+0000 7f2bf8e00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f2bf415e5a0 0x7f2bf41f3df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3301/0 says I am v2:172.21.15.1:48248/0 (socket says 172.21.15.1:48248) 2024-05-09T08:05:44.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.777+0000 7f2bf8e00640 1 -- 172.21.15.1:0/2191496339 learned_addr learned my addr 172.21.15.1:0/2191496339 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:05:44.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.778+0000 7f2bf8e00640 1 -- 172.21.15.1:0/2191496339 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2bf406e400 msgr2=0x7f2bf41f2110 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-09T08:05:44.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.778+0000 7f2bf8e00640 1 --2- 172.21.15.1:0/2191496339 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2bf406e400 0x7f2bf41f2110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:05:44.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.778+0000 7f2bf8e00640 1 -- 172.21.15.1:0/2191496339 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2bf41547e0 msgr2=0x7f2bf415c3e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T08:05:44.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.778+0000 7f2bf3400640 1 --2- 172.21.15.1:0/2191496339 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2bf41547e0 0x7f2bf415c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:05:44.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.778+0000 7f2bf8e00640 1 --2- 172.21.15.1:0/2191496339 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2bf41547e0 0x7f2bf415c3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:05:44.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.778+0000 7f2bf8e00640 1 -- 172.21.15.1:0/2191496339 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bf41eebd0 con 0x7f2bf415e5a0 2024-05-09T08:05:44.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.778+0000 7f2bf3400640 1 --2- 172.21.15.1:0/2191496339 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2bf41547e0 0x7f2bf415c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T08:05:44.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.779+0000 7f2bf8e00640 1 --2- 172.21.15.1:0/2191496339 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f2bf415e5a0 0x7f2bf41f3df0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f2bec00c780 tx=0x7f2bec00cc40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:05:44.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.779+0000 7f2bebe00640 1 -- 172.21.15.1:0/2191496339 <== mon.2 v2:172.21.15.7:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bec018070 con 0x7f2bf415e5a0 2024-05-09T08:05:44.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.779+0000 7f2bfae00640 1 -- 172.21.15.1:0/2191496339 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bf41eeeb0 con 0x7f2bf415e5a0 2024-05-09T08:05:44.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.779+0000 7f2bfae00640 1 -- 172.21.15.1:0/2191496339 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bf41f1910 con 0x7f2bf415e5a0 2024-05-09T08:05:44.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.780+0000 7f2bebe00640 1 -- 172.21.15.1:0/2191496339 <== mon.2 v2:172.21.15.7:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2bec00a640 con 0x7f2bf415e5a0 2024-05-09T08:05:44.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.781+0000 7f2bebe00640 1 -- 172.21.15.1:0/2191496339 <== mon.2 v2:172.21.15.7:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bec018070 con 0x7f2bf415e5a0 2024-05-09T08:05:44.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.787+0000 7f2bfae00640 1 --2- 172.21.15.1:0/2191496339 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2bcc002a70 0x7f2bcc004f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:05:44.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.787+0000 7f2bfae00640 1 -- 172.21.15.1:0/2191496339 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2bcc007ec0 con 0x7f2bcc002a70 2024-05-09T08:05:44.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.788+0000 7f2bebe00640 1 -- 172.21.15.1:0/2191496339 <== mon.2 v2:172.21.15.7:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2bec00a7f0 con 0x7f2bf415e5a0 2024-05-09T08:05:44.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.788+0000 7f2bf3e00640 1 --2- 172.21.15.1:0/2191496339 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2bcc002a70 0x7f2bcc004f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:05:44.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.788+0000 7f2bf3e00640 1 -- 172.21.15.1:0/2191496339 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2bcc0080c0 con 0x7f2bcc002a70 2024-05-09T08:05:44.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.789+0000 7f2bf3e00640 1 --2- 172.21.15.1:0/2191496339 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2bcc002a70 0x7f2bcc004f30 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f2bf40b3480 tx=0x7f2be40046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:05:44.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.793+0000 7f2bebe00640 1 --2- 172.21.15.1:0/2191496339 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f2bd406c230 0x7f2bd406e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:05:44.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.794+0000 7f2bebe00640 1 -- 172.21.15.1:0/2191496339 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2be4041070 con 0x7f2bcc002a70 2024-05-09T08:05:44.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.794+0000 7f2bebe00640 1 -- 172.21.15.1:0/2191496339 <== mon.2 v2:172.21.15.7:3301/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7f2bec004d70 con 0x7f2bf415e5a0 2024-05-09T08:05:44.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.794+0000 7f2bebe00640 1 -- 172.21.15.1:0/2191496339 <== mon.1 v2:172.21.15.7:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f2be403c460 con 0x7f2bcc002a70 2024-05-09T08:05:44.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.794+0000 7f2bebe00640 1 -- 172.21.15.1:0/2191496339 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2bcc002a70 msgr2=0x7f2bcc004f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:05:44.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.794+0000 7f2bebe00640 1 --2- 172.21.15.1:0/2191496339 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2bcc002a70 0x7f2bcc004f30 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f2bf40b3480 tx=0x7f2be40046c0 comp rx=0 tx=0).stop 2024-05-09T08:05:44.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.794+0000 7f2bebe00640 1 -- 172.21.15.1:0/2191496339 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2bcc002a70 msgr2=0x7f2bcc004f30 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-05-09T08:05:44.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.794+0000 7f2bebe00640 1 --2- 172.21.15.1:0/2191496339 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2bcc002a70 0x7f2bcc004f30 secure :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f2bf40b3480 tx=0x7f2be40046c0 comp rx=0 tx=0).stop 2024-05-09T08:05:44.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.795+0000 7f2bf3400640 1 --2- 172.21.15.1:0/2191496339 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f2bd406c230 0x7f2bd406e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:05:44.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.798+0000 7f2bf3400640 1 --2- 172.21.15.1:0/2191496339 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f2bd406c230 0x7f2bd406e6f0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f2bf40c7490 tx=0x7f2be0008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:05:44.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.809+0000 7f2bfae00640 1 --2- 172.21.15.1:0/2191496339 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2bcc006f50 0x7f2bcc0073e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:05:44.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.809+0000 7f2bfae00640 1 -- 172.21.15.1:0/2191496339 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2bcc00a4a0 con 0x7f2bcc006f50 2024-05-09T08:05:44.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.814+0000 7f2bf3e00640 1 --2- 172.21.15.1:0/2191496339 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2bcc006f50 0x7f2bcc0073e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:05:44.833 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-05-09T08:05:44.833 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "b", 2024-05-09T08:05:44.833 INFO:teuthology.orchestra.run.smithi001.stdout: "rank": 1, 2024-05-09T08:05:44.833 INFO:teuthology.orchestra.run.smithi001.stdout: "state": "peon", 2024-05-09T08:05:44.833 INFO:teuthology.orchestra.run.smithi001.stdout: "election_epoch": 30, 2024-05-09T08:05:44.833 INFO:teuthology.orchestra.run.smithi001.stdout: "uptime": 112478, 2024-05-09T08:05:44.833 INFO:teuthology.orchestra.run.smithi001.stdout: "quorum": [ 2024-05-09T08:05:44.833 INFO:teuthology.orchestra.run.smithi001.stdout: 0, 2024-05-09T08:05:44.833 INFO:teuthology.orchestra.run.smithi001.stdout: 1, 2024-05-09T08:05:44.833 INFO:teuthology.orchestra.run.smithi001.stdout: 2 2024-05-09T08:05:44.834 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:05:44.834 INFO:teuthology.orchestra.run.smithi001.stdout: "quorum_age": 1, 2024-05-09T08:05:44.834 INFO:teuthology.orchestra.run.smithi001.stdout: "features": { 2024-05-09T08:05:44.834 INFO:teuthology.orchestra.run.smithi001.stdout: "required_con": "2450521708007866372", 2024-05-09T08:05:44.834 INFO:teuthology.orchestra.run.smithi001.stdout: "required_mon": [ 2024-05-09T08:05:44.834 INFO:teuthology.orchestra.run.smithi001.stdout: "kraken", 2024-05-09T08:05:44.834 INFO:teuthology.orchestra.run.smithi001.stdout: "luminous", 2024-05-09T08:05:44.834 INFO:teuthology.orchestra.run.smithi001.stdout: "mimic", 2024-05-09T08:05:44.834 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap-prune", 2024-05-09T08:05:44.834 INFO:teuthology.orchestra.run.smithi001.stdout: "nautilus", 2024-05-09T08:05:44.834 INFO:teuthology.orchestra.run.smithi001.stdout: "octopus", 2024-05-09T08:05:44.834 INFO:teuthology.orchestra.run.smithi001.stdout: "pacific", 2024-05-09T08:05:44.834 INFO:teuthology.orchestra.run.smithi001.stdout: "elector-pinging", 2024-05-09T08:05:44.835 INFO:teuthology.orchestra.run.smithi001.stdout: "quincy", 2024-05-09T08:05:44.835 INFO:teuthology.orchestra.run.smithi001.stdout: "reef", 2024-05-09T08:05:44.835 INFO:teuthology.orchestra.run.smithi001.stdout: "squid" 2024-05-09T08:05:44.835 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:05:44.835 INFO:teuthology.orchestra.run.smithi001.stdout: "quorum_con": "4540701547738038271", 2024-05-09T08:05:44.835 INFO:teuthology.orchestra.run.smithi001.stdout: "quorum_mon": [ 2024-05-09T08:05:44.835 INFO:teuthology.orchestra.run.smithi001.stdout: "kraken", 2024-05-09T08:05:44.837 INFO:teuthology.orchestra.run.smithi001.stdout: "luminous", 2024-05-09T08:05:44.837 INFO:teuthology.orchestra.run.smithi001.stdout: "mimic", 2024-05-09T08:05:44.837 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap-prune", 2024-05-09T08:05:44.837 INFO:teuthology.orchestra.run.smithi001.stdout: "nautilus", 2024-05-09T08:05:44.837 INFO:teuthology.orchestra.run.smithi001.stdout: "octopus", 2024-05-09T08:05:44.837 INFO:teuthology.orchestra.run.smithi001.stdout: "pacific", 2024-05-09T08:05:44.837 INFO:teuthology.orchestra.run.smithi001.stdout: "elector-pinging", 2024-05-09T08:05:44.837 INFO:teuthology.orchestra.run.smithi001.stdout: "quincy", 2024-05-09T08:05:44.837 INFO:teuthology.orchestra.run.smithi001.stdout: "reef", 2024-05-09T08:05:44.837 INFO:teuthology.orchestra.run.smithi001.stdout: "squid" 2024-05-09T08:05:44.838 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:05:44.838 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:05:44.838 INFO:teuthology.orchestra.run.smithi001.stdout: "outside_quorum": [], 2024-05-09T08:05:44.838 INFO:teuthology.orchestra.run.smithi001.stdout: "extra_probe_peers": [], 2024-05-09T08:05:44.838 INFO:teuthology.orchestra.run.smithi001.stdout: "sync_provider": [], 2024-05-09T08:05:44.838 INFO:teuthology.orchestra.run.smithi001.stdout: "monmap": { 2024-05-09T08:05:44.838 INFO:teuthology.orchestra.run.smithi001.stdout: "epoch": 1, 2024-05-09T08:05:44.838 INFO:teuthology.orchestra.run.smithi001.stdout: "fsid": "54329ec8-3270-4043-9a82-c6cd7420aa7f", 2024-05-09T08:05:44.838 INFO:teuthology.orchestra.run.smithi001.stdout: "modified": "2024-05-09T08:01:21.819270Z", 2024-05-09T08:05:44.838 INFO:teuthology.orchestra.run.smithi001.stdout: "created": "2024-05-09T08:01:21.819270Z", 2024-05-09T08:05:44.838 INFO:teuthology.orchestra.run.smithi001.stdout: "min_mon_release": 19, 2024-05-09T08:05:44.838 INFO:teuthology.orchestra.run.smithi001.stdout: "min_mon_release_name": "squid", 2024-05-09T08:05:44.838 INFO:teuthology.orchestra.run.smithi001.stdout: "election_strategy": 1, 2024-05-09T08:05:44.838 INFO:teuthology.orchestra.run.smithi001.stdout: "disallowed_leaders: ": "", 2024-05-09T08:05:44.839 INFO:teuthology.orchestra.run.smithi001.stdout: "stretch_mode": false, 2024-05-09T08:05:44.839 INFO:teuthology.orchestra.run.smithi001.stdout: "tiebreaker_mon": "", 2024-05-09T08:05:44.839 INFO:teuthology.orchestra.run.smithi001.stdout: "removed_ranks: ": "", 2024-05-09T08:05:44.839 INFO:teuthology.orchestra.run.smithi001.stdout: "features": { 2024-05-09T08:05:44.839 INFO:teuthology.orchestra.run.smithi001.stdout: "persistent": [ 2024-05-09T08:05:44.839 INFO:teuthology.orchestra.run.smithi001.stdout: "kraken", 2024-05-09T08:05:44.839 INFO:teuthology.orchestra.run.smithi001.stdout: "luminous", 2024-05-09T08:05:44.839 INFO:teuthology.orchestra.run.smithi001.stdout: "mimic", 2024-05-09T08:05:44.839 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap-prune", 2024-05-09T08:05:44.839 INFO:teuthology.orchestra.run.smithi001.stdout: "nautilus", 2024-05-09T08:05:44.839 INFO:teuthology.orchestra.run.smithi001.stdout: "octopus", 2024-05-09T08:05:44.839 INFO:teuthology.orchestra.run.smithi001.stdout: "pacific", 2024-05-09T08:05:44.839 INFO:teuthology.orchestra.run.smithi001.stdout: "elector-pinging", 2024-05-09T08:05:44.839 INFO:teuthology.orchestra.run.smithi001.stdout: "quincy", 2024-05-09T08:05:44.840 INFO:teuthology.orchestra.run.smithi001.stdout: "reef", 2024-05-09T08:05:44.840 INFO:teuthology.orchestra.run.smithi001.stdout: "squid" 2024-05-09T08:05:44.840 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:05:44.840 INFO:teuthology.orchestra.run.smithi001.stdout: "optional": [] 2024-05-09T08:05:44.840 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:05:44.840 INFO:teuthology.orchestra.run.smithi001.stdout: "mons": [ 2024-05-09T08:05:44.840 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:05:44.840 INFO:teuthology.orchestra.run.smithi001.stdout: "rank": 0, 2024-05-09T08:05:44.840 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "a", 2024-05-09T08:05:44.840 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-05-09T08:05:44.840 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-05-09T08:05:44.840 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:05:44.840 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-05-09T08:05:44.840 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:3300", 2024-05-09T08:05:44.840 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:05:44.841 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:05:44.841 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:05:44.841 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-05-09T08:05:44.841 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6789", 2024-05-09T08:05:44.841 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:05:44.841 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:05:44.841 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:05:44.841 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:05:44.841 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6789/0", 2024-05-09T08:05:44.841 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.1:6789/0", 2024-05-09T08:05:44.841 INFO:teuthology.orchestra.run.smithi001.stdout: "priority": 0, 2024-05-09T08:05:44.841 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 0, 2024-05-09T08:05:44.841 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_location": "{}" 2024-05-09T08:05:44.841 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:05:44.842 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:05:44.842 INFO:teuthology.orchestra.run.smithi001.stdout: "rank": 1, 2024-05-09T08:05:44.842 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "b", 2024-05-09T08:05:44.842 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-05-09T08:05:44.842 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-05-09T08:05:44.842 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:05:44.842 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-05-09T08:05:44.842 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:3300", 2024-05-09T08:05:44.842 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:05:44.842 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:05:44.842 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:05:44.842 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-05-09T08:05:44.842 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:6789", 2024-05-09T08:05:44.842 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:05:44.843 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:05:44.843 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:05:44.843 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:05:44.843 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:6789/0", 2024-05-09T08:05:44.843 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.7:6789/0", 2024-05-09T08:05:44.843 INFO:teuthology.orchestra.run.smithi001.stdout: "priority": 0, 2024-05-09T08:05:44.843 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 0, 2024-05-09T08:05:44.843 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_location": "{}" 2024-05-09T08:05:44.843 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:05:44.843 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:05:44.843 INFO:teuthology.orchestra.run.smithi001.stdout: "rank": 2, 2024-05-09T08:05:44.843 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "c", 2024-05-09T08:05:44.843 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-05-09T08:05:44.844 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-05-09T08:05:44.844 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:05:44.844 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-05-09T08:05:44.844 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:3301", 2024-05-09T08:05:44.844 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:05:44.844 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:05:44.844 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:05:44.844 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-05-09T08:05:44.844 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:6790", 2024-05-09T08:05:44.844 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:05:44.844 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:05:44.844 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:05:44.844 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:05:44.844 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:6790/0", 2024-05-09T08:05:44.844 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.7:6790/0", 2024-05-09T08:05:44.845 INFO:teuthology.orchestra.run.smithi001.stdout: "priority": 0, 2024-05-09T08:05:44.845 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 0, 2024-05-09T08:05:44.845 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_location": "{}" 2024-05-09T08:05:44.845 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:05:44.845 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:05:44.845 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:05:44.845 INFO:teuthology.orchestra.run.smithi001.stdout: "feature_map": { 2024-05-09T08:05:44.845 INFO:teuthology.orchestra.run.smithi001.stdout: "mon": [ 2024-05-09T08:05:44.845 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:05:44.845 INFO:teuthology.orchestra.run.smithi001.stdout: "features": "0x3f03cffffffdffff", 2024-05-09T08:05:44.845 INFO:teuthology.orchestra.run.smithi001.stdout: "release": "luminous", 2024-05-09T08:05:44.845 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 1 2024-05-09T08:05:44.845 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:05:44.845 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:05:44.846 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": [ 2024-05-09T08:05:44.846 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:05:44.846 INFO:teuthology.orchestra.run.smithi001.stdout: "features": "0x3f03cffffffdffff", 2024-05-09T08:05:44.846 INFO:teuthology.orchestra.run.smithi001.stdout: "release": "luminous", 2024-05-09T08:05:44.846 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 1 2024-05-09T08:05:44.846 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:05:44.846 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:05:44.846 INFO:teuthology.orchestra.run.smithi001.stdout: "client": [ 2024-05-09T08:05:44.846 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:05:44.846 INFO:teuthology.orchestra.run.smithi001.stdout: "features": "0x3f03cffffffdffff", 2024-05-09T08:05:44.846 INFO:teuthology.orchestra.run.smithi001.stdout: "release": "luminous", 2024-05-09T08:05:44.846 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 1 2024-05-09T08:05:44.846 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:05:44.846 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:05:44.846 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:05:44.847 INFO:teuthology.orchestra.run.smithi001.stdout: "stretch_mode": false 2024-05-09T08:05:44.847 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-05-09T08:05:44.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.814+0000 7f2bf3e00640 1 -- 172.21.15.1:0/2191496339 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f2bcc007920 con 0x7f2bcc006f50 2024-05-09T08:05:44.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.816+0000 7f2bf3e00640 1 --2- 172.21.15.1:0/2191496339 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2bcc006f50 0x7f2bcc0073e0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f2be402a2b0 tx=0x7f2be4009190 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:05:44.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.816+0000 7f2bebe00640 1 -- 172.21.15.1:0/2191496339 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2be4037040 con 0x7f2bcc006f50 2024-05-09T08:05:44.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.816+0000 7f2bebe00640 1 -- 172.21.15.1:0/2191496339 <== mon.1 v2:172.21.15.7:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4705 (secure 0 0 0) 0x7f2be403c610 con 0x7f2bcc006f50 2024-05-09T08:05:44.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.816+0000 7f2bebe00640 1 -- 172.21.15.1:0/2191496339 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2bcc006f50 msgr2=0x7f2bcc0073e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:05:44.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.816+0000 7f2bebe00640 1 --2- 172.21.15.1:0/2191496339 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2bcc006f50 0x7f2bcc0073e0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f2be402a2b0 tx=0x7f2be4009190 comp rx=0 tx=0).stop 2024-05-09T08:05:44.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.816+0000 7f2bebe00640 1 -- 172.21.15.1:0/2191496339 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2bcc006f50 msgr2=0x7f2bcc0073e0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-05-09T08:05:44.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.816+0000 7f2bebe00640 1 --2- 172.21.15.1:0/2191496339 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2bcc006f50 0x7f2bcc0073e0 secure :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f2be402a2b0 tx=0x7f2be4009190 comp rx=0 tx=0).stop 2024-05-09T08:05:44.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.820+0000 7f2bfae00640 1 -- 172.21.15.1:0/2191496339 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f2bd406c230 msgr2=0x7f2bd406e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:05:44.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.820+0000 7f2bfae00640 1 --2- 172.21.15.1:0/2191496339 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f2bd406c230 0x7f2bd406e6f0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f2bf40c7490 tx=0x7f2be0008040 comp rx=0 tx=0).stop 2024-05-09T08:05:44.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.820+0000 7f2bf8e00640 1 -- 172.21.15.1:0/2191496339 reap_dead start 2024-05-09T08:05:44.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.820+0000 7f2bfae00640 1 -- 172.21.15.1:0/2191496339 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f2bf415e5a0 msgr2=0x7f2bf41f3df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:05:44.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.820+0000 7f2bfae00640 1 --2- 172.21.15.1:0/2191496339 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f2bf415e5a0 0x7f2bf41f3df0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f2bec00c780 tx=0x7f2bec00cc40 comp rx=0 tx=0).stop 2024-05-09T08:05:44.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.820+0000 7f2bfae00640 1 -- 172.21.15.1:0/2191496339 shutdown_connections 2024-05-09T08:05:44.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.820+0000 7f2bfae00640 1 --2- 172.21.15.1:0/2191496339 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f2bf415e5a0 0x7f2bf41f3df0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:05:44.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.820+0000 7f2bfae00640 1 -- 172.21.15.1:0/2191496339 >> 172.21.15.1:0/2191496339 conn(0x7f2bf40708a0 msgr2=0x7f2bf41527f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:05:44.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.820+0000 7f2bfae00640 1 -- 172.21.15.1:0/2191496339 shutdown_connections 2024-05-09T08:05:44.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:44.821+0000 7f2bfae00640 1 -- 172.21.15.1:0/2191496339 wait complete. 2024-05-09T08:05:44.862 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-05-09T08:05:44.862 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-05-09T08:05:44.862 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-05-09T08:05:45.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:45.156+0000 7f1964a00640 1 Processor -- start 2024-05-09T08:05:45.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:45.156+0000 7f1964a00640 1 -- start start 2024-05-09T08:05:45.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:45.158+0000 7f1964a00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f19600c6a00 0x7f19600c70a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:05:45.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:45.158+0000 7f1964a00640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1960151170 con 0x7f1960150d90 2024-05-09T08:05:45.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:45.158+0000 7f1964a00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1960157930 con 0x7f1960071fd0 2024-05-09T08:05:45.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:45.158+0000 7f1964a00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f19600c2880 con 0x7f19600c6a00 2024-05-09T08:05:45.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:45.158+0000 7f195d400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f19600c6a00 0x7f19600c70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:05:45.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:45.158+0000 7f195d400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f19600c6a00 0x7f19600c70a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.1:60220/0 (socket says 172.21.15.1:60220) 2024-05-09T08:05:45.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:45.158+0000 7f195d400640 1 -- 172.21.15.1:0/586712782 learned_addr learned my addr 172.21.15.1:0/586712782 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:05:45.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:45.159+0000 7f195d400640 1 -- 172.21.15.1:0/586712782 >> v1:172.21.15.7:6789/0 conn(0x7f1960071fd0 legacy=0x7f19601612f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:05:45.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:45.159+0000 7f195d400640 1 -- 172.21.15.1:0/586712782 >> v1:172.21.15.1:6789/0 conn(0x7f1960150d90 legacy=0x7f1960157220 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T08:05:45.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:45.160+0000 7f195d400640 1 -- 172.21.15.1:0/586712782 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19600c2b40 con 0x7f19600c6a00 2024-05-09T08:05:45.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:45.162+0000 7f195d400640 1 --2- 172.21.15.1:0/586712782 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f19600c6a00 0x7f19600c70a0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f194c009870 tx=0x7f194c02b550 comp rx=0 tx=0).ready entity=mon.1 client_cookie=21fceedd4158c90b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:05:45.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:45.162+0000 7f195ca00640 1 -- 172.21.15.1:0/586712782 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f194c037070 con 0x7f19600c6a00 2024-05-09T08:05:45.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:45.162+0000 7f195ca00640 1 -- 172.21.15.1:0/586712782 <== mon.1 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f194c030710 con 0x7f19600c6a00 2024-05-09T08:05:45.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:45.162+0000 7f195ca00640 1 -- 172.21.15.1:0/586712782 <== mon.1 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f194c030ae0 con 0x7f19600c6a00 2024-05-09T08:05:45.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:45.163+0000 7f1964a00640 1 -- 172.21.15.1:0/586712782 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f19600c6a00 msgr2=0x7f19600c70a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:05:45.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:45.163+0000 7f1964a00640 1 --2- 172.21.15.1:0/586712782 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f19600c6a00 0x7f19600c70a0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f194c009870 tx=0x7f194c02b550 comp rx=0 tx=0).stop 2024-05-09T08:05:45.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:45.164+0000 7f1964a00640 1 -- 172.21.15.1:0/586712782 shutdown_connections 2024-05-09T08:05:45.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:45.164+0000 7f1964a00640 1 --2- 172.21.15.1:0/586712782 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f19600c6a00 0x7f19600c70a0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:05:45.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:45.164+0000 7f1964a00640 1 -- 172.21.15.1:0/586712782 >> 172.21.15.1:0/586712782 conn(0x7f196006dad0 msgr2=0x7f196006deb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:05:45.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:45.165+0000 7f1964a00640 1 -- 172.21.15.1:0/586712782 shutdown_connections 2024-05-09T08:05:45.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:45.165+0000 7f1964a00640 1 -- 172.21.15.1:0/586712782 wait complete. 2024-05-09T08:05:45.181 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:45.168+0000 7f1964a00640 1 Processor -- start 2024-05-09T08:05:45.181 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:45.168+0000 7f1964a00640 1 -- start start 2024-05-09T08:05:45.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:45.169+0000 7f1964a00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f1960071fd0 0x7f19601f45e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:05:45.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:45.170+0000 7f1964a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f19600c6a00 0x7f19600c8860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:05:45.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:45.170+0000 7f195de00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f1960071fd0 0x7f19601f45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:05:45.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:45.170+0000 7f195de00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f1960071fd0 0x7f19601f45e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3301/0 says I am v2:172.21.15.1:48252/0 (socket says 172.21.15.1:48252) 2024-05-09T08:05:45.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:45.170+0000 7f1964a00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1960150d90 0x7f19600ca8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:05:45.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:45.171+0000 7f1964a00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f1960156d00 con 0x7f1960150d90 2024-05-09T08:05:45.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:45.171+0000 7f1964a00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f19600c3100 con 0x7f19600c6a00 2024-05-09T08:05:45.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:45.171+0000 7f1964a00640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f19601569d0 con 0x7f1960071fd0 2024-05-09T08:05:45.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:45.171+0000 7f195de00640 1 -- 172.21.15.1:0/1838643517 learned_addr learned my addr 172.21.15.1:0/1838643517 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:05:45.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:45.171+0000 7f195e800640 1 --2- 172.21.15.1:0/1838643517 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1960150d90 0x7f19600ca8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:05:45.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:45.171+0000 7f195de00640 1 -- 172.21.15.1:0/1838643517 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f19600c6a00 msgr2=0x7f19600c8860 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-09T08:05:45.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:45.171+0000 7f195de00640 1 --2- 172.21.15.1:0/1838643517 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f19600c6a00 0x7f19600c8860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:05:45.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:45.171+0000 7f195de00640 1 -- 172.21.15.1:0/1838643517 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1960150d90 msgr2=0x7f19600ca8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:05:45.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:45.171+0000 7f195de00640 1 --2- 172.21.15.1:0/1838643517 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1960150d90 0x7f19600ca8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:05:45.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:45.171+0000 7f195de00640 1 -- 172.21.15.1:0/1838643517 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19601f4b20 con 0x7f1960071fd0 2024-05-09T08:05:45.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:45.172+0000 7f195de00640 1 --2- 172.21.15.1:0/1838643517 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f1960071fd0 0x7f19601f45e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f195000c7f0 tx=0x7f195000ccb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:05:45.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:45.172+0000 7f1956000640 1 -- 172.21.15.1:0/1838643517 <== mon.2 v2:172.21.15.7:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1950016020 con 0x7f1960071fd0 2024-05-09T08:05:45.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:45.172+0000 7f1956000640 1 -- 172.21.15.1:0/1838643517 <== mon.2 v2:172.21.15.7:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f195000aa50 con 0x7f1960071fd0 2024-05-09T08:05:45.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:45.172+0000 7f1956000640 1 -- 172.21.15.1:0/1838643517 <== mon.2 v2:172.21.15.7:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f195000ae40 con 0x7f1960071fd0 2024-05-09T08:05:45.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:45.172+0000 7f1964a00640 1 -- 172.21.15.1:0/1838643517 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19600cb9a0 con 0x7f1960071fd0 2024-05-09T08:05:45.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:45.173+0000 7f1964a00640 1 -- 172.21.15.1:0/1838643517 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f19600cbf60 con 0x7f1960071fd0 2024-05-09T08:05:45.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:45.175+0000 7f1956000640 1 -- 172.21.15.1:0/1838643517 <== mon.2 v2:172.21.15.7:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1950019020 con 0x7f1960071fd0 2024-05-09T08:05:45.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:45.175+0000 7f1964a00640 1 --2- 172.21.15.1:0/1838643517 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f1938002a70 0x7f1938004f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:05:45.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:45.176+0000 7f1964a00640 1 -- 172.21.15.1:0/1838643517 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f1938007ec0 con 0x7f1938002a70 2024-05-09T08:05:45.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:45.176+0000 7f195d400640 1 --2- 172.21.15.1:0/1838643517 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f1938002a70 0x7f1938004f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:05:45.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:45.176+0000 7f195d400640 1 -- 172.21.15.1:0/1838643517 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f19380080c0 con 0x7f1938002a70 2024-05-09T08:05:45.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:45.176+0000 7f1956000640 1 --2- 172.21.15.1:0/1838643517 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f194006be30 0x7f194006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:05:45.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:45.177+0000 7f1956000640 1 -- 172.21.15.1:0/1838643517 <== mon.2 v2:172.21.15.7:3301/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7f195008b650 con 0x7f1960071fd0 2024-05-09T08:05:45.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:45.177+0000 7f195e800640 1 --2- 172.21.15.1:0/1838643517 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f194006be30 0x7f194006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:05:45.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:45.177+0000 7f195d400640 1 --2- 172.21.15.1:0/1838643517 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f1938002a70 0x7f1938004f30 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f19600ccab0 tx=0x7f194c0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:05:45.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:45.178+0000 7f1956000640 1 -- 172.21.15.1:0/1838643517 <== mon.2 v2:172.21.15.7:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f194c041020 con 0x7f1938002a70 2024-05-09T08:05:45.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:45.178+0000 7f1956000640 1 -- 172.21.15.1:0/1838643517 <== mon.2 v2:172.21.15.7:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f194c03d650 con 0x7f1938002a70 2024-05-09T08:05:45.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:45.178+0000 7f1956000640 1 -- 172.21.15.1:0/1838643517 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f1938002a70 msgr2=0x7f1938004f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:05:45.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:45.178+0000 7f1956000640 1 --2- 172.21.15.1:0/1838643517 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f1938002a70 0x7f1938004f30 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f19600ccab0 tx=0x7f194c0046c0 comp rx=0 tx=0).stop 2024-05-09T08:05:45.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:45.178+0000 7f1956000640 1 -- 172.21.15.1:0/1838643517 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f1938002a70 msgr2=0x7f1938004f30 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-05-09T08:05:45.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:45.178+0000 7f1956000640 1 --2- 172.21.15.1:0/1838643517 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f1938002a70 0x7f1938004f30 secure :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f19600ccab0 tx=0x7f194c0046c0 comp rx=0 tx=0).stop 2024-05-09T08:05:45.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:45.178+0000 7f195e800640 1 --2- 172.21.15.1:0/1838643517 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f194006be30 0x7f194006e2f0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f1958007fd0 tx=0x7f1958009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:05:45.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:45.195+0000 7f1964a00640 1 --2- 172.21.15.1:0/1838643517 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f1938006ee0 0x7f1938007370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:05:45.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:45.195+0000 7f1964a00640 1 -- 172.21.15.1:0/1838643517 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f193800a4a0 con 0x7f1938006ee0 2024-05-09T08:05:45.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:45.195+0000 7f195d400640 1 --2- 172.21.15.1:0/1838643517 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f1938006ee0 0x7f1938007370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:05:45.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:45.196+0000 7f195d400640 1 -- 172.21.15.1:0/1838643517 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f19380078b0 con 0x7f1938006ee0 2024-05-09T08:05:45.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:45.196+0000 7f195d400640 1 --2- 172.21.15.1:0/1838643517 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f1938006ee0 0x7f1938007370 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f194c02b590 tx=0x7f194c009240 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:05:45.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:45.197+0000 7f1956000640 1 -- 172.21.15.1:0/1838643517 <== mon.2 v2:172.21.15.7:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f194c041070 con 0x7f1938006ee0 2024-05-09T08:05:45.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:45.197+0000 7f1956000640 1 -- 172.21.15.1:0/1838643517 <== mon.2 v2:172.21.15.7:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5188 (secure 0 0 0) 0x7f194c003610 con 0x7f1938006ee0 2024-05-09T08:05:45.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:45.197+0000 7f1956000640 1 -- 172.21.15.1:0/1838643517 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f1938006ee0 msgr2=0x7f1938007370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:05:45.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:45.197+0000 7f1956000640 1 --2- 172.21.15.1:0/1838643517 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f1938006ee0 0x7f1938007370 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f194c02b590 tx=0x7f194c009240 comp rx=0 tx=0).stop 2024-05-09T08:05:45.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:45.197+0000 7f1956000640 1 -- 172.21.15.1:0/1838643517 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f1938006ee0 msgr2=0x7f1938007370 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-05-09T08:05:45.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:45.197+0000 7f1956000640 1 --2- 172.21.15.1:0/1838643517 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f1938006ee0 0x7f1938007370 secure :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f194c02b590 tx=0x7f194c009240 comp rx=0 tx=0).stop 2024-05-09T08:05:45.211 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-05-09T08:05:45.211 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "c", 2024-05-09T08:05:45.211 INFO:teuthology.orchestra.run.smithi001.stdout: "rank": 2, 2024-05-09T08:05:45.211 INFO:teuthology.orchestra.run.smithi001.stdout: "state": "peon", 2024-05-09T08:05:45.211 INFO:teuthology.orchestra.run.smithi001.stdout: "election_epoch": 30, 2024-05-09T08:05:45.211 INFO:teuthology.orchestra.run.smithi001.stdout: "uptime": 228613, 2024-05-09T08:05:45.212 INFO:teuthology.orchestra.run.smithi001.stdout: "quorum": [ 2024-05-09T08:05:45.212 INFO:teuthology.orchestra.run.smithi001.stdout: 0, 2024-05-09T08:05:45.212 INFO:teuthology.orchestra.run.smithi001.stdout: 1, 2024-05-09T08:05:45.212 INFO:teuthology.orchestra.run.smithi001.stdout: 2 2024-05-09T08:05:45.212 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:05:45.212 INFO:teuthology.orchestra.run.smithi001.stdout: "quorum_age": 1, 2024-05-09T08:05:45.212 INFO:teuthology.orchestra.run.smithi001.stdout: "features": { 2024-05-09T08:05:45.212 INFO:teuthology.orchestra.run.smithi001.stdout: "required_con": "2450521708007866372", 2024-05-09T08:05:45.212 INFO:teuthology.orchestra.run.smithi001.stdout: "required_mon": [ 2024-05-09T08:05:45.212 INFO:teuthology.orchestra.run.smithi001.stdout: "kraken", 2024-05-09T08:05:45.213 INFO:teuthology.orchestra.run.smithi001.stdout: "luminous", 2024-05-09T08:05:45.213 INFO:teuthology.orchestra.run.smithi001.stdout: "mimic", 2024-05-09T08:05:45.213 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap-prune", 2024-05-09T08:05:45.213 INFO:teuthology.orchestra.run.smithi001.stdout: "nautilus", 2024-05-09T08:05:45.213 INFO:teuthology.orchestra.run.smithi001.stdout: "octopus", 2024-05-09T08:05:45.213 INFO:teuthology.orchestra.run.smithi001.stdout: "pacific", 2024-05-09T08:05:45.213 INFO:teuthology.orchestra.run.smithi001.stdout: "elector-pinging", 2024-05-09T08:05:45.213 INFO:teuthology.orchestra.run.smithi001.stdout: "quincy", 2024-05-09T08:05:45.213 INFO:teuthology.orchestra.run.smithi001.stdout: "reef", 2024-05-09T08:05:45.213 INFO:teuthology.orchestra.run.smithi001.stdout: "squid" 2024-05-09T08:05:45.213 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:05:45.214 INFO:teuthology.orchestra.run.smithi001.stdout: "quorum_con": "4540701547738038271", 2024-05-09T08:05:45.214 INFO:teuthology.orchestra.run.smithi001.stdout: "quorum_mon": [ 2024-05-09T08:05:45.214 INFO:teuthology.orchestra.run.smithi001.stdout: "kraken", 2024-05-09T08:05:45.214 INFO:teuthology.orchestra.run.smithi001.stdout: "luminous", 2024-05-09T08:05:45.214 INFO:teuthology.orchestra.run.smithi001.stdout: "mimic", 2024-05-09T08:05:45.214 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap-prune", 2024-05-09T08:05:45.214 INFO:teuthology.orchestra.run.smithi001.stdout: "nautilus", 2024-05-09T08:05:45.214 INFO:teuthology.orchestra.run.smithi001.stdout: "octopus", 2024-05-09T08:05:45.214 INFO:teuthology.orchestra.run.smithi001.stdout: "pacific", 2024-05-09T08:05:45.214 INFO:teuthology.orchestra.run.smithi001.stdout: "elector-pinging", 2024-05-09T08:05:45.214 INFO:teuthology.orchestra.run.smithi001.stdout: "quincy", 2024-05-09T08:05:45.214 INFO:teuthology.orchestra.run.smithi001.stdout: "reef", 2024-05-09T08:05:45.214 INFO:teuthology.orchestra.run.smithi001.stdout: "squid" 2024-05-09T08:05:45.215 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:05:45.215 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:05:45.215 INFO:teuthology.orchestra.run.smithi001.stdout: "outside_quorum": [], 2024-05-09T08:05:45.215 INFO:teuthology.orchestra.run.smithi001.stdout: "extra_probe_peers": [], 2024-05-09T08:05:45.215 INFO:teuthology.orchestra.run.smithi001.stdout: "sync_provider": [], 2024-05-09T08:05:45.215 INFO:teuthology.orchestra.run.smithi001.stdout: "monmap": { 2024-05-09T08:05:45.215 INFO:teuthology.orchestra.run.smithi001.stdout: "epoch": 1, 2024-05-09T08:05:45.216 INFO:teuthology.orchestra.run.smithi001.stdout: "fsid": "54329ec8-3270-4043-9a82-c6cd7420aa7f", 2024-05-09T08:05:45.216 INFO:teuthology.orchestra.run.smithi001.stdout: "modified": "2024-05-09T08:01:21.819270Z", 2024-05-09T08:05:45.216 INFO:teuthology.orchestra.run.smithi001.stdout: "created": "2024-05-09T08:01:21.819270Z", 2024-05-09T08:05:45.216 INFO:teuthology.orchestra.run.smithi001.stdout: "min_mon_release": 19, 2024-05-09T08:05:45.216 INFO:teuthology.orchestra.run.smithi001.stdout: "min_mon_release_name": "squid", 2024-05-09T08:05:45.216 INFO:teuthology.orchestra.run.smithi001.stdout: "election_strategy": 1, 2024-05-09T08:05:45.216 INFO:teuthology.orchestra.run.smithi001.stdout: "disallowed_leaders: ": "", 2024-05-09T08:05:45.216 INFO:teuthology.orchestra.run.smithi001.stdout: "stretch_mode": false, 2024-05-09T08:05:45.216 INFO:teuthology.orchestra.run.smithi001.stdout: "tiebreaker_mon": "", 2024-05-09T08:05:45.216 INFO:teuthology.orchestra.run.smithi001.stdout: "removed_ranks: ": "", 2024-05-09T08:05:45.216 INFO:teuthology.orchestra.run.smithi001.stdout: "features": { 2024-05-09T08:05:45.217 INFO:teuthology.orchestra.run.smithi001.stdout: "persistent": [ 2024-05-09T08:05:45.217 INFO:teuthology.orchestra.run.smithi001.stdout: "kraken", 2024-05-09T08:05:45.217 INFO:teuthology.orchestra.run.smithi001.stdout: "luminous", 2024-05-09T08:05:45.217 INFO:teuthology.orchestra.run.smithi001.stdout: "mimic", 2024-05-09T08:05:45.217 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap-prune", 2024-05-09T08:05:45.217 INFO:teuthology.orchestra.run.smithi001.stdout: "nautilus", 2024-05-09T08:05:45.217 INFO:teuthology.orchestra.run.smithi001.stdout: "octopus", 2024-05-09T08:05:45.217 INFO:teuthology.orchestra.run.smithi001.stdout: "pacific", 2024-05-09T08:05:45.217 INFO:teuthology.orchestra.run.smithi001.stdout: "elector-pinging", 2024-05-09T08:05:45.217 INFO:teuthology.orchestra.run.smithi001.stdout: "quincy", 2024-05-09T08:05:45.217 INFO:teuthology.orchestra.run.smithi001.stdout: "reef", 2024-05-09T08:05:45.217 INFO:teuthology.orchestra.run.smithi001.stdout: "squid" 2024-05-09T08:05:45.217 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:05:45.218 INFO:teuthology.orchestra.run.smithi001.stdout: "optional": [] 2024-05-09T08:05:45.218 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:05:45.218 INFO:teuthology.orchestra.run.smithi001.stdout: "mons": [ 2024-05-09T08:05:45.218 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:05:45.218 INFO:teuthology.orchestra.run.smithi001.stdout: "rank": 0, 2024-05-09T08:05:45.218 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "a", 2024-05-09T08:05:45.218 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-05-09T08:05:45.218 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-05-09T08:05:45.218 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:05:45.218 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-05-09T08:05:45.218 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:3300", 2024-05-09T08:05:45.218 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:05:45.218 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:05:45.218 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:05:45.219 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-05-09T08:05:45.219 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6789", 2024-05-09T08:05:45.219 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:05:45.219 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:05:45.219 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:05:45.219 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:05:45.219 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6789/0", 2024-05-09T08:05:45.219 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.1:6789/0", 2024-05-09T08:05:45.219 INFO:teuthology.orchestra.run.smithi001.stdout: "priority": 0, 2024-05-09T08:05:45.219 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 0, 2024-05-09T08:05:45.219 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_location": "{}" 2024-05-09T08:05:45.219 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:05:45.219 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:05:45.219 INFO:teuthology.orchestra.run.smithi001.stdout: "rank": 1, 2024-05-09T08:05:45.220 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "b", 2024-05-09T08:05:45.220 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-05-09T08:05:45.220 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-05-09T08:05:45.220 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:05:45.220 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-05-09T08:05:45.220 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:3300", 2024-05-09T08:05:45.220 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:05:45.220 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:05:45.220 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:05:45.220 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-05-09T08:05:45.220 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:6789", 2024-05-09T08:05:45.220 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:05:45.220 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:05:45.220 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:05:45.221 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:05:45.221 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:6789/0", 2024-05-09T08:05:45.221 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.7:6789/0", 2024-05-09T08:05:45.221 INFO:teuthology.orchestra.run.smithi001.stdout: "priority": 0, 2024-05-09T08:05:45.221 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 0, 2024-05-09T08:05:45.221 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_location": "{}" 2024-05-09T08:05:45.221 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:05:45.221 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:05:45.221 INFO:teuthology.orchestra.run.smithi001.stdout: "rank": 2, 2024-05-09T08:05:45.221 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "c", 2024-05-09T08:05:45.221 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-05-09T08:05:45.221 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-05-09T08:05:45.221 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:05:45.222 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-05-09T08:05:45.222 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:3301", 2024-05-09T08:05:45.222 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:05:45.222 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:05:45.222 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:05:45.222 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-05-09T08:05:45.222 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:6790", 2024-05-09T08:05:45.222 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:05:45.222 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:05:45.222 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:05:45.222 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:05:45.222 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:6790/0", 2024-05-09T08:05:45.222 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.7:6790/0", 2024-05-09T08:05:45.223 INFO:teuthology.orchestra.run.smithi001.stdout: "priority": 0, 2024-05-09T08:05:45.223 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 0, 2024-05-09T08:05:45.223 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_location": "{}" 2024-05-09T08:05:45.223 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:05:45.223 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:05:45.223 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:05:45.223 INFO:teuthology.orchestra.run.smithi001.stdout: "feature_map": { 2024-05-09T08:05:45.223 INFO:teuthology.orchestra.run.smithi001.stdout: "mon": [ 2024-05-09T08:05:45.223 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:05:45.223 INFO:teuthology.orchestra.run.smithi001.stdout: "features": "0x3f03cffffffdffff", 2024-05-09T08:05:45.223 INFO:teuthology.orchestra.run.smithi001.stdout: "release": "luminous", 2024-05-09T08:05:45.223 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 1 2024-05-09T08:05:45.223 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:05:45.223 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:05:45.224 INFO:teuthology.orchestra.run.smithi001.stdout: "mds": [ 2024-05-09T08:05:45.224 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:05:45.224 INFO:teuthology.orchestra.run.smithi001.stdout: "features": "0x3f03cffffffdffff", 2024-05-09T08:05:45.224 INFO:teuthology.orchestra.run.smithi001.stdout: "release": "luminous", 2024-05-09T08:05:45.224 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 6 2024-05-09T08:05:45.224 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:05:45.224 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:05:45.224 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": [ 2024-05-09T08:05:45.224 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:05:45.224 INFO:teuthology.orchestra.run.smithi001.stdout: "features": "0x3f03cffffffdffff", 2024-05-09T08:05:45.224 INFO:teuthology.orchestra.run.smithi001.stdout: "release": "luminous", 2024-05-09T08:05:45.224 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 7 2024-05-09T08:05:45.224 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:05:45.225 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:05:45.225 INFO:teuthology.orchestra.run.smithi001.stdout: "client": [ 2024-05-09T08:05:45.225 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:05:45.225 INFO:teuthology.orchestra.run.smithi001.stdout: "features": "0x2f018fb87aa4aafe", 2024-05-09T08:05:45.225 INFO:teuthology.orchestra.run.smithi001.stdout: "release": "luminous", 2024-05-09T08:05:45.225 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 1 2024-05-09T08:05:45.225 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:05:45.225 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:05:45.225 INFO:teuthology.orchestra.run.smithi001.stdout: "features": "0x3f03cffffffdffff", 2024-05-09T08:05:45.225 INFO:teuthology.orchestra.run.smithi001.stdout: "release": "luminous", 2024-05-09T08:05:45.225 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 6 2024-05-09T08:05:45.225 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:05:45.225 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:05:45.225 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr": [ 2024-05-09T08:05:45.226 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:05:45.226 INFO:teuthology.orchestra.run.smithi001.stdout: "features": "0x3f03cffffffdffff", 2024-05-09T08:05:45.226 INFO:teuthology.orchestra.run.smithi001.stdout: "release": "luminous", 2024-05-09T08:05:45.226 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 2 2024-05-09T08:05:45.226 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:05:45.226 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:05:45.226 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:05:45.226 INFO:teuthology.orchestra.run.smithi001.stdout: "stretch_mode": false 2024-05-09T08:05:45.226 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-05-09T08:05:45.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:45.198+0000 7f1964a00640 1 -- 172.21.15.1:0/1838643517 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f194006be30 msgr2=0x7f194006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:05:45.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:45.198+0000 7f1964a00640 1 --2- 172.21.15.1:0/1838643517 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f194006be30 0x7f194006e2f0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f1958007fd0 tx=0x7f1958009040 comp rx=0 tx=0).stop 2024-05-09T08:05:45.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:45.198+0000 7f1964a00640 1 -- 172.21.15.1:0/1838643517 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f1960071fd0 msgr2=0x7f19601f45e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:05:45.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:45.198+0000 7f1964a00640 1 --2- 172.21.15.1:0/1838643517 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f1960071fd0 0x7f19601f45e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f195000c7f0 tx=0x7f195000ccb0 comp rx=0 tx=0).stop 2024-05-09T08:05:45.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:45.198+0000 7f195e800640 1 -- 172.21.15.1:0/1838643517 reap_dead start 2024-05-09T08:05:45.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:45.198+0000 7f195e800640 1 -- 172.21.15.1:0/1838643517 reap_dead start 2024-05-09T08:05:45.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:45.198+0000 7f1964a00640 1 -- 172.21.15.1:0/1838643517 shutdown_connections 2024-05-09T08:05:45.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:45.198+0000 7f1964a00640 1 -- 172.21.15.1:0/1838643517 >> 172.21.15.1:0/1838643517 conn(0x7f196006dad0 msgr2=0x7f196006b9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:05:45.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:45.199+0000 7f1964a00640 1 -- 172.21.15.1:0/1838643517 shutdown_connections 2024-05-09T08:05:45.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:45.199+0000 7f1964a00640 1 -- 172.21.15.1:0/1838643517 wait complete. 2024-05-09T08:05:45.245 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-05-09T08:05:45.246 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-05-09T08:05:45.246 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-05-09T08:05:45.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:45.489+0000 7f5ca5800640 1 Processor -- start 2024-05-09T08:05:45.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:45.489+0000 7f5ca5800640 1 -- start start 2024-05-09T08:05:45.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:45.491+0000 7f5ca5800640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f5ca006dca0 0x7f5ca0072900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:05:45.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:45.491+0000 7f5ca5800640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f5ca006ece0 0x7f5ca0159010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:05:45.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:45.491+0000 7f5ca5800640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ca0073200 con 0x7f5ca00c6ea0 2024-05-09T08:05:45.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:45.491+0000 7f5ca5800640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f5ca0072e40 con 0x7f5ca006dca0 2024-05-09T08:05:45.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:45.491+0000 7f5ca5800640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f5ca00737c0 con 0x7f5ca006ece0 2024-05-09T08:05:45.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:45.492+0000 7f5c9ea00640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f5ca006dca0 0x7f5ca0072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:05:45.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:45.492+0000 7f5c9f400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f5ca006ece0 0x7f5ca0159010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:05:45.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:45.492+0000 7f5c9ea00640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f5ca006dca0 0x7f5ca0072900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:35568/0 (socket says 172.21.15.1:35568) 2024-05-09T08:05:45.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:45.492+0000 7f5c9ea00640 1 -- 172.21.15.1:0/3401006143 learned_addr learned my addr 172.21.15.1:0/3401006143 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:05:45.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:45.492+0000 7f5c9e000640 1 --1- 172.21.15.1:0/3401006143 >> v1:172.21.15.7:6789/0 conn(0x7f5ca00c6ea0 0x7f5ca006e080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.1:54050/0 (socket says 172.21.15.1:54050) 2024-05-09T08:05:45.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:45.492+0000 7f5c9f400640 1 -- 172.21.15.1:0/3401006143 >> v2:172.21.15.1:3300/0 conn(0x7f5ca006dca0 msgr2=0x7f5ca0072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:05:45.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:45.492+0000 7f5c9f400640 1 --2- 172.21.15.1:0/3401006143 >> v2:172.21.15.1:3300/0 conn(0x7f5ca006dca0 0x7f5ca0072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:05:45.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:45.492+0000 7f5c9f400640 1 -- 172.21.15.1:0/3401006143 >> v1:172.21.15.7:6789/0 conn(0x7f5ca00c6ea0 legacy=0x7f5ca006e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:05:45.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:45.492+0000 7f5c9f400640 1 -- 172.21.15.1:0/3401006143 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ca00734e0 con 0x7f5ca006ece0 2024-05-09T08:05:45.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:45.494+0000 7f5c9f400640 1 --2- 172.21.15.1:0/3401006143 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5ca006ece0 0x7f5ca0159010 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f5ca00c9f00 tx=0x7f5c9802a710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a295471262877529 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:05:45.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:45.494+0000 7f5c9d600640 1 -- 172.21.15.1:0/3401006143 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c98007740 con 0x7f5ca006ece0 2024-05-09T08:05:45.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:45.494+0000 7f5c9d600640 1 -- 172.21.15.1:0/3401006143 <== mon.1 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c980078e0 con 0x7f5ca006ece0 2024-05-09T08:05:45.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:45.494+0000 7f5c9d600640 1 -- 172.21.15.1:0/3401006143 <== mon.1 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c98007cb0 con 0x7f5ca006ece0 2024-05-09T08:05:45.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:45.495+0000 7f5ca5800640 1 -- 172.21.15.1:0/3401006143 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5ca006ece0 msgr2=0x7f5ca0159010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:05:45.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:45.495+0000 7f5ca5800640 1 --2- 172.21.15.1:0/3401006143 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5ca006ece0 0x7f5ca0159010 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f5ca00c9f00 tx=0x7f5c9802a710 comp rx=0 tx=0).stop 2024-05-09T08:05:45.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:45.495+0000 7f5ca5800640 1 -- 172.21.15.1:0/3401006143 shutdown_connections 2024-05-09T08:05:45.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:45.495+0000 7f5ca5800640 1 --2- 172.21.15.1:0/3401006143 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5ca006ece0 0x7f5ca0159010 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:05:45.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:45.495+0000 7f5ca5800640 1 --2- 172.21.15.1:0/3401006143 >> v2:172.21.15.1:3300/0 conn(0x7f5ca006dca0 0x7f5ca0072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:05:45.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:45.495+0000 7f5ca5800640 1 -- 172.21.15.1:0/3401006143 >> 172.21.15.1:0/3401006143 conn(0x7f5ca00cc070 msgr2=0x7f5ca00cc480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:05:45.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:45.496+0000 7f5ca5800640 1 -- 172.21.15.1:0/3401006143 shutdown_connections 2024-05-09T08:05:45.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:45.496+0000 7f5ca5800640 1 -- 172.21.15.1:0/3401006143 wait complete. 2024-05-09T08:05:45.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:45.499+0000 7f5ca5800640 1 Processor -- start 2024-05-09T08:05:45.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:45.499+0000 7f5ca5800640 1 -- start start 2024-05-09T08:05:45.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:45.501+0000 7f5ca5800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5ca006dca0 0x7f5ca01573f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:05:45.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:45.502+0000 7f5ca5800640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f5ca006ece0 0x7f5ca00c2620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:05:45.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:45.502+0000 7f5ca5800640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5ca00c6ea0 0x7f5ca01449d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:05:45.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:45.502+0000 7f5ca5800640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f5ca0072470 con 0x7f5ca00c6ea0 2024-05-09T08:05:45.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:45.502+0000 7f5ca5800640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5ca0159550 con 0x7f5ca006dca0 2024-05-09T08:05:45.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:45.502+0000 7f5ca5800640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f5ca0072100 con 0x7f5ca006ece0 2024-05-09T08:05:45.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:45.502+0000 7f5c9e000640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f5ca006ece0 0x7f5ca00c2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:05:45.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:45.503+0000 7f5c9e000640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f5ca006ece0 0x7f5ca00c2620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3301/0 says I am v2:172.21.15.1:48280/0 (socket says 172.21.15.1:48280) 2024-05-09T08:05:45.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:45.503+0000 7f5c9e000640 1 -- 172.21.15.1:0/507704621 learned_addr learned my addr 172.21.15.1:0/507704621 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:05:45.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:45.503+0000 7f5c9ea00640 1 --2- 172.21.15.1:0/507704621 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5ca006dca0 0x7f5ca01573f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:05:45.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:45.504+0000 7f5c9e000640 1 -- 172.21.15.1:0/507704621 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5ca006dca0 msgr2=0x7f5ca01573f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:05:45.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:45.504+0000 7f5c9e000640 1 --2- 172.21.15.1:0/507704621 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5ca006dca0 0x7f5ca01573f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:05:45.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:45.504+0000 7f5c9e000640 1 -- 172.21.15.1:0/507704621 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5ca00c6ea0 msgr2=0x7f5ca01449d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:05:45.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:45.504+0000 7f5c9e000640 1 --2- 172.21.15.1:0/507704621 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5ca00c6ea0 0x7f5ca01449d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:05:45.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:45.504+0000 7f5c9e000640 1 -- 172.21.15.1:0/507704621 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ca00ca5e0 con 0x7f5ca006ece0 2024-05-09T08:05:45.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:45.505+0000 7f5c9e000640 1 --2- 172.21.15.1:0/507704621 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f5ca006ece0 0x7f5ca00c2620 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f5c8c00c410 tx=0x7f5c8c00c8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:05:45.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:45.506+0000 7f5c96a00640 1 -- 172.21.15.1:0/507704621 <== mon.2 v2:172.21.15.7:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c8c0074e0 con 0x7f5ca006ece0 2024-05-09T08:05:45.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:45.506+0000 7f5c96a00640 1 -- 172.21.15.1:0/507704621 <== mon.2 v2:172.21.15.7:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c8c007680 con 0x7f5ca006ece0 2024-05-09T08:05:45.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:45.506+0000 7f5c96a00640 1 -- 172.21.15.1:0/507704621 <== mon.2 v2:172.21.15.7:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c8c0049e0 con 0x7f5ca006ece0 2024-05-09T08:05:45.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:45.506+0000 7f5ca5800640 1 -- 172.21.15.1:0/507704621 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ca00c1b40 con 0x7f5ca006ece0 2024-05-09T08:05:45.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:45.508+0000 7f5ca5800640 1 -- 172.21.15.1:0/507704621 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ca00c2100 con 0x7f5ca006ece0 2024-05-09T08:05:45.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:45.512+0000 7f5ca5800640 1 -- 172.21.15.1:0/507704621 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ca0074d70 con 0x7f5ca006ece0 2024-05-09T08:05:45.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:45.515+0000 7f5c96a00640 1 -- 172.21.15.1:0/507704621 <== mon.2 v2:172.21.15.7:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5c8c01a020 con 0x7f5ca006ece0 2024-05-09T08:05:45.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:45.516+0000 7f5c96a00640 1 --2- 172.21.15.1:0/507704621 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f5c8006be80 0x7f5c8006e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:05:45.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:45.516+0000 7f5c96a00640 1 -- 172.21.15.1:0/507704621 <== mon.2 v2:172.21.15.7:3301/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7f5c8c08ae60 con 0x7f5ca006ece0 2024-05-09T08:05:45.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:45.516+0000 7f5c9ea00640 1 --2- 172.21.15.1:0/507704621 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f5c8006be80 0x7f5c8006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:05:45.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:45.517+0000 7f5c9ea00640 1 --2- 172.21.15.1:0/507704621 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f5c8006be80 0x7f5c8006e340 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f5c90009b60 tx=0x7f5c90008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:05:45.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:45.522+0000 7f5c96a00640 1 -- 172.21.15.1:0/507704621 <== mon.2 v2:172.21.15.7:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f5c8c059400 con 0x7f5ca006ece0 2024-05-09T08:05:45.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:45.972+0000 7f5ca5800640 1 -- 172.21.15.1:0/507704621 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_command({"prefix": "mon scrub"} v 0) -- 0x7f5ca00ccd20 con 0x7f5ca006ece0 2024-05-09T08:05:45.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:45.974+0000 7f5c96a00640 1 -- 172.21.15.1:0/507704621 <== mon.2 v2:172.21.15.7:3301/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) ==== 57+0+0 (secure 0 0 0) 0x7f5c8c058d00 con 0x7f5ca006ece0 2024-05-09T08:05:45.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:45.976+0000 7f5ca5800640 1 -- 172.21.15.1:0/507704621 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f5c8006be80 msgr2=0x7f5c8006e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:05:45.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:45.976+0000 7f5ca5800640 1 --2- 172.21.15.1:0/507704621 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f5c8006be80 0x7f5c8006e340 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f5c90009b60 tx=0x7f5c90008040 comp rx=0 tx=0).stop 2024-05-09T08:05:45.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:45.976+0000 7f5ca5800640 1 -- 172.21.15.1:0/507704621 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f5ca006ece0 msgr2=0x7f5ca00c2620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:05:45.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:45.976+0000 7f5ca5800640 1 --2- 172.21.15.1:0/507704621 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f5ca006ece0 0x7f5ca00c2620 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f5c8c00c410 tx=0x7f5c8c00c8d0 comp rx=0 tx=0).stop 2024-05-09T08:05:45.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:45.977+0000 7f5ca5800640 1 -- 172.21.15.1:0/507704621 shutdown_connections 2024-05-09T08:05:45.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:45.977+0000 7f5ca5800640 1 --2- 172.21.15.1:0/507704621 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f5c8006be80 0x7f5c8006e340 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:05:45.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:45.977+0000 7f5ca5800640 1 --2- 172.21.15.1:0/507704621 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5ca00c6ea0 0x7f5ca01449d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:05:45.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:45.977+0000 7f5ca5800640 1 --2- 172.21.15.1:0/507704621 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f5ca006ece0 0x7f5ca00c2620 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:05:45.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:45.977+0000 7f5ca5800640 1 --2- 172.21.15.1:0/507704621 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5ca006dca0 0x7f5ca01573f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:05:45.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:45.977+0000 7f5ca5800640 1 -- 172.21.15.1:0/507704621 >> 172.21.15.1:0/507704621 conn(0x7f5ca00cc070 msgr2=0x7f5ca01521e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:05:45.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:45.977+0000 7f5ca5800640 1 -- 172.21.15.1:0/507704621 shutdown_connections 2024-05-09T08:05:45.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:45.977+0000 7f5ca5800640 1 -- 172.21.15.1:0/507704621 wait complete. 2024-05-09T08:05:46.008 INFO:tasks.mon_thrash.mon_thrasher:waiting for 10.0 secs before continuing thrashing 2024-05-09T08:05:50.993 DEBUG:teuthology.orchestra.run.smithi001:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T08:05:50.998 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T08:05:56.010 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-05-09T08:05:56.010 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-05-09T08:05:56.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:56.214+0000 7f91b2600640 1 Processor -- start 2024-05-09T08:05:56.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:56.215+0000 7f91b2600640 1 -- start start 2024-05-09T08:05:56.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:56.216+0000 7f91b2600640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f91ac0ce160 0x7f91ac0707c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:05:56.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:56.216+0000 7f91b2600640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f91ac0c35d0 con 0x7f91ac15aee0 2024-05-09T08:05:56.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:56.216+0000 7f91b2600640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f91ac0c8820 con 0x7f91ac0ce540 2024-05-09T08:05:56.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:56.216+0000 7f91b2600640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f91ac06ce30 con 0x7f91ac0ce160 2024-05-09T08:05:56.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:56.216+0000 7f91b1200640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f91ac0ce160 0x7f91ac0707c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:05:56.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:56.216+0000 7f91a3e00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f91ac0ce540 0x7f91ac0c2ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.1:60768/0 (socket says 172.21.15.1:60768) 2024-05-09T08:05:56.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:56.216+0000 7f91a3e00640 1 -- 172.21.15.1:0/1883723605 learned_addr learned my addr 172.21.15.1:0/1883723605 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T08:05:56.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:56.217+0000 7f91b1200640 1 -- 172.21.15.1:0/1883723605 >> v1:172.21.15.7:6789/0 conn(0x7f91ac0ce540 legacy=0x7f91ac0c2ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:05:56.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:56.217+0000 7f91b1200640 1 -- 172.21.15.1:0/1883723605 >> v1:172.21.15.1:6789/0 conn(0x7f91ac15aee0 legacy=0x7f91ac153da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:05:56.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:56.217+0000 7f91b1200640 1 -- 172.21.15.1:0/1883723605 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91ac06c500 con 0x7f91ac0ce160 2024-05-09T08:05:56.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:56.217+0000 7f91abe00640 1 -- 172.21.15.1:0/1883723605 <== mon.1 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1831624841 0 0) 0x7f91ac0c8820 con 0x7f91ac0ce540 2024-05-09T08:05:56.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:56.218+0000 7f91b1200640 1 --2- 172.21.15.1:0/1883723605 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f91ac0ce160 0x7f91ac0707c0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f919c0097d0 tx=0x7f919c02a2a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ea65d3495c917251 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:05:56.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:56.218+0000 7f91abe00640 1 -- 172.21.15.1:0/1883723605 <== mon.2 v2:172.21.15.7:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f919c037070 con 0x7f91ac0ce160 2024-05-09T08:05:56.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:56.218+0000 7f91abe00640 1 -- 172.21.15.1:0/1883723605 <== mon.2 v2:172.21.15.7:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f919c032440 con 0x7f91ac0ce160 2024-05-09T08:05:56.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:56.218+0000 7f91abe00640 1 -- 172.21.15.1:0/1883723605 <== mon.2 v2:172.21.15.7:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f919c032810 con 0x7f91ac0ce160 2024-05-09T08:05:56.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:56.218+0000 7f91b2600640 1 -- 172.21.15.1:0/1883723605 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f91ac0ce160 msgr2=0x7f91ac0707c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:05:56.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:56.219+0000 7f91b2600640 1 --2- 172.21.15.1:0/1883723605 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f91ac0ce160 0x7f91ac0707c0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f919c0097d0 tx=0x7f919c02a2a0 comp rx=0 tx=0).stop 2024-05-09T08:05:56.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:56.219+0000 7f91b2600640 1 -- 172.21.15.1:0/1883723605 shutdown_connections 2024-05-09T08:05:56.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:56.219+0000 7f91b2600640 1 --2- 172.21.15.1:0/1883723605 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f91ac0ce160 0x7f91ac0707c0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:05:56.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:56.219+0000 7f91b2600640 1 -- 172.21.15.1:0/1883723605 >> 172.21.15.1:0/1883723605 conn(0x7f91ac0672c0 msgr2=0x7f91ac0676d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:05:56.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:56.219+0000 7f91b2600640 1 -- 172.21.15.1:0/1883723605 shutdown_connections 2024-05-09T08:05:56.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:56.219+0000 7f91b2600640 1 -- 172.21.15.1:0/1883723605 wait complete. 2024-05-09T08:05:56.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:56.220+0000 7f91b2600640 1 Processor -- start 2024-05-09T08:05:56.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:56.221+0000 7f91b2600640 1 -- start start 2024-05-09T08:05:56.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:56.221+0000 7f91b2600640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f91ac0ce160 0x7f91ac1e21f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:05:56.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:56.221+0000 7f91b2600640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f91ac0ce540 0x7f91ac1e0580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:05:56.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:56.221+0000 7f91b2600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f91ac15aee0 0x7f91ac1f04f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:05:56.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:56.222+0000 7f91b2600640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f91ac070d00 con 0x7f91ac0ce160 2024-05-09T08:05:56.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:56.222+0000 7f91b2600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f91ac06cac0 con 0x7f91ac15aee0 2024-05-09T08:05:56.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:56.222+0000 7f91b2600640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f91ac06d2c0 con 0x7f91ac0ce540 2024-05-09T08:05:56.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:56.222+0000 7f91a3e00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f91ac0ce540 0x7f91ac1e0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:05:56.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:56.222+0000 7f91b1200640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f91ac0ce160 0x7f91ac1e21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:05:56.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:56.222+0000 7f91b1c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f91ac15aee0 0x7f91ac1f04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:05:56.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:56.222+0000 7f91a3e00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f91ac0ce540 0x7f91ac1e0580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3301/0 says I am v2:172.21.15.1:34214/0 (socket says 172.21.15.1:34214) 2024-05-09T08:05:56.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:56.222+0000 7f91a3e00640 1 -- 172.21.15.1:0/4088680012 learned_addr learned my addr 172.21.15.1:0/4088680012 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:05:56.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:56.222+0000 7f91b1c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f91ac15aee0 0x7f91ac1f04f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.1:52122/0 (socket says 172.21.15.1:52122) 2024-05-09T08:05:56.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:56.222+0000 7f91a3e00640 1 -- 172.21.15.1:0/4088680012 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f91ac15aee0 msgr2=0x7f91ac1f04f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:05:56.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:56.222+0000 7f91a3e00640 1 --2- 172.21.15.1:0/4088680012 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f91ac15aee0 0x7f91ac1f04f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:05:56.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:56.222+0000 7f91a3e00640 1 -- 172.21.15.1:0/4088680012 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f91ac0ce160 msgr2=0x7f91ac1e21f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:05:56.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:56.222+0000 7f91a3e00640 1 --2- 172.21.15.1:0/4088680012 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f91ac0ce160 0x7f91ac1e21f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:05:56.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:56.222+0000 7f91a3e00640 1 -- 172.21.15.1:0/4088680012 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91ac1e2970 con 0x7f91ac0ce540 2024-05-09T08:05:56.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:56.223+0000 7f91a3e00640 1 --2- 172.21.15.1:0/4088680012 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f91ac0ce540 0x7f91ac1e0580 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f919400c4d0 tx=0x7f919400c990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:05:56.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:56.223+0000 7f91a9600640 1 -- 172.21.15.1:0/4088680012 <== mon.2 v2:172.21.15.7:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9194018070 con 0x7f91ac0ce540 2024-05-09T08:05:56.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:56.223+0000 7f91a9600640 1 -- 172.21.15.1:0/4088680012 <== mon.2 v2:172.21.15.7:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91940074e0 con 0x7f91ac0ce540 2024-05-09T08:05:56.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:56.223+0000 7f91b2600640 1 -- 172.21.15.1:0/4088680012 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91ac1f3c00 con 0x7f91ac0ce540 2024-05-09T08:05:56.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:56.223+0000 7f91a9600640 1 -- 172.21.15.1:0/4088680012 <== mon.2 v2:172.21.15.7:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9194004a20 con 0x7f91ac0ce540 2024-05-09T08:05:56.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:56.223+0000 7f91b2600640 1 -- 172.21.15.1:0/4088680012 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f91ac1f41c0 con 0x7f91ac0ce540 2024-05-09T08:05:56.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:56.224+0000 7f91b2600640 1 -- 172.21.15.1:0/4088680012 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91ac04f660 con 0x7f91ac0ce540 2024-05-09T08:05:56.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:56.225+0000 7f91a9600640 1 -- 172.21.15.1:0/4088680012 <== mon.2 v2:172.21.15.7:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f919401b070 con 0x7f91ac0ce540 2024-05-09T08:05:56.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:56.226+0000 7f91a9600640 1 --2- 172.21.15.1:0/4088680012 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f918806be30 0x7f918806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:05:56.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:56.226+0000 7f91a9600640 1 -- 172.21.15.1:0/4088680012 <== mon.2 v2:172.21.15.7:3301/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7f919408a5c0 con 0x7f91ac0ce540 2024-05-09T08:05:56.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:56.226+0000 7f91b1200640 1 --2- 172.21.15.1:0/4088680012 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f918806be30 0x7f918806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:05:56.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:56.227+0000 7f91b1200640 1 --2- 172.21.15.1:0/4088680012 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f918806be30 0x7f918806e2f0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f919c034000 tx=0x7f919c02e000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:05:56.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:56.230+0000 7f91a9600640 1 -- 172.21.15.1:0/4088680012 <== mon.2 v2:172.21.15.7:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f9194058490 con 0x7f91ac0ce540 2024-05-09T08:05:56.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:56.712+0000 7f91b2600640 1 -- 172.21.15.1:0/4088680012 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f91ac157e10 con 0x7f91ac0ce540 2024-05-09T08:05:56.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:56.713+0000 7f91a9600640 1 -- 172.21.15.1:0/4088680012 <== mon.2 v2:172.21.15.7:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1545 (secure 0 0 0) 0x7f919405c790 con 0x7f91ac0ce540 2024-05-09T08:05:56.725 INFO:teuthology.orchestra.run.smithi001.stdout:{"election_epoch":30,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":13,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"54329ec8-3270-4043-9a82-c6cd7420aa7f","modified":"2024-05-09T08:01:21.819270Z","created":"2024-05-09T08:01:21.819270Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:3300","nonce":0},{"type":"v1","addr":"172.21.15.1:6789","nonce":0}]},"addr":"172.21.15.1:6789/0","public_addr":"172.21.15.1:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:3300","nonce":0},{"type":"v1","addr":"172.21.15.7:6789","nonce":0}]},"addr":"172.21.15.7:6789/0","public_addr":"172.21.15.7:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:3301","nonce":0},{"type":"v1","addr":"172.21.15.7:6790","nonce":0}]},"addr":"172.21.15.7:6790/0","public_addr":"172.21.15.7:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-05-09T08:05:56.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:56.715+0000 7f91b2600640 1 -- 172.21.15.1:0/4088680012 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f918806be30 msgr2=0x7f918806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:05:56.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:56.715+0000 7f91b2600640 1 --2- 172.21.15.1:0/4088680012 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f918806be30 0x7f918806e2f0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f919c034000 tx=0x7f919c02e000 comp rx=0 tx=0).stop 2024-05-09T08:05:56.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:56.715+0000 7f91b2600640 1 -- 172.21.15.1:0/4088680012 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f91ac0ce540 msgr2=0x7f91ac1e0580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:05:56.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:56.715+0000 7f91b2600640 1 --2- 172.21.15.1:0/4088680012 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f91ac0ce540 0x7f91ac1e0580 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f919400c4d0 tx=0x7f919400c990 comp rx=0 tx=0).stop 2024-05-09T08:05:56.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:56.716+0000 7f91b2600640 1 -- 172.21.15.1:0/4088680012 shutdown_connections 2024-05-09T08:05:56.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:56.716+0000 7f91b2600640 1 --2- 172.21.15.1:0/4088680012 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f918806be30 0x7f918806e2f0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:05:56.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:56.716+0000 7f91b2600640 1 --2- 172.21.15.1:0/4088680012 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f91ac15aee0 0x7f91ac1f04f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:05:56.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:56.716+0000 7f91b2600640 1 --2- 172.21.15.1:0/4088680012 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f91ac0ce540 0x7f91ac1e0580 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:05:56.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:56.716+0000 7f91b2600640 1 --2- 172.21.15.1:0/4088680012 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f91ac0ce160 0x7f91ac1e21f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:05:56.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:56.716+0000 7f91b2600640 1 -- 172.21.15.1:0/4088680012 >> 172.21.15.1:0/4088680012 conn(0x7f91ac0672c0 msgr2=0x7f91ac0744e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:05:56.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:56.716+0000 7f91b2600640 1 -- 172.21.15.1:0/4088680012 shutdown_connections 2024-05-09T08:05:56.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:56.717+0000 7f91b2600640 1 -- 172.21.15.1:0/4088680012 wait complete. 2024-05-09T08:05:56.773 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-05-09T08:05:56.773 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-05-09T08:05:56.773 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-05-09T08:05:56.773 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-05-09T08:05:57.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.100+0000 7f9c28400640 1 Processor -- start 2024-05-09T08:05:57.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.101+0000 7f9c28400640 1 -- start start 2024-05-09T08:05:57.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.102+0000 7f9c28400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f9c2015c360 0x7f9c200c9570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:05:57.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.102+0000 7f9c28400640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f9c20072c20 0x7f9c20073050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:05:57.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.102+0000 7f9c28400640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9c20071c80 con 0x7f9c200c5e40 2024-05-09T08:05:57.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.102+0000 7f9c28400640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f9c2015b980 con 0x7f9c20072c20 2024-05-09T08:05:57.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.102+0000 7f9c28400640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f9c20071f20 con 0x7f9c2015c360 2024-05-09T08:05:57.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.102+0000 7f9c25000640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f9c200c5e40 0x7f9c200cc5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.1:60780/0 (socket says 172.21.15.1:60780) 2024-05-09T08:05:57.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.102+0000 7f9c25000640 1 -- 172.21.15.1:0/3838090031 learned_addr learned my addr 172.21.15.1:0/3838090031 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T08:05:57.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.102+0000 7f9c26400640 1 --2- 172.21.15.1:0/3838090031 >> v2:172.21.15.1:3300/0 conn(0x7f9c20072c20 0x7f9c20073050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:05:57.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.103+0000 7f9c25a00640 1 --2- 172.21.15.1:0/3838090031 >> v2:172.21.15.7:3300/0 conn(0x7f9c2015c360 0x7f9c200c9570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:05:57.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.103+0000 7f9c1fe00640 1 -- 172.21.15.1:0/3838090031 <== mon.1 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3381874077 0 0) 0x7f9c20071c80 con 0x7f9c200c5e40 2024-05-09T08:05:57.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.103+0000 7f9c1fe00640 1 -- 172.21.15.1:0/3838090031 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9c08003520 con 0x7f9c200c5e40 2024-05-09T08:05:57.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.103+0000 7f9c25a00640 1 -- 172.21.15.1:0/3838090031 >> v2:172.21.15.1:3300/0 conn(0x7f9c20072c20 msgr2=0x7f9c20073050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:05:57.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.103+0000 7f9c25a00640 1 --2- 172.21.15.1:0/3838090031 >> v2:172.21.15.1:3300/0 conn(0x7f9c20072c20 0x7f9c20073050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:05:57.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.103+0000 7f9c25a00640 1 -- 172.21.15.1:0/3838090031 >> v1:172.21.15.7:6789/0 conn(0x7f9c200c5e40 legacy=0x7f9c200cc5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:05:57.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.103+0000 7f9c1fe00640 1 -- 172.21.15.1:0/3838090031 <== mon.1 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3721959640 0 0) 0x7f9c08003520 con 0x7f9c200c5e40 2024-05-09T08:05:57.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.103+0000 7f9c25a00640 1 -- 172.21.15.1:0/3838090031 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c200721e0 con 0x7f9c2015c360 2024-05-09T08:05:57.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.103+0000 7f9c26400640 1 --2- 172.21.15.1:0/3838090031 >> v2:172.21.15.1:3300/0 conn(0x7f9c20072c20 0x7f9c20073050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:05:57.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.104+0000 7f9c25a00640 1 --2- 172.21.15.1:0/3838090031 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c2015c360 0x7f9c200c9570 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f9c200c8c60 tx=0x7f9c1402a010 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2a5c035a5450b4c3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:05:57.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.104+0000 7f9c1fe00640 1 -- 172.21.15.1:0/3838090031 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c14037070 con 0x7f9c2015c360 2024-05-09T08:05:57.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.104+0000 7f9c1fe00640 1 -- 172.21.15.1:0/3838090031 <== mon.1 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c1402aaa0 con 0x7f9c2015c360 2024-05-09T08:05:57.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.104+0000 7f9c1fe00640 1 -- 172.21.15.1:0/3838090031 <== mon.1 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c0c001690 con 0x7f9c2015c360 2024-05-09T08:05:57.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.105+0000 7f9c28400640 1 -- 172.21.15.1:0/3838090031 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c2015c360 msgr2=0x7f9c200c9570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:05:57.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.105+0000 7f9c28400640 1 --2- 172.21.15.1:0/3838090031 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c2015c360 0x7f9c200c9570 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f9c200c8c60 tx=0x7f9c1402a010 comp rx=0 tx=0).stop 2024-05-09T08:05:57.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.105+0000 7f9c28400640 1 -- 172.21.15.1:0/3838090031 shutdown_connections 2024-05-09T08:05:57.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.105+0000 7f9c28400640 1 --2- 172.21.15.1:0/3838090031 >> v2:172.21.15.1:3300/0 conn(0x7f9c20072c20 0x7f9c20073050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:05:57.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.105+0000 7f9c28400640 1 --2- 172.21.15.1:0/3838090031 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c2015c360 0x7f9c200c9570 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:05:57.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.105+0000 7f9c28400640 1 -- 172.21.15.1:0/3838090031 >> 172.21.15.1:0/3838090031 conn(0x7f9c2006dad0 msgr2=0x7f9c2006deb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:05:57.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.105+0000 7f9c28400640 1 -- 172.21.15.1:0/3838090031 shutdown_connections 2024-05-09T08:05:57.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.105+0000 7f9c28400640 1 -- 172.21.15.1:0/3838090031 wait complete. 2024-05-09T08:05:57.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.107+0000 7f9c28400640 1 Processor -- start 2024-05-09T08:05:57.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.107+0000 7f9c28400640 1 -- start start 2024-05-09T08:05:57.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.108+0000 7f9c28400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9c20072c20 0x7f9c2015a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:05:57.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.108+0000 7f9c28400640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f9c200c5e40 0x7f9c2014daa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:05:57.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.108+0000 7f9c28400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c2015c360 0x7f9c201f01e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:05:57.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.108+0000 7f9c28400640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f9c200cc0d0 con 0x7f9c20072c20 2024-05-09T08:05:57.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.108+0000 7f9c28400640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9c200727a0 con 0x7f9c2015c360 2024-05-09T08:05:57.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.108+0000 7f9c28400640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f9c200ccd00 con 0x7f9c200c5e40 2024-05-09T08:05:57.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.108+0000 7f9c25a00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9c20072c20 0x7f9c2015a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:05:57.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.108+0000 7f9c25000640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f9c200c5e40 0x7f9c2014daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:05:57.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.108+0000 7f9c25a00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9c20072c20 0x7f9c2015a430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45252/0 (socket says 172.21.15.1:45252) 2024-05-09T08:05:57.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.108+0000 7f9c25a00640 1 -- 172.21.15.1:0/2524670957 learned_addr learned my addr 172.21.15.1:0/2524670957 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:05:57.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.108+0000 7f9c26400640 1 --2- 172.21.15.1:0/2524670957 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c2015c360 0x7f9c201f01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:05:57.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.108+0000 7f9c25a00640 1 -- 172.21.15.1:0/2524670957 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f9c200c5e40 msgr2=0x7f9c2014daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:05:57.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.108+0000 7f9c25a00640 1 --2- 172.21.15.1:0/2524670957 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f9c200c5e40 0x7f9c2014daa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:05:57.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.108+0000 7f9c25a00640 1 -- 172.21.15.1:0/2524670957 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c2015c360 msgr2=0x7f9c201f01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:05:57.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.108+0000 7f9c25a00640 1 --2- 172.21.15.1:0/2524670957 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c2015c360 0x7f9c201f01e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:05:57.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.109+0000 7f9c25a00640 1 -- 172.21.15.1:0/2524670957 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c2015a970 con 0x7f9c20072c20 2024-05-09T08:05:57.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.109+0000 7f9c25000640 1 --2- 172.21.15.1:0/2524670957 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f9c200c5e40 0x7f9c2014daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T08:05:57.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.109+0000 7f9c25a00640 1 --2- 172.21.15.1:0/2524670957 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9c20072c20 0x7f9c2015a430 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f9c1402a600 tx=0x7f9c140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:05:57.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.109+0000 7f9c1d600640 1 -- 172.21.15.1:0/2524670957 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c14037050 con 0x7f9c20072c20 2024-05-09T08:05:57.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.109+0000 7f9c1d600640 1 -- 172.21.15.1:0/2524670957 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c14043070 con 0x7f9c20072c20 2024-05-09T08:05:57.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.109+0000 7f9c28400640 1 -- 172.21.15.1:0/2524670957 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c201599f0 con 0x7f9c20072c20 2024-05-09T08:05:57.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.109+0000 7f9c1d600640 1 -- 172.21.15.1:0/2524670957 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c14006e10 con 0x7f9c20072c20 2024-05-09T08:05:57.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.109+0000 7f9c28400640 1 -- 172.21.15.1:0/2524670957 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c2015b100 con 0x7f9c20072c20 2024-05-09T08:05:57.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.110+0000 7f9c28400640 1 --2- 172.21.15.1:0/2524670957 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9c2015b300 0x7f9c201f0720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:05:57.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.110+0000 7f9c28400640 1 -- 172.21.15.1:0/2524670957 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f9c2015b750 con 0x7f9c2015b300 2024-05-09T08:05:57.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.111+0000 7f9c25000640 1 --2- 172.21.15.1:0/2524670957 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9c2015b300 0x7f9c201f0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:05:57.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.111+0000 7f9c25000640 1 -- 172.21.15.1:0/2524670957 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9c20074d70 con 0x7f9c2015b300 2024-05-09T08:05:57.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.111+0000 7f9c25000640 1 --2- 172.21.15.1:0/2524670957 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9c2015b300 0x7f9c201f0720 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f9c2014e8e0 tx=0x7f9c0c00eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:05:57.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.112+0000 7f9c1d600640 1 -- 172.21.15.1:0/2524670957 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9c14048460 con 0x7f9c20072c20 2024-05-09T08:05:57.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.113+0000 7f9c1d600640 1 --2- 172.21.15.1:0/2524670957 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f9c0006be30 0x7f9c0006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:05:57.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.113+0000 7f9c1d600640 1 -- 172.21.15.1:0/2524670957 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7f9c140ab830 con 0x7f9c20072c20 2024-05-09T08:05:57.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.113+0000 7f9c26400640 1 --2- 172.21.15.1:0/2524670957 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f9c0006be30 0x7f9c0006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:05:57.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.113+0000 7f9c1d600640 1 -- 172.21.15.1:0/2524670957 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c0c00c680 con 0x7f9c2015b300 2024-05-09T08:05:57.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.113+0000 7f9c1d600640 1 -- 172.21.15.1:0/2524670957 <== mon.0 v2:172.21.15.1:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f9c0c018070 con 0x7f9c2015b300 2024-05-09T08:05:57.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.113+0000 7f9c1d600640 1 -- 172.21.15.1:0/2524670957 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9c2015b300 msgr2=0x7f9c201f0720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:05:57.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.113+0000 7f9c1d600640 1 --2- 172.21.15.1:0/2524670957 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9c2015b300 0x7f9c201f0720 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f9c2014e8e0 tx=0x7f9c0c00eb20 comp rx=0 tx=0).stop 2024-05-09T08:05:57.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.113+0000 7f9c1d600640 1 -- 172.21.15.1:0/2524670957 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9c2015b300 msgr2=0x7f9c201f0720 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-09T08:05:57.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.113+0000 7f9c1d600640 1 --2- 172.21.15.1:0/2524670957 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9c2015b300 0x7f9c201f0720 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:05:57.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.114+0000 7f9c26400640 1 --2- 172.21.15.1:0/2524670957 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f9c0006be30 0x7f9c0006e2f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f9c18009c10 tx=0x7f9c1800b040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:05:57.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.120+0000 7f9c28400640 1 --2- 172.21.15.1:0/2524670957 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9c201e6600 0x7f9c201f3400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:05:57.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.120+0000 7f9c28400640 1 -- 172.21.15.1:0/2524670957 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f9c200c8e70 con 0x7f9c201e6600 2024-05-09T08:05:57.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.120+0000 7f9c25000640 1 --2- 172.21.15.1:0/2524670957 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9c201e6600 0x7f9c201f3400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:05:57.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.121+0000 7f9c25000640 1 -- 172.21.15.1:0/2524670957 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f9c2006c890 con 0x7f9c201e6600 2024-05-09T08:05:57.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.121+0000 7f9c25000640 1 --2- 172.21.15.1:0/2524670957 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9c201e6600 0x7f9c201f3400 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f9c2015b8d0 tx=0x7f9c0c0011c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:05:57.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.121+0000 7f9c1d600640 1 -- 172.21.15.1:0/2524670957 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c0c00cda0 con 0x7f9c201e6600 2024-05-09T08:05:57.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.122+0000 7f9c1d600640 1 -- 172.21.15.1:0/2524670957 <== mon.0 v2:172.21.15.1:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4537 (secure 0 0 0) 0x7f9c0c00cda0 con 0x7f9c201e6600 2024-05-09T08:05:57.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.122+0000 7f9c1d600640 1 -- 172.21.15.1:0/2524670957 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9c201e6600 msgr2=0x7f9c201f3400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:05:57.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.122+0000 7f9c1d600640 1 --2- 172.21.15.1:0/2524670957 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9c201e6600 0x7f9c201f3400 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f9c2015b8d0 tx=0x7f9c0c0011c0 comp rx=0 tx=0).stop 2024-05-09T08:05:57.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.122+0000 7f9c1d600640 1 -- 172.21.15.1:0/2524670957 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9c201e6600 msgr2=0x7f9c201f3400 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-05-09T08:05:57.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.122+0000 7f9c1d600640 1 --2- 172.21.15.1:0/2524670957 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9c201e6600 0x7f9c201f3400 secure :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f9c2015b8d0 tx=0x7f9c0c0011c0 comp rx=0 tx=0).stop 2024-05-09T08:05:57.134 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-05-09T08:05:57.134 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "a", 2024-05-09T08:05:57.134 INFO:teuthology.orchestra.run.smithi001.stdout: "rank": 0, 2024-05-09T08:05:57.135 INFO:teuthology.orchestra.run.smithi001.stdout: "state": "leader", 2024-05-09T08:05:57.135 INFO:teuthology.orchestra.run.smithi001.stdout: "election_epoch": 30, 2024-05-09T08:05:57.135 INFO:teuthology.orchestra.run.smithi001.stdout: "uptime": 19737, 2024-05-09T08:05:57.135 INFO:teuthology.orchestra.run.smithi001.stdout: "quorum": [ 2024-05-09T08:05:57.135 INFO:teuthology.orchestra.run.smithi001.stdout: 0, 2024-05-09T08:05:57.135 INFO:teuthology.orchestra.run.smithi001.stdout: 1, 2024-05-09T08:05:57.135 INFO:teuthology.orchestra.run.smithi001.stdout: 2 2024-05-09T08:05:57.135 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:05:57.135 INFO:teuthology.orchestra.run.smithi001.stdout: "quorum_age": 13, 2024-05-09T08:05:57.135 INFO:teuthology.orchestra.run.smithi001.stdout: "features": { 2024-05-09T08:05:57.135 INFO:teuthology.orchestra.run.smithi001.stdout: "required_con": "2450521708007866372", 2024-05-09T08:05:57.135 INFO:teuthology.orchestra.run.smithi001.stdout: "required_mon": [ 2024-05-09T08:05:57.135 INFO:teuthology.orchestra.run.smithi001.stdout: "kraken", 2024-05-09T08:05:57.135 INFO:teuthology.orchestra.run.smithi001.stdout: "luminous", 2024-05-09T08:05:57.136 INFO:teuthology.orchestra.run.smithi001.stdout: "mimic", 2024-05-09T08:05:57.136 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap-prune", 2024-05-09T08:05:57.136 INFO:teuthology.orchestra.run.smithi001.stdout: "nautilus", 2024-05-09T08:05:57.136 INFO:teuthology.orchestra.run.smithi001.stdout: "octopus", 2024-05-09T08:05:57.136 INFO:teuthology.orchestra.run.smithi001.stdout: "pacific", 2024-05-09T08:05:57.136 INFO:teuthology.orchestra.run.smithi001.stdout: "elector-pinging", 2024-05-09T08:05:57.136 INFO:teuthology.orchestra.run.smithi001.stdout: "quincy", 2024-05-09T08:05:57.136 INFO:teuthology.orchestra.run.smithi001.stdout: "reef", 2024-05-09T08:05:57.136 INFO:teuthology.orchestra.run.smithi001.stdout: "squid" 2024-05-09T08:05:57.136 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:05:57.136 INFO:teuthology.orchestra.run.smithi001.stdout: "quorum_con": "4540701547738038271", 2024-05-09T08:05:57.136 INFO:teuthology.orchestra.run.smithi001.stdout: "quorum_mon": [ 2024-05-09T08:05:57.136 INFO:teuthology.orchestra.run.smithi001.stdout: "kraken", 2024-05-09T08:05:57.136 INFO:teuthology.orchestra.run.smithi001.stdout: "luminous", 2024-05-09T08:05:57.137 INFO:teuthology.orchestra.run.smithi001.stdout: "mimic", 2024-05-09T08:05:57.137 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap-prune", 2024-05-09T08:05:57.137 INFO:teuthology.orchestra.run.smithi001.stdout: "nautilus", 2024-05-09T08:05:57.137 INFO:teuthology.orchestra.run.smithi001.stdout: "octopus", 2024-05-09T08:05:57.137 INFO:teuthology.orchestra.run.smithi001.stdout: "pacific", 2024-05-09T08:05:57.137 INFO:teuthology.orchestra.run.smithi001.stdout: "elector-pinging", 2024-05-09T08:05:57.137 INFO:teuthology.orchestra.run.smithi001.stdout: "quincy", 2024-05-09T08:05:57.137 INFO:teuthology.orchestra.run.smithi001.stdout: "reef", 2024-05-09T08:05:57.137 INFO:teuthology.orchestra.run.smithi001.stdout: "squid" 2024-05-09T08:05:57.137 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:05:57.137 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:05:57.137 INFO:teuthology.orchestra.run.smithi001.stdout: "outside_quorum": [], 2024-05-09T08:05:57.137 INFO:teuthology.orchestra.run.smithi001.stdout: "extra_probe_peers": [], 2024-05-09T08:05:57.138 INFO:teuthology.orchestra.run.smithi001.stdout: "sync_provider": [], 2024-05-09T08:05:57.138 INFO:teuthology.orchestra.run.smithi001.stdout: "monmap": { 2024-05-09T08:05:57.138 INFO:teuthology.orchestra.run.smithi001.stdout: "epoch": 1, 2024-05-09T08:05:57.138 INFO:teuthology.orchestra.run.smithi001.stdout: "fsid": "54329ec8-3270-4043-9a82-c6cd7420aa7f", 2024-05-09T08:05:57.138 INFO:teuthology.orchestra.run.smithi001.stdout: "modified": "2024-05-09T08:01:21.819270Z", 2024-05-09T08:05:57.138 INFO:teuthology.orchestra.run.smithi001.stdout: "created": "2024-05-09T08:01:21.819270Z", 2024-05-09T08:05:57.138 INFO:teuthology.orchestra.run.smithi001.stdout: "min_mon_release": 19, 2024-05-09T08:05:57.138 INFO:teuthology.orchestra.run.smithi001.stdout: "min_mon_release_name": "squid", 2024-05-09T08:05:57.138 INFO:teuthology.orchestra.run.smithi001.stdout: "election_strategy": 1, 2024-05-09T08:05:57.138 INFO:teuthology.orchestra.run.smithi001.stdout: "disallowed_leaders: ": "", 2024-05-09T08:05:57.138 INFO:teuthology.orchestra.run.smithi001.stdout: "stretch_mode": false, 2024-05-09T08:05:57.138 INFO:teuthology.orchestra.run.smithi001.stdout: "tiebreaker_mon": "", 2024-05-09T08:05:57.138 INFO:teuthology.orchestra.run.smithi001.stdout: "removed_ranks: ": "", 2024-05-09T08:05:57.138 INFO:teuthology.orchestra.run.smithi001.stdout: "features": { 2024-05-09T08:05:57.139 INFO:teuthology.orchestra.run.smithi001.stdout: "persistent": [ 2024-05-09T08:05:57.139 INFO:teuthology.orchestra.run.smithi001.stdout: "kraken", 2024-05-09T08:05:57.139 INFO:teuthology.orchestra.run.smithi001.stdout: "luminous", 2024-05-09T08:05:57.139 INFO:teuthology.orchestra.run.smithi001.stdout: "mimic", 2024-05-09T08:05:57.139 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap-prune", 2024-05-09T08:05:57.139 INFO:teuthology.orchestra.run.smithi001.stdout: "nautilus", 2024-05-09T08:05:57.139 INFO:teuthology.orchestra.run.smithi001.stdout: "octopus", 2024-05-09T08:05:57.139 INFO:teuthology.orchestra.run.smithi001.stdout: "pacific", 2024-05-09T08:05:57.139 INFO:teuthology.orchestra.run.smithi001.stdout: "elector-pinging", 2024-05-09T08:05:57.139 INFO:teuthology.orchestra.run.smithi001.stdout: "quincy", 2024-05-09T08:05:57.139 INFO:teuthology.orchestra.run.smithi001.stdout: "reef", 2024-05-09T08:05:57.139 INFO:teuthology.orchestra.run.smithi001.stdout: "squid" 2024-05-09T08:05:57.139 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:05:57.140 INFO:teuthology.orchestra.run.smithi001.stdout: "optional": [] 2024-05-09T08:05:57.140 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:05:57.140 INFO:teuthology.orchestra.run.smithi001.stdout: "mons": [ 2024-05-09T08:05:57.140 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:05:57.140 INFO:teuthology.orchestra.run.smithi001.stdout: "rank": 0, 2024-05-09T08:05:57.140 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "a", 2024-05-09T08:05:57.140 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-05-09T08:05:57.140 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-05-09T08:05:57.140 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:05:57.140 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-05-09T08:05:57.140 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:3300", 2024-05-09T08:05:57.140 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:05:57.140 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:05:57.140 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:05:57.141 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-05-09T08:05:57.141 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6789", 2024-05-09T08:05:57.141 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:05:57.141 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:05:57.141 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:05:57.141 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:05:57.141 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6789/0", 2024-05-09T08:05:57.141 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.1:6789/0", 2024-05-09T08:05:57.141 INFO:teuthology.orchestra.run.smithi001.stdout: "priority": 0, 2024-05-09T08:05:57.141 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 0, 2024-05-09T08:05:57.141 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_location": "{}" 2024-05-09T08:05:57.141 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:05:57.141 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:05:57.142 INFO:teuthology.orchestra.run.smithi001.stdout: "rank": 1, 2024-05-09T08:05:57.142 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "b", 2024-05-09T08:05:57.142 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-05-09T08:05:57.142 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-05-09T08:05:57.142 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:05:57.142 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-05-09T08:05:57.142 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:3300", 2024-05-09T08:05:57.142 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:05:57.142 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:05:57.142 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:05:57.142 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-05-09T08:05:57.142 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:6789", 2024-05-09T08:05:57.142 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:05:57.143 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:05:57.143 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:05:57.143 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:05:57.143 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:6789/0", 2024-05-09T08:05:57.143 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.7:6789/0", 2024-05-09T08:05:57.143 INFO:teuthology.orchestra.run.smithi001.stdout: "priority": 0, 2024-05-09T08:05:57.143 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 0, 2024-05-09T08:05:57.143 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_location": "{}" 2024-05-09T08:05:57.143 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:05:57.143 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:05:57.143 INFO:teuthology.orchestra.run.smithi001.stdout: "rank": 2, 2024-05-09T08:05:57.143 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "c", 2024-05-09T08:05:57.143 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-05-09T08:05:57.143 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-05-09T08:05:57.144 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:05:57.144 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-05-09T08:05:57.144 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:3301", 2024-05-09T08:05:57.144 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:05:57.144 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:05:57.144 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:05:57.144 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-05-09T08:05:57.144 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:6790", 2024-05-09T08:05:57.144 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:05:57.144 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:05:57.144 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:05:57.144 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:05:57.145 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:6790/0", 2024-05-09T08:05:57.145 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.7:6790/0", 2024-05-09T08:05:57.145 INFO:teuthology.orchestra.run.smithi001.stdout: "priority": 0, 2024-05-09T08:05:57.145 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 0, 2024-05-09T08:05:57.145 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_location": "{}" 2024-05-09T08:05:57.145 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:05:57.145 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:05:57.145 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:05:57.145 INFO:teuthology.orchestra.run.smithi001.stdout: "feature_map": { 2024-05-09T08:05:57.145 INFO:teuthology.orchestra.run.smithi001.stdout: "mon": [ 2024-05-09T08:05:57.145 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:05:57.145 INFO:teuthology.orchestra.run.smithi001.stdout: "features": "0x3f03cffffffdffff", 2024-05-09T08:05:57.146 INFO:teuthology.orchestra.run.smithi001.stdout: "release": "luminous", 2024-05-09T08:05:57.146 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 1 2024-05-09T08:05:57.146 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:05:57.146 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:05:57.146 INFO:teuthology.orchestra.run.smithi001.stdout: "client": [ 2024-05-09T08:05:57.146 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:05:57.146 INFO:teuthology.orchestra.run.smithi001.stdout: "features": "0x3f03cffffffdffff", 2024-05-09T08:05:57.146 INFO:teuthology.orchestra.run.smithi001.stdout: "release": "luminous", 2024-05-09T08:05:57.146 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 2 2024-05-09T08:05:57.146 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:05:57.146 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:05:57.146 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:05:57.147 INFO:teuthology.orchestra.run.smithi001.stdout: "stretch_mode": false 2024-05-09T08:05:57.147 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-05-09T08:05:57.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.122+0000 7f9c28400640 1 -- 172.21.15.1:0/2524670957 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f9c0006be30 msgr2=0x7f9c0006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:05:57.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.122+0000 7f9c28400640 1 --2- 172.21.15.1:0/2524670957 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f9c0006be30 0x7f9c0006e2f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f9c18009c10 tx=0x7f9c1800b040 comp rx=0 tx=0).stop 2024-05-09T08:05:57.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.122+0000 7f9c28400640 1 -- 172.21.15.1:0/2524670957 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9c20072c20 msgr2=0x7f9c2015a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:05:57.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.122+0000 7f9c28400640 1 --2- 172.21.15.1:0/2524670957 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9c20072c20 0x7f9c2015a430 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f9c1402a600 tx=0x7f9c140046c0 comp rx=0 tx=0).stop 2024-05-09T08:05:57.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.122+0000 7f9c26400640 1 -- 172.21.15.1:0/2524670957 reap_dead start 2024-05-09T08:05:57.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.122+0000 7f9c26400640 1 -- 172.21.15.1:0/2524670957 reap_dead start 2024-05-09T08:05:57.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.122+0000 7f9c28400640 1 -- 172.21.15.1:0/2524670957 shutdown_connections 2024-05-09T08:05:57.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.122+0000 7f9c28400640 1 -- 172.21.15.1:0/2524670957 >> 172.21.15.1:0/2524670957 conn(0x7f9c2006dad0 msgr2=0x7f9c2006deb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:05:57.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.123+0000 7f9c28400640 1 -- 172.21.15.1:0/2524670957 shutdown_connections 2024-05-09T08:05:57.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.123+0000 7f9c28400640 1 -- 172.21.15.1:0/2524670957 wait complete. 2024-05-09T08:05:57.152 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-05-09T08:05:57.152 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-05-09T08:05:57.153 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-05-09T08:05:57.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.380+0000 7f9969400640 1 Processor -- start 2024-05-09T08:05:57.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.380+0000 7f9969400640 1 -- start start 2024-05-09T08:05:57.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.382+0000 7f9969400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f9964154870 0x7f996415b190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:05:57.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.382+0000 7f9969400640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f9964154c50 0x7f99640c63a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:05:57.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.382+0000 7f9969400640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f99640c68e0 0x7f99640d01d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:05:57.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.382+0000 7f9969400640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f996415b6d0 con 0x7f99640c68e0 2024-05-09T08:05:57.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.382+0000 7f9969400640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f99640cafc0 con 0x7f9964154870 2024-05-09T08:05:57.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.382+0000 7f9969400640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f99640c4e50 con 0x7f9964154c50 2024-05-09T08:05:57.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.383+0000 7f9962800640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f9964154870 0x7f996415b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:05:57.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.383+0000 7f9963200640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f99640c68e0 0x7f99640d01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:05:57.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.383+0000 7f9962800640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f9964154870 0x7f996415b190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.1:52142/0 (socket says 172.21.15.1:52142) 2024-05-09T08:05:57.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.383+0000 7f9962800640 1 -- 172.21.15.1:0/4010608838 learned_addr learned my addr 172.21.15.1:0/4010608838 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:05:57.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.384+0000 7f9962800640 1 -- 172.21.15.1:0/4010608838 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f9964154c50 msgr2=0x7f99640c63a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T08:05:57.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.384+0000 7f9962800640 1 --2- 172.21.15.1:0/4010608838 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f9964154c50 0x7f99640c63a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:05:57.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.384+0000 7f9962800640 1 -- 172.21.15.1:0/4010608838 >> v2:172.21.15.1:3300/0 conn(0x7f99640c68e0 msgr2=0x7f99640d01d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:05:57.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.384+0000 7f9962800640 1 --2- 172.21.15.1:0/4010608838 >> v2:172.21.15.1:3300/0 conn(0x7f99640c68e0 0x7f99640d01d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:05:57.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.384+0000 7f9962800640 1 -- 172.21.15.1:0/4010608838 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99640cb260 con 0x7f9964154870 2024-05-09T08:05:57.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.386+0000 7f9962800640 1 --2- 172.21.15.1:0/4010608838 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9964154870 0x7f996415b190 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f996408d500 tx=0x7f995402a210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=18cb8b0b52fd6ee9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:05:57.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.386+0000 7f9961400640 1 -- 172.21.15.1:0/4010608838 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9954037070 con 0x7f9964154870 2024-05-09T08:05:57.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.386+0000 7f9961400640 1 -- 172.21.15.1:0/4010608838 <== mon.1 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9954032440 con 0x7f9964154870 2024-05-09T08:05:57.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.386+0000 7f9961400640 1 -- 172.21.15.1:0/4010608838 <== mon.1 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9954032810 con 0x7f9964154870 2024-05-09T08:05:57.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.387+0000 7f9969400640 1 -- 172.21.15.1:0/4010608838 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9964154870 msgr2=0x7f996415b190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:05:57.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.387+0000 7f9969400640 1 --2- 172.21.15.1:0/4010608838 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9964154870 0x7f996415b190 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f996408d500 tx=0x7f995402a210 comp rx=0 tx=0).stop 2024-05-09T08:05:57.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.387+0000 7f9969400640 1 -- 172.21.15.1:0/4010608838 shutdown_connections 2024-05-09T08:05:57.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.387+0000 7f9969400640 1 --2- 172.21.15.1:0/4010608838 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f9964154c50 0x7f99640c63a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:05:57.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.387+0000 7f9969400640 1 --2- 172.21.15.1:0/4010608838 >> v2:172.21.15.1:3300/0 conn(0x7f99640c68e0 0x7f99640d01d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:05:57.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.387+0000 7f9969400640 1 --2- 172.21.15.1:0/4010608838 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9964154870 0x7f996415b190 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:05:57.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.387+0000 7f9969400640 1 -- 172.21.15.1:0/4010608838 >> 172.21.15.1:0/4010608838 conn(0x7f996408bff0 msgr2=0x7f996408c400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:05:57.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.388+0000 7f9969400640 1 -- 172.21.15.1:0/4010608838 shutdown_connections 2024-05-09T08:05:57.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.388+0000 7f9969400640 1 -- 172.21.15.1:0/4010608838 wait complete. 2024-05-09T08:05:57.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.389+0000 7f9969400640 1 Processor -- start 2024-05-09T08:05:57.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.389+0000 7f9969400640 1 -- start start 2024-05-09T08:05:57.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.390+0000 7f9969400640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f99640c68e0 0x7f99641f3510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:05:57.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.390+0000 7f9969400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9964154870 0x7f99641f0a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:05:57.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.390+0000 7f9969400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f99641e6500 0x7f99640c9cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:05:57.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.390+0000 7f9969400640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f99640c58c0 con 0x7f99641e6500 2024-05-09T08:05:57.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.390+0000 7f9969400640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f99640c55e0 con 0x7f9964154870 2024-05-09T08:05:57.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.390+0000 7f9969400640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f99640c5ba0 con 0x7f99640c68e0 2024-05-09T08:05:57.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.390+0000 7f9962800640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f99640c68e0 0x7f99641f3510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:05:57.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.391+0000 7f9962800640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f99640c68e0 0x7f99641f3510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3301/0 says I am v2:172.21.15.1:34248/0 (socket says 172.21.15.1:34248) 2024-05-09T08:05:57.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.391+0000 7f9962800640 1 -- 172.21.15.1:0/1506687181 learned_addr learned my addr 172.21.15.1:0/1506687181 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:05:57.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.391+0000 7f9961e00640 1 --2- 172.21.15.1:0/1506687181 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9964154870 0x7f99641f0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:05:57.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.391+0000 7f9963200640 1 --2- 172.21.15.1:0/1506687181 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f99641e6500 0x7f99640c9cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:05:57.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.391+0000 7f9962800640 1 -- 172.21.15.1:0/1506687181 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9964154870 msgr2=0x7f99641f0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:05:57.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.391+0000 7f9962800640 1 --2- 172.21.15.1:0/1506687181 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9964154870 0x7f99641f0a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:05:57.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.391+0000 7f9962800640 1 -- 172.21.15.1:0/1506687181 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f99641e6500 msgr2=0x7f99640c9cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:05:57.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.391+0000 7f9962800640 1 --2- 172.21.15.1:0/1506687181 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f99641e6500 0x7f99640c9cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:05:57.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.391+0000 7f9962800640 1 -- 172.21.15.1:0/1506687181 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99640c7250 con 0x7f99640c68e0 2024-05-09T08:05:57.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.391+0000 7f9963200640 1 --2- 172.21.15.1:0/1506687181 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f99641e6500 0x7f99640c9cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T08:05:57.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.391+0000 7f9962800640 1 --2- 172.21.15.1:0/1506687181 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f99640c68e0 0x7f99641f3510 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f9954000c00 tx=0x7f99540046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:05:57.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.392+0000 7f995aa00640 1 -- 172.21.15.1:0/1506687181 <== mon.2 v2:172.21.15.7:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9954037040 con 0x7f99640c68e0 2024-05-09T08:05:57.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.392+0000 7f995aa00640 1 -- 172.21.15.1:0/1506687181 <== mon.2 v2:172.21.15.7:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9954042070 con 0x7f99640c68e0 2024-05-09T08:05:57.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.392+0000 7f995aa00640 1 -- 172.21.15.1:0/1506687181 <== mon.2 v2:172.21.15.7:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9954030840 con 0x7f99640c68e0 2024-05-09T08:05:57.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.392+0000 7f9969400640 1 -- 172.21.15.1:0/1506687181 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99641e3c40 con 0x7f99640c68e0 2024-05-09T08:05:57.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.392+0000 7f9969400640 1 -- 172.21.15.1:0/1506687181 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f99640c74c0 con 0x7f99640c68e0 2024-05-09T08:05:57.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.393+0000 7f9969400640 1 --2- 172.21.15.1:0/1506687181 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9964154c50 0x7f99640cca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:05:57.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.393+0000 7f9969400640 1 -- 172.21.15.1:0/1506687181 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f99640736e0 con 0x7f9964154c50 2024-05-09T08:05:57.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.394+0000 7f9961e00640 1 --2- 172.21.15.1:0/1506687181 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9964154c50 0x7f99640cca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:05:57.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.394+0000 7f9961e00640 1 -- 172.21.15.1:0/1506687181 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f99641f0570 con 0x7f9964154c50 2024-05-09T08:05:57.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.394+0000 7f9961e00640 1 --2- 172.21.15.1:0/1506687181 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9964154c50 0x7f99640cca20 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f99641e6b40 tx=0x7f995000d980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:05:57.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.395+0000 7f995aa00640 1 -- 172.21.15.1:0/1506687181 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9950018020 con 0x7f9964154c50 2024-05-09T08:05:57.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.395+0000 7f995aa00640 1 -- 172.21.15.1:0/1506687181 <== mon.1 v2:172.21.15.7:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f9950014620 con 0x7f9964154c50 2024-05-09T08:05:57.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.395+0000 7f995aa00640 1 -- 172.21.15.1:0/1506687181 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9964154c50 msgr2=0x7f99640cca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:05:57.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.395+0000 7f995aa00640 1 --2- 172.21.15.1:0/1506687181 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9964154c50 0x7f99640cca20 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f99641e6b40 tx=0x7f995000d980 comp rx=0 tx=0).stop 2024-05-09T08:05:57.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.395+0000 7f995aa00640 1 -- 172.21.15.1:0/1506687181 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9964154c50 msgr2=0x7f99640cca20 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-09T08:05:57.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.395+0000 7f995aa00640 1 --2- 172.21.15.1:0/1506687181 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9964154c50 0x7f99640cca20 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:05:57.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.395+0000 7f995aa00640 1 -- 172.21.15.1:0/1506687181 <== mon.2 v2:172.21.15.7:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f99540309e0 con 0x7f99640c68e0 2024-05-09T08:05:57.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.396+0000 7f995aa00640 1 --2- 172.21.15.1:0/1506687181 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f994406b8d0 0x7f994406dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:05:57.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.396+0000 7f995aa00640 1 -- 172.21.15.1:0/1506687181 <== mon.2 v2:172.21.15.7:3301/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7f99540aba60 con 0x7f99640c68e0 2024-05-09T08:05:57.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.396+0000 7f9961e00640 1 --2- 172.21.15.1:0/1506687181 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f994406b8d0 0x7f994406dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:05:57.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.397+0000 7f9961e00640 1 --2- 172.21.15.1:0/1506687181 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f994406b8d0 0x7f994406dd90 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f99641e6b40 tx=0x7f995001f040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:05:57.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.410+0000 7f9969400640 1 --2- 172.21.15.1:0/1506687181 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f996414bd20 0x7f996414c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:05:57.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.410+0000 7f9969400640 1 -- 172.21.15.1:0/1506687181 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9964074d70 con 0x7f996414bd20 2024-05-09T08:05:57.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.410+0000 7f9963200640 1 --2- 172.21.15.1:0/1506687181 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f996414bd20 0x7f996414c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:05:57.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.411+0000 7f9963200640 1 -- 172.21.15.1:0/1506687181 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f99640cbde0 con 0x7f996414bd20 2024-05-09T08:05:57.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.411+0000 7f9963200640 1 --2- 172.21.15.1:0/1506687181 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f996414bd20 0x7f996414c100 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f995c009f80 tx=0x7f995c00f520 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:05:57.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.411+0000 7f995aa00640 1 -- 172.21.15.1:0/1506687181 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f995c00fe80 con 0x7f996414bd20 2024-05-09T08:05:57.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.412+0000 7f995aa00640 1 -- 172.21.15.1:0/1506687181 <== mon.1 v2:172.21.15.7:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4706 (secure 0 0 0) 0x7f995c00fe80 con 0x7f996414bd20 2024-05-09T08:05:57.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.412+0000 7f995aa00640 1 -- 172.21.15.1:0/1506687181 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f996414bd20 msgr2=0x7f996414c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:05:57.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.412+0000 7f995aa00640 1 --2- 172.21.15.1:0/1506687181 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f996414bd20 0x7f996414c100 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f995c009f80 tx=0x7f995c00f520 comp rx=0 tx=0).stop 2024-05-09T08:05:57.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.412+0000 7f995aa00640 1 -- 172.21.15.1:0/1506687181 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f996414bd20 msgr2=0x7f996414c100 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-09T08:05:57.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.412+0000 7f995aa00640 1 --2- 172.21.15.1:0/1506687181 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f996414bd20 0x7f996414c100 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:05:57.425 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-05-09T08:05:57.425 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "b", 2024-05-09T08:05:57.426 INFO:teuthology.orchestra.run.smithi001.stdout: "rank": 1, 2024-05-09T08:05:57.426 INFO:teuthology.orchestra.run.smithi001.stdout: "state": "peon", 2024-05-09T08:05:57.426 INFO:teuthology.orchestra.run.smithi001.stdout: "election_epoch": 30, 2024-05-09T08:05:57.426 INFO:teuthology.orchestra.run.smithi001.stdout: "uptime": 125074, 2024-05-09T08:05:57.426 INFO:teuthology.orchestra.run.smithi001.stdout: "quorum": [ 2024-05-09T08:05:57.426 INFO:teuthology.orchestra.run.smithi001.stdout: 0, 2024-05-09T08:05:57.426 INFO:teuthology.orchestra.run.smithi001.stdout: 1, 2024-05-09T08:05:57.426 INFO:teuthology.orchestra.run.smithi001.stdout: 2 2024-05-09T08:05:57.426 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:05:57.426 INFO:teuthology.orchestra.run.smithi001.stdout: "quorum_age": 13, 2024-05-09T08:05:57.426 INFO:teuthology.orchestra.run.smithi001.stdout: "features": { 2024-05-09T08:05:57.426 INFO:teuthology.orchestra.run.smithi001.stdout: "required_con": "2450521708007866372", 2024-05-09T08:05:57.426 INFO:teuthology.orchestra.run.smithi001.stdout: "required_mon": [ 2024-05-09T08:05:57.427 INFO:teuthology.orchestra.run.smithi001.stdout: "kraken", 2024-05-09T08:05:57.427 INFO:teuthology.orchestra.run.smithi001.stdout: "luminous", 2024-05-09T08:05:57.427 INFO:teuthology.orchestra.run.smithi001.stdout: "mimic", 2024-05-09T08:05:57.427 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap-prune", 2024-05-09T08:05:57.427 INFO:teuthology.orchestra.run.smithi001.stdout: "nautilus", 2024-05-09T08:05:57.427 INFO:teuthology.orchestra.run.smithi001.stdout: "octopus", 2024-05-09T08:05:57.427 INFO:teuthology.orchestra.run.smithi001.stdout: "pacific", 2024-05-09T08:05:57.427 INFO:teuthology.orchestra.run.smithi001.stdout: "elector-pinging", 2024-05-09T08:05:57.427 INFO:teuthology.orchestra.run.smithi001.stdout: "quincy", 2024-05-09T08:05:57.427 INFO:teuthology.orchestra.run.smithi001.stdout: "reef", 2024-05-09T08:05:57.427 INFO:teuthology.orchestra.run.smithi001.stdout: "squid" 2024-05-09T08:05:57.427 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:05:57.427 INFO:teuthology.orchestra.run.smithi001.stdout: "quorum_con": "4540701547738038271", 2024-05-09T08:05:57.427 INFO:teuthology.orchestra.run.smithi001.stdout: "quorum_mon": [ 2024-05-09T08:05:57.428 INFO:teuthology.orchestra.run.smithi001.stdout: "kraken", 2024-05-09T08:05:57.428 INFO:teuthology.orchestra.run.smithi001.stdout: "luminous", 2024-05-09T08:05:57.428 INFO:teuthology.orchestra.run.smithi001.stdout: "mimic", 2024-05-09T08:05:57.428 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap-prune", 2024-05-09T08:05:57.428 INFO:teuthology.orchestra.run.smithi001.stdout: "nautilus", 2024-05-09T08:05:57.428 INFO:teuthology.orchestra.run.smithi001.stdout: "octopus", 2024-05-09T08:05:57.428 INFO:teuthology.orchestra.run.smithi001.stdout: "pacific", 2024-05-09T08:05:57.428 INFO:teuthology.orchestra.run.smithi001.stdout: "elector-pinging", 2024-05-09T08:05:57.428 INFO:teuthology.orchestra.run.smithi001.stdout: "quincy", 2024-05-09T08:05:57.428 INFO:teuthology.orchestra.run.smithi001.stdout: "reef", 2024-05-09T08:05:57.428 INFO:teuthology.orchestra.run.smithi001.stdout: "squid" 2024-05-09T08:05:57.428 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:05:57.428 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:05:57.428 INFO:teuthology.orchestra.run.smithi001.stdout: "outside_quorum": [], 2024-05-09T08:05:57.429 INFO:teuthology.orchestra.run.smithi001.stdout: "extra_probe_peers": [], 2024-05-09T08:05:57.429 INFO:teuthology.orchestra.run.smithi001.stdout: "sync_provider": [], 2024-05-09T08:05:57.429 INFO:teuthology.orchestra.run.smithi001.stdout: "monmap": { 2024-05-09T08:05:57.429 INFO:teuthology.orchestra.run.smithi001.stdout: "epoch": 1, 2024-05-09T08:05:57.429 INFO:teuthology.orchestra.run.smithi001.stdout: "fsid": "54329ec8-3270-4043-9a82-c6cd7420aa7f", 2024-05-09T08:05:57.429 INFO:teuthology.orchestra.run.smithi001.stdout: "modified": "2024-05-09T08:01:21.819270Z", 2024-05-09T08:05:57.429 INFO:teuthology.orchestra.run.smithi001.stdout: "created": "2024-05-09T08:01:21.819270Z", 2024-05-09T08:05:57.429 INFO:teuthology.orchestra.run.smithi001.stdout: "min_mon_release": 19, 2024-05-09T08:05:57.429 INFO:teuthology.orchestra.run.smithi001.stdout: "min_mon_release_name": "squid", 2024-05-09T08:05:57.429 INFO:teuthology.orchestra.run.smithi001.stdout: "election_strategy": 1, 2024-05-09T08:05:57.429 INFO:teuthology.orchestra.run.smithi001.stdout: "disallowed_leaders: ": "", 2024-05-09T08:05:57.429 INFO:teuthology.orchestra.run.smithi001.stdout: "stretch_mode": false, 2024-05-09T08:05:57.429 INFO:teuthology.orchestra.run.smithi001.stdout: "tiebreaker_mon": "", 2024-05-09T08:05:57.429 INFO:teuthology.orchestra.run.smithi001.stdout: "removed_ranks: ": "", 2024-05-09T08:05:57.430 INFO:teuthology.orchestra.run.smithi001.stdout: "features": { 2024-05-09T08:05:57.430 INFO:teuthology.orchestra.run.smithi001.stdout: "persistent": [ 2024-05-09T08:05:57.430 INFO:teuthology.orchestra.run.smithi001.stdout: "kraken", 2024-05-09T08:05:57.430 INFO:teuthology.orchestra.run.smithi001.stdout: "luminous", 2024-05-09T08:05:57.430 INFO:teuthology.orchestra.run.smithi001.stdout: "mimic", 2024-05-09T08:05:57.430 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap-prune", 2024-05-09T08:05:57.430 INFO:teuthology.orchestra.run.smithi001.stdout: "nautilus", 2024-05-09T08:05:57.430 INFO:teuthology.orchestra.run.smithi001.stdout: "octopus", 2024-05-09T08:05:57.430 INFO:teuthology.orchestra.run.smithi001.stdout: "pacific", 2024-05-09T08:05:57.430 INFO:teuthology.orchestra.run.smithi001.stdout: "elector-pinging", 2024-05-09T08:05:57.430 INFO:teuthology.orchestra.run.smithi001.stdout: "quincy", 2024-05-09T08:05:57.430 INFO:teuthology.orchestra.run.smithi001.stdout: "reef", 2024-05-09T08:05:57.431 INFO:teuthology.orchestra.run.smithi001.stdout: "squid" 2024-05-09T08:05:57.431 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:05:57.431 INFO:teuthology.orchestra.run.smithi001.stdout: "optional": [] 2024-05-09T08:05:57.431 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:05:57.431 INFO:teuthology.orchestra.run.smithi001.stdout: "mons": [ 2024-05-09T08:05:57.431 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:05:57.431 INFO:teuthology.orchestra.run.smithi001.stdout: "rank": 0, 2024-05-09T08:05:57.431 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "a", 2024-05-09T08:05:57.431 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-05-09T08:05:57.431 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-05-09T08:05:57.431 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:05:57.431 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-05-09T08:05:57.431 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:3300", 2024-05-09T08:05:57.431 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:05:57.432 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:05:57.432 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:05:57.432 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-05-09T08:05:57.432 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6789", 2024-05-09T08:05:57.432 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:05:57.432 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:05:57.432 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:05:57.432 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:05:57.432 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6789/0", 2024-05-09T08:05:57.432 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.1:6789/0", 2024-05-09T08:05:57.432 INFO:teuthology.orchestra.run.smithi001.stdout: "priority": 0, 2024-05-09T08:05:57.432 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 0, 2024-05-09T08:05:57.432 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_location": "{}" 2024-05-09T08:05:57.432 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:05:57.433 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:05:57.433 INFO:teuthology.orchestra.run.smithi001.stdout: "rank": 1, 2024-05-09T08:05:57.433 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "b", 2024-05-09T08:05:57.433 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-05-09T08:05:57.433 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-05-09T08:05:57.433 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:05:57.433 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-05-09T08:05:57.433 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:3300", 2024-05-09T08:05:57.433 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:05:57.433 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:05:57.433 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:05:57.433 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-05-09T08:05:57.433 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:6789", 2024-05-09T08:05:57.433 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:05:57.434 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:05:57.434 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:05:57.434 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:05:57.434 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:6789/0", 2024-05-09T08:05:57.434 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.7:6789/0", 2024-05-09T08:05:57.434 INFO:teuthology.orchestra.run.smithi001.stdout: "priority": 0, 2024-05-09T08:05:57.434 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 0, 2024-05-09T08:05:57.434 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_location": "{}" 2024-05-09T08:05:57.434 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:05:57.434 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:05:57.434 INFO:teuthology.orchestra.run.smithi001.stdout: "rank": 2, 2024-05-09T08:05:57.434 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "c", 2024-05-09T08:05:57.434 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-05-09T08:05:57.435 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-05-09T08:05:57.435 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:05:57.435 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-05-09T08:05:57.435 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:3301", 2024-05-09T08:05:57.435 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:05:57.435 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:05:57.435 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:05:57.435 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-05-09T08:05:57.435 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:6790", 2024-05-09T08:05:57.435 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:05:57.435 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:05:57.435 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:05:57.435 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:05:57.435 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:6790/0", 2024-05-09T08:05:57.436 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.7:6790/0", 2024-05-09T08:05:57.436 INFO:teuthology.orchestra.run.smithi001.stdout: "priority": 0, 2024-05-09T08:05:57.436 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 0, 2024-05-09T08:05:57.436 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_location": "{}" 2024-05-09T08:05:57.436 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:05:57.436 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:05:57.436 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:05:57.436 INFO:teuthology.orchestra.run.smithi001.stdout: "feature_map": { 2024-05-09T08:05:57.436 INFO:teuthology.orchestra.run.smithi001.stdout: "mon": [ 2024-05-09T08:05:57.436 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:05:57.436 INFO:teuthology.orchestra.run.smithi001.stdout: "features": "0x3f03cffffffdffff", 2024-05-09T08:05:57.436 INFO:teuthology.orchestra.run.smithi001.stdout: "release": "luminous", 2024-05-09T08:05:57.436 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 1 2024-05-09T08:05:57.436 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:05:57.437 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:05:57.437 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": [ 2024-05-09T08:05:57.437 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:05:57.437 INFO:teuthology.orchestra.run.smithi001.stdout: "features": "0x3f03cffffffdffff", 2024-05-09T08:05:57.437 INFO:teuthology.orchestra.run.smithi001.stdout: "release": "luminous", 2024-05-09T08:05:57.437 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 1 2024-05-09T08:05:57.437 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:05:57.437 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:05:57.437 INFO:teuthology.orchestra.run.smithi001.stdout: "client": [ 2024-05-09T08:05:57.437 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:05:57.437 INFO:teuthology.orchestra.run.smithi001.stdout: "features": "0x3f03cffffffdffff", 2024-05-09T08:05:57.437 INFO:teuthology.orchestra.run.smithi001.stdout: "release": "luminous", 2024-05-09T08:05:57.437 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 1 2024-05-09T08:05:57.437 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:05:57.438 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:05:57.438 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:05:57.438 INFO:teuthology.orchestra.run.smithi001.stdout: "stretch_mode": false 2024-05-09T08:05:57.438 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-05-09T08:05:57.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.413+0000 7f9969400640 1 -- 172.21.15.1:0/1506687181 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f994406b8d0 msgr2=0x7f994406dd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:05:57.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.413+0000 7f9969400640 1 --2- 172.21.15.1:0/1506687181 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f994406b8d0 0x7f994406dd90 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f99641e6b40 tx=0x7f995001f040 comp rx=0 tx=0).stop 2024-05-09T08:05:57.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.413+0000 7f9969400640 1 -- 172.21.15.1:0/1506687181 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f99640c68e0 msgr2=0x7f99641f3510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:05:57.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.413+0000 7f9969400640 1 --2- 172.21.15.1:0/1506687181 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f99640c68e0 0x7f99641f3510 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f9954000c00 tx=0x7f99540046c0 comp rx=0 tx=0).stop 2024-05-09T08:05:57.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.413+0000 7f9963200640 1 -- 172.21.15.1:0/1506687181 reap_dead start 2024-05-09T08:05:57.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.413+0000 7f9969400640 1 -- 172.21.15.1:0/1506687181 shutdown_connections 2024-05-09T08:05:57.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.413+0000 7f9969400640 1 -- 172.21.15.1:0/1506687181 >> 172.21.15.1:0/1506687181 conn(0x7f996408bff0 msgr2=0x7f996408c3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:05:57.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.414+0000 7f9969400640 1 -- 172.21.15.1:0/1506687181 shutdown_connections 2024-05-09T08:05:57.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.414+0000 7f9969400640 1 -- 172.21.15.1:0/1506687181 wait complete. 2024-05-09T08:05:57.459 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-05-09T08:05:57.459 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-05-09T08:05:57.459 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-05-09T08:05:57.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.709+0000 7f06aae00640 1 Processor -- start 2024-05-09T08:05:58.671 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.710+0000 7f06aae00640 1 -- start start 2024-05-09T08:05:58.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.712+0000 7f06aae00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f06a40c9750 0x7f06a40c7d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:05:58.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.712+0000 7f06aae00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f06a4072890 0x7f06a415a880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:05:58.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.712+0000 7f06aae00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f06a4073750 con 0x7f06a4154910 2024-05-09T08:05:58.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.712+0000 7f06aae00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f06a4073190 con 0x7f06a4072890 2024-05-09T08:05:58.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.712+0000 7f06aae00640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f06a4073470 con 0x7f06a40c9750 2024-05-09T08:05:58.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.712+0000 7f06a8e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f06a4072890 0x7f06a415a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:05:58.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.712+0000 7f06a8e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f06a4072890 0x7f06a415a880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.1:52184/0 (socket says 172.21.15.1:52184) 2024-05-09T08:05:58.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.712+0000 7f06a8e00640 1 -- 172.21.15.1:0/3314335347 learned_addr learned my addr 172.21.15.1:0/3314335347 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:05:58.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.712+0000 7f069be00640 1 --2- 172.21.15.1:0/3314335347 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f06a40c9750 0x7f06a40c7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:05:58.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.713+0000 7f069be00640 1 -- 172.21.15.1:0/3314335347 >> v2:172.21.15.7:3300/0 conn(0x7f06a4072890 msgr2=0x7f06a415a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:05:58.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.713+0000 7f069be00640 1 --2- 172.21.15.1:0/3314335347 >> v2:172.21.15.7:3300/0 conn(0x7f06a4072890 0x7f06a415a880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:05:58.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.713+0000 7f069be00640 1 -- 172.21.15.1:0/3314335347 >> v1:172.21.15.7:6789/0 conn(0x7f06a4154910 legacy=0x7f06a406f720 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T08:05:58.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.713+0000 7f069be00640 1 -- 172.21.15.1:0/3314335347 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06a4071d90 con 0x7f06a40c9750 2024-05-09T08:05:58.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.715+0000 7f069be00640 1 --2- 172.21.15.1:0/3314335347 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f06a40c9750 0x7f06a40c7d20 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f06940097a0 tx=0x7f069402a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=92d3da2fd9e60407 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:05:58.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.715+0000 7f06a3400640 1 -- 172.21.15.1:0/3314335347 <== mon.2 v2:172.21.15.7:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0694037070 con 0x7f06a40c9750 2024-05-09T08:05:58.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.716+0000 7f06a3400640 1 -- 172.21.15.1:0/3314335347 <== mon.2 v2:172.21.15.7:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0694032440 con 0x7f06a40c9750 2024-05-09T08:05:58.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.716+0000 7f06a3400640 1 -- 172.21.15.1:0/3314335347 <== mon.2 v2:172.21.15.7:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0694032810 con 0x7f06a40c9750 2024-05-09T08:05:58.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.717+0000 7f06aae00640 1 -- 172.21.15.1:0/3314335347 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f06a40c9750 msgr2=0x7f06a40c7d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:05:58.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.717+0000 7f06aae00640 1 --2- 172.21.15.1:0/3314335347 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f06a40c9750 0x7f06a40c7d20 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f06940097a0 tx=0x7f069402a1c0 comp rx=0 tx=0).stop 2024-05-09T08:05:58.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.720+0000 7f06aae00640 1 -- 172.21.15.1:0/3314335347 shutdown_connections 2024-05-09T08:05:58.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.720+0000 7f06aae00640 1 --2- 172.21.15.1:0/3314335347 >> v2:172.21.15.7:3300/0 conn(0x7f06a4072890 0x7f06a415a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:05:58.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.720+0000 7f06aae00640 1 --2- 172.21.15.1:0/3314335347 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f06a40c9750 0x7f06a40c7d20 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:05:58.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.720+0000 7f06aae00640 1 -- 172.21.15.1:0/3314335347 >> 172.21.15.1:0/3314335347 conn(0x7f06a406dad0 msgr2=0x7f06a406deb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:05:58.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.723+0000 7f06aae00640 1 -- 172.21.15.1:0/3314335347 shutdown_connections 2024-05-09T08:05:58.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.723+0000 7f06aae00640 1 -- 172.21.15.1:0/3314335347 wait complete. 2024-05-09T08:05:58.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.726+0000 7f06aae00640 1 Processor -- start 2024-05-09T08:05:58.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.731+0000 7f06aae00640 1 -- start start 2024-05-09T08:05:58.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.739+0000 7f06aae00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06a4072890 0x7f06a41550f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:05:58.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.739+0000 7f06aae00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f06a40c9750 0x7f06a414f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:05:58.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.739+0000 7f06aae00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f06a4154910 0x7f06a40c51a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:05:58.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.739+0000 7f06aae00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f06a406ffb0 con 0x7f06a4154910 2024-05-09T08:05:58.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.739+0000 7f06aae00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f06a406fe30 con 0x7f06a4072890 2024-05-09T08:05:58.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.739+0000 7f06aae00640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f06a415adc0 con 0x7f06a40c9750 2024-05-09T08:05:58.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.739+0000 7f069be00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f06a40c9750 0x7f06a414f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:05:58.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.739+0000 7f069be00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f06a40c9750 0x7f06a414f810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3301/0 says I am v2:172.21.15.1:34280/0 (socket says 172.21.15.1:34280) 2024-05-09T08:05:58.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.739+0000 7f069be00640 1 -- 172.21.15.1:0/855219089 learned_addr learned my addr 172.21.15.1:0/855219089 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:05:58.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.740+0000 7f06a3e00640 1 --2- 172.21.15.1:0/855219089 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06a4072890 0x7f06a41550f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:05:58.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.740+0000 7f06a8e00640 1 --2- 172.21.15.1:0/855219089 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f06a4154910 0x7f06a40c51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:05:58.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.741+0000 7f06a8e00640 1 -- 172.21.15.1:0/855219089 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f06a40c9750 msgr2=0x7f06a414f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:05:58.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.741+0000 7f06a8e00640 1 --2- 172.21.15.1:0/855219089 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f06a40c9750 0x7f06a414f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:05:58.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.741+0000 7f06a8e00640 1 -- 172.21.15.1:0/855219089 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06a4072890 msgr2=0x7f06a41550f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:05:58.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.741+0000 7f06a8e00640 1 --2- 172.21.15.1:0/855219089 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06a4072890 0x7f06a41550f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:05:58.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.741+0000 7f06a8e00640 1 -- 172.21.15.1:0/855219089 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06a40cc9e0 con 0x7f06a4154910 2024-05-09T08:05:58.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.741+0000 7f06a8e00640 1 --2- 172.21.15.1:0/855219089 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f06a4154910 0x7f06a40c51a0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f069c007a70 tx=0x7f069c007f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:05:58.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.741+0000 7f06a0c00640 1 -- 172.21.15.1:0/855219089 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f069c01a070 con 0x7f06a4154910 2024-05-09T08:05:58.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.741+0000 7f06a0c00640 1 -- 172.21.15.1:0/855219089 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f069c00dc40 con 0x7f06a4154910 2024-05-09T08:05:58.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.741+0000 7f06a0c00640 1 -- 172.21.15.1:0/855219089 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f069c015400 con 0x7f06a4154910 2024-05-09T08:05:58.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.741+0000 7f06aae00640 1 -- 172.21.15.1:0/855219089 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06a40cccc0 con 0x7f06a4154910 2024-05-09T08:05:58.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.742+0000 7f06aae00640 1 -- 172.21.15.1:0/855219089 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06a40c46c0 con 0x7f06a4154910 2024-05-09T08:05:58.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.744+0000 7f06aae00640 1 --2- 172.21.15.1:0/855219089 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f06a40c61d0 0x7f06a40c56e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:05:58.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.744+0000 7f06aae00640 1 -- 172.21.15.1:0/855219089 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f06a40c4ec0 con 0x7f06a40c61d0 2024-05-09T08:05:58.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.745+0000 7f06a3e00640 1 --2- 172.21.15.1:0/855219089 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f06a40c61d0 0x7f06a40c56e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:05:58.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.745+0000 7f06a0c00640 1 -- 172.21.15.1:0/855219089 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f069c01d070 con 0x7f06a4154910 2024-05-09T08:05:58.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.746+0000 7f06a3e00640 1 -- 172.21.15.1:0/855219089 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f06a4074d70 con 0x7f06a40c61d0 2024-05-09T08:05:58.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.746+0000 7f06a3e00640 1 --2- 172.21.15.1:0/855219089 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f06a40c61d0 0x7f06a40c56e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f068c00e7a0 tx=0x7f068c00ec60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:05:58.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.747+0000 7f06a0c00640 1 --2- 172.21.15.1:0/855219089 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f068006be80 0x7f068006e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:05:58.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.747+0000 7f06a0c00640 1 -- 172.21.15.1:0/855219089 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7f069c08aff0 con 0x7f06a4154910 2024-05-09T08:05:58.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.747+0000 7f069be00640 1 --2- 172.21.15.1:0/855219089 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f068006be80 0x7f068006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:05:58.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.747+0000 7f06a0c00640 1 -- 172.21.15.1:0/855219089 <== mon.2 v2:172.21.15.7:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f068c017070 con 0x7f06a40c61d0 2024-05-09T08:05:58.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.747+0000 7f06a0c00640 1 -- 172.21.15.1:0/855219089 <== mon.2 v2:172.21.15.7:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f068c00c740 con 0x7f06a40c61d0 2024-05-09T08:05:58.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.747+0000 7f06a0c00640 1 -- 172.21.15.1:0/855219089 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f06a40c61d0 msgr2=0x7f06a40c56e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:05:58.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.747+0000 7f06a0c00640 1 --2- 172.21.15.1:0/855219089 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f06a40c61d0 0x7f06a40c56e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f068c00e7a0 tx=0x7f068c00ec60 comp rx=0 tx=0).stop 2024-05-09T08:05:58.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.747+0000 7f06a0c00640 1 -- 172.21.15.1:0/855219089 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f06a40c61d0 msgr2=0x7f06a40c56e0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-05-09T08:05:58.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.747+0000 7f06a0c00640 1 --2- 172.21.15.1:0/855219089 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f06a40c61d0 0x7f06a40c56e0 secure :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f068c00e7a0 tx=0x7f068c00ec60 comp rx=0 tx=0).stop 2024-05-09T08:05:58.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.748+0000 7f069be00640 1 --2- 172.21.15.1:0/855219089 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f068006be80 0x7f068006e340 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f069402a200 tx=0x7f069402e000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:05:58.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.758+0000 7f06aae00640 1 --2- 172.21.15.1:0/855219089 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f06a40c4890 0x7f06a41f8330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:05:58.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.758+0000 7f06aae00640 1 -- 172.21.15.1:0/855219089 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f06a40c2eb0 con 0x7f06a40c4890 2024-05-09T08:05:58.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.759+0000 7f06a3e00640 1 --2- 172.21.15.1:0/855219089 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f06a40c4890 0x7f06a41f8330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:05:58.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.759+0000 7f06a3e00640 1 -- 172.21.15.1:0/855219089 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f06a40c16c0 con 0x7f06a40c4890 2024-05-09T08:05:58.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.760+0000 7f06a3e00640 1 --2- 172.21.15.1:0/855219089 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f06a40c4890 0x7f06a41f8330 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f068c00e7a0 tx=0x7f068c0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:05:58.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.760+0000 7f06a0c00640 1 -- 172.21.15.1:0/855219089 <== mon.2 v2:172.21.15.7:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f068c017040 con 0x7f06a40c4890 2024-05-09T08:05:58.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.760+0000 7f06a0c00640 1 -- 172.21.15.1:0/855219089 <== mon.2 v2:172.21.15.7:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5189 (secure 0 0 0) 0x7f068c013c00 con 0x7f06a40c4890 2024-05-09T08:05:58.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.760+0000 7f06a0c00640 1 -- 172.21.15.1:0/855219089 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f06a40c4890 msgr2=0x7f06a41f8330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:05:58.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.760+0000 7f06a0c00640 1 --2- 172.21.15.1:0/855219089 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f06a40c4890 0x7f06a41f8330 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f068c00e7a0 tx=0x7f068c0046c0 comp rx=0 tx=0).stop 2024-05-09T08:05:58.685 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-05-09T08:05:58.685 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "c", 2024-05-09T08:05:58.686 INFO:teuthology.orchestra.run.smithi001.stdout: "rank": 2, 2024-05-09T08:05:58.686 INFO:teuthology.orchestra.run.smithi001.stdout: "state": "peon", 2024-05-09T08:05:58.686 INFO:teuthology.orchestra.run.smithi001.stdout: "election_epoch": 30, 2024-05-09T08:05:58.686 INFO:teuthology.orchestra.run.smithi001.stdout: "uptime": 241176, 2024-05-09T08:05:58.686 INFO:teuthology.orchestra.run.smithi001.stdout: "quorum": [ 2024-05-09T08:05:58.686 INFO:teuthology.orchestra.run.smithi001.stdout: 0, 2024-05-09T08:05:58.686 INFO:teuthology.orchestra.run.smithi001.stdout: 1, 2024-05-09T08:05:58.686 INFO:teuthology.orchestra.run.smithi001.stdout: 2 2024-05-09T08:05:58.686 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:05:58.686 INFO:teuthology.orchestra.run.smithi001.stdout: "quorum_age": 14, 2024-05-09T08:05:58.686 INFO:teuthology.orchestra.run.smithi001.stdout: "features": { 2024-05-09T08:05:58.686 INFO:teuthology.orchestra.run.smithi001.stdout: "required_con": "2450521708007866372", 2024-05-09T08:05:58.686 INFO:teuthology.orchestra.run.smithi001.stdout: "required_mon": [ 2024-05-09T08:05:58.687 INFO:teuthology.orchestra.run.smithi001.stdout: "kraken", 2024-05-09T08:05:58.687 INFO:teuthology.orchestra.run.smithi001.stdout: "luminous", 2024-05-09T08:05:58.687 INFO:teuthology.orchestra.run.smithi001.stdout: "mimic", 2024-05-09T08:05:58.687 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap-prune", 2024-05-09T08:05:58.687 INFO:teuthology.orchestra.run.smithi001.stdout: "nautilus", 2024-05-09T08:05:58.687 INFO:teuthology.orchestra.run.smithi001.stdout: "octopus", 2024-05-09T08:05:58.687 INFO:teuthology.orchestra.run.smithi001.stdout: "pacific", 2024-05-09T08:05:58.687 INFO:teuthology.orchestra.run.smithi001.stdout: "elector-pinging", 2024-05-09T08:05:58.687 INFO:teuthology.orchestra.run.smithi001.stdout: "quincy", 2024-05-09T08:05:58.687 INFO:teuthology.orchestra.run.smithi001.stdout: "reef", 2024-05-09T08:05:58.687 INFO:teuthology.orchestra.run.smithi001.stdout: "squid" 2024-05-09T08:05:58.687 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:05:58.687 INFO:teuthology.orchestra.run.smithi001.stdout: "quorum_con": "4540701547738038271", 2024-05-09T08:05:58.687 INFO:teuthology.orchestra.run.smithi001.stdout: "quorum_mon": [ 2024-05-09T08:05:58.688 INFO:teuthology.orchestra.run.smithi001.stdout: "kraken", 2024-05-09T08:05:58.688 INFO:teuthology.orchestra.run.smithi001.stdout: "luminous", 2024-05-09T08:05:58.688 INFO:teuthology.orchestra.run.smithi001.stdout: "mimic", 2024-05-09T08:05:58.688 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap-prune", 2024-05-09T08:05:58.688 INFO:teuthology.orchestra.run.smithi001.stdout: "nautilus", 2024-05-09T08:05:58.688 INFO:teuthology.orchestra.run.smithi001.stdout: "octopus", 2024-05-09T08:05:58.688 INFO:teuthology.orchestra.run.smithi001.stdout: "pacific", 2024-05-09T08:05:58.688 INFO:teuthology.orchestra.run.smithi001.stdout: "elector-pinging", 2024-05-09T08:05:58.688 INFO:teuthology.orchestra.run.smithi001.stdout: "quincy", 2024-05-09T08:05:58.688 INFO:teuthology.orchestra.run.smithi001.stdout: "reef", 2024-05-09T08:05:58.688 INFO:teuthology.orchestra.run.smithi001.stdout: "squid" 2024-05-09T08:05:58.688 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:05:58.688 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:05:58.688 INFO:teuthology.orchestra.run.smithi001.stdout: "outside_quorum": [], 2024-05-09T08:05:58.688 INFO:teuthology.orchestra.run.smithi001.stdout: "extra_probe_peers": [], 2024-05-09T08:05:58.689 INFO:teuthology.orchestra.run.smithi001.stdout: "sync_provider": [], 2024-05-09T08:05:58.689 INFO:teuthology.orchestra.run.smithi001.stdout: "monmap": { 2024-05-09T08:05:58.689 INFO:teuthology.orchestra.run.smithi001.stdout: "epoch": 1, 2024-05-09T08:05:58.689 INFO:teuthology.orchestra.run.smithi001.stdout: "fsid": "54329ec8-3270-4043-9a82-c6cd7420aa7f", 2024-05-09T08:05:58.689 INFO:teuthology.orchestra.run.smithi001.stdout: "modified": "2024-05-09T08:01:21.819270Z", 2024-05-09T08:05:58.689 INFO:teuthology.orchestra.run.smithi001.stdout: "created": "2024-05-09T08:01:21.819270Z", 2024-05-09T08:05:58.689 INFO:teuthology.orchestra.run.smithi001.stdout: "min_mon_release": 19, 2024-05-09T08:05:58.689 INFO:teuthology.orchestra.run.smithi001.stdout: "min_mon_release_name": "squid", 2024-05-09T08:05:58.689 INFO:teuthology.orchestra.run.smithi001.stdout: "election_strategy": 1, 2024-05-09T08:05:58.689 INFO:teuthology.orchestra.run.smithi001.stdout: "disallowed_leaders: ": "", 2024-05-09T08:05:58.689 INFO:teuthology.orchestra.run.smithi001.stdout: "stretch_mode": false, 2024-05-09T08:05:58.689 INFO:teuthology.orchestra.run.smithi001.stdout: "tiebreaker_mon": "", 2024-05-09T08:05:58.689 INFO:teuthology.orchestra.run.smithi001.stdout: "removed_ranks: ": "", 2024-05-09T08:05:58.689 INFO:teuthology.orchestra.run.smithi001.stdout: "features": { 2024-05-09T08:05:58.690 INFO:teuthology.orchestra.run.smithi001.stdout: "persistent": [ 2024-05-09T08:05:58.690 INFO:teuthology.orchestra.run.smithi001.stdout: "kraken", 2024-05-09T08:05:58.690 INFO:teuthology.orchestra.run.smithi001.stdout: "luminous", 2024-05-09T08:05:58.690 INFO:teuthology.orchestra.run.smithi001.stdout: "mimic", 2024-05-09T08:05:58.690 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap-prune", 2024-05-09T08:05:58.690 INFO:teuthology.orchestra.run.smithi001.stdout: "nautilus", 2024-05-09T08:05:58.690 INFO:teuthology.orchestra.run.smithi001.stdout: "octopus", 2024-05-09T08:05:58.690 INFO:teuthology.orchestra.run.smithi001.stdout: "pacific", 2024-05-09T08:05:58.690 INFO:teuthology.orchestra.run.smithi001.stdout: "elector-pinging", 2024-05-09T08:05:58.690 INFO:teuthology.orchestra.run.smithi001.stdout: "quincy", 2024-05-09T08:05:58.690 INFO:teuthology.orchestra.run.smithi001.stdout: "reef", 2024-05-09T08:05:58.690 INFO:teuthology.orchestra.run.smithi001.stdout: "squid" 2024-05-09T08:05:58.690 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:05:58.690 INFO:teuthology.orchestra.run.smithi001.stdout: "optional": [] 2024-05-09T08:05:58.691 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:05:58.691 INFO:teuthology.orchestra.run.smithi001.stdout: "mons": [ 2024-05-09T08:05:58.691 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:05:58.691 INFO:teuthology.orchestra.run.smithi001.stdout: "rank": 0, 2024-05-09T08:05:58.691 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "a", 2024-05-09T08:05:58.691 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-05-09T08:05:58.691 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-05-09T08:05:58.691 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:05:58.691 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-05-09T08:05:58.691 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:3300", 2024-05-09T08:05:58.691 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:05:58.691 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:05:58.691 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:05:58.691 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-05-09T08:05:58.692 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6789", 2024-05-09T08:05:58.692 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:05:58.692 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:05:58.692 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:05:58.692 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:05:58.692 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6789/0", 2024-05-09T08:05:58.692 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.1:6789/0", 2024-05-09T08:05:58.692 INFO:teuthology.orchestra.run.smithi001.stdout: "priority": 0, 2024-05-09T08:05:58.692 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 0, 2024-05-09T08:05:58.692 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_location": "{}" 2024-05-09T08:05:58.692 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:05:58.692 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:05:58.692 INFO:teuthology.orchestra.run.smithi001.stdout: "rank": 1, 2024-05-09T08:05:58.692 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "b", 2024-05-09T08:05:58.693 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-05-09T08:05:58.693 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-05-09T08:05:58.693 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:05:58.693 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-05-09T08:05:58.693 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:3300", 2024-05-09T08:05:58.693 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:05:58.693 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:05:58.693 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:05:58.693 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-05-09T08:05:58.693 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:6789", 2024-05-09T08:05:58.693 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:05:58.693 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:05:58.693 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:05:58.693 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:05:58.694 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:6789/0", 2024-05-09T08:05:58.694 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.7:6789/0", 2024-05-09T08:05:58.694 INFO:teuthology.orchestra.run.smithi001.stdout: "priority": 0, 2024-05-09T08:05:58.694 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 0, 2024-05-09T08:05:58.694 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_location": "{}" 2024-05-09T08:05:58.694 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:05:58.694 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:05:58.694 INFO:teuthology.orchestra.run.smithi001.stdout: "rank": 2, 2024-05-09T08:05:58.694 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "c", 2024-05-09T08:05:58.694 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-05-09T08:05:58.694 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-05-09T08:05:58.694 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:05:58.694 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-05-09T08:05:58.694 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:3301", 2024-05-09T08:05:58.695 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:05:58.695 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:05:58.695 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:05:58.695 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-05-09T08:05:58.695 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:6790", 2024-05-09T08:05:58.695 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:05:58.695 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:05:58.695 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:05:58.695 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:05:58.695 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:6790/0", 2024-05-09T08:05:58.695 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.7:6790/0", 2024-05-09T08:05:58.695 INFO:teuthology.orchestra.run.smithi001.stdout: "priority": 0, 2024-05-09T08:05:58.695 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 0, 2024-05-09T08:05:58.695 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_location": "{}" 2024-05-09T08:05:58.696 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:05:58.696 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:05:58.696 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:05:58.696 INFO:teuthology.orchestra.run.smithi001.stdout: "feature_map": { 2024-05-09T08:05:58.696 INFO:teuthology.orchestra.run.smithi001.stdout: "mon": [ 2024-05-09T08:05:58.696 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:05:58.696 INFO:teuthology.orchestra.run.smithi001.stdout: "features": "0x3f03cffffffdffff", 2024-05-09T08:05:58.696 INFO:teuthology.orchestra.run.smithi001.stdout: "release": "luminous", 2024-05-09T08:05:58.696 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 1 2024-05-09T08:05:58.696 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:05:58.696 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:05:58.696 INFO:teuthology.orchestra.run.smithi001.stdout: "mds": [ 2024-05-09T08:05:58.696 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:05:58.696 INFO:teuthology.orchestra.run.smithi001.stdout: "features": "0x3f03cffffffdffff", 2024-05-09T08:05:58.697 INFO:teuthology.orchestra.run.smithi001.stdout: "release": "luminous", 2024-05-09T08:05:58.697 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 6 2024-05-09T08:05:58.697 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:05:58.697 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:05:58.697 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": [ 2024-05-09T08:05:58.697 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:05:58.697 INFO:teuthology.orchestra.run.smithi001.stdout: "features": "0x3f03cffffffdffff", 2024-05-09T08:05:58.697 INFO:teuthology.orchestra.run.smithi001.stdout: "release": "luminous", 2024-05-09T08:05:58.697 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 7 2024-05-09T08:05:58.697 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:05:58.697 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:05:58.697 INFO:teuthology.orchestra.run.smithi001.stdout: "client": [ 2024-05-09T08:05:58.697 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:05:58.697 INFO:teuthology.orchestra.run.smithi001.stdout: "features": "0x2f018fb87aa4aafe", 2024-05-09T08:05:58.697 INFO:teuthology.orchestra.run.smithi001.stdout: "release": "luminous", 2024-05-09T08:05:58.698 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 1 2024-05-09T08:05:58.698 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:05:58.698 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:05:58.698 INFO:teuthology.orchestra.run.smithi001.stdout: "features": "0x3f03cffffffdffff", 2024-05-09T08:05:58.698 INFO:teuthology.orchestra.run.smithi001.stdout: "release": "luminous", 2024-05-09T08:05:58.698 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 5 2024-05-09T08:05:58.698 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:05:58.698 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:05:58.698 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr": [ 2024-05-09T08:05:58.698 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:05:58.698 INFO:teuthology.orchestra.run.smithi001.stdout: "features": "0x3f03cffffffdffff", 2024-05-09T08:05:58.698 INFO:teuthology.orchestra.run.smithi001.stdout: "release": "luminous", 2024-05-09T08:05:58.698 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 2 2024-05-09T08:05:58.698 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:05:58.699 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:05:58.699 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:05:58.699 INFO:teuthology.orchestra.run.smithi001.stdout: "stretch_mode": false 2024-05-09T08:05:58.699 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-05-09T08:05:58.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.760+0000 7f06a0c00640 1 -- 172.21.15.1:0/855219089 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f06a40c4890 msgr2=0x7f06a41f8330 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-05-09T08:05:58.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.760+0000 7f06a0c00640 1 --2- 172.21.15.1:0/855219089 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f06a40c4890 0x7f06a41f8330 secure :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f068c00e7a0 tx=0x7f068c0046c0 comp rx=0 tx=0).stop 2024-05-09T08:05:58.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.761+0000 7f06aae00640 1 -- 172.21.15.1:0/855219089 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f068006be80 msgr2=0x7f068006e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:05:58.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.761+0000 7f06aae00640 1 --2- 172.21.15.1:0/855219089 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f068006be80 0x7f068006e340 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f069402a200 tx=0x7f069402e000 comp rx=0 tx=0).stop 2024-05-09T08:05:58.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.761+0000 7f06aae00640 1 -- 172.21.15.1:0/855219089 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f06a4154910 msgr2=0x7f06a40c51a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:05:58.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.761+0000 7f06aae00640 1 --2- 172.21.15.1:0/855219089 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f06a4154910 0x7f06a40c51a0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f069c007a70 tx=0x7f069c007f30 comp rx=0 tx=0).stop 2024-05-09T08:05:58.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.761+0000 7f06a8e00640 1 -- 172.21.15.1:0/855219089 reap_dead start 2024-05-09T08:05:58.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.761+0000 7f06a8e00640 1 -- 172.21.15.1:0/855219089 reap_dead start 2024-05-09T08:05:58.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.762+0000 7f06aae00640 1 -- 172.21.15.1:0/855219089 shutdown_connections 2024-05-09T08:05:58.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.762+0000 7f06aae00640 1 -- 172.21.15.1:0/855219089 >> 172.21.15.1:0/855219089 conn(0x7f06a406dad0 msgr2=0x7f06a406deb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:05:58.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.762+0000 7f06aae00640 1 -- 172.21.15.1:0/855219089 shutdown_connections 2024-05-09T08:05:58.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:57.762+0000 7f06aae00640 1 -- 172.21.15.1:0/855219089 wait complete. 2024-05-09T08:05:58.701 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-05-09T08:05:58.702 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['c'] 2024-05-09T08:05:58.702 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2024-05-09T08:05:58.702 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.c 2024-05-09T08:05:58.702 INFO:tasks.mon_thrash.mon_thrasher:killing mon.c 2024-05-09T08:05:58.703 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-05-09T08:05:58.703 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T08:05:58.720 INFO:tasks.ceph.mon.c:Stopped 2024-05-09T08:05:58.720 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2024-05-09T08:05:58.720 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-05-09T08:05:58.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:58.977+0000 7fc4c4800640 1 Processor -- start 2024-05-09T08:05:58.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:58.977+0000 7fc4c4800640 1 -- start start 2024-05-09T08:05:58.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:58.978+0000 7fc4c4800640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fc4bc0c6ea0 0x7fc4bc072900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:05:58.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:58.978+0000 7fc4c4800640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc4bc06ece0 0x7fc4bc159010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:05:58.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:58.979+0000 7fc4c4800640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc4bc073200 con 0x7fc4bc06dca0 2024-05-09T08:05:58.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:58.979+0000 7fc4c4800640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fc4bc072e40 con 0x7fc4bc06ece0 2024-05-09T08:05:58.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:58.979+0000 7fc4c4800640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7fc4bc0737c0 con 0x7fc4bc0c6ea0 2024-05-09T08:05:58.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:58.979+0000 7fc4c1400640 1 -- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fc4bc0c6ea0 msgr2=0x7fc4bc072900 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.7:3301/0 2024-05-09T08:05:58.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:58.979+0000 7fc4c1400640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fc4bc0c6ea0 0x7fc4bc072900 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T08:05:58.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:58.979+0000 7fc4c2800640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc4bc06ece0 0x7fc4bc159010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:05:58.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:58.979+0000 7fc4c1e00640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7fc4bc06dca0 0x7fc4bc06e080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:42206/0 (socket says 172.21.15.1:42206) 2024-05-09T08:05:58.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:58.979+0000 7fc4c1e00640 1 -- 172.21.15.1:0/523854181 learned_addr learned my addr 172.21.15.1:0/523854181 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T08:05:58.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:58.979+0000 7fc4c2800640 1 -- 172.21.15.1:0/523854181 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fc4bc0c6ea0 msgr2=0x7fc4bc072900 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T08:05:58.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:58.979+0000 7fc4c2800640 1 --2- 172.21.15.1:0/523854181 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fc4bc0c6ea0 0x7fc4bc072900 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:05:58.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:58.979+0000 7fc4c2800640 1 -- 172.21.15.1:0/523854181 >> v1:172.21.15.1:6789/0 conn(0x7fc4bc06dca0 legacy=0x7fc4bc06e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:05:58.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:58.979+0000 7fc4c0a00640 1 -- 172.21.15.1:0/523854181 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2449969911 0 0) 0x7fc4bc073200 con 0x7fc4bc06dca0 2024-05-09T08:05:58.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:58.979+0000 7fc4c2800640 1 -- 172.21.15.1:0/523854181 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4bc0734e0 con 0x7fc4bc06ece0 2024-05-09T08:05:58.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:58.981+0000 7fc4c2800640 1 --2- 172.21.15.1:0/523854181 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc4bc06ece0 0x7fc4bc159010 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fc4b8009d90 tx=0x7fc4b802a710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=295c76c9c5842078 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:05:58.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:58.981+0000 7fc4c0a00640 1 -- 172.21.15.1:0/523854181 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4b8007740 con 0x7fc4bc06ece0 2024-05-09T08:05:58.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:58.981+0000 7fc4c0a00640 1 -- 172.21.15.1:0/523854181 <== mon.1 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4b80078e0 con 0x7fc4bc06ece0 2024-05-09T08:05:58.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:58.981+0000 7fc4c0a00640 1 -- 172.21.15.1:0/523854181 <== mon.1 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4b8007cb0 con 0x7fc4bc06ece0 2024-05-09T08:05:58.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:58.982+0000 7fc4c4800640 1 -- 172.21.15.1:0/523854181 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc4bc06ece0 msgr2=0x7fc4bc159010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:05:58.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:58.982+0000 7fc4c4800640 1 --2- 172.21.15.1:0/523854181 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc4bc06ece0 0x7fc4bc159010 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fc4b8009d90 tx=0x7fc4b802a710 comp rx=0 tx=0).stop 2024-05-09T08:05:58.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:58.982+0000 7fc4c4800640 1 -- 172.21.15.1:0/523854181 shutdown_connections 2024-05-09T08:05:58.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:58.982+0000 7fc4c4800640 1 --2- 172.21.15.1:0/523854181 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fc4bc0c6ea0 0x7fc4bc072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:05:58.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:58.982+0000 7fc4c4800640 1 --2- 172.21.15.1:0/523854181 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc4bc06ece0 0x7fc4bc159010 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:05:58.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:58.982+0000 7fc4c4800640 1 -- 172.21.15.1:0/523854181 >> 172.21.15.1:0/523854181 conn(0x7fc4bc0cc070 msgr2=0x7fc4bc0cc480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:05:58.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:58.982+0000 7fc4c4800640 1 -- 172.21.15.1:0/523854181 shutdown_connections 2024-05-09T08:05:58.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:58.982+0000 7fc4c4800640 1 -- 172.21.15.1:0/523854181 wait complete. 2024-05-09T08:05:58.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:58.986+0000 7fc4c4800640 1 Processor -- start 2024-05-09T08:05:58.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:58.987+0000 7fc4c4800640 1 -- start start 2024-05-09T08:05:59.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:58.988+0000 7fc4c4800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc4bc06dca0 0x7fc4bc1f1050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:05:59.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:58.988+0000 7fc4c4800640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fc4bc06ece0 0x7fc4bc0cb320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:05:59.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:58.988+0000 7fc4c4800640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc4bc0c6ea0 0x7fc4bc1570f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:05:59.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:58.988+0000 7fc4c4800640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fc4bc072470 con 0x7fc4bc0c6ea0 2024-05-09T08:05:59.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:58.988+0000 7fc4c4800640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc4bc159550 con 0x7fc4bc06dca0 2024-05-09T08:05:59.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:58.988+0000 7fc4c4800640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7fc4bc072100 con 0x7fc4bc06ece0 2024-05-09T08:05:59.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:58.988+0000 7fc4c1400640 1 -- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fc4bc06ece0 msgr2=0x7fc4bc0cb320 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.7:3301/0 2024-05-09T08:05:59.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:58.988+0000 7fc4c1400640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fc4bc06ece0 0x7fc4bc0cb320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T08:05:59.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:58.989+0000 7fc4c2800640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc4bc0c6ea0 0x7fc4bc1570f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:05:59.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:58.989+0000 7fc4c2800640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc4bc0c6ea0 0x7fc4bc1570f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45368/0 (socket says 172.21.15.1:45368) 2024-05-09T08:05:59.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:58.989+0000 7fc4c2800640 1 -- 172.21.15.1:0/4093236358 learned_addr learned my addr 172.21.15.1:0/4093236358 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:05:59.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:58.989+0000 7fc4c2800640 1 -- 172.21.15.1:0/4093236358 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fc4bc06ece0 msgr2=0x7fc4bc0cb320 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T08:05:59.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:58.989+0000 7fc4c2800640 1 --2- 172.21.15.1:0/4093236358 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fc4bc06ece0 0x7fc4bc0cb320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:05:59.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:58.989+0000 7fc4c2800640 1 -- 172.21.15.1:0/4093236358 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc4bc06dca0 msgr2=0x7fc4bc1f1050 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T08:05:59.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:58.989+0000 7fc4c2800640 1 --2- 172.21.15.1:0/4093236358 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc4bc06dca0 0x7fc4bc1f1050 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:05:59.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:58.989+0000 7fc4c2800640 1 -- 172.21.15.1:0/4093236358 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4bc156cf0 con 0x7fc4bc0c6ea0 2024-05-09T08:05:59.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:58.989+0000 7fc4c2800640 1 --2- 172.21.15.1:0/4093236358 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc4bc0c6ea0 0x7fc4bc1570f0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fc4b8009d90 tx=0x7fc4b8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:05:59.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:58.990+0000 7fc4b6000640 1 -- 172.21.15.1:0/4093236358 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4b8040070 con 0x7fc4bc0c6ea0 2024-05-09T08:05:59.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:58.990+0000 7fc4b6000640 1 -- 172.21.15.1:0/4093236358 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4b8031c40 con 0x7fc4bc0c6ea0 2024-05-09T08:05:59.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:58.990+0000 7fc4b6000640 1 -- 172.21.15.1:0/4093236358 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4b8003990 con 0x7fc4bc0c6ea0 2024-05-09T08:05:59.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:58.990+0000 7fc4c4800640 1 -- 172.21.15.1:0/4093236358 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4bc1f3c00 con 0x7fc4bc0c6ea0 2024-05-09T08:05:59.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:58.990+0000 7fc4c4800640 1 -- 172.21.15.1:0/4093236358 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4bc1f41c0 con 0x7fc4bc0c6ea0 2024-05-09T08:05:59.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:58.992+0000 7fc4c4800640 1 -- 172.21.15.1:0/4093236358 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4bc0b23e0 con 0x7fc4bc0c6ea0 2024-05-09T08:05:59.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:58.994+0000 7fc4b6000640 1 -- 172.21.15.1:0/4093236358 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc4b8039040 con 0x7fc4bc0c6ea0 2024-05-09T08:05:59.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:58.996+0000 7fc4b6000640 1 --2- 172.21.15.1:0/4093236358 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fc4a006be30 0x7fc4a006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:05:59.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:58.996+0000 7fc4b6000640 1 -- 172.21.15.1:0/4093236358 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7fc4b80ac590 con 0x7fc4bc0c6ea0 2024-05-09T08:05:59.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:58.998+0000 7fc4c1e00640 1 --2- 172.21.15.1:0/4093236358 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fc4a006be30 0x7fc4a006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:05:59.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:58.998+0000 7fc4c1e00640 1 --2- 172.21.15.1:0/4093236358 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fc4a006be30 0x7fc4a006e2f0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fc4b00011e0 tx=0x7fc4b00078b0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:05:59.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:59.002+0000 7fc4b6000640 1 -- 172.21.15.1:0/4093236358 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fc4bc0b23e0 con 0x7fc4bc0c6ea0 2024-05-09T08:05:59.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:59.465+0000 7fc4c4800640 1 -- 172.21.15.1:0/4093236358 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fc4bc06d760 con 0x7fc4bc0c6ea0 2024-05-09T08:05:59.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:59.466+0000 7fc4b6000640 1 -- 172.21.15.1:0/4093236358 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1545 (secure 0 0 0) 0x7fc4bc06d760 con 0x7fc4bc0c6ea0 2024-05-09T08:05:59.479 INFO:teuthology.orchestra.run.smithi001.stdout:{"election_epoch":30,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":15,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"54329ec8-3270-4043-9a82-c6cd7420aa7f","modified":"2024-05-09T08:01:21.819270Z","created":"2024-05-09T08:01:21.819270Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:3300","nonce":0},{"type":"v1","addr":"172.21.15.1:6789","nonce":0}]},"addr":"172.21.15.1:6789/0","public_addr":"172.21.15.1:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:3300","nonce":0},{"type":"v1","addr":"172.21.15.7:6789","nonce":0}]},"addr":"172.21.15.7:6789/0","public_addr":"172.21.15.7:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:3301","nonce":0},{"type":"v1","addr":"172.21.15.7:6790","nonce":0}]},"addr":"172.21.15.7:6790/0","public_addr":"172.21.15.7:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-05-09T08:05:59.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:59.469+0000 7fc4c4800640 1 -- 172.21.15.1:0/4093236358 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fc4a006be30 msgr2=0x7fc4a006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:05:59.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:59.469+0000 7fc4c4800640 1 --2- 172.21.15.1:0/4093236358 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fc4a006be30 0x7fc4a006e2f0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fc4b00011e0 tx=0x7fc4b00078b0 comp rx=0 tx=0).stop 2024-05-09T08:05:59.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:59.469+0000 7fc4c4800640 1 -- 172.21.15.1:0/4093236358 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc4bc0c6ea0 msgr2=0x7fc4bc1570f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:05:59.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:59.469+0000 7fc4c4800640 1 --2- 172.21.15.1:0/4093236358 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc4bc0c6ea0 0x7fc4bc1570f0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fc4b8009d90 tx=0x7fc4b8004a40 comp rx=0 tx=0).stop 2024-05-09T08:05:59.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:59.469+0000 7fc4c4800640 1 -- 172.21.15.1:0/4093236358 shutdown_connections 2024-05-09T08:05:59.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:59.469+0000 7fc4c4800640 1 --2- 172.21.15.1:0/4093236358 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fc4a006be30 0x7fc4a006e2f0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:05:59.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:59.469+0000 7fc4c4800640 1 --2- 172.21.15.1:0/4093236358 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc4bc0c6ea0 0x7fc4bc1570f0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:05:59.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:59.469+0000 7fc4c4800640 1 --2- 172.21.15.1:0/4093236358 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fc4bc06ece0 0x7fc4bc0cb320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:05:59.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:59.469+0000 7fc4c4800640 1 --2- 172.21.15.1:0/4093236358 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc4bc06dca0 0x7fc4bc1f1050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:05:59.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:59.469+0000 7fc4c4800640 1 -- 172.21.15.1:0/4093236358 >> 172.21.15.1:0/4093236358 conn(0x7fc4bc0cc070 msgr2=0x7fc4bc0744e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:05:59.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:59.469+0000 7fc4c4800640 1 -- 172.21.15.1:0/4093236358 shutdown_connections 2024-05-09T08:05:59.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:05:59.469+0000 7fc4c4800640 1 -- 172.21.15.1:0/4093236358 wait complete. 2024-05-09T08:06:02.506 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-05-09T08:06:02.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:02.824+0000 7f0d42a00640 1 Processor -- start 2024-05-09T08:06:02.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:02.824+0000 7f0d42a00640 1 -- start start 2024-05-09T08:06:02.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:02.825+0000 7f0d42a00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f0d3c0b1d30 0x7f0d3c0c6c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:06:02.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:02.825+0000 7f0d42a00640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0d3c0c72f0 con 0x7f0d3c0c1ba0 2024-05-09T08:06:02.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:02.826+0000 7f0d42a00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0d3c157620 con 0x7f0d3c151660 2024-05-09T08:06:02.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:02.826+0000 7f0d42a00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f0d3c151a40 con 0x7f0d3c0b1d30 2024-05-09T08:06:02.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:02.826+0000 7f0d3be00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f0d3c151660 0x7f0d3c072d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.1:60786/0 (socket says 172.21.15.1:60786) 2024-05-09T08:06:02.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:02.826+0000 7f0d3be00640 1 -- 172.21.15.1:0/1771390623 learned_addr learned my addr 172.21.15.1:0/1771390623 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T08:06:02.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:02.826+0000 7f0d40a00640 1 --2- 172.21.15.1:0/1771390623 >> v2:172.21.15.7:3300/0 conn(0x7f0d3c0b1d30 0x7f0d3c0c6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:06:02.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:02.826+0000 7f0d40a00640 1 -- 172.21.15.1:0/1771390623 >> v1:172.21.15.7:6789/0 conn(0x7f0d3c151660 legacy=0x7f0d3c072d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:06:02.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:02.826+0000 7f0d40a00640 1 -- 172.21.15.1:0/1771390623 >> v1:172.21.15.1:6789/0 conn(0x7f0d3c0c1ba0 legacy=0x7f0d3c0c7a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:06:02.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:02.826+0000 7f0d40a00640 1 -- 172.21.15.1:0/1771390623 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d3c157900 con 0x7f0d3c0b1d30 2024-05-09T08:06:02.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:02.827+0000 7f0d40a00640 1 --2- 172.21.15.1:0/1771390623 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d3c0b1d30 0x7f0d3c0c6c90 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f0d340097a0 tx=0x7f0d3402a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=552f244b3f347e4d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:06:02.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:02.827+0000 7f0d3b400640 1 -- 172.21.15.1:0/1771390623 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d34037070 con 0x7f0d3c0b1d30 2024-05-09T08:06:02.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:02.828+0000 7f0d3b400640 1 -- 172.21.15.1:0/1771390623 <== mon.1 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d34032440 con 0x7f0d3c0b1d30 2024-05-09T08:06:02.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:02.828+0000 7f0d3b400640 1 -- 172.21.15.1:0/1771390623 <== mon.1 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d34032810 con 0x7f0d3c0b1d30 2024-05-09T08:06:02.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:02.828+0000 7f0d42a00640 1 -- 172.21.15.1:0/1771390623 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d3c0b1d30 msgr2=0x7f0d3c0c6c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:06:02.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:02.828+0000 7f0d42a00640 1 --2- 172.21.15.1:0/1771390623 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d3c0b1d30 0x7f0d3c0c6c90 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f0d340097a0 tx=0x7f0d3402a230 comp rx=0 tx=0).stop 2024-05-09T08:06:02.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:02.828+0000 7f0d42a00640 1 -- 172.21.15.1:0/1771390623 shutdown_connections 2024-05-09T08:06:02.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:02.828+0000 7f0d42a00640 1 --2- 172.21.15.1:0/1771390623 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d3c0b1d30 0x7f0d3c0c6c90 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:06:02.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:02.828+0000 7f0d42a00640 1 -- 172.21.15.1:0/1771390623 >> 172.21.15.1:0/1771390623 conn(0x7f0d3c0cc070 msgr2=0x7f0d3c0cc480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:06:02.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:02.828+0000 7f0d42a00640 1 -- 172.21.15.1:0/1771390623 shutdown_connections 2024-05-09T08:06:02.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:02.828+0000 7f0d42a00640 1 -- 172.21.15.1:0/1771390623 wait complete. 2024-05-09T08:06:02.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:02.830+0000 7f0d42a00640 1 Processor -- start 2024-05-09T08:06:02.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:02.830+0000 7f0d42a00640 1 -- start start 2024-05-09T08:06:02.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:02.831+0000 7f0d42a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d3c0b1d30 0x7f0d3c158e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:06:02.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:02.831+0000 7f0d42a00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0d3c0c1ba0 0x7f0d3c15ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:06:02.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:02.831+0000 7f0d42a00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f0d3c151660 0x7f0d3c15b060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:06:02.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:02.831+0000 7f0d42a00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f0d3c1581a0 con 0x7f0d3c0c1ba0 2024-05-09T08:06:02.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:02.831+0000 7f0d42a00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0d3c157ec0 con 0x7f0d3c0b1d30 2024-05-09T08:06:02.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:02.831+0000 7f0d42a00640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f0d3c1572d0 con 0x7f0d3c151660 2024-05-09T08:06:02.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:02.831+0000 7f0d3be00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d3c0b1d30 0x7f0d3c158e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:06:02.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:02.831+0000 7f0d40a00640 1 -- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f0d3c151660 msgr2=0x7f0d3c15b060 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.7:3301/0 2024-05-09T08:06:02.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:02.831+0000 7f0d40a00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f0d3c151660 0x7f0d3c15b060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T08:06:02.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:02.831+0000 7f0d3be00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d3c0b1d30 0x7f0d3c158e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.1:52280/0 (socket says 172.21.15.1:52280) 2024-05-09T08:06:02.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:02.831+0000 7f0d3be00640 1 -- 172.21.15.1:0/4110208523 learned_addr learned my addr 172.21.15.1:0/4110208523 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:06:02.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:02.831+0000 7f0d33400640 1 --2- 172.21.15.1:0/4110208523 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0d3c0c1ba0 0x7f0d3c15ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:06:02.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:02.832+0000 7f0d3be00640 1 -- 172.21.15.1:0/4110208523 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f0d3c151660 msgr2=0x7f0d3c15b060 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T08:06:02.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:02.832+0000 7f0d3be00640 1 --2- 172.21.15.1:0/4110208523 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f0d3c151660 0x7f0d3c15b060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:06:02.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:02.832+0000 7f0d3be00640 1 -- 172.21.15.1:0/4110208523 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0d3c0c1ba0 msgr2=0x7f0d3c15ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:06:02.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:02.832+0000 7f0d3be00640 1 --2- 172.21.15.1:0/4110208523 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0d3c0c1ba0 0x7f0d3c15ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:06:02.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:02.832+0000 7f0d3be00640 1 -- 172.21.15.1:0/4110208523 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d3c1f0a80 con 0x7f0d3c0b1d30 2024-05-09T08:06:02.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:02.832+0000 7f0d3be00640 1 --2- 172.21.15.1:0/4110208523 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d3c0b1d30 0x7f0d3c158e70 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f0d2c00cbf0 tx=0x7f0d2c00a060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:06:02.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:02.832+0000 7f0d38c00640 1 -- 172.21.15.1:0/4110208523 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d2c017070 con 0x7f0d3c0b1d30 2024-05-09T08:06:02.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:02.832+0000 7f0d38c00640 1 -- 172.21.15.1:0/4110208523 <== mon.1 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d2c00aca0 con 0x7f0d3c0b1d30 2024-05-09T08:06:02.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:02.832+0000 7f0d38c00640 1 -- 172.21.15.1:0/4110208523 <== mon.1 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d2c012600 con 0x7f0d3c0b1d30 2024-05-09T08:06:02.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:02.832+0000 7f0d42a00640 1 -- 172.21.15.1:0/4110208523 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d3c1f0cf0 con 0x7f0d3c0b1d30 2024-05-09T08:06:02.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:02.832+0000 7f0d42a00640 1 -- 172.21.15.1:0/4110208523 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d3c1f12b0 con 0x7f0d3c0b1d30 2024-05-09T08:06:02.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:02.833+0000 7f0d42a00640 1 -- 172.21.15.1:0/4110208523 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d3c074d70 con 0x7f0d3c0b1d30 2024-05-09T08:06:02.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:02.834+0000 7f0d38c00640 1 -- 172.21.15.1:0/4110208523 <== mon.1 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0d2c01a070 con 0x7f0d3c0b1d30 2024-05-09T08:06:02.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:02.835+0000 7f0d38c00640 1 --2- 172.21.15.1:0/4110208523 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f0d1806bde0 0x7f0d1806e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:06:02.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:02.835+0000 7f0d38c00640 1 -- 172.21.15.1:0/4110208523 <== mon.1 v2:172.21.15.7:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7f0d2c08afb0 con 0x7f0d3c0b1d30 2024-05-09T08:06:02.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:02.835+0000 7f0d33400640 1 --2- 172.21.15.1:0/4110208523 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f0d1806bde0 0x7f0d1806e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:06:02.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:02.836+0000 7f0d33400640 1 --2- 172.21.15.1:0/4110208523 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f0d1806bde0 0x7f0d1806e2a0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f0d3c0c9bf0 tx=0x7f0d24009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:06:02.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:02.841+0000 7f0d38c00640 1 -- 172.21.15.1:0/4110208523 <== mon.1 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f0d2c058e80 con 0x7f0d3c0b1d30 2024-05-09T08:06:03.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:03.448+0000 7f0d42a00640 1 -- 172.21.15.1:0/4110208523 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f0d3c0c81c0 con 0x7f0d3c0b1d30 2024-05-09T08:06:03.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:03.448+0000 7f0d38c00640 1 -- 172.21.15.1:0/4110208523 <== mon.1 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1545 (secure 0 0 0) 0x7f0d2c05d180 con 0x7f0d3c0b1d30 2024-05-09T08:06:03.461 INFO:teuthology.orchestra.run.smithi001.stdout:{"election_epoch":30,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":19,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"54329ec8-3270-4043-9a82-c6cd7420aa7f","modified":"2024-05-09T08:01:21.819270Z","created":"2024-05-09T08:01:21.819270Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:3300","nonce":0},{"type":"v1","addr":"172.21.15.1:6789","nonce":0}]},"addr":"172.21.15.1:6789/0","public_addr":"172.21.15.1:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:3300","nonce":0},{"type":"v1","addr":"172.21.15.7:6789","nonce":0}]},"addr":"172.21.15.7:6789/0","public_addr":"172.21.15.7:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:3301","nonce":0},{"type":"v1","addr":"172.21.15.7:6790","nonce":0}]},"addr":"172.21.15.7:6790/0","public_addr":"172.21.15.7:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-05-09T08:06:03.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:03.452+0000 7f0d42a00640 1 -- 172.21.15.1:0/4110208523 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f0d1806bde0 msgr2=0x7f0d1806e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:06:03.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:03.452+0000 7f0d42a00640 1 --2- 172.21.15.1:0/4110208523 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f0d1806bde0 0x7f0d1806e2a0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f0d3c0c9bf0 tx=0x7f0d24009040 comp rx=0 tx=0).stop 2024-05-09T08:06:03.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:03.452+0000 7f0d42a00640 1 -- 172.21.15.1:0/4110208523 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d3c0b1d30 msgr2=0x7f0d3c158e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:06:03.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:03.452+0000 7f0d42a00640 1 --2- 172.21.15.1:0/4110208523 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d3c0b1d30 0x7f0d3c158e70 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f0d2c00cbf0 tx=0x7f0d2c00a060 comp rx=0 tx=0).stop 2024-05-09T08:06:03.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:03.452+0000 7f0d42a00640 1 -- 172.21.15.1:0/4110208523 shutdown_connections 2024-05-09T08:06:03.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:03.452+0000 7f0d42a00640 1 --2- 172.21.15.1:0/4110208523 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f0d1806bde0 0x7f0d1806e2a0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:06:03.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:03.452+0000 7f0d42a00640 1 --2- 172.21.15.1:0/4110208523 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f0d3c151660 0x7f0d3c15b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:06:03.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:03.452+0000 7f0d42a00640 1 --2- 172.21.15.1:0/4110208523 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0d3c0c1ba0 0x7f0d3c15ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:06:03.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:03.452+0000 7f0d42a00640 1 --2- 172.21.15.1:0/4110208523 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d3c0b1d30 0x7f0d3c158e70 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:06:03.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:03.452+0000 7f0d42a00640 1 -- 172.21.15.1:0/4110208523 >> 172.21.15.1:0/4110208523 conn(0x7f0d3c0cc070 msgr2=0x7f0d3c14ab10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:06:03.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:03.452+0000 7f0d42a00640 1 -- 172.21.15.1:0/4110208523 shutdown_connections 2024-05-09T08:06:03.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:03.452+0000 7f0d42a00640 1 -- 172.21.15.1:0/4110208523 wait complete. 2024-05-09T08:06:06.499 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-05-09T08:06:06.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:06.800+0000 7f5aa5200640 1 Processor -- start 2024-05-09T08:06:06.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:06.800+0000 7f5aa5200640 1 -- start start 2024-05-09T08:06:06.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:06.804+0000 7f5a9f400640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f5aa00ca5c0 0x7f5aa015a5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.1:55926/0 (socket says 172.21.15.1:55926) 2024-05-09T08:06:06.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:06.804+0000 7f5a9f400640 1 -- 172.21.15.1:0/900740866 learned_addr learned my addr 172.21.15.1:0/900740866 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T08:06:06.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:06.804+0000 7f5aa5200640 1 --2- 172.21.15.1:0/900740866 >> v2:172.21.15.1:3300/0 conn(0x7f5aa00ccf70 0x7f5aa00ca9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:06:06.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:06.808+0000 7f5aa5200640 1 -- 172.21.15.1:0/900740866 --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5aa015ad80 con 0x7f5aa00cf370 2024-05-09T08:06:06.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:06.808+0000 7f5aa5200640 1 -- 172.21.15.1:0/900740866 --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5aa015b060 con 0x7f5aa00ca5c0 2024-05-09T08:06:06.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:06.808+0000 7f5aa5200640 1 -- 172.21.15.1:0/900740866 --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f5aa006c7d0 con 0x7f5aa00ccf70 2024-05-09T08:06:06.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:06.808+0000 7f5a9ea00640 1 --2- 172.21.15.1:0/900740866 >> v2:172.21.15.1:3300/0 conn(0x7f5aa00ccf70 0x7f5aa00ca9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:06:06.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:06.809+0000 7f5a9ea00640 1 -- 172.21.15.1:0/900740866 >> v1:172.21.15.7:6789/0 conn(0x7f5aa00ca5c0 legacy=0x7f5aa015a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:06:06.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:06.810+0000 7f5a9ea00640 1 -- 172.21.15.1:0/900740866 >> v1:172.21.15.1:6789/0 conn(0x7f5aa00cf370 legacy=0x7f5aa006d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:06:06.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:06.812+0000 7f5a9ea00640 1 -- 172.21.15.1:0/900740866 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5aa015b340 con 0x7f5aa00ccf70 2024-05-09T08:06:06.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:06.818+0000 7f5a9ea00640 1 --2- 172.21.15.1:0/900740866 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5aa00ccf70 0x7f5aa00ca9a0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f5a8c0097a0 tx=0x7f5a8c02a2e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35105932345f59d4 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:06:06.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:06.819+0000 7f5a9e000640 1 -- 172.21.15.1:0/900740866 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a8c037070 con 0x7f5aa00ccf70 2024-05-09T08:06:06.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:06.819+0000 7f5a9e000640 1 -- 172.21.15.1:0/900740866 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a8c032640 con 0x7f5aa00ccf70 2024-05-09T08:06:06.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:06.819+0000 7f5a9e000640 1 -- 172.21.15.1:0/900740866 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a8c032a50 con 0x7f5aa00ccf70 2024-05-09T08:06:06.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:06.822+0000 7f5aa5200640 1 -- 172.21.15.1:0/900740866 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5aa00ccf70 msgr2=0x7f5aa00ca9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:06:06.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:06.822+0000 7f5aa5200640 1 --2- 172.21.15.1:0/900740866 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5aa00ccf70 0x7f5aa00ca9a0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f5a8c0097a0 tx=0x7f5a8c02a2e0 comp rx=0 tx=0).stop 2024-05-09T08:06:06.836 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:06.823+0000 7f5aa5200640 1 -- 172.21.15.1:0/900740866 shutdown_connections 2024-05-09T08:06:06.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:06.823+0000 7f5aa5200640 1 --2- 172.21.15.1:0/900740866 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5aa00ccf70 0x7f5aa00ca9a0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:06:06.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:06.823+0000 7f5aa5200640 1 -- 172.21.15.1:0/900740866 >> 172.21.15.1:0/900740866 conn(0x7f5aa00cfc40 msgr2=0x7f5aa00d0050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:06:06.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:06.824+0000 7f5aa5200640 1 -- 172.21.15.1:0/900740866 shutdown_connections 2024-05-09T08:06:06.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:06.824+0000 7f5aa5200640 1 -- 172.21.15.1:0/900740866 wait complete. 2024-05-09T08:06:06.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:06.830+0000 7f5aa5200640 1 Processor -- start 2024-05-09T08:06:06.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:06.831+0000 7f5aa5200640 1 -- start start 2024-05-09T08:06:06.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:06.833+0000 7f5aa5200640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f5aa00ca5c0 0x7f5aa0200b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:06:06.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:06.833+0000 7f5aa5200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5aa00ccf70 0x7f5aa00c2620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:06:06.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:06.833+0000 7f5aa5200640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5aa00cf370 0x7f5aa00c2b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:06:06.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:06.834+0000 7f5aa5200640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f5aa006dab0 con 0x7f5aa00cf370 2024-05-09T08:06:06.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:06.834+0000 7f5aa5200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5aa006d930 con 0x7f5aa00ccf70 2024-05-09T08:06:06.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:06.834+0000 7f5aa5200640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f5aa00b5240 con 0x7f5aa00ca5c0 2024-05-09T08:06:06.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:06.835+0000 7f5a9ea00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5aa00ccf70 0x7f5aa00c2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:06:06.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:06.835+0000 7f5a9ea00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5aa00ccf70 0x7f5aa00c2620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.1:35178/0 (socket says 172.21.15.1:35178) 2024-05-09T08:06:06.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:06.835+0000 7f5a9ea00640 1 -- 172.21.15.1:0/4201561865 learned_addr learned my addr 172.21.15.1:0/4201561865 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:06:06.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:06.835+0000 7f5a9f400640 1 -- 172.21.15.1:0/4201561865 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f5aa00ca5c0 msgr2=0x7f5aa0200b80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.7:3301/0 2024-05-09T08:06:06.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:06.835+0000 7f5a9f400640 1 --2- 172.21.15.1:0/4201561865 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f5aa00ca5c0 0x7f5aa0200b80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T08:06:06.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:06.835+0000 7f5a9ea00640 1 -- 172.21.15.1:0/4201561865 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f5aa00ca5c0 msgr2=0x7f5aa0200b80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T08:06:06.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:06.835+0000 7f5a9ea00640 1 --2- 172.21.15.1:0/4201561865 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f5aa00ca5c0 0x7f5aa0200b80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:06:06.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:06.835+0000 7f5a9ea00640 1 -- 172.21.15.1:0/4201561865 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5aa00cf370 msgr2=0x7f5aa00c2b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:06:06.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:06.835+0000 7f5a9fe00640 1 --2- 172.21.15.1:0/4201561865 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5aa00cf370 0x7f5aa00c2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:06:06.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:06.836+0000 7f5a9ea00640 1 --2- 172.21.15.1:0/4201561865 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5aa00cf370 0x7f5aa00c2b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:06:06.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:06.836+0000 7f5a9ea00640 1 -- 172.21.15.1:0/4201561865 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5aa01ff220 con 0x7f5aa00ccf70 2024-05-09T08:06:06.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:06.837+0000 7f5a9fe00640 1 --2- 172.21.15.1:0/4201561865 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5aa00cf370 0x7f5aa00c2b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T08:06:06.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:06.840+0000 7f5a9ea00640 1 --2- 172.21.15.1:0/4201561865 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5aa00ccf70 0x7f5aa00c2620 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f5a8c009890 tx=0x7f5a8c0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:06:06.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:06.841+0000 7f5a97400640 1 -- 172.21.15.1:0/4201561865 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a8c037050 con 0x7f5aa00ccf70 2024-05-09T08:06:06.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:06.841+0000 7f5a97400640 1 -- 172.21.15.1:0/4201561865 <== mon.1 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a8c043070 con 0x7f5aa00ccf70 2024-05-09T08:06:06.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:06.841+0000 7f5a97400640 1 -- 172.21.15.1:0/4201561865 <== mon.1 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a8c030d50 con 0x7f5aa00ccf70 2024-05-09T08:06:06.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:06.841+0000 7f5aa5200640 1 -- 172.21.15.1:0/4201561865 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5aa0201640 con 0x7f5aa00ccf70 2024-05-09T08:06:06.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:06.841+0000 7f5aa5200640 1 -- 172.21.15.1:0/4201561865 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5aa00b58a0 con 0x7f5aa00ccf70 2024-05-09T08:06:06.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:06.843+0000 7f5aa5200640 1 -- 172.21.15.1:0/4201561865 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5aa004f660 con 0x7f5aa00ccf70 2024-05-09T08:06:06.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:06.844+0000 7f5a97400640 1 -- 172.21.15.1:0/4201561865 <== mon.1 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5a8c032500 con 0x7f5aa00ccf70 2024-05-09T08:06:06.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:06.846+0000 7f5a97400640 1 --2- 172.21.15.1:0/4201561865 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f5a8006be60 0x7f5a8006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:06:06.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:06.846+0000 7f5a97400640 1 -- 172.21.15.1:0/4201561865 <== mon.1 v2:172.21.15.7:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7f5a8c0abff0 con 0x7f5aa00ccf70 2024-05-09T08:06:06.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:06.846+0000 7f5a9f400640 1 --2- 172.21.15.1:0/4201561865 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f5a8006be60 0x7f5a8006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:06:06.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:06.847+0000 7f5a9f400640 1 --2- 172.21.15.1:0/4201561865 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f5a8006be60 0x7f5a8006e320 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f5a90004810 tx=0x7f5a90007ae0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:06:06.863 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:06.850+0000 7f5a97400640 1 -- 172.21.15.1:0/4201561865 <== mon.1 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f5a8c07a620 con 0x7f5aa00ccf70 2024-05-09T08:06:07.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:07.356+0000 7f5aa5200640 1 -- 172.21.15.1:0/4201561865 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f5aa00cb120 con 0x7f5aa00ccf70 2024-05-09T08:06:07.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:07.357+0000 7f5a97400640 1 -- 172.21.15.1:0/4201561865 <== mon.1 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1545 (secure 0 0 0) 0x7f5a8c079f20 con 0x7f5aa00ccf70 2024-05-09T08:06:07.370 INFO:teuthology.orchestra.run.smithi001.stdout:{"election_epoch":30,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":23,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"54329ec8-3270-4043-9a82-c6cd7420aa7f","modified":"2024-05-09T08:01:21.819270Z","created":"2024-05-09T08:01:21.819270Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:3300","nonce":0},{"type":"v1","addr":"172.21.15.1:6789","nonce":0}]},"addr":"172.21.15.1:6789/0","public_addr":"172.21.15.1:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:3300","nonce":0},{"type":"v1","addr":"172.21.15.7:6789","nonce":0}]},"addr":"172.21.15.7:6789/0","public_addr":"172.21.15.7:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:3301","nonce":0},{"type":"v1","addr":"172.21.15.7:6790","nonce":0}]},"addr":"172.21.15.7:6790/0","public_addr":"172.21.15.7:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-05-09T08:06:07.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:07.366+0000 7f5aa5200640 1 -- 172.21.15.1:0/4201561865 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f5a8006be60 msgr2=0x7f5a8006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:06:07.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:07.366+0000 7f5aa5200640 1 --2- 172.21.15.1:0/4201561865 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f5a8006be60 0x7f5a8006e320 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f5a90004810 tx=0x7f5a90007ae0 comp rx=0 tx=0).stop 2024-05-09T08:06:07.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:07.366+0000 7f5aa5200640 1 -- 172.21.15.1:0/4201561865 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5aa00ccf70 msgr2=0x7f5aa00c2620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:06:07.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:07.366+0000 7f5aa5200640 1 --2- 172.21.15.1:0/4201561865 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5aa00ccf70 0x7f5aa00c2620 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f5a8c009890 tx=0x7f5a8c0046c0 comp rx=0 tx=0).stop 2024-05-09T08:06:07.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:07.366+0000 7f5aa5200640 1 -- 172.21.15.1:0/4201561865 shutdown_connections 2024-05-09T08:06:07.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:07.366+0000 7f5aa5200640 1 --2- 172.21.15.1:0/4201561865 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f5a8006be60 0x7f5a8006e320 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:06:07.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:07.366+0000 7f5aa5200640 1 --2- 172.21.15.1:0/4201561865 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5aa00ccf70 0x7f5aa00c2620 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:06:07.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:07.366+0000 7f5aa5200640 1 --2- 172.21.15.1:0/4201561865 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5aa00cf370 0x7f5aa00c2b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:06:07.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:07.366+0000 7f5aa5200640 1 --2- 172.21.15.1:0/4201561865 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f5aa00ca5c0 0x7f5aa0200b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:06:07.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:07.366+0000 7f5aa5200640 1 -- 172.21.15.1:0/4201561865 >> 172.21.15.1:0/4201561865 conn(0x7f5aa00cfc40 msgr2=0x7f5aa0159e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:06:07.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:07.368+0000 7f5aa5200640 1 -- 172.21.15.1:0/4201561865 shutdown_connections 2024-05-09T08:06:07.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:07.368+0000 7f5aa5200640 1 -- 172.21.15.1:0/4201561865 wait complete. 2024-05-09T08:06:08.655 INFO:tasks.workunit.client.0.smithi001.stdout: 2024-05-09T08:06:08.656 INFO:tasks.workunit.client.0.smithi001.stdout: Children see throughput for 1 initial writers = 249478.39 kB/sec 2024-05-09T08:06:08.656 INFO:tasks.workunit.client.0.smithi001.stdout: Parent sees throughput for 1 initial writers = 249473.59 kB/sec 2024-05-09T08:06:08.656 INFO:tasks.workunit.client.0.smithi001.stdout: Min throughput per process = 249478.39 kB/sec 2024-05-09T08:06:08.656 INFO:tasks.workunit.client.0.smithi001.stdout: Max throughput per process = 249478.39 kB/sec 2024-05-09T08:06:08.656 INFO:tasks.workunit.client.0.smithi001.stdout: Avg throughput per process = 249478.39 kB/sec 2024-05-09T08:06:08.656 INFO:tasks.workunit.client.0.smithi001.stdout: Min xfer = 10485760.00 kB 2024-05-09T08:06:10.416 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-05-09T08:06:10.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:10.625+0000 7f57fbe00640 1 Processor -- start 2024-05-09T08:06:10.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:10.626+0000 7f57fbe00640 1 -- start start 2024-05-09T08:06:10.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:10.627+0000 7f57fbe00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f57fc158cf0 0x7f57fc162850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:06:10.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:10.627+0000 7f57fbe00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f57fc15f610 0x7f57fc1691d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:06:10.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:10.627+0000 7f57fbe00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f57fc06bfd0 con 0x7f57fc15f1f0 2024-05-09T08:06:10.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:10.627+0000 7f57fbe00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f57fc15ed60 con 0x7f57fc15f610 2024-05-09T08:06:10.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:10.627+0000 7f57fbe00640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f57fc0c8210 con 0x7f57fc158cf0 2024-05-09T08:06:10.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:10.628+0000 7f57faa00640 1 -- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f57fc158cf0 msgr2=0x7f57fc162850 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.7:3301/0 2024-05-09T08:06:10.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:10.628+0000 7f57faa00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f57fc158cf0 0x7f57fc162850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T08:06:10.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:10.628+0000 7f57fb400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f57fc15f610 0x7f57fc1691d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:06:10.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:10.628+0000 7f57fa000640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f57fc15f1f0 0x7f57fc06b250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.1:55936/0 (socket says 172.21.15.1:55936) 2024-05-09T08:06:10.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:10.628+0000 7f57fa000640 1 -- 172.21.15.1:0/2638620245 learned_addr learned my addr 172.21.15.1:0/2638620245 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T08:06:10.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:10.628+0000 7f57fb400640 1 -- 172.21.15.1:0/2638620245 >> v2:172.21.15.7:3300/0 conn(0x7f57fc15f610 msgr2=0x7f57fc1691d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-05-09T08:06:10.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:10.628+0000 7f57fb400640 1 -- 172.21.15.1:0/2638620245 >> v2:172.21.15.7:3300/0 conn(0x7f57fc15f610 msgr2=0x7f57fc1691d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-09T08:06:10.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:10.628+0000 7f57fb400640 1 --2- 172.21.15.1:0/2638620245 >> v2:172.21.15.7:3300/0 conn(0x7f57fc15f610 0x7f57fc1691d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-09T08:06:10.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:10.628+0000 7f57fb400640 1 --2- 172.21.15.1:0/2638620245 >> v2:172.21.15.7:3300/0 conn(0x7f57fc15f610 0x7f57fc1691d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T08:06:10.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:10.829+0000 7f57faa00640 1 -- 172.21.15.1:0/2638620245 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f57fc158cf0 msgr2=0x7f57fc162850 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.7:3301/0 2024-05-09T08:06:10.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:10.829+0000 7f57faa00640 1 --2- 172.21.15.1:0/2638620245 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f57fc158cf0 0x7f57fc162850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-09T08:06:10.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:10.829+0000 7f57fb400640 1 --2- 172.21.15.1:0/2638620245 >> v2:172.21.15.7:3300/0 conn(0x7f57fc15f610 0x7f57fc1691d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:06:10.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:10.829+0000 7f57fb400640 1 -- 172.21.15.1:0/2638620245 >> v2:172.21.15.7:3300/0 conn(0x7f57fc15f610 msgr2=0x7f57fc1691d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-05-09T08:06:10.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:10.829+0000 7f57fb400640 1 -- 172.21.15.1:0/2638620245 >> v2:172.21.15.7:3300/0 conn(0x7f57fc15f610 msgr2=0x7f57fc1691d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-09T08:06:10.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:10.829+0000 7f57fb400640 1 --2- 172.21.15.1:0/2638620245 >> v2:172.21.15.7:3300/0 conn(0x7f57fc15f610 0x7f57fc1691d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-09T08:06:10.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:10.829+0000 7f57fb400640 1 --2- 172.21.15.1:0/2638620245 >> v2:172.21.15.7:3300/0 conn(0x7f57fc15f610 0x7f57fc1691d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-09T08:06:11.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:11.230+0000 7f57faa00640 1 -- 172.21.15.1:0/2638620245 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f57fc158cf0 msgr2=0x7f57fc162850 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.7:3301/0 2024-05-09T08:06:11.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:11.230+0000 7f57faa00640 1 --2- 172.21.15.1:0/2638620245 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f57fc158cf0 0x7f57fc162850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-09T08:06:11.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:11.230+0000 7f57fb400640 1 --2- 172.21.15.1:0/2638620245 >> v2:172.21.15.7:3300/0 conn(0x7f57fc15f610 0x7f57fc1691d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:06:11.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:11.231+0000 7f57fb400640 1 -- 172.21.15.1:0/2638620245 >> v2:172.21.15.7:3300/0 conn(0x7f57fc15f610 msgr2=0x7f57fc1691d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-05-09T08:06:11.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:11.231+0000 7f57fb400640 1 -- 172.21.15.1:0/2638620245 >> v2:172.21.15.7:3300/0 conn(0x7f57fc15f610 msgr2=0x7f57fc1691d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-09T08:06:11.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:11.231+0000 7f57fb400640 1 --2- 172.21.15.1:0/2638620245 >> v2:172.21.15.7:3300/0 conn(0x7f57fc15f610 0x7f57fc1691d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-09T08:06:11.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:11.231+0000 7f57fb400640 1 --2- 172.21.15.1:0/2638620245 >> v2:172.21.15.7:3300/0 conn(0x7f57fc15f610 0x7f57fc1691d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-09T08:06:12.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:12.031+0000 7f57faa00640 1 -- 172.21.15.1:0/2638620245 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f57fc158cf0 msgr2=0x7f57fc162850 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.7:3301/0 2024-05-09T08:06:12.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:12.031+0000 7f57faa00640 1 --2- 172.21.15.1:0/2638620245 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f57fc158cf0 0x7f57fc162850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-09T08:06:12.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:12.032+0000 7f57fb400640 1 --2- 172.21.15.1:0/2638620245 >> v2:172.21.15.7:3300/0 conn(0x7f57fc15f610 0x7f57fc1691d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:06:12.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:12.033+0000 7f57fb400640 1 -- 172.21.15.1:0/2638620245 >> v2:172.21.15.7:3300/0 conn(0x7f57fc15f610 msgr2=0x7f57fc1691d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-05-09T08:06:12.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:12.033+0000 7f57fb400640 1 -- 172.21.15.1:0/2638620245 >> v2:172.21.15.7:3300/0 conn(0x7f57fc15f610 msgr2=0x7f57fc1691d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-09T08:06:12.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:12.033+0000 7f57fb400640 1 --2- 172.21.15.1:0/2638620245 >> v2:172.21.15.7:3300/0 conn(0x7f57fc15f610 0x7f57fc1691d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-09T08:06:12.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:12.033+0000 7f57fb400640 1 --2- 172.21.15.1:0/2638620245 >> v2:172.21.15.7:3300/0 conn(0x7f57fc15f610 0x7f57fc1691d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-09T08:06:12.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:12.776+0000 7f57f9600640 1 -- 172.21.15.1:0/2638620245 <== mon.1 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1236392244 0 0) 0x7f57fc06bfd0 con 0x7f57fc15f1f0 2024-05-09T08:06:12.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:12.776+0000 7f57f9600640 1 -- 172.21.15.1:0/2638620245 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f57e0003520 con 0x7f57fc15f1f0 2024-05-09T08:06:12.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:12.777+0000 7f57f9600640 1 -- 172.21.15.1:0/2638620245 <== mon.1 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 978329507 0 0) 0x7f57e0003520 con 0x7f57fc15f1f0 2024-05-09T08:06:12.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:12.777+0000 7f57f9600640 1 -- 172.21.15.1:0/2638620245 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f57fc158cf0 msgr2=0x7f57fc162850 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T08:06:12.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:12.777+0000 7f57f9600640 1 --2- 172.21.15.1:0/2638620245 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f57fc158cf0 0x7f57fc162850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:06:12.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:12.777+0000 7f57f9600640 1 -- 172.21.15.1:0/2638620245 >> v2:172.21.15.7:3300/0 conn(0x7f57fc15f610 msgr2=0x7f57fc1691d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T08:06:12.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:12.777+0000 7f57f9600640 1 --2- 172.21.15.1:0/2638620245 >> v2:172.21.15.7:3300/0 conn(0x7f57fc15f610 0x7f57fc1691d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:06:12.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:12.777+0000 7f57f9600640 1 -- 172.21.15.1:0/2638620245 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57fc162d90 con 0x7f57fc15f1f0 2024-05-09T08:06:12.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:12.777+0000 7f57f9600640 1 -- 172.21.15.1:0/2638620245 <== mon.1 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4179846320 0 0) 0x7f57e8001650 con 0x7f57fc15f1f0 2024-05-09T08:06:12.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:12.777+0000 7f57f9600640 1 -- 172.21.15.1:0/2638620245 >> v1:172.21.15.7:6789/0 conn(0x7f57fc15f1f0 legacy=0x7f57fc06b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:06:12.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:12.777+0000 7f57f9600640 1 --2- 172.21.15.1:0/2638620245 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f57e00039b0 0x7f57e0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:06:12.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:12.777+0000 7f57f9600640 1 --2- 172.21.15.1:0/2638620245 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57e0004800 0x7f57e0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:06:12.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:12.777+0000 7f57f9600640 1 --2- 172.21.15.1:0/2638620245 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f57fc158cf0 0x7f57e00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:06:12.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:12.777+0000 7f57f9600640 1 -- 172.21.15.1:0/2638620245 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f57fc15ed60 con 0x7f57e00039b0 2024-05-09T08:06:12.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:12.777+0000 7f57f9600640 1 -- 172.21.15.1:0/2638620245 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f57fc0c8210 con 0x7f57e0004800 2024-05-09T08:06:12.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:12.777+0000 7f57f9600640 1 -- 172.21.15.1:0/2638620245 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f57e000bb20 con 0x7f57fc158cf0 2024-05-09T08:06:12.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:12.778+0000 7f57fb400640 1 -- 172.21.15.1:0/2638620245 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f57fc158cf0 msgr2=0x7f57e00051c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.7:3301/0 2024-05-09T08:06:12.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:12.778+0000 7f57fb400640 1 --2- 172.21.15.1:0/2638620245 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f57fc158cf0 0x7f57e00051c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T08:06:12.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:12.778+0000 7f57fa000640 1 --2- 172.21.15.1:0/2638620245 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57e0004800 0x7f57e0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:06:12.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:12.779+0000 7f57fa000640 1 -- 172.21.15.1:0/2638620245 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f57fc158cf0 msgr2=0x7f57e00051c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T08:06:12.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:12.779+0000 7f57fa000640 1 --2- 172.21.15.1:0/2638620245 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f57fc158cf0 0x7f57e00051c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:06:12.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:12.779+0000 7f57fa000640 1 -- 172.21.15.1:0/2638620245 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f57e00039b0 msgr2=0x7f57e0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:06:12.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:12.779+0000 7f57fa000640 1 --2- 172.21.15.1:0/2638620245 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f57e00039b0 0x7f57e0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:06:12.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:12.779+0000 7f57fa000640 1 -- 172.21.15.1:0/2638620245 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57fc162d90 con 0x7f57e0004800 2024-05-09T08:06:12.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:12.782+0000 7f57fa000640 1 --2- 172.21.15.1:0/2638620245 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57e0004800 0x7f57e0004c80 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f57e8001eb0 tx=0x7f57e0005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=25754ba3599cefd6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:06:12.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:12.782+0000 7f57f9600640 1 -- 172.21.15.1:0/2638620245 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57e8007d00 con 0x7f57e0004800 2024-05-09T08:06:12.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:12.782+0000 7f57f9600640 1 -- 172.21.15.1:0/2638620245 <== mon.1 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57e8005ad0 con 0x7f57e0004800 2024-05-09T08:06:12.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:12.782+0000 7f57f9600640 1 -- 172.21.15.1:0/2638620245 <== mon.1 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57e8007070 con 0x7f57e0004800 2024-05-09T08:06:12.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:12.784+0000 7f57fbe00640 1 -- 172.21.15.1:0/2638620245 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57e0004800 msgr2=0x7f57e0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:06:12.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:12.784+0000 7f57fbe00640 1 --2- 172.21.15.1:0/2638620245 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57e0004800 0x7f57e0004c80 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f57e8001eb0 tx=0x7f57e0005830 comp rx=0 tx=0).stop 2024-05-09T08:06:12.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:12.784+0000 7f57fb400640 1 -- 172.21.15.1:0/2638620245 reap_dead start 2024-05-09T08:06:12.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:12.784+0000 7f57fbe00640 1 -- 172.21.15.1:0/2638620245 shutdown_connections 2024-05-09T08:06:12.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:12.784+0000 7f57fbe00640 1 -- 172.21.15.1:0/2638620245 >> 172.21.15.1:0/2638620245 conn(0x7f57fc0cbc10 msgr2=0x7f57fc0cc020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:06:12.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:12.784+0000 7f57fbe00640 1 -- 172.21.15.1:0/2638620245 shutdown_connections 2024-05-09T08:06:12.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:12.784+0000 7f57fbe00640 1 -- 172.21.15.1:0/2638620245 wait complete. 2024-05-09T08:06:12.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:12.788+0000 7f57fbe00640 1 Processor -- start 2024-05-09T08:06:12.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:12.788+0000 7f57fbe00640 1 -- start start 2024-05-09T08:06:12.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:12.789+0000 7f57fbe00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f57e0004800 0x7f57fc15d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:06:12.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:12.789+0000 7f57fbe00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f57fc15a360 0x7f57fc150160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:06:12.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:12.789+0000 7f57fbe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57fc15fa10 0x7f57fc14d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:06:12.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:12.789+0000 7f57fbe00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f57fc0c7c50 con 0x7f57fc15a360 2024-05-09T08:06:12.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:12.789+0000 7f57fbe00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f57fc0c7970 con 0x7f57fc15fa10 2024-05-09T08:06:12.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:12.789+0000 7f57fbe00640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f57fc0c7f30 con 0x7f57e0004800 2024-05-09T08:06:12.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:12.789+0000 7f57faa00640 1 -- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f57e0004800 msgr2=0x7f57fc15d0d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.7:3301/0 2024-05-09T08:06:12.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:12.789+0000 7f57fb400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57fc15fa10 0x7f57fc14d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:06:12.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:12.789+0000 7f57fa000640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f57fc15a360 0x7f57fc150160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:06:12.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:12.789+0000 7f57faa00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f57e0004800 0x7f57fc15d0d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T08:06:12.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:12.789+0000 7f57fb400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57fc15fa10 0x7f57fc14d9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.1:35228/0 (socket says 172.21.15.1:35228) 2024-05-09T08:06:12.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:12.789+0000 7f57fb400640 1 -- 172.21.15.1:0/3854922142 learned_addr learned my addr 172.21.15.1:0/3854922142 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:06:12.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:12.790+0000 7f57fb400640 1 -- 172.21.15.1:0/3854922142 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f57e0004800 msgr2=0x7f57fc15d0d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T08:06:12.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:12.790+0000 7f57fb400640 1 --2- 172.21.15.1:0/3854922142 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f57e0004800 0x7f57fc15d0d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:06:12.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:12.790+0000 7f57fb400640 1 -- 172.21.15.1:0/3854922142 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f57fc15a360 msgr2=0x7f57fc150160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:06:12.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:12.790+0000 7f57fb400640 1 --2- 172.21.15.1:0/3854922142 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f57fc15a360 0x7f57fc150160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:06:12.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:12.790+0000 7f57fb400640 1 -- 172.21.15.1:0/3854922142 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57fc1592c0 con 0x7f57fc15fa10 2024-05-09T08:06:12.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:12.790+0000 7f57fb400640 1 --2- 172.21.15.1:0/3854922142 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57fc15fa10 0x7f57fc14d9d0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f57fc06cfc0 tx=0x7f57f400ad90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:06:12.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:12.790+0000 7f57f3400640 1 -- 172.21.15.1:0/3854922142 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57f4007cf0 con 0x7f57fc15fa10 2024-05-09T08:06:12.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:12.790+0000 7f57f3400640 1 -- 172.21.15.1:0/3854922142 <== mon.1 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57f401c070 con 0x7f57fc15fa10 2024-05-09T08:06:12.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:12.790+0000 7f57f3400640 1 -- 172.21.15.1:0/3854922142 <== mon.1 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57f4015650 con 0x7f57fc15fa10 2024-05-09T08:06:12.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:12.790+0000 7f57fbe00640 1 -- 172.21.15.1:0/3854922142 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57fc159560 con 0x7f57fc15fa10 2024-05-09T08:06:12.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:12.791+0000 7f57fbe00640 1 -- 172.21.15.1:0/3854922142 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57fc15d610 con 0x7f57fc15fa10 2024-05-09T08:06:12.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:12.792+0000 7f57fbe00640 1 -- 172.21.15.1:0/3854922142 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57fc15c6c0 con 0x7f57fc15fa10 2024-05-09T08:06:12.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:12.792+0000 7f57f3400640 1 -- 172.21.15.1:0/3854922142 <== mon.1 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f57f40157f0 con 0x7f57fc15fa10 2024-05-09T08:06:12.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:12.793+0000 7f57f3400640 1 --2- 172.21.15.1:0/3854922142 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f57dc06be10 0x7f57dc06e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:06:12.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:12.793+0000 7f57f3400640 1 -- 172.21.15.1:0/3854922142 <== mon.1 v2:172.21.15.7:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7f57f408bc90 con 0x7f57fc15fa10 2024-05-09T08:06:12.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:12.794+0000 7f57faa00640 1 --2- 172.21.15.1:0/3854922142 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f57dc06be10 0x7f57dc06e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:06:12.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:12.794+0000 7f57faa00640 1 --2- 172.21.15.1:0/3854922142 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f57dc06be10 0x7f57dc06e2d0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f57ec003bd0 tx=0x7f57ec0090c0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:06:12.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:12.798+0000 7f57f3400640 1 -- 172.21.15.1:0/3854922142 <== mon.1 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f57f4010040 con 0x7f57fc15fa10 2024-05-09T08:06:13.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.068+0000 7f57fbe00640 1 -- 172.21.15.1:0/3854922142 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f57fc04f660 con 0x7f57fc15fa10 2024-05-09T08:06:13.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.068+0000 7f57f3400640 1 -- 172.21.15.1:0/3854922142 <== mon.1 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1538 (secure 0 0 0) 0x7f57f405a1b0 con 0x7f57fc15fa10 2024-05-09T08:06:13.081 INFO:teuthology.orchestra.run.smithi001.stdout:{"election_epoch":34,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"54329ec8-3270-4043-9a82-c6cd7420aa7f","modified":"2024-05-09T08:01:21.819270Z","created":"2024-05-09T08:01:21.819270Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:3300","nonce":0},{"type":"v1","addr":"172.21.15.1:6789","nonce":0}]},"addr":"172.21.15.1:6789/0","public_addr":"172.21.15.1:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:3300","nonce":0},{"type":"v1","addr":"172.21.15.7:6789","nonce":0}]},"addr":"172.21.15.7:6789/0","public_addr":"172.21.15.7:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:3301","nonce":0},{"type":"v1","addr":"172.21.15.7:6790","nonce":0}]},"addr":"172.21.15.7:6790/0","public_addr":"172.21.15.7:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-05-09T08:06:13.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.071+0000 7f57fbe00640 1 -- 172.21.15.1:0/3854922142 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f57dc06be10 msgr2=0x7f57dc06e2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:06:13.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.071+0000 7f57fbe00640 1 --2- 172.21.15.1:0/3854922142 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f57dc06be10 0x7f57dc06e2d0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f57ec003bd0 tx=0x7f57ec0090c0 comp rx=0 tx=0).stop 2024-05-09T08:06:13.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.071+0000 7f57fbe00640 1 -- 172.21.15.1:0/3854922142 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57fc15fa10 msgr2=0x7f57fc14d9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:06:13.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.071+0000 7f57fbe00640 1 --2- 172.21.15.1:0/3854922142 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57fc15fa10 0x7f57fc14d9d0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f57fc06cfc0 tx=0x7f57f400ad90 comp rx=0 tx=0).stop 2024-05-09T08:06:13.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.071+0000 7f57fbe00640 1 -- 172.21.15.1:0/3854922142 shutdown_connections 2024-05-09T08:06:13.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.071+0000 7f57fbe00640 1 --2- 172.21.15.1:0/3854922142 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f57dc06be10 0x7f57dc06e2d0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:06:13.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.071+0000 7f57fbe00640 1 --2- 172.21.15.1:0/3854922142 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57fc15fa10 0x7f57fc14d9d0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:06:13.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.071+0000 7f57fbe00640 1 --2- 172.21.15.1:0/3854922142 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f57fc15a360 0x7f57fc150160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:06:13.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.071+0000 7f57fbe00640 1 --2- 172.21.15.1:0/3854922142 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f57e0004800 0x7f57fc15d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:06:13.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.071+0000 7f57fbe00640 1 -- 172.21.15.1:0/3854922142 >> 172.21.15.1:0/3854922142 conn(0x7f57fc0cbc10 msgr2=0x7f57fc0ca920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:06:13.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.071+0000 7f57fbe00640 1 -- 172.21.15.1:0/3854922142 shutdown_connections 2024-05-09T08:06:13.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.071+0000 7f57fbe00640 1 -- 172.21.15.1:0/3854922142 wait complete. 2024-05-09T08:06:13.104 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2024-05-09T08:06:13.105 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon a 2024-05-09T08:06:13.105 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-05-09T08:06:13.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.433+0000 7fede4a00640 1 Processor -- start 2024-05-09T08:06:13.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.434+0000 7fede4a00640 1 -- start start 2024-05-09T08:06:13.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.436+0000 7fede4a00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fede00c6c50 0x7fede0071af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:06:13.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.436+0000 7fede4a00640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fede015c5c0 0x7fede00c8280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:06:13.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.436+0000 7fede4a00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fede00c8f60 con 0x7fede00c7030 2024-05-09T08:06:13.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.436+0000 7fede4a00640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7fede00c89a0 con 0x7fede015c5c0 2024-05-09T08:06:13.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.436+0000 7fede4a00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fede00c8c80 con 0x7fede00c6c50 2024-05-09T08:06:13.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.440+0000 7fedde800640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fede00c6c50 0x7fede0071af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:06:13.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.446+0000 7feddf200640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fede015c5c0 0x7fede00c8280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:06:13.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.446+0000 7feddde00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fede00c7030 0x7fede015beb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.1:55946/0 (socket says 172.21.15.1:55946) 2024-05-09T08:06:13.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.448+0000 7feddde00640 1 -- 172.21.15.1:0/1483521804 learned_addr learned my addr 172.21.15.1:0/1483521804 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T08:06:13.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.449+0000 7feddd400640 1 -- 172.21.15.1:0/1483521804 <== mon.1 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2738629558 0 0) 0x7fede00c8f60 con 0x7fede00c7030 2024-05-09T08:06:13.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.449+0000 7feddd400640 1 -- 172.21.15.1:0/1483521804 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fedc4003520 con 0x7fede00c7030 2024-05-09T08:06:13.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.451+0000 7fedde800640 1 -- 172.21.15.1:0/1483521804 >> v2:172.21.15.1:3300/0 conn(0x7fede015c5c0 msgr2=0x7fede00c8280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:06:13.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.451+0000 7fedde800640 1 --2- 172.21.15.1:0/1483521804 >> v2:172.21.15.1:3300/0 conn(0x7fede015c5c0 0x7fede00c8280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:06:13.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.451+0000 7fedde800640 1 -- 172.21.15.1:0/1483521804 >> v1:172.21.15.7:6789/0 conn(0x7fede00c7030 legacy=0x7fede015beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:06:13.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.451+0000 7feddd400640 1 -- 172.21.15.1:0/1483521804 <== mon.1 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 716988403 0 0) 0x7fedc4003520 con 0x7fede00c7030 2024-05-09T08:06:13.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.451+0000 7fedde800640 1 -- 172.21.15.1:0/1483521804 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fede006f0f0 con 0x7fede00c6c50 2024-05-09T08:06:13.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.455+0000 7fedde800640 1 --2- 172.21.15.1:0/1483521804 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fede00c6c50 0x7fede0071af0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fede00c6700 tx=0x7fedd802a630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8770aca7ceedf563 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:06:13.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.456+0000 7feddd400640 1 -- 172.21.15.1:0/1483521804 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedcc001670 con 0x7fede00c6c50 2024-05-09T08:06:13.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.460+0000 7feddd400640 1 -- 172.21.15.1:0/1483521804 <== mon.1 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fedd8007540 con 0x7fede00c6c50 2024-05-09T08:06:13.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.460+0000 7feddd400640 1 -- 172.21.15.1:0/1483521804 <== mon.1 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedcc001670 con 0x7fede00c6c50 2024-05-09T08:06:13.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.461+0000 7fede4a00640 1 -- 172.21.15.1:0/1483521804 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fede00c6c50 msgr2=0x7fede0071af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:06:13.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.461+0000 7fede4a00640 1 --2- 172.21.15.1:0/1483521804 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fede00c6c50 0x7fede0071af0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fede00c6700 tx=0x7fedd802a630 comp rx=0 tx=0).stop 2024-05-09T08:06:13.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.463+0000 7fede4a00640 1 -- 172.21.15.1:0/1483521804 shutdown_connections 2024-05-09T08:06:13.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.463+0000 7fede4a00640 1 --2- 172.21.15.1:0/1483521804 >> v2:172.21.15.1:3300/0 conn(0x7fede015c5c0 0x7fede00c8280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:06:13.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.464+0000 7fede4a00640 1 --2- 172.21.15.1:0/1483521804 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fede00c6c50 0x7fede0071af0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:06:13.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.464+0000 7fede4a00640 1 -- 172.21.15.1:0/1483521804 >> 172.21.15.1:0/1483521804 conn(0x7fede006b250 msgr2=0x7fede006b630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:06:13.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.466+0000 7fede4a00640 1 -- 172.21.15.1:0/1483521804 shutdown_connections 2024-05-09T08:06:13.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.471+0000 7fede4a00640 1 -- 172.21.15.1:0/1483521804 wait complete. 2024-05-09T08:06:13.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.473+0000 7fede4a00640 1 Processor -- start 2024-05-09T08:06:13.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.479+0000 7fede4a00640 1 -- start start 2024-05-09T08:06:13.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.480+0000 7fede4a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fede00c6c50 0x7fede00b14e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:06:13.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.480+0000 7fede4a00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fede00c7030 0x7fede00a94d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:06:13.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.480+0000 7fede4a00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fede015c5c0 0x7fede02003b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:06:13.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.480+0000 7fede4a00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fede00c7740 con 0x7fede00c7030 2024-05-09T08:06:13.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.480+0000 7fede4a00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fede006f6b0 con 0x7fede00c6c50 2024-05-09T08:06:13.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.480+0000 7fede4a00640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7fede006fbd0 con 0x7fede015c5c0 2024-05-09T08:06:13.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.482+0000 7fedde800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fede00c6c50 0x7fede00b14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:06:13.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.482+0000 7fedde800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fede00c6c50 0x7fede00b14e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.1:35242/0 (socket says 172.21.15.1:35242) 2024-05-09T08:06:13.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.482+0000 7fedde800640 1 -- 172.21.15.1:0/919039184 learned_addr learned my addr 172.21.15.1:0/919039184 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:06:13.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.482+0000 7feddde00640 1 --2- 172.21.15.1:0/919039184 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fede00c7030 0x7fede00a94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:06:13.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.483+0000 7feddf200640 1 -- 172.21.15.1:0/919039184 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fede015c5c0 msgr2=0x7fede02003b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.7:3301/0 2024-05-09T08:06:13.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.483+0000 7feddf200640 1 --2- 172.21.15.1:0/919039184 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fede015c5c0 0x7fede02003b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T08:06:13.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.484+0000 7fedde800640 1 -- 172.21.15.1:0/919039184 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fede015c5c0 msgr2=0x7fede02003b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T08:06:13.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.484+0000 7fedde800640 1 --2- 172.21.15.1:0/919039184 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fede015c5c0 0x7fede02003b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:06:13.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.484+0000 7fedde800640 1 -- 172.21.15.1:0/919039184 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fede00c7030 msgr2=0x7fede00a94d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:06:13.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.484+0000 7fedde800640 1 --2- 172.21.15.1:0/919039184 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fede00c7030 0x7fede00a94d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:06:13.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.484+0000 7fedde800640 1 -- 172.21.15.1:0/919039184 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fede0149710 con 0x7fede00c6c50 2024-05-09T08:06:13.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.484+0000 7fedde800640 1 --2- 172.21.15.1:0/919039184 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fede00c6c50 0x7fede00b14e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fedd8009e50 tx=0x7fedd8004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:06:13.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.484+0000 7fedd6a00640 1 -- 172.21.15.1:0/919039184 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedd8004450 con 0x7fede00c6c50 2024-05-09T08:06:13.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.485+0000 7fedd6a00640 1 -- 172.21.15.1:0/919039184 <== mon.1 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fedd80045f0 con 0x7fede00c6c50 2024-05-09T08:06:13.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.485+0000 7fedd6a00640 1 -- 172.21.15.1:0/919039184 <== mon.1 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedd8007e50 con 0x7fede00c6c50 2024-05-09T08:06:13.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.486+0000 7fede4a00640 1 -- 172.21.15.1:0/919039184 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fede01499b0 con 0x7fede00c6c50 2024-05-09T08:06:13.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.486+0000 7fede4a00640 1 -- 172.21.15.1:0/919039184 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fede0149f70 con 0x7fede00c6c50 2024-05-09T08:06:13.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.491+0000 7fedd6a00640 1 -- 172.21.15.1:0/919039184 <== mon.1 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fedd8039040 con 0x7fede00c6c50 2024-05-09T08:06:13.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.493+0000 7fede4a00640 1 --2- 172.21.15.1:0/919039184 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fede0200c40 0x7fede014a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:06:13.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.493+0000 7fede4a00640 1 -- 172.21.15.1:0/919039184 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fede01428d0 con 0x7fede0200c40 2024-05-09T08:06:13.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.493+0000 7fedd6a00640 1 --2- 172.21.15.1:0/919039184 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fedc006be30 0x7fedc006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:06:13.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.494+0000 7fedd6a00640 1 -- 172.21.15.1:0/919039184 <== mon.1 v2:172.21.15.7:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7fedd80ac570 con 0x7fede00c6c50 2024-05-09T08:06:13.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.494+0000 7feddde00640 1 --2- 172.21.15.1:0/919039184 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fede0200c40 0x7fede014a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:06:13.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.494+0000 7feddf200640 1 --2- 172.21.15.1:0/919039184 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fedc006be30 0x7fedc006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:06:13.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.494+0000 7feddde00640 1 -- 172.21.15.1:0/919039184 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fede004f660 con 0x7fede0200c40 2024-05-09T08:06:13.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.495+0000 7feddde00640 1 --2- 172.21.15.1:0/919039184 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fede0200c40 0x7fede014a770 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fedcc001380 tx=0x7fedcc00db80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:06:13.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.495+0000 7fedd6a00640 1 -- 172.21.15.1:0/919039184 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedcc005ad0 con 0x7fede0200c40 2024-05-09T08:06:13.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.497+0000 7feddf200640 1 --2- 172.21.15.1:0/919039184 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fedc006be30 0x7fedc006e2f0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fedd0006b40 tx=0x7fedd0008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:06:13.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.509+0000 7fedd6a00640 1 -- 172.21.15.1:0/919039184 <== mon.0 v2:172.21.15.1:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fedcc017070 con 0x7fede0200c40 2024-05-09T08:06:13.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.509+0000 7fedd6a00640 1 -- 172.21.15.1:0/919039184 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fede0200c40 msgr2=0x7fede014a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:06:13.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.509+0000 7fedd6a00640 1 --2- 172.21.15.1:0/919039184 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fede0200c40 0x7fede014a770 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fedcc001380 tx=0x7fedcc00db80 comp rx=0 tx=0).stop 2024-05-09T08:06:13.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.509+0000 7fedd6a00640 1 -- 172.21.15.1:0/919039184 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fede0200c40 msgr2=0x7fede014a770 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-05-09T08:06:13.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.509+0000 7fedd6a00640 1 --2- 172.21.15.1:0/919039184 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fede0200c40 0x7fede014a770 secure :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fedcc001380 tx=0x7fedcc00db80 comp rx=0 tx=0).stop 2024-05-09T08:06:13.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.528+0000 7fede4a00640 1 --2- 172.21.15.1:0/919039184 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fede014a170 0x7fede014acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:06:13.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.528+0000 7fede4a00640 1 -- 172.21.15.1:0/919039184 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fede00c7520 con 0x7fede014a170 2024-05-09T08:06:13.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.529+0000 7feddde00640 1 --2- 172.21.15.1:0/919039184 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fede014a170 0x7fede014acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:06:13.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.529+0000 7feddde00640 1 -- 172.21.15.1:0/919039184 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fede0071910 con 0x7fede014a170 2024-05-09T08:06:13.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.530+0000 7feddde00640 1 --2- 172.21.15.1:0/919039184 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fede014a170 0x7fede014acb0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fede01498e0 tx=0x7fedcc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:06:13.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.531+0000 7fedd6a00640 1 -- 172.21.15.1:0/919039184 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedcc01ce20 con 0x7fede014a170 2024-05-09T08:06:13.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.531+0000 7fedd6a00640 1 -- 172.21.15.1:0/919039184 <== mon.0 v2:172.21.15.1:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4865 (secure 0 0 0) 0x7fedcc007280 con 0x7fede014a170 2024-05-09T08:06:13.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.531+0000 7fedd6a00640 1 -- 172.21.15.1:0/919039184 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fede014a170 msgr2=0x7fede014acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:06:13.546 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-05-09T08:06:13.546 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "a", 2024-05-09T08:06:13.546 INFO:teuthology.orchestra.run.smithi001.stdout: "rank": 0, 2024-05-09T08:06:13.547 INFO:teuthology.orchestra.run.smithi001.stdout: "state": "leader", 2024-05-09T08:06:13.547 INFO:teuthology.orchestra.run.smithi001.stdout: "election_epoch": 34, 2024-05-09T08:06:13.547 INFO:teuthology.orchestra.run.smithi001.stdout: "uptime": 36148, 2024-05-09T08:06:13.547 INFO:teuthology.orchestra.run.smithi001.stdout: "quorum": [ 2024-05-09T08:06:13.547 INFO:teuthology.orchestra.run.smithi001.stdout: 0, 2024-05-09T08:06:13.547 INFO:teuthology.orchestra.run.smithi001.stdout: 1 2024-05-09T08:06:13.547 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:06:13.547 INFO:teuthology.orchestra.run.smithi001.stdout: "quorum_age": 0, 2024-05-09T08:06:13.547 INFO:teuthology.orchestra.run.smithi001.stdout: "features": { 2024-05-09T08:06:13.547 INFO:teuthology.orchestra.run.smithi001.stdout: "required_con": "2450521708007866372", 2024-05-09T08:06:13.547 INFO:teuthology.orchestra.run.smithi001.stdout: "required_mon": [ 2024-05-09T08:06:13.547 INFO:teuthology.orchestra.run.smithi001.stdout: "kraken", 2024-05-09T08:06:13.548 INFO:teuthology.orchestra.run.smithi001.stdout: "luminous", 2024-05-09T08:06:13.548 INFO:teuthology.orchestra.run.smithi001.stdout: "mimic", 2024-05-09T08:06:13.548 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap-prune", 2024-05-09T08:06:13.549 INFO:teuthology.orchestra.run.smithi001.stdout: "nautilus", 2024-05-09T08:06:13.549 INFO:teuthology.orchestra.run.smithi001.stdout: "octopus", 2024-05-09T08:06:13.549 INFO:teuthology.orchestra.run.smithi001.stdout: "pacific", 2024-05-09T08:06:13.549 INFO:teuthology.orchestra.run.smithi001.stdout: "elector-pinging", 2024-05-09T08:06:13.549 INFO:teuthology.orchestra.run.smithi001.stdout: "quincy", 2024-05-09T08:06:13.549 INFO:teuthology.orchestra.run.smithi001.stdout: "reef", 2024-05-09T08:06:13.549 INFO:teuthology.orchestra.run.smithi001.stdout: "squid" 2024-05-09T08:06:13.549 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:06:13.549 INFO:teuthology.orchestra.run.smithi001.stdout: "quorum_con": "4540701547738038271", 2024-05-09T08:06:13.550 INFO:teuthology.orchestra.run.smithi001.stdout: "quorum_mon": [ 2024-05-09T08:06:13.550 INFO:teuthology.orchestra.run.smithi001.stdout: "kraken", 2024-05-09T08:06:13.550 INFO:teuthology.orchestra.run.smithi001.stdout: "luminous", 2024-05-09T08:06:13.550 INFO:teuthology.orchestra.run.smithi001.stdout: "mimic", 2024-05-09T08:06:13.550 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap-prune", 2024-05-09T08:06:13.550 INFO:teuthology.orchestra.run.smithi001.stdout: "nautilus", 2024-05-09T08:06:13.550 INFO:teuthology.orchestra.run.smithi001.stdout: "octopus", 2024-05-09T08:06:13.550 INFO:teuthology.orchestra.run.smithi001.stdout: "pacific", 2024-05-09T08:06:13.550 INFO:teuthology.orchestra.run.smithi001.stdout: "elector-pinging", 2024-05-09T08:06:13.550 INFO:teuthology.orchestra.run.smithi001.stdout: "quincy", 2024-05-09T08:06:13.550 INFO:teuthology.orchestra.run.smithi001.stdout: "reef", 2024-05-09T08:06:13.550 INFO:teuthology.orchestra.run.smithi001.stdout: "squid" 2024-05-09T08:06:13.551 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:06:13.551 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:06:13.551 INFO:teuthology.orchestra.run.smithi001.stdout: "outside_quorum": [], 2024-05-09T08:06:13.551 INFO:teuthology.orchestra.run.smithi001.stdout: "extra_probe_peers": [], 2024-05-09T08:06:13.551 INFO:teuthology.orchestra.run.smithi001.stdout: "sync_provider": [], 2024-05-09T08:06:13.551 INFO:teuthology.orchestra.run.smithi001.stdout: "monmap": { 2024-05-09T08:06:13.551 INFO:teuthology.orchestra.run.smithi001.stdout: "epoch": 1, 2024-05-09T08:06:13.551 INFO:teuthology.orchestra.run.smithi001.stdout: "fsid": "54329ec8-3270-4043-9a82-c6cd7420aa7f", 2024-05-09T08:06:13.551 INFO:teuthology.orchestra.run.smithi001.stdout: "modified": "2024-05-09T08:01:21.819270Z", 2024-05-09T08:06:13.551 INFO:teuthology.orchestra.run.smithi001.stdout: "created": "2024-05-09T08:01:21.819270Z", 2024-05-09T08:06:13.551 INFO:teuthology.orchestra.run.smithi001.stdout: "min_mon_release": 19, 2024-05-09T08:06:13.551 INFO:teuthology.orchestra.run.smithi001.stdout: "min_mon_release_name": "squid", 2024-05-09T08:06:13.551 INFO:teuthology.orchestra.run.smithi001.stdout: "election_strategy": 1, 2024-05-09T08:06:13.551 INFO:teuthology.orchestra.run.smithi001.stdout: "disallowed_leaders: ": "", 2024-05-09T08:06:13.552 INFO:teuthology.orchestra.run.smithi001.stdout: "stretch_mode": false, 2024-05-09T08:06:13.552 INFO:teuthology.orchestra.run.smithi001.stdout: "tiebreaker_mon": "", 2024-05-09T08:06:13.552 INFO:teuthology.orchestra.run.smithi001.stdout: "removed_ranks: ": "", 2024-05-09T08:06:13.552 INFO:teuthology.orchestra.run.smithi001.stdout: "features": { 2024-05-09T08:06:13.552 INFO:teuthology.orchestra.run.smithi001.stdout: "persistent": [ 2024-05-09T08:06:13.552 INFO:teuthology.orchestra.run.smithi001.stdout: "kraken", 2024-05-09T08:06:13.552 INFO:teuthology.orchestra.run.smithi001.stdout: "luminous", 2024-05-09T08:06:13.552 INFO:teuthology.orchestra.run.smithi001.stdout: "mimic", 2024-05-09T08:06:13.552 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap-prune", 2024-05-09T08:06:13.552 INFO:teuthology.orchestra.run.smithi001.stdout: "nautilus", 2024-05-09T08:06:13.552 INFO:teuthology.orchestra.run.smithi001.stdout: "octopus", 2024-05-09T08:06:13.552 INFO:teuthology.orchestra.run.smithi001.stdout: "pacific", 2024-05-09T08:06:13.552 INFO:teuthology.orchestra.run.smithi001.stdout: "elector-pinging", 2024-05-09T08:06:13.553 INFO:teuthology.orchestra.run.smithi001.stdout: "quincy", 2024-05-09T08:06:13.553 INFO:teuthology.orchestra.run.smithi001.stdout: "reef", 2024-05-09T08:06:13.553 INFO:teuthology.orchestra.run.smithi001.stdout: "squid" 2024-05-09T08:06:13.553 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:06:13.553 INFO:teuthology.orchestra.run.smithi001.stdout: "optional": [] 2024-05-09T08:06:13.553 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:06:13.553 INFO:teuthology.orchestra.run.smithi001.stdout: "mons": [ 2024-05-09T08:06:13.553 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:06:13.554 INFO:teuthology.orchestra.run.smithi001.stdout: "rank": 0, 2024-05-09T08:06:13.554 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "a", 2024-05-09T08:06:13.554 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-05-09T08:06:13.554 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-05-09T08:06:13.554 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:06:13.555 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-05-09T08:06:13.555 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:3300", 2024-05-09T08:06:13.555 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:06:13.555 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:06:13.555 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:06:13.555 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-05-09T08:06:13.555 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6789", 2024-05-09T08:06:13.556 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:06:13.556 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:06:13.556 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:06:13.556 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:06:13.556 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6789/0", 2024-05-09T08:06:13.556 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.1:6789/0", 2024-05-09T08:06:13.557 INFO:teuthology.orchestra.run.smithi001.stdout: "priority": 0, 2024-05-09T08:06:13.557 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 0, 2024-05-09T08:06:13.557 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_location": "{}" 2024-05-09T08:06:13.557 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:06:13.557 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:06:13.557 INFO:teuthology.orchestra.run.smithi001.stdout: "rank": 1, 2024-05-09T08:06:13.557 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "b", 2024-05-09T08:06:13.557 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-05-09T08:06:13.557 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-05-09T08:06:13.557 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:06:13.558 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-05-09T08:06:13.558 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:3300", 2024-05-09T08:06:13.558 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:06:13.558 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:06:13.558 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:06:13.558 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-05-09T08:06:13.558 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:6789", 2024-05-09T08:06:13.558 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:06:13.558 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:06:13.558 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:06:13.558 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:06:13.559 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:6789/0", 2024-05-09T08:06:13.559 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.7:6789/0", 2024-05-09T08:06:13.559 INFO:teuthology.orchestra.run.smithi001.stdout: "priority": 0, 2024-05-09T08:06:13.559 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 0, 2024-05-09T08:06:13.559 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_location": "{}" 2024-05-09T08:06:13.559 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:06:13.560 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:06:13.560 INFO:teuthology.orchestra.run.smithi001.stdout: "rank": 2, 2024-05-09T08:06:13.560 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "c", 2024-05-09T08:06:13.560 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-05-09T08:06:13.560 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-05-09T08:06:13.560 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:06:13.560 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-05-09T08:06:13.560 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:3301", 2024-05-09T08:06:13.560 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:06:13.560 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:06:13.560 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:06:13.561 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-05-09T08:06:13.561 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:6790", 2024-05-09T08:06:13.561 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:06:13.561 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:06:13.561 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:06:13.561 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:06:13.561 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:6790/0", 2024-05-09T08:06:13.561 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.7:6790/0", 2024-05-09T08:06:13.561 INFO:teuthology.orchestra.run.smithi001.stdout: "priority": 0, 2024-05-09T08:06:13.561 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 0, 2024-05-09T08:06:13.561 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_location": "{}" 2024-05-09T08:06:13.562 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:06:13.562 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:06:13.562 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:06:13.562 INFO:teuthology.orchestra.run.smithi001.stdout: "feature_map": { 2024-05-09T08:06:13.562 INFO:teuthology.orchestra.run.smithi001.stdout: "mon": [ 2024-05-09T08:06:13.562 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:06:13.563 INFO:teuthology.orchestra.run.smithi001.stdout: "features": "0x3f03cffffffdffff", 2024-05-09T08:06:13.563 INFO:teuthology.orchestra.run.smithi001.stdout: "release": "luminous", 2024-05-09T08:06:13.563 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 1 2024-05-09T08:06:13.563 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:06:13.563 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:06:13.563 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": [ 2024-05-09T08:06:13.563 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:06:13.564 INFO:teuthology.orchestra.run.smithi001.stdout: "features": "0x3f03cffffffdffff", 2024-05-09T08:06:13.564 INFO:teuthology.orchestra.run.smithi001.stdout: "release": "luminous", 2024-05-09T08:06:13.564 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 1 2024-05-09T08:06:13.564 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:06:13.564 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:06:13.564 INFO:teuthology.orchestra.run.smithi001.stdout: "client": [ 2024-05-09T08:06:13.564 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:06:13.565 INFO:teuthology.orchestra.run.smithi001.stdout: "features": "0x3f03cffffffdffff", 2024-05-09T08:06:13.565 INFO:teuthology.orchestra.run.smithi001.stdout: "release": "luminous", 2024-05-09T08:06:13.565 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 1 2024-05-09T08:06:13.565 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:06:13.565 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:06:13.565 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr": [ 2024-05-09T08:06:13.565 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:06:13.565 INFO:teuthology.orchestra.run.smithi001.stdout: "features": "0x3f03cffffffdffff", 2024-05-09T08:06:13.565 INFO:teuthology.orchestra.run.smithi001.stdout: "release": "luminous", 2024-05-09T08:06:13.565 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 1 2024-05-09T08:06:13.565 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:06:13.565 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:06:13.565 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:06:13.566 INFO:teuthology.orchestra.run.smithi001.stdout: "stretch_mode": false 2024-05-09T08:06:13.566 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-05-09T08:06:13.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.531+0000 7fedd6a00640 1 --2- 172.21.15.1:0/919039184 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fede014a170 0x7fede014acb0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fede01498e0 tx=0x7fedcc0057f0 comp rx=0 tx=0).stop 2024-05-09T08:06:13.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.531+0000 7fedd6a00640 1 -- 172.21.15.1:0/919039184 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fede014a170 msgr2=0x7fede014acb0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-05-09T08:06:13.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.531+0000 7fedd6a00640 1 --2- 172.21.15.1:0/919039184 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fede014a170 0x7fede014acb0 secure :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fede01498e0 tx=0x7fedcc0057f0 comp rx=0 tx=0).stop 2024-05-09T08:06:13.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.532+0000 7fede4a00640 1 -- 172.21.15.1:0/919039184 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fedc006be30 msgr2=0x7fedc006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:06:13.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.532+0000 7fede4a00640 1 --2- 172.21.15.1:0/919039184 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fedc006be30 0x7fedc006e2f0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fedd0006b40 tx=0x7fedd0008040 comp rx=0 tx=0).stop 2024-05-09T08:06:13.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.532+0000 7fede4a00640 1 -- 172.21.15.1:0/919039184 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fede00c6c50 msgr2=0x7fede00b14e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:06:13.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.532+0000 7fede4a00640 1 --2- 172.21.15.1:0/919039184 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fede00c6c50 0x7fede00b14e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fedd8009e50 tx=0x7fedd8004a40 comp rx=0 tx=0).stop 2024-05-09T08:06:13.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.532+0000 7feddf200640 1 -- 172.21.15.1:0/919039184 reap_dead start 2024-05-09T08:06:13.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.532+0000 7feddf200640 1 -- 172.21.15.1:0/919039184 reap_dead start 2024-05-09T08:06:13.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.533+0000 7fede4a00640 1 -- 172.21.15.1:0/919039184 shutdown_connections 2024-05-09T08:06:13.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.533+0000 7fede4a00640 1 -- 172.21.15.1:0/919039184 >> 172.21.15.1:0/919039184 conn(0x7fede006b250 msgr2=0x7fede00c9050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:06:13.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.533+0000 7fede4a00640 1 -- 172.21.15.1:0/919039184 shutdown_connections 2024-05-09T08:06:13.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.534+0000 7fede4a00640 1 -- 172.21.15.1:0/919039184 wait complete. 2024-05-09T08:06:13.581 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-05-09T08:06:13.581 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon b 2024-05-09T08:06:13.582 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-05-09T08:06:13.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.889+0000 7efd0ec00640 1 Processor -- start 2024-05-09T08:06:13.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.890+0000 7efd0ec00640 1 -- start start 2024-05-09T08:06:13.907 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.891+0000 7efd0ec00640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7efd08159b40 0x7efd080728e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:06:13.907 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.892+0000 7efd0ec00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7efd08072e20 0x7efd08150540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:06:13.907 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.892+0000 7efd0ec00640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efd08073520 con 0x7efd08150a80 2024-05-09T08:06:13.907 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.892+0000 7efd0ec00640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7efd081568f0 con 0x7efd08159b40 2024-05-09T08:06:13.907 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.892+0000 7efd0ec00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7efd080c2080 con 0x7efd08072e20 2024-05-09T08:06:13.907 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.892+0000 7efd07400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7efd08072e20 0x7efd08150540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:06:13.907 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.892+0000 7efd07400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7efd08072e20 0x7efd08150540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.1:41736/0 (socket says 172.21.15.1:41736) 2024-05-09T08:06:13.907 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.892+0000 7efd07400640 1 -- 172.21.15.1:0/2135783638 learned_addr learned my addr 172.21.15.1:0/2135783638 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:06:13.907 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.893+0000 7efd07400640 1 -- 172.21.15.1:0/2135783638 >> v2:172.21.15.1:3300/0 conn(0x7efd08159b40 msgr2=0x7efd080728e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T08:06:13.907 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.893+0000 7efd07e00640 1 --2- 172.21.15.1:0/2135783638 >> v2:172.21.15.1:3300/0 conn(0x7efd08159b40 0x7efd080728e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:06:13.907 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.893+0000 7efd07400640 1 --2- 172.21.15.1:0/2135783638 >> v2:172.21.15.1:3300/0 conn(0x7efd08159b40 0x7efd080728e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:06:13.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.894+0000 7efd07400640 1 -- 172.21.15.1:0/2135783638 >> v1:172.21.15.1:6789/0 conn(0x7efd08150a80 legacy=0x7efd08156d80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T08:06:13.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.894+0000 7efd07400640 1 -- 172.21.15.1:0/2135783638 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd0815a0d0 con 0x7efd08072e20 2024-05-09T08:06:13.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.895+0000 7efd07e00640 1 --2- 172.21.15.1:0/2135783638 >> v2:172.21.15.1:3300/0 conn(0x7efd08159b40 0x7efd080728e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T08:06:13.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.896+0000 7efd07400640 1 --2- 172.21.15.1:0/2135783638 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efd08072e20 0x7efd08150540 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7efd080c9790 tx=0x7efd0002a780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b0fb4277bba71fc5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:06:13.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.896+0000 7efd06a00640 1 -- 172.21.15.1:0/2135783638 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd00036070 con 0x7efd08072e20 2024-05-09T08:06:13.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.896+0000 7efd06a00640 1 -- 172.21.15.1:0/2135783638 <== mon.1 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd00031600 con 0x7efd08072e20 2024-05-09T08:06:13.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.896+0000 7efd06a00640 1 -- 172.21.15.1:0/2135783638 <== mon.1 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd000319d0 con 0x7efd08072e20 2024-05-09T08:06:13.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.897+0000 7efd0ec00640 1 -- 172.21.15.1:0/2135783638 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efd08072e20 msgr2=0x7efd08150540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:06:13.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.897+0000 7efd0ec00640 1 --2- 172.21.15.1:0/2135783638 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efd08072e20 0x7efd08150540 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7efd080c9790 tx=0x7efd0002a780 comp rx=0 tx=0).stop 2024-05-09T08:06:13.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.897+0000 7efd0ec00640 1 -- 172.21.15.1:0/2135783638 shutdown_connections 2024-05-09T08:06:13.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.897+0000 7efd0ec00640 1 --2- 172.21.15.1:0/2135783638 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efd08072e20 0x7efd08150540 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:06:13.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.897+0000 7efd0ec00640 1 --2- 172.21.15.1:0/2135783638 >> v2:172.21.15.1:3300/0 conn(0x7efd08159b40 0x7efd080728e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:06:13.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.897+0000 7efd0ec00640 1 -- 172.21.15.1:0/2135783638 >> 172.21.15.1:0/2135783638 conn(0x7efd0806dad0 msgr2=0x7efd0806deb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:06:13.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.897+0000 7efd0ec00640 1 -- 172.21.15.1:0/2135783638 shutdown_connections 2024-05-09T08:06:13.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.898+0000 7efd0ec00640 1 -- 172.21.15.1:0/2135783638 wait complete. 2024-05-09T08:06:13.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.899+0000 7efd0ec00640 1 Processor -- start 2024-05-09T08:06:13.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.900+0000 7efd0ec00640 1 -- start start 2024-05-09T08:06:13.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.901+0000 7efd0ec00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efd08150a80 0x7efd0814aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:06:13.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.901+0000 7efd0ec00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efd08159b40 0x7efd080c3a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:06:13.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.901+0000 7efd0ec00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7efd081403b0 0x7efd080c47a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:06:13.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.901+0000 7efd0ec00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7efd0815a970 con 0x7efd08159b40 2024-05-09T08:06:13.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.901+0000 7efd0ec00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7efd0815a690 con 0x7efd08150a80 2024-05-09T08:06:13.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.901+0000 7efd0ec00640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7efd08157490 con 0x7efd081403b0 2024-05-09T08:06:13.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.902+0000 7efd07e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efd08150a80 0x7efd0814aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:06:13.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.902+0000 7efd07e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efd08150a80 0x7efd0814aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.1:41748/0 (socket says 172.21.15.1:41748) 2024-05-09T08:06:13.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.902+0000 7efd07e00640 1 -- 172.21.15.1:0/3426933753 learned_addr learned my addr 172.21.15.1:0/3426933753 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:06:13.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.902+0000 7efd07e00640 1 -- 172.21.15.1:0/3426933753 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7efd081403b0 msgr2=0x7efd080c47a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T08:06:13.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.902+0000 7efd07e00640 1 --2- 172.21.15.1:0/3426933753 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7efd081403b0 0x7efd080c47a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:06:13.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.902+0000 7efd07e00640 1 -- 172.21.15.1:0/3426933753 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efd08159b40 msgr2=0x7efd080c3a90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T08:06:13.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.902+0000 7efd07e00640 1 --2- 172.21.15.1:0/3426933753 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efd08159b40 0x7efd080c3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:06:13.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.902+0000 7efd07e00640 1 -- 172.21.15.1:0/3426933753 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd0813dea0 con 0x7efd08150a80 2024-05-09T08:06:13.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.903+0000 7efd07e00640 1 --2- 172.21.15.1:0/3426933753 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efd08150a80 0x7efd0814aa70 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7efcf400c7f0 tx=0x7efcf400ccb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:06:13.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.903+0000 7efcffe00640 1 -- 172.21.15.1:0/3426933753 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcf4015070 con 0x7efd08150a80 2024-05-09T08:06:13.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.904+0000 7efcffe00640 1 -- 172.21.15.1:0/3426933753 <== mon.1 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efcf4007820 con 0x7efd08150a80 2024-05-09T08:06:13.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.904+0000 7efcffe00640 1 -- 172.21.15.1:0/3426933753 <== mon.1 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcf4007bd0 con 0x7efd08150a80 2024-05-09T08:06:13.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.904+0000 7efd0ec00640 1 -- 172.21.15.1:0/3426933753 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd0813f110 con 0x7efd08150a80 2024-05-09T08:06:13.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.904+0000 7efd0ec00640 1 -- 172.21.15.1:0/3426933753 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd0813f6d0 con 0x7efd08150a80 2024-05-09T08:06:13.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.906+0000 7efd0ec00640 1 --2- 172.21.15.1:0/3426933753 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efd0813f910 0x7efd081545a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:06:13.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.906+0000 7efd0ec00640 1 -- 172.21.15.1:0/3426933753 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7efd0813e110 con 0x7efd0813f910 2024-05-09T08:06:13.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.908+0000 7efd07400640 1 --2- 172.21.15.1:0/3426933753 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efd0813f910 0x7efd081545a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:06:13.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.908+0000 7efcffe00640 1 -- 172.21.15.1:0/3426933753 <== mon.1 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7efcf4018070 con 0x7efd08150a80 2024-05-09T08:06:13.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.909+0000 7efd07400640 1 -- 172.21.15.1:0/3426933753 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7efd08074d70 con 0x7efd0813f910 2024-05-09T08:06:13.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.911+0000 7efd07400640 1 --2- 172.21.15.1:0/3426933753 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efd0813f910 0x7efd081545a0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7efd080c33b0 tx=0x7efd00031ec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:06:13.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.911+0000 7efcffe00640 1 --2- 172.21.15.1:0/3426933753 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7efce806be30 0x7efce806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:06:13.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.913+0000 7efcffe00640 1 -- 172.21.15.1:0/3426933753 <== mon.1 v2:172.21.15.7:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7efcf408b460 con 0x7efd08150a80 2024-05-09T08:06:13.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.913+0000 7efcffe00640 1 -- 172.21.15.1:0/3426933753 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd00041020 con 0x7efd0813f910 2024-05-09T08:06:13.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.913+0000 7efcffe00640 1 -- 172.21.15.1:0/3426933753 <== mon.1 v2:172.21.15.7:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7efd00037040 con 0x7efd0813f910 2024-05-09T08:06:13.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.913+0000 7efcffe00640 1 -- 172.21.15.1:0/3426933753 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efd0813f910 msgr2=0x7efd081545a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:06:13.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.913+0000 7efcffe00640 1 --2- 172.21.15.1:0/3426933753 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efd0813f910 0x7efd081545a0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7efd080c33b0 tx=0x7efd00031ec0 comp rx=0 tx=0).stop 2024-05-09T08:06:13.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.913+0000 7efcffe00640 1 -- 172.21.15.1:0/3426933753 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efd0813f910 msgr2=0x7efd081545a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-05-09T08:06:13.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.913+0000 7efcffe00640 1 --2- 172.21.15.1:0/3426933753 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efd0813f910 0x7efd081545a0 secure :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7efd080c33b0 tx=0x7efd00031ec0 comp rx=0 tx=0).stop 2024-05-09T08:06:13.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.914+0000 7efd0cc00640 1 --2- 172.21.15.1:0/3426933753 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7efce806be30 0x7efce806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:06:13.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.915+0000 7efd0cc00640 1 --2- 172.21.15.1:0/3426933753 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7efce806be30 0x7efce806e2f0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7efd080c1fe0 tx=0x7efcf8006bd0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:06:13.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.924+0000 7efd0ec00640 1 --2- 172.21.15.1:0/3426933753 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efd08154ae0 0x7efd08154f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:06:13.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.924+0000 7efd0ec00640 1 -- 172.21.15.1:0/3426933753 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7efd08156380 con 0x7efd08154ae0 2024-05-09T08:06:13.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.924+0000 7efd07400640 1 --2- 172.21.15.1:0/3426933753 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efd08154ae0 0x7efd08154f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:06:13.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.925+0000 7efd07400640 1 -- 172.21.15.1:0/3426933753 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7efd080726c0 con 0x7efd08154ae0 2024-05-09T08:06:13.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.925+0000 7efd07400640 1 --2- 172.21.15.1:0/3426933753 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efd08154ae0 0x7efd08154f00 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7efd0002a7c0 tx=0x7efd0002f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:06:13.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.926+0000 7efcffe00640 1 -- 172.21.15.1:0/3426933753 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd00041030 con 0x7efd08154ae0 2024-05-09T08:06:13.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.927+0000 7efcffe00640 1 -- 172.21.15.1:0/3426933753 <== mon.1 v2:172.21.15.7:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5177 (secure 0 0 0) 0x7efd00009af0 con 0x7efd08154ae0 2024-05-09T08:06:13.940 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-05-09T08:06:13.940 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "b", 2024-05-09T08:06:13.940 INFO:teuthology.orchestra.run.smithi001.stdout: "rank": 1, 2024-05-09T08:06:13.940 INFO:teuthology.orchestra.run.smithi001.stdout: "state": "peon", 2024-05-09T08:06:13.940 INFO:teuthology.orchestra.run.smithi001.stdout: "election_epoch": 34, 2024-05-09T08:06:13.940 INFO:teuthology.orchestra.run.smithi001.stdout: "uptime": 141588, 2024-05-09T08:06:13.941 INFO:teuthology.orchestra.run.smithi001.stdout: "quorum": [ 2024-05-09T08:06:13.941 INFO:teuthology.orchestra.run.smithi001.stdout: 0, 2024-05-09T08:06:13.941 INFO:teuthology.orchestra.run.smithi001.stdout: 1 2024-05-09T08:06:13.941 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:06:13.941 INFO:teuthology.orchestra.run.smithi001.stdout: "quorum_age": 1, 2024-05-09T08:06:13.942 INFO:teuthology.orchestra.run.smithi001.stdout: "features": { 2024-05-09T08:06:13.942 INFO:teuthology.orchestra.run.smithi001.stdout: "required_con": "2450521708007866372", 2024-05-09T08:06:13.942 INFO:teuthology.orchestra.run.smithi001.stdout: "required_mon": [ 2024-05-09T08:06:13.942 INFO:teuthology.orchestra.run.smithi001.stdout: "kraken", 2024-05-09T08:06:13.942 INFO:teuthology.orchestra.run.smithi001.stdout: "luminous", 2024-05-09T08:06:13.942 INFO:teuthology.orchestra.run.smithi001.stdout: "mimic", 2024-05-09T08:06:13.942 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap-prune", 2024-05-09T08:06:13.942 INFO:teuthology.orchestra.run.smithi001.stdout: "nautilus", 2024-05-09T08:06:13.942 INFO:teuthology.orchestra.run.smithi001.stdout: "octopus", 2024-05-09T08:06:13.942 INFO:teuthology.orchestra.run.smithi001.stdout: "pacific", 2024-05-09T08:06:13.942 INFO:teuthology.orchestra.run.smithi001.stdout: "elector-pinging", 2024-05-09T08:06:13.943 INFO:teuthology.orchestra.run.smithi001.stdout: "quincy", 2024-05-09T08:06:13.943 INFO:teuthology.orchestra.run.smithi001.stdout: "reef", 2024-05-09T08:06:13.943 INFO:teuthology.orchestra.run.smithi001.stdout: "squid" 2024-05-09T08:06:13.943 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:06:13.943 INFO:teuthology.orchestra.run.smithi001.stdout: "quorum_con": "4540701547738038271", 2024-05-09T08:06:13.943 INFO:teuthology.orchestra.run.smithi001.stdout: "quorum_mon": [ 2024-05-09T08:06:13.943 INFO:teuthology.orchestra.run.smithi001.stdout: "kraken", 2024-05-09T08:06:13.943 INFO:teuthology.orchestra.run.smithi001.stdout: "luminous", 2024-05-09T08:06:13.943 INFO:teuthology.orchestra.run.smithi001.stdout: "mimic", 2024-05-09T08:06:13.943 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap-prune", 2024-05-09T08:06:13.943 INFO:teuthology.orchestra.run.smithi001.stdout: "nautilus", 2024-05-09T08:06:13.944 INFO:teuthology.orchestra.run.smithi001.stdout: "octopus", 2024-05-09T08:06:13.944 INFO:teuthology.orchestra.run.smithi001.stdout: "pacific", 2024-05-09T08:06:13.944 INFO:teuthology.orchestra.run.smithi001.stdout: "elector-pinging", 2024-05-09T08:06:13.944 INFO:teuthology.orchestra.run.smithi001.stdout: "quincy", 2024-05-09T08:06:13.944 INFO:teuthology.orchestra.run.smithi001.stdout: "reef", 2024-05-09T08:06:13.944 INFO:teuthology.orchestra.run.smithi001.stdout: "squid" 2024-05-09T08:06:13.944 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:06:13.944 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:06:13.944 INFO:teuthology.orchestra.run.smithi001.stdout: "outside_quorum": [], 2024-05-09T08:06:13.944 INFO:teuthology.orchestra.run.smithi001.stdout: "extra_probe_peers": [], 2024-05-09T08:06:13.944 INFO:teuthology.orchestra.run.smithi001.stdout: "sync_provider": [], 2024-05-09T08:06:13.945 INFO:teuthology.orchestra.run.smithi001.stdout: "monmap": { 2024-05-09T08:06:13.945 INFO:teuthology.orchestra.run.smithi001.stdout: "epoch": 1, 2024-05-09T08:06:13.945 INFO:teuthology.orchestra.run.smithi001.stdout: "fsid": "54329ec8-3270-4043-9a82-c6cd7420aa7f", 2024-05-09T08:06:13.945 INFO:teuthology.orchestra.run.smithi001.stdout: "modified": "2024-05-09T08:01:21.819270Z", 2024-05-09T08:06:13.945 INFO:teuthology.orchestra.run.smithi001.stdout: "created": "2024-05-09T08:01:21.819270Z", 2024-05-09T08:06:13.945 INFO:teuthology.orchestra.run.smithi001.stdout: "min_mon_release": 19, 2024-05-09T08:06:13.945 INFO:teuthology.orchestra.run.smithi001.stdout: "min_mon_release_name": "squid", 2024-05-09T08:06:13.945 INFO:teuthology.orchestra.run.smithi001.stdout: "election_strategy": 1, 2024-05-09T08:06:13.945 INFO:teuthology.orchestra.run.smithi001.stdout: "disallowed_leaders: ": "", 2024-05-09T08:06:13.945 INFO:teuthology.orchestra.run.smithi001.stdout: "stretch_mode": false, 2024-05-09T08:06:13.945 INFO:teuthology.orchestra.run.smithi001.stdout: "tiebreaker_mon": "", 2024-05-09T08:06:13.946 INFO:teuthology.orchestra.run.smithi001.stdout: "removed_ranks: ": "", 2024-05-09T08:06:13.946 INFO:teuthology.orchestra.run.smithi001.stdout: "features": { 2024-05-09T08:06:13.946 INFO:teuthology.orchestra.run.smithi001.stdout: "persistent": [ 2024-05-09T08:06:13.946 INFO:teuthology.orchestra.run.smithi001.stdout: "kraken", 2024-05-09T08:06:13.946 INFO:teuthology.orchestra.run.smithi001.stdout: "luminous", 2024-05-09T08:06:13.946 INFO:teuthology.orchestra.run.smithi001.stdout: "mimic", 2024-05-09T08:06:13.946 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap-prune", 2024-05-09T08:06:13.946 INFO:teuthology.orchestra.run.smithi001.stdout: "nautilus", 2024-05-09T08:06:13.946 INFO:teuthology.orchestra.run.smithi001.stdout: "octopus", 2024-05-09T08:06:13.946 INFO:teuthology.orchestra.run.smithi001.stdout: "pacific", 2024-05-09T08:06:13.946 INFO:teuthology.orchestra.run.smithi001.stdout: "elector-pinging", 2024-05-09T08:06:13.947 INFO:teuthology.orchestra.run.smithi001.stdout: "quincy", 2024-05-09T08:06:13.947 INFO:teuthology.orchestra.run.smithi001.stdout: "reef", 2024-05-09T08:06:13.947 INFO:teuthology.orchestra.run.smithi001.stdout: "squid" 2024-05-09T08:06:13.947 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:06:13.947 INFO:teuthology.orchestra.run.smithi001.stdout: "optional": [] 2024-05-09T08:06:13.947 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:06:13.947 INFO:teuthology.orchestra.run.smithi001.stdout: "mons": [ 2024-05-09T08:06:13.947 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:06:13.947 INFO:teuthology.orchestra.run.smithi001.stdout: "rank": 0, 2024-05-09T08:06:13.947 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "a", 2024-05-09T08:06:13.947 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-05-09T08:06:13.947 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-05-09T08:06:13.948 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:06:13.948 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-05-09T08:06:13.948 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:3300", 2024-05-09T08:06:13.948 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:06:13.948 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:06:13.948 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:06:13.948 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-05-09T08:06:13.948 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6789", 2024-05-09T08:06:13.948 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:06:13.948 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:06:13.948 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:06:13.948 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:06:13.948 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6789/0", 2024-05-09T08:06:13.949 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.1:6789/0", 2024-05-09T08:06:13.949 INFO:teuthology.orchestra.run.smithi001.stdout: "priority": 0, 2024-05-09T08:06:13.949 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 0, 2024-05-09T08:06:13.949 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_location": "{}" 2024-05-09T08:06:13.949 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:06:13.949 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:06:13.949 INFO:teuthology.orchestra.run.smithi001.stdout: "rank": 1, 2024-05-09T08:06:13.949 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "b", 2024-05-09T08:06:13.949 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-05-09T08:06:13.949 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-05-09T08:06:13.949 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:06:13.949 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-05-09T08:06:13.950 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:3300", 2024-05-09T08:06:13.950 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:06:13.950 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:06:13.950 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:06:13.950 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-05-09T08:06:13.950 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:6789", 2024-05-09T08:06:13.950 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:06:13.950 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:06:13.950 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:06:13.950 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:06:13.950 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:6789/0", 2024-05-09T08:06:13.950 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.7:6789/0", 2024-05-09T08:06:13.950 INFO:teuthology.orchestra.run.smithi001.stdout: "priority": 0, 2024-05-09T08:06:13.951 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 0, 2024-05-09T08:06:13.951 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_location": "{}" 2024-05-09T08:06:13.951 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:06:13.951 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:06:13.951 INFO:teuthology.orchestra.run.smithi001.stdout: "rank": 2, 2024-05-09T08:06:13.951 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "c", 2024-05-09T08:06:13.951 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-05-09T08:06:13.951 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-05-09T08:06:13.951 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:06:13.951 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-05-09T08:06:13.951 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:3301", 2024-05-09T08:06:13.951 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:06:13.952 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:06:13.952 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:06:13.952 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-05-09T08:06:13.952 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:6790", 2024-05-09T08:06:13.952 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:06:13.952 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:06:13.952 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:06:13.952 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:06:13.952 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:6790/0", 2024-05-09T08:06:13.952 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.7:6790/0", 2024-05-09T08:06:13.952 INFO:teuthology.orchestra.run.smithi001.stdout: "priority": 0, 2024-05-09T08:06:13.952 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 0, 2024-05-09T08:06:13.952 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_location": "{}" 2024-05-09T08:06:13.952 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:06:13.953 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:06:13.953 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:06:13.953 INFO:teuthology.orchestra.run.smithi001.stdout: "feature_map": { 2024-05-09T08:06:13.953 INFO:teuthology.orchestra.run.smithi001.stdout: "mon": [ 2024-05-09T08:06:13.953 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:06:13.953 INFO:teuthology.orchestra.run.smithi001.stdout: "features": "0x3f03cffffffdffff", 2024-05-09T08:06:13.953 INFO:teuthology.orchestra.run.smithi001.stdout: "release": "luminous", 2024-05-09T08:06:13.953 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 1 2024-05-09T08:06:13.953 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:06:13.953 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:06:13.953 INFO:teuthology.orchestra.run.smithi001.stdout: "mds": [ 2024-05-09T08:06:13.953 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:06:13.953 INFO:teuthology.orchestra.run.smithi001.stdout: "features": "0x3f03cffffffdffff", 2024-05-09T08:06:13.954 INFO:teuthology.orchestra.run.smithi001.stdout: "release": "luminous", 2024-05-09T08:06:13.954 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 6 2024-05-09T08:06:13.954 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:06:13.954 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:06:13.954 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": [ 2024-05-09T08:06:13.954 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:06:13.954 INFO:teuthology.orchestra.run.smithi001.stdout: "features": "0x3f03cffffffdffff", 2024-05-09T08:06:13.954 INFO:teuthology.orchestra.run.smithi001.stdout: "release": "luminous", 2024-05-09T08:06:13.954 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 7 2024-05-09T08:06:13.954 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:06:13.954 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:06:13.954 INFO:teuthology.orchestra.run.smithi001.stdout: "client": [ 2024-05-09T08:06:13.954 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:06:13.955 INFO:teuthology.orchestra.run.smithi001.stdout: "features": "0x2f018fb87aa4aafe", 2024-05-09T08:06:13.955 INFO:teuthology.orchestra.run.smithi001.stdout: "release": "luminous", 2024-05-09T08:06:13.955 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 1 2024-05-09T08:06:13.955 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:06:13.956 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:06:13.956 INFO:teuthology.orchestra.run.smithi001.stdout: "features": "0x3f03cffffffdffff", 2024-05-09T08:06:13.956 INFO:teuthology.orchestra.run.smithi001.stdout: "release": "luminous", 2024-05-09T08:06:13.956 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 6 2024-05-09T08:06:13.956 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:06:13.956 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:06:13.956 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr": [ 2024-05-09T08:06:13.956 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:06:13.956 INFO:teuthology.orchestra.run.smithi001.stdout: "features": "0x3f03cffffffdffff", 2024-05-09T08:06:13.956 INFO:teuthology.orchestra.run.smithi001.stdout: "release": "luminous", 2024-05-09T08:06:13.956 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 1 2024-05-09T08:06:13.957 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:06:13.957 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:06:13.957 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:06:13.957 INFO:teuthology.orchestra.run.smithi001.stdout: "stretch_mode": false 2024-05-09T08:06:13.957 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-05-09T08:06:13.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.927+0000 7efcffe00640 1 -- 172.21.15.1:0/3426933753 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efd08154ae0 msgr2=0x7efd08154f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:06:13.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.927+0000 7efcffe00640 1 --2- 172.21.15.1:0/3426933753 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efd08154ae0 0x7efd08154f00 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7efd0002a7c0 tx=0x7efd0002f000 comp rx=0 tx=0).stop 2024-05-09T08:06:13.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.927+0000 7efcffe00640 1 -- 172.21.15.1:0/3426933753 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efd08154ae0 msgr2=0x7efd08154f00 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-05-09T08:06:13.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.927+0000 7efcffe00640 1 --2- 172.21.15.1:0/3426933753 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efd08154ae0 0x7efd08154f00 secure :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7efd0002a7c0 tx=0x7efd0002f000 comp rx=0 tx=0).stop 2024-05-09T08:06:13.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.928+0000 7efd0ec00640 1 -- 172.21.15.1:0/3426933753 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7efce806be30 msgr2=0x7efce806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:06:13.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.928+0000 7efd0ec00640 1 --2- 172.21.15.1:0/3426933753 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7efce806be30 0x7efce806e2f0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7efd080c1fe0 tx=0x7efcf8006bd0 comp rx=0 tx=0).stop 2024-05-09T08:06:13.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.928+0000 7efd0ec00640 1 -- 172.21.15.1:0/3426933753 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efd08150a80 msgr2=0x7efd0814aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:06:13.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.928+0000 7efd0ec00640 1 --2- 172.21.15.1:0/3426933753 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efd08150a80 0x7efd0814aa70 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7efcf400c7f0 tx=0x7efcf400ccb0 comp rx=0 tx=0).stop 2024-05-09T08:06:13.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.928+0000 7efd0cc00640 1 -- 172.21.15.1:0/3426933753 reap_dead start 2024-05-09T08:06:13.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.928+0000 7efd0cc00640 1 -- 172.21.15.1:0/3426933753 reap_dead start 2024-05-09T08:06:13.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.928+0000 7efd0ec00640 1 -- 172.21.15.1:0/3426933753 shutdown_connections 2024-05-09T08:06:13.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.928+0000 7efd0ec00640 1 -- 172.21.15.1:0/3426933753 >> 172.21.15.1:0/3426933753 conn(0x7efd0806dad0 msgr2=0x7efd08141120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:06:13.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.928+0000 7efd0ec00640 1 -- 172.21.15.1:0/3426933753 shutdown_connections 2024-05-09T08:06:13.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:13.929+0000 7efd0ec00640 1 -- 172.21.15.1:0/3426933753 wait complete. 2024-05-09T08:06:13.985 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-05-09T08:06:13.985 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2024-05-09T08:06:21.249 DEBUG:teuthology.orchestra.run.smithi001:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T08:06:21.254 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T08:06:33.986 INFO:tasks.mon_thrash.mon_thrasher:killing mon.c 2024-05-09T08:06:33.987 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.c 2024-05-09T08:06:33.987 INFO:tasks.ceph.mon.c:Restarting daemon 2024-05-09T08:06:33.987 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-05-09T08:06:33.990 INFO:tasks.ceph.mon.c:Started 2024-05-09T08:06:33.990 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-05-09T08:06:33.990 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-05-09T08:06:34.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:34.320+0000 7f3bcb600640 1 Processor -- start 2024-05-09T08:06:34.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:34.321+0000 7f3bcb600640 1 -- start start 2024-05-09T08:06:34.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:34.323+0000 7f3bcb600640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f3bbc0d90c0 0x7f3bbc0dedd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:06:34.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:34.323+0000 7f3bcb600640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f3bbc0378a0 0x7f3bbc0505f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:06:34.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:34.323+0000 7f3bcb600640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3bbc0df4d0 con 0x7f3bbc0501b0 2024-05-09T08:06:34.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:34.323+0000 7f3bcb600640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f3bbc0380c0 con 0x7f3bbc0378a0 2024-05-09T08:06:34.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:34.323+0000 7f3bcb600640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f3bbc0df7b0 con 0x7f3bbc0d90c0 2024-05-09T08:06:34.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:34.323+0000 7f3bc8c00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f3bbc0d90c0 0x7f3bbc0dedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:06:34.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:34.323+0000 7f3bc8c00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f3bbc0d90c0 0x7f3bbc0dedd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.1:50644/0 (socket says 172.21.15.1:50644) 2024-05-09T08:06:34.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:34.323+0000 7f3bc8c00640 1 -- 172.21.15.1:0/4273027242 learned_addr learned my addr 172.21.15.1:0/4273027242 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:06:34.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:34.324+0000 7f3bc9600640 1 --2- 172.21.15.1:0/4273027242 >> v2:172.21.15.1:3300/0 conn(0x7f3bbc0378a0 0x7f3bbc0505f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:06:34.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:34.324+0000 7f3bc8c00640 1 -- 172.21.15.1:0/4273027242 >> v2:172.21.15.1:3300/0 conn(0x7f3bbc0378a0 msgr2=0x7f3bbc0505f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:06:34.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:34.324+0000 7f3bc8c00640 1 --2- 172.21.15.1:0/4273027242 >> v2:172.21.15.1:3300/0 conn(0x7f3bbc0378a0 0x7f3bbc0505f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:06:34.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:34.324+0000 7f3bc8c00640 1 -- 172.21.15.1:0/4273027242 >> v1:172.21.15.1:6789/0 conn(0x7f3bbc0501b0 legacy=0x7f3bbc037190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:06:34.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:34.324+0000 7f3bc8c00640 1 -- 172.21.15.1:0/4273027242 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bbc0dfa90 con 0x7f3bbc0d90c0 2024-05-09T08:06:34.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:34.326+0000 7f3bc8c00640 1 --2- 172.21.15.1:0/4273027242 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3bbc0d90c0 0x7f3bbc0dedd0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f3bbc011cb0 tx=0x7f3bb802a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=10f513edd00adfd8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:06:34.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:34.326+0000 7f3bc3400640 1 -- 172.21.15.1:0/4273027242 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bb8004030 con 0x7f3bbc0d90c0 2024-05-09T08:06:34.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:34.326+0000 7f3bc3400640 1 -- 172.21.15.1:0/4273027242 <== mon.1 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3bb80041d0 con 0x7f3bbc0d90c0 2024-05-09T08:06:34.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:34.326+0000 7f3bc3400640 1 -- 172.21.15.1:0/4273027242 <== mon.1 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bb8003610 con 0x7f3bbc0d90c0 2024-05-09T08:06:34.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:34.327+0000 7f3bcb600640 1 -- 172.21.15.1:0/4273027242 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3bbc0d90c0 msgr2=0x7f3bbc0dedd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:06:34.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:34.327+0000 7f3bcb600640 1 --2- 172.21.15.1:0/4273027242 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3bbc0d90c0 0x7f3bbc0dedd0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f3bbc011cb0 tx=0x7f3bb802a270 comp rx=0 tx=0).stop 2024-05-09T08:06:34.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:34.328+0000 7f3bcb600640 1 -- 172.21.15.1:0/4273027242 shutdown_connections 2024-05-09T08:06:34.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:34.328+0000 7f3bcb600640 1 --2- 172.21.15.1:0/4273027242 >> v2:172.21.15.1:3300/0 conn(0x7f3bbc0378a0 0x7f3bbc0505f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:06:34.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:34.328+0000 7f3bcb600640 1 --2- 172.21.15.1:0/4273027242 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3bbc0d90c0 0x7f3bbc0dedd0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:06:34.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:34.328+0000 7f3bcb600640 1 -- 172.21.15.1:0/4273027242 >> 172.21.15.1:0/4273027242 conn(0x7f3bbc054640 msgr2=0x7f3bbc054a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:06:34.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:34.328+0000 7f3bcb600640 1 -- 172.21.15.1:0/4273027242 shutdown_connections 2024-05-09T08:06:34.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:34.328+0000 7f3bcb600640 1 -- 172.21.15.1:0/4273027242 wait complete. 2024-05-09T08:06:34.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:34.331+0000 7f3bcb600640 1 Processor -- start 2024-05-09T08:06:34.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:34.331+0000 7f3bcb600640 1 -- start start 2024-05-09T08:06:34.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:34.333+0000 7f3bcb600640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3bbc0378a0 0x7f3bbc053250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:06:34.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:34.333+0000 7f3bcb600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3bbc0501b0 0x7f3bbc0c9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:06:34.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:34.333+0000 7f3bcb600640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f3bbc0d90c0 0x7f3bbc0ca230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:06:34.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:34.333+0000 7f3bc9600640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f3bbc0d90c0 0x7f3bbc0ca230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:06:34.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:34.333+0000 7f3bc9600640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f3bbc0d90c0 0x7f3bbc0ca230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3301/0 says I am v2:172.21.15.1:33702/0 (socket says 172.21.15.1:33702) 2024-05-09T08:06:34.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:34.333+0000 7f3bc9600640 1 -- 172.21.15.1:0/3592199520 learned_addr learned my addr 172.21.15.1:0/3592199520 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:06:34.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:34.333+0000 7f3bcb600640 1 -- 172.21.15.1:0/3592199520 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f3bbc038550 con 0x7f3bbc0378a0 2024-05-09T08:06:34.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:34.333+0000 7f3bcb600640 1 -- 172.21.15.1:0/3592199520 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f3bbc0d94a0 con 0x7f3bbc0501b0 2024-05-09T08:06:34.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:34.333+0000 7f3bcb600640 1 -- 172.21.15.1:0/3592199520 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f3bbc0386d0 con 0x7f3bbc0d90c0 2024-05-09T08:06:34.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:34.334+0000 7f3bc9600640 1 -- 172.21.15.1:0/3592199520 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f3bbc0d90c0 msgr2=0x7f3bbc0ca230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-05-09T08:06:34.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:34.334+0000 7f3bc9600640 1 -- 172.21.15.1:0/3592199520 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f3bbc0d90c0 msgr2=0x7f3bbc0ca230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-09T08:06:34.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:34.334+0000 7f3bc9600640 1 --2- 172.21.15.1:0/3592199520 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f3bbc0d90c0 0x7f3bbc0ca230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-09T08:06:34.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:34.334+0000 7f3bc9600640 1 --2- 172.21.15.1:0/3592199520 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f3bbc0d90c0 0x7f3bbc0ca230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T08:06:34.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:34.334+0000 7f3bc8c00640 1 --2- 172.21.15.1:0/3592199520 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3bbc0378a0 0x7f3bbc053250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:06:34.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:34.334+0000 7f3bc3e00640 1 --2- 172.21.15.1:0/3592199520 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3bbc0501b0 0x7f3bbc0c9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:06:34.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:34.335+0000 7f3bc3e00640 1 -- 172.21.15.1:0/3592199520 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f3bbc0d90c0 msgr2=0x7f3bbc0ca230 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T08:06:34.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:34.335+0000 7f3bc3e00640 1 --2- 172.21.15.1:0/3592199520 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f3bbc0d90c0 0x7f3bbc0ca230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:06:34.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:34.335+0000 7f3bc3e00640 1 -- 172.21.15.1:0/3592199520 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3bbc0378a0 msgr2=0x7f3bbc053250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:06:34.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:34.335+0000 7f3bc3e00640 1 --2- 172.21.15.1:0/3592199520 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3bbc0378a0 0x7f3bbc053250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:06:34.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:34.335+0000 7f3bc3e00640 1 -- 172.21.15.1:0/3592199520 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bbc0dcde0 con 0x7f3bbc0501b0 2024-05-09T08:06:34.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:34.336+0000 7f3bc3e00640 1 --2- 172.21.15.1:0/3592199520 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3bbc0501b0 0x7f3bbc0c9cf0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f3bb000c3f0 tx=0x7f3bb000c8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:06:34.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:34.337+0000 7f3bc0c00640 1 -- 172.21.15.1:0/3592199520 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bb0017070 con 0x7f3bbc0501b0 2024-05-09T08:06:34.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:34.337+0000 7f3bc0c00640 1 -- 172.21.15.1:0/3592199520 <== mon.1 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3bb00052d0 con 0x7f3bbc0501b0 2024-05-09T08:06:34.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:34.337+0000 7f3bc0c00640 1 -- 172.21.15.1:0/3592199520 <== mon.1 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bb0003830 con 0x7f3bbc0501b0 2024-05-09T08:06:34.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:34.337+0000 7f3bcb600640 1 -- 172.21.15.1:0/3592199520 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3bbc0dd0c0 con 0x7f3bbc0501b0 2024-05-09T08:06:34.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:34.337+0000 7f3bcb600640 1 -- 172.21.15.1:0/3592199520 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3bbc053e00 con 0x7f3bbc0501b0 2024-05-09T08:06:34.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:34.342+0000 7f3bcb600640 1 -- 172.21.15.1:0/3592199520 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3bbc039110 con 0x7f3bbc0501b0 2024-05-09T08:06:34.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:34.343+0000 7f3bc0c00640 1 -- 172.21.15.1:0/3592199520 <== mon.1 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3bb001a070 con 0x7f3bbc0501b0 2024-05-09T08:06:34.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:34.345+0000 7f3bc0c00640 1 --2- 172.21.15.1:0/3592199520 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f3ba406bf00 0x7f3ba406e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:06:34.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:34.345+0000 7f3bc8c00640 1 --2- 172.21.15.1:0/3592199520 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f3ba406bf00 0x7f3ba406e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:06:34.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:34.345+0000 7f3bc0c00640 1 -- 172.21.15.1:0/3592199520 <== mon.1 v2:172.21.15.7:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7f3bb008a5b0 con 0x7f3bbc0501b0 2024-05-09T08:06:34.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:34.346+0000 7f3bc8c00640 1 --2- 172.21.15.1:0/3592199520 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f3ba406bf00 0x7f3ba406e3c0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f3bb8033000 tx=0x7f3bb802e000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:06:34.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:34.349+0000 7f3bc0c00640 1 -- 172.21.15.1:0/3592199520 <== mon.1 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f3bbc039110 con 0x7f3bbc0501b0 2024-05-09T08:06:34.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:34.916+0000 7f3bcb600640 1 -- 172.21.15.1:0/3592199520 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f3bbc047be0 con 0x7f3bbc0501b0 2024-05-09T08:06:34.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:34.917+0000 7f3bc0c00640 1 -- 172.21.15.1:0/3592199520 <== mon.1 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1539 (secure 0 0 0) 0x7f3bbc047be0 con 0x7f3bbc0501b0 2024-05-09T08:06:34.929 INFO:teuthology.orchestra.run.smithi001.stdout:{"election_epoch":34,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":22,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"54329ec8-3270-4043-9a82-c6cd7420aa7f","modified":"2024-05-09T08:01:21.819270Z","created":"2024-05-09T08:01:21.819270Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:3300","nonce":0},{"type":"v1","addr":"172.21.15.1:6789","nonce":0}]},"addr":"172.21.15.1:6789/0","public_addr":"172.21.15.1:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:3300","nonce":0},{"type":"v1","addr":"172.21.15.7:6789","nonce":0}]},"addr":"172.21.15.7:6789/0","public_addr":"172.21.15.7:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:3301","nonce":0},{"type":"v1","addr":"172.21.15.7:6790","nonce":0}]},"addr":"172.21.15.7:6790/0","public_addr":"172.21.15.7:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-05-09T08:06:34.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:34.919+0000 7f3bcb600640 1 -- 172.21.15.1:0/3592199520 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f3ba406bf00 msgr2=0x7f3ba406e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:06:34.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:34.919+0000 7f3bcb600640 1 --2- 172.21.15.1:0/3592199520 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f3ba406bf00 0x7f3ba406e3c0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f3bb8033000 tx=0x7f3bb802e000 comp rx=0 tx=0).stop 2024-05-09T08:06:34.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:34.919+0000 7f3bcb600640 1 -- 172.21.15.1:0/3592199520 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3bbc0501b0 msgr2=0x7f3bbc0c9cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:06:34.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:34.919+0000 7f3bcb600640 1 --2- 172.21.15.1:0/3592199520 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3bbc0501b0 0x7f3bbc0c9cf0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f3bb000c3f0 tx=0x7f3bb000c8b0 comp rx=0 tx=0).stop 2024-05-09T08:06:34.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:34.920+0000 7f3bcb600640 1 -- 172.21.15.1:0/3592199520 shutdown_connections 2024-05-09T08:06:34.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:34.920+0000 7f3bcb600640 1 --2- 172.21.15.1:0/3592199520 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f3ba406bf00 0x7f3ba406e3c0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:06:34.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:34.920+0000 7f3bcb600640 1 --2- 172.21.15.1:0/3592199520 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3bbc0501b0 0x7f3bbc0c9cf0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:06:34.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:34.920+0000 7f3bcb600640 1 --2- 172.21.15.1:0/3592199520 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f3bbc0d90c0 0x7f3bbc0ca230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:06:34.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:34.920+0000 7f3bcb600640 1 --2- 172.21.15.1:0/3592199520 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3bbc0378a0 0x7f3bbc053250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:06:34.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:34.920+0000 7f3bcb600640 1 -- 172.21.15.1:0/3592199520 >> 172.21.15.1:0/3592199520 conn(0x7f3bbc054640 msgr2=0x7f3bbc038a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:06:34.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:34.920+0000 7f3bcb600640 1 -- 172.21.15.1:0/3592199520 shutdown_connections 2024-05-09T08:06:34.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:34.920+0000 7f3bcb600640 1 -- 172.21.15.1:0/3592199520 wait complete. 2024-05-09T08:06:37.975 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-05-09T08:06:38.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:38.208+0000 7fb476e00640 1 Processor -- start 2024-05-09T08:06:38.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:38.208+0000 7fb476e00640 1 -- start start 2024-05-09T08:06:38.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:38.210+0000 7fb476e00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fb47006dca0 0x7fb470072900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:06:38.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:38.210+0000 7fb476e00640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fb47006ece0 0x7fb470159010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:06:38.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:38.210+0000 7fb476e00640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb470073200 con 0x7fb4700c6ea0 2024-05-09T08:06:38.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:38.210+0000 7fb476e00640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7fb470072e40 con 0x7fb47006ece0 2024-05-09T08:06:38.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:38.210+0000 7fb476e00640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7fb4700737c0 con 0x7fb47006dca0 2024-05-09T08:06:38.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:38.210+0000 7fb46fe00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fb47006dca0 0x7fb470072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:06:38.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:38.211+0000 7fb46f400640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7fb4700c6ea0 0x7fb47006e080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:45208/0 (socket says 172.21.15.1:45208) 2024-05-09T08:06:38.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:38.211+0000 7fb46f400640 1 -- 172.21.15.1:0/2492337350 learned_addr learned my addr 172.21.15.1:0/2492337350 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T08:06:38.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:38.211+0000 7fb474e00640 1 --2- 172.21.15.1:0/2492337350 >> v2:172.21.15.1:3300/0 conn(0x7fb47006ece0 0x7fb470159010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:06:38.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:38.211+0000 7fb46fe00640 1 -- 172.21.15.1:0/2492337350 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fb47006dca0 msgr2=0x7fb470072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-09T08:06:38.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:38.211+0000 7fb46fe00640 1 -- 172.21.15.1:0/2492337350 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fb47006dca0 msgr2=0x7fb470072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-09T08:06:38.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:38.211+0000 7fb46fe00640 1 --2- 172.21.15.1:0/2492337350 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fb47006dca0 0x7fb470072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-09T08:06:38.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:38.211+0000 7fb46fe00640 1 --2- 172.21.15.1:0/2492337350 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fb47006dca0 0x7fb470072900 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T08:06:38.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:38.211+0000 7fb474e00640 1 -- 172.21.15.1:0/2492337350 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fb47006dca0 msgr2=0x7fb470072900 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T08:06:38.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:38.211+0000 7fb474e00640 1 --2- 172.21.15.1:0/2492337350 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fb47006dca0 0x7fb470072900 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:06:38.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:38.211+0000 7fb474e00640 1 -- 172.21.15.1:0/2492337350 >> v1:172.21.15.1:6789/0 conn(0x7fb4700c6ea0 legacy=0x7fb47006e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:06:38.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:38.211+0000 7fb474e00640 1 -- 172.21.15.1:0/2492337350 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4700734e0 con 0x7fb47006ece0 2024-05-09T08:06:38.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:38.211+0000 7fb46ea00640 1 -- 172.21.15.1:0/2492337350 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1583954361 0 0) 0x7fb470073200 con 0x7fb4700c6ea0 2024-05-09T08:06:38.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:38.212+0000 7fb474e00640 1 --2- 172.21.15.1:0/2492337350 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb47006ece0 0x7fb470159010 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fb4680097a0 tx=0x7fb46802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7a19dffa8952a16 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:06:38.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:38.212+0000 7fb46ea00640 1 -- 172.21.15.1:0/2492337350 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb468037070 con 0x7fb47006ece0 2024-05-09T08:06:38.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:38.213+0000 7fb46ea00640 1 -- 172.21.15.1:0/2492337350 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb468032440 con 0x7fb47006ece0 2024-05-09T08:06:38.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:38.213+0000 7fb46ea00640 1 -- 172.21.15.1:0/2492337350 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb468032810 con 0x7fb47006ece0 2024-05-09T08:06:38.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:38.213+0000 7fb476e00640 1 -- 172.21.15.1:0/2492337350 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb47006ece0 msgr2=0x7fb470159010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:06:38.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:38.213+0000 7fb476e00640 1 --2- 172.21.15.1:0/2492337350 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb47006ece0 0x7fb470159010 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fb4680097a0 tx=0x7fb46802a210 comp rx=0 tx=0).stop 2024-05-09T08:06:38.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:38.213+0000 7fb476e00640 1 -- 172.21.15.1:0/2492337350 shutdown_connections 2024-05-09T08:06:38.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:38.213+0000 7fb476e00640 1 --2- 172.21.15.1:0/2492337350 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb47006ece0 0x7fb470159010 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:06:38.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:38.213+0000 7fb476e00640 1 --2- 172.21.15.1:0/2492337350 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fb47006dca0 0x7fb470072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:06:38.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:38.213+0000 7fb476e00640 1 -- 172.21.15.1:0/2492337350 >> 172.21.15.1:0/2492337350 conn(0x7fb4700cc070 msgr2=0x7fb4700cc480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:06:38.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:38.213+0000 7fb476e00640 1 -- 172.21.15.1:0/2492337350 shutdown_connections 2024-05-09T08:06:38.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:38.213+0000 7fb476e00640 1 -- 172.21.15.1:0/2492337350 wait complete. 2024-05-09T08:06:38.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:38.215+0000 7fb476e00640 1 Processor -- start 2024-05-09T08:06:38.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:38.215+0000 7fb476e00640 1 -- start start 2024-05-09T08:06:38.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:38.216+0000 7fb476e00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fb47006dca0 0x7fb4700c4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:06:38.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:38.216+0000 7fb476e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb47006ece0 0x7fb470158170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:06:38.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:38.216+0000 7fb476e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb4700c6ea0 0x7fb470153f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:06:38.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:38.216+0000 7fb476e00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fb470072470 con 0x7fb47006ece0 2024-05-09T08:06:38.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:38.216+0000 7fb476e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb470159550 con 0x7fb4700c6ea0 2024-05-09T08:06:38.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:38.216+0000 7fb476e00640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7fb470072100 con 0x7fb47006dca0 2024-05-09T08:06:38.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:38.216+0000 7fb46fe00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fb47006dca0 0x7fb4700c4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:06:38.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:38.216+0000 7fb46f400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb47006ece0 0x7fb470158170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:06:38.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:38.216+0000 7fb46fe00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fb47006dca0 0x7fb4700c4610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3301/0 says I am v2:172.21.15.1:33716/0 (socket says 172.21.15.1:33716) 2024-05-09T08:06:38.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:38.216+0000 7fb46fe00640 1 -- 172.21.15.1:0/429336464 learned_addr learned my addr 172.21.15.1:0/429336464 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:06:38.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:38.216+0000 7fb474e00640 1 --2- 172.21.15.1:0/429336464 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb4700c6ea0 0x7fb470153f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:06:38.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:38.217+0000 7fb46fe00640 1 -- 172.21.15.1:0/429336464 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fb47006dca0 msgr2=0x7fb4700c4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-05-09T08:06:38.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:38.217+0000 7fb46fe00640 1 -- 172.21.15.1:0/429336464 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fb47006dca0 msgr2=0x7fb4700c4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-09T08:06:38.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:38.217+0000 7fb46fe00640 1 --2- 172.21.15.1:0/429336464 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fb47006dca0 0x7fb4700c4610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-09T08:06:38.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:38.217+0000 7fb46fe00640 1 --2- 172.21.15.1:0/429336464 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fb47006dca0 0x7fb4700c4610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T08:06:38.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:38.217+0000 7fb474e00640 1 -- 172.21.15.1:0/429336464 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fb47006dca0 msgr2=0x7fb4700c4610 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T08:06:38.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:38.217+0000 7fb474e00640 1 --2- 172.21.15.1:0/429336464 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fb47006dca0 0x7fb4700c4610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:06:38.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:38.217+0000 7fb474e00640 1 -- 172.21.15.1:0/429336464 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb47006ece0 msgr2=0x7fb470158170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:06:38.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:38.217+0000 7fb474e00640 1 --2- 172.21.15.1:0/429336464 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb47006ece0 0x7fb470158170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:06:38.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:38.217+0000 7fb474e00640 1 -- 172.21.15.1:0/429336464 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb470155980 con 0x7fb4700c6ea0 2024-05-09T08:06:38.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:38.217+0000 7fb46f400640 1 --2- 172.21.15.1:0/429336464 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb47006ece0 0x7fb470158170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:06:38.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:38.217+0000 7fb474e00640 1 --2- 172.21.15.1:0/429336464 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb4700c6ea0 0x7fb470153f10 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fb468031000 tx=0x7fb4680046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:06:38.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:38.217+0000 7fb467e00640 1 -- 172.21.15.1:0/429336464 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb468032e60 con 0x7fb4700c6ea0 2024-05-09T08:06:38.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:38.217+0000 7fb467e00640 1 -- 172.21.15.1:0/429336464 <== mon.1 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4680306f0 con 0x7fb4700c6ea0 2024-05-09T08:06:38.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:38.217+0000 7fb476e00640 1 -- 172.21.15.1:0/429336464 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb470155c60 con 0x7fb4700c6ea0 2024-05-09T08:06:38.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:38.218+0000 7fb467e00640 1 -- 172.21.15.1:0/429336464 <== mon.1 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb468037040 con 0x7fb4700c6ea0 2024-05-09T08:06:38.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:38.218+0000 7fb476e00640 1 -- 172.21.15.1:0/429336464 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4700c6390 con 0x7fb4700c6ea0 2024-05-09T08:06:38.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:38.219+0000 7fb476e00640 1 -- 172.21.15.1:0/429336464 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4701472d0 con 0x7fb4700c6ea0 2024-05-09T08:06:38.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:38.220+0000 7fb467e00640 1 -- 172.21.15.1:0/429336464 <== mon.1 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb468044020 con 0x7fb4700c6ea0 2024-05-09T08:06:38.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:38.221+0000 7fb467e00640 1 --2- 172.21.15.1:0/429336464 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fb44c06be60 0x7fb44c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:06:38.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:38.221+0000 7fb467e00640 1 -- 172.21.15.1:0/429336464 <== mon.1 v2:172.21.15.7:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7fb4680ab510 con 0x7fb4700c6ea0 2024-05-09T08:06:38.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:38.222+0000 7fb46fe00640 1 --2- 172.21.15.1:0/429336464 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fb44c06be60 0x7fb44c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:06:38.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:38.223+0000 7fb46fe00640 1 --2- 172.21.15.1:0/429336464 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fb44c06be60 0x7fb44c06e320 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fb4600094c0 tx=0x7fb46000b040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:06:38.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:38.229+0000 7fb467e00640 1 -- 172.21.15.1:0/429336464 <== mon.1 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fb468079c40 con 0x7fb4700c6ea0 2024-05-09T08:06:38.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:38.606+0000 7fb476e00640 1 -- 172.21.15.1:0/429336464 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fb470074d70 con 0x7fb4700c6ea0 2024-05-09T08:06:38.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:38.607+0000 7fb467e00640 1 -- 172.21.15.1:0/429336464 <== mon.1 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1539 (secure 0 0 0) 0x7fb468079540 con 0x7fb4700c6ea0 2024-05-09T08:06:38.619 INFO:teuthology.orchestra.run.smithi001.stdout:{"election_epoch":34,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":25,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"54329ec8-3270-4043-9a82-c6cd7420aa7f","modified":"2024-05-09T08:01:21.819270Z","created":"2024-05-09T08:01:21.819270Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:3300","nonce":0},{"type":"v1","addr":"172.21.15.1:6789","nonce":0}]},"addr":"172.21.15.1:6789/0","public_addr":"172.21.15.1:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:3300","nonce":0},{"type":"v1","addr":"172.21.15.7:6789","nonce":0}]},"addr":"172.21.15.7:6789/0","public_addr":"172.21.15.7:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:3301","nonce":0},{"type":"v1","addr":"172.21.15.7:6790","nonce":0}]},"addr":"172.21.15.7:6790/0","public_addr":"172.21.15.7:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-05-09T08:06:38.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:38.611+0000 7fb476e00640 1 -- 172.21.15.1:0/429336464 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fb44c06be60 msgr2=0x7fb44c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:06:38.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:38.612+0000 7fb476e00640 1 --2- 172.21.15.1:0/429336464 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fb44c06be60 0x7fb44c06e320 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fb4600094c0 tx=0x7fb46000b040 comp rx=0 tx=0).stop 2024-05-09T08:06:38.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:38.612+0000 7fb476e00640 1 -- 172.21.15.1:0/429336464 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb4700c6ea0 msgr2=0x7fb470153f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:06:38.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:38.612+0000 7fb476e00640 1 --2- 172.21.15.1:0/429336464 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb4700c6ea0 0x7fb470153f10 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fb468031000 tx=0x7fb4680046c0 comp rx=0 tx=0).stop 2024-05-09T08:06:38.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:38.612+0000 7fb476e00640 1 -- 172.21.15.1:0/429336464 shutdown_connections 2024-05-09T08:06:38.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:38.612+0000 7fb476e00640 1 --2- 172.21.15.1:0/429336464 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fb44c06be60 0x7fb44c06e320 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:06:38.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:38.612+0000 7fb476e00640 1 --2- 172.21.15.1:0/429336464 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb4700c6ea0 0x7fb470153f10 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:06:38.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:38.612+0000 7fb476e00640 1 --2- 172.21.15.1:0/429336464 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb47006ece0 0x7fb470158170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:06:38.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:38.612+0000 7fb476e00640 1 --2- 172.21.15.1:0/429336464 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fb47006dca0 0x7fb4700c4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:06:38.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:38.612+0000 7fb476e00640 1 -- 172.21.15.1:0/429336464 >> 172.21.15.1:0/429336464 conn(0x7fb4700cc070 msgr2=0x7fb470150bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:06:38.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:38.613+0000 7fb476e00640 1 -- 172.21.15.1:0/429336464 shutdown_connections 2024-05-09T08:06:38.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:38.613+0000 7fb476e00640 1 -- 172.21.15.1:0/429336464 wait complete. 2024-05-09T08:06:41.647 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-05-09T08:06:41.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:41.982+0000 7f7443000640 1 Processor -- start 2024-05-09T08:06:41.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:41.982+0000 7f7443000640 1 -- start start 2024-05-09T08:06:41.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:41.984+0000 7f7443000640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f743c15f0a0 0x7f743c06b250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:06:41.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:41.984+0000 7f7443000640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f743c15f4a0 0x7f743c06b790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:06:41.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:41.984+0000 7f7443000640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f743c06bfb0 con 0x7f743c158c40 2024-05-09T08:06:41.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:41.985+0000 7f7443000640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f743c0c7830 con 0x7f743c15f0a0 2024-05-09T08:06:41.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:41.985+0000 7f7443000640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f743c0c7b10 con 0x7f743c15f4a0 2024-05-09T08:06:41.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:41.986+0000 7f7442600640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f743c15f4a0 0x7f743c06b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:06:41.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:41.987+0000 7f7442600640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f743c15f4a0 0x7f743c06b790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3301/0 says I am v2:172.21.15.1:33724/0 (socket says 172.21.15.1:33724) 2024-05-09T08:06:41.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:41.987+0000 7f7442600640 1 -- 172.21.15.1:0/1697010290 learned_addr learned my addr 172.21.15.1:0/1697010290 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:06:41.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:41.987+0000 7f7442600640 1 -- 172.21.15.1:0/1697010290 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f743c15f4a0 msgr2=0x7f743c06b790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-05-09T08:06:41.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:41.987+0000 7f7442600640 1 -- 172.21.15.1:0/1697010290 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f743c15f4a0 msgr2=0x7f743c06b790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-09T08:06:41.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:41.987+0000 7f7442600640 1 --2- 172.21.15.1:0/1697010290 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f743c15f4a0 0x7f743c06b790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-09T08:06:41.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:41.987+0000 7f7442600640 1 --2- 172.21.15.1:0/1697010290 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f743c15f4a0 0x7f743c06b790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T08:06:42.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:41.988+0000 7f7441200640 1 --2- 172.21.15.1:0/1697010290 >> v2:172.21.15.1:3300/0 conn(0x7f743c15f0a0 0x7f743c06b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:06:42.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:41.989+0000 7f7441200640 1 -- 172.21.15.1:0/1697010290 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f743c15f4a0 msgr2=0x7f743c06b790 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T08:06:42.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:41.989+0000 7f7441200640 1 --2- 172.21.15.1:0/1697010290 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f743c15f4a0 0x7f743c06b790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:06:42.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:41.989+0000 7f7441200640 1 -- 172.21.15.1:0/1697010290 >> v1:172.21.15.1:6789/0 conn(0x7f743c158c40 legacy=0x7f743c162850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:06:42.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:41.989+0000 7f7441200640 1 -- 172.21.15.1:0/1697010290 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f743c0c7df0 con 0x7f743c15f0a0 2024-05-09T08:06:42.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:41.992+0000 7f7441200640 1 --2- 172.21.15.1:0/1697010290 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f743c15f0a0 0x7f743c06b250 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f74280097a0 tx=0x7f742802a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b57914ad6aa1070 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:06:42.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:41.992+0000 7f743be00640 1 -- 172.21.15.1:0/1697010290 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7428037070 con 0x7f743c15f0a0 2024-05-09T08:06:42.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:41.992+0000 7f743be00640 1 -- 172.21.15.1:0/1697010290 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7428032440 con 0x7f743c15f0a0 2024-05-09T08:06:42.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:41.993+0000 7f743be00640 1 -- 172.21.15.1:0/1697010290 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7428037070 con 0x7f743c15f0a0 2024-05-09T08:06:42.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:41.994+0000 7f7443000640 1 -- 172.21.15.1:0/1697010290 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f743c15f0a0 msgr2=0x7f743c06b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:06:42.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:41.994+0000 7f7443000640 1 --2- 172.21.15.1:0/1697010290 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f743c15f0a0 0x7f743c06b250 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f74280097a0 tx=0x7f742802a250 comp rx=0 tx=0).stop 2024-05-09T08:06:42.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:41.995+0000 7f7443000640 1 -- 172.21.15.1:0/1697010290 shutdown_connections 2024-05-09T08:06:42.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:41.995+0000 7f7443000640 1 --2- 172.21.15.1:0/1697010290 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f743c15f4a0 0x7f743c06b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:06:42.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:41.995+0000 7f7443000640 1 --2- 172.21.15.1:0/1697010290 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f743c15f0a0 0x7f743c06b250 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:06:42.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:41.995+0000 7f7443000640 1 -- 172.21.15.1:0/1697010290 >> 172.21.15.1:0/1697010290 conn(0x7f743c0cbc10 msgr2=0x7f743c0cc020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:06:42.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:41.995+0000 7f7443000640 1 -- 172.21.15.1:0/1697010290 shutdown_connections 2024-05-09T08:06:42.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:41.995+0000 7f7443000640 1 -- 172.21.15.1:0/1697010290 wait complete. 2024-05-09T08:06:42.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:41.998+0000 7f7443000640 1 Processor -- start 2024-05-09T08:06:42.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:41.998+0000 7f7443000640 1 -- start start 2024-05-09T08:06:42.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:41.999+0000 7f7443000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f743c158c40 0x7f743c0c9d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:06:42.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:41.999+0000 7f7443000640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f743c15f0a0 0x7f743c148c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:06:42.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:41.999+0000 7f7443000640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f743c15f4a0 0x7f743c149150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:06:42.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:41.999+0000 7f7443000640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f743c15e850 con 0x7f743c15f4a0 2024-05-09T08:06:42.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:41.999+0000 7f7443000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f743c15eb80 con 0x7f743c158c40 2024-05-09T08:06:42.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:41.999+0000 7f7443000640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f743c0718a0 con 0x7f743c15f0a0 2024-05-09T08:06:42.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:42.000+0000 7f7442600640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f743c15f4a0 0x7f743c149150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:06:42.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:42.000+0000 7f7442600640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f743c15f4a0 0x7f743c149150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:47324/0 (socket says 172.21.15.1:47324) 2024-05-09T08:06:42.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:42.000+0000 7f7442600640 1 -- 172.21.15.1:0/3456711967 learned_addr learned my addr 172.21.15.1:0/3456711967 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:06:42.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:42.000+0000 7f7441c00640 1 --2- 172.21.15.1:0/3456711967 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f743c158c40 0x7f743c0c9d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:06:42.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:42.000+0000 7f7441200640 1 --2- 172.21.15.1:0/3456711967 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f743c15f0a0 0x7f743c148c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:06:42.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:42.001+0000 7f7441200640 1 -- 172.21.15.1:0/3456711967 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f743c15f0a0 msgr2=0x7f743c148c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-05-09T08:06:42.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:42.001+0000 7f7441200640 1 -- 172.21.15.1:0/3456711967 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f743c15f0a0 msgr2=0x7f743c148c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-09T08:06:42.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:42.001+0000 7f7441200640 1 --2- 172.21.15.1:0/3456711967 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f743c15f0a0 0x7f743c148c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-09T08:06:42.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:42.001+0000 7f7441200640 1 --2- 172.21.15.1:0/3456711967 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f743c15f0a0 0x7f743c148c10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T08:06:42.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:42.001+0000 7f7442600640 1 -- 172.21.15.1:0/3456711967 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f743c15f0a0 msgr2=0x7f743c148c10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T08:06:42.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:42.001+0000 7f7442600640 1 --2- 172.21.15.1:0/3456711967 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f743c15f0a0 0x7f743c148c10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:06:42.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:42.001+0000 7f7442600640 1 -- 172.21.15.1:0/3456711967 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f743c158c40 msgr2=0x7f743c0c9d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:06:42.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:42.001+0000 7f7442600640 1 --2- 172.21.15.1:0/3456711967 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f743c158c40 0x7f743c0c9d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:06:42.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:42.001+0000 7f7442600640 1 -- 172.21.15.1:0/3456711967 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f743c14e5b0 con 0x7f743c15f4a0 2024-05-09T08:06:42.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:42.002+0000 7f7442600640 1 --2- 172.21.15.1:0/3456711967 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f743c15f4a0 0x7f743c149150 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f7434007b00 tx=0x7f743400d0d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:06:42.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:42.002+0000 7f7439600640 1 -- 172.21.15.1:0/3456711967 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f743401a070 con 0x7f743c15f4a0 2024-05-09T08:06:42.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:42.002+0000 7f7439600640 1 -- 172.21.15.1:0/3456711967 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f743400dcb0 con 0x7f743c15f4a0 2024-05-09T08:06:42.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:42.002+0000 7f7439600640 1 -- 172.21.15.1:0/3456711967 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f743400de60 con 0x7f743c15f4a0 2024-05-09T08:06:42.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:42.003+0000 7f7443000640 1 -- 172.21.15.1:0/3456711967 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f743c14e890 con 0x7f743c15f4a0 2024-05-09T08:06:42.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:42.004+0000 7f7443000640 1 -- 172.21.15.1:0/3456711967 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f743c156790 con 0x7f743c15f4a0 2024-05-09T08:06:42.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:42.006+0000 7f7443000640 1 -- 172.21.15.1:0/3456711967 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f743c04f660 con 0x7f743c15f4a0 2024-05-09T08:06:42.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:42.007+0000 7f7439600640 1 -- 172.21.15.1:0/3456711967 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f743401d070 con 0x7f743c15f4a0 2024-05-09T08:06:42.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:42.009+0000 7f7439600640 1 --2- 172.21.15.1:0/3456711967 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f741c06be30 0x7f741c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:06:42.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:42.009+0000 7f7439600640 1 -- 172.21.15.1:0/3456711967 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7f743408b150 con 0x7f743c15f4a0 2024-05-09T08:06:42.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:42.010+0000 7f7441c00640 1 --2- 172.21.15.1:0/3456711967 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f741c06be30 0x7f741c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:06:42.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:42.011+0000 7f7441c00640 1 --2- 172.21.15.1:0/3456711967 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f741c06be30 0x7f741c06e2f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f7430000f60 tx=0x7f7430009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:06:42.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:42.016+0000 7f7439600640 1 -- 172.21.15.1:0/3456711967 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f7434010070 con 0x7f743c15f4a0 2024-05-09T08:06:42.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:42.415+0000 7f7443000640 1 -- 172.21.15.1:0/3456711967 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f743c0c36d0 con 0x7f743c15f4a0 2024-05-09T08:06:42.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:42.416+0000 7f7439600640 1 -- 172.21.15.1:0/3456711967 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1544 (secure 0 0 0) 0x7f7434059020 con 0x7f743c15f4a0 2024-05-09T08:06:42.433 INFO:teuthology.orchestra.run.smithi001.stdout:{"election_epoch":36,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":3,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"54329ec8-3270-4043-9a82-c6cd7420aa7f","modified":"2024-05-09T08:01:21.819270Z","created":"2024-05-09T08:01:21.819270Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:3300","nonce":0},{"type":"v1","addr":"172.21.15.1:6789","nonce":0}]},"addr":"172.21.15.1:6789/0","public_addr":"172.21.15.1:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:3300","nonce":0},{"type":"v1","addr":"172.21.15.7:6789","nonce":0}]},"addr":"172.21.15.7:6789/0","public_addr":"172.21.15.7:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:3301","nonce":0},{"type":"v1","addr":"172.21.15.7:6790","nonce":0}]},"addr":"172.21.15.7:6790/0","public_addr":"172.21.15.7:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-05-09T08:06:42.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:42.424+0000 7f742ea00640 1 -- 172.21.15.1:0/3456711967 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f741c06be30 msgr2=0x7f741c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:06:42.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:42.424+0000 7f742ea00640 1 --2- 172.21.15.1:0/3456711967 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f741c06be30 0x7f741c06e2f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f7430000f60 tx=0x7f7430009040 comp rx=0 tx=0).stop 2024-05-09T08:06:42.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:42.424+0000 7f742ea00640 1 -- 172.21.15.1:0/3456711967 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f743c15f4a0 msgr2=0x7f743c149150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:06:42.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:42.424+0000 7f742ea00640 1 --2- 172.21.15.1:0/3456711967 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f743c15f4a0 0x7f743c149150 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f7434007b00 tx=0x7f743400d0d0 comp rx=0 tx=0).stop 2024-05-09T08:06:42.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:42.424+0000 7f742ea00640 1 -- 172.21.15.1:0/3456711967 shutdown_connections 2024-05-09T08:06:42.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:42.424+0000 7f742ea00640 1 --2- 172.21.15.1:0/3456711967 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f741c06be30 0x7f741c06e2f0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:06:42.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:42.424+0000 7f742ea00640 1 --2- 172.21.15.1:0/3456711967 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f743c15f4a0 0x7f743c149150 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:06:42.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:42.424+0000 7f742ea00640 1 --2- 172.21.15.1:0/3456711967 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f743c15f0a0 0x7f743c148c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:06:42.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:42.424+0000 7f742ea00640 1 --2- 172.21.15.1:0/3456711967 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f743c158c40 0x7f743c0c9d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:06:42.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:42.424+0000 7f742ea00640 1 -- 172.21.15.1:0/3456711967 >> 172.21.15.1:0/3456711967 conn(0x7f743c0cbc10 msgr2=0x7f743c0cb380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:06:42.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:42.425+0000 7f742ea00640 1 -- 172.21.15.1:0/3456711967 shutdown_connections 2024-05-09T08:06:42.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:42.425+0000 7f742ea00640 1 -- 172.21.15.1:0/3456711967 wait complete. 2024-05-09T08:06:42.464 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-05-09T08:06:42.464 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-05-09T08:06:42.464 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-05-09T08:06:42.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:42.799+0000 7f698c200640 1 Processor -- start 2024-05-09T08:06:42.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:42.799+0000 7f698c200640 1 -- start start 2024-05-09T08:06:42.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:42.800+0000 7f698c200640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f6984159b40 0x7f6984156d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:06:42.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:42.800+0000 7f698c200640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f69841565a0 con 0x7f69841507b0 2024-05-09T08:06:42.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:42.801+0000 7f698c200640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6984073520 con 0x7f69841572c0 2024-05-09T08:06:42.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:42.801+0000 7f698c200640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f69841568f0 con 0x7f6984159b40 2024-05-09T08:06:42.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:42.801+0000 7f698a200640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f69841572c0 0x7f698406fc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.1:42774/0 (socket says 172.21.15.1:42774) 2024-05-09T08:06:42.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:42.801+0000 7f698a200640 1 -- 172.21.15.1:0/3970614249 learned_addr learned my addr 172.21.15.1:0/3970614249 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T08:06:42.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:42.801+0000 7f6983e00640 1 -- 172.21.15.1:0/3970614249 <== mon.1 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3297577948 0 0) 0x7f6984073520 con 0x7f69841572c0 2024-05-09T08:06:42.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:42.801+0000 7f6983e00640 1 -- 172.21.15.1:0/3970614249 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f696c003520 con 0x7f69841572c0 2024-05-09T08:06:42.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:42.802+0000 7f6983e00640 1 -- 172.21.15.1:0/3970614249 <== mon.1 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 276659754 0 0) 0x7f696c003520 con 0x7f69841572c0 2024-05-09T08:06:42.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:42.802+0000 7f6983e00640 1 -- 172.21.15.1:0/3970614249 >> v2:172.21.15.1:3300/0 conn(0x7f6984159b40 msgr2=0x7f6984156d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:06:42.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:42.802+0000 7f6988e00640 1 --2- 172.21.15.1:0/3970614249 >> v2:172.21.15.1:3300/0 conn(0x7f6984159b40 0x7f6984156d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:06:42.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:42.802+0000 7f6983e00640 1 --2- 172.21.15.1:0/3970614249 >> v2:172.21.15.1:3300/0 conn(0x7f6984159b40 0x7f6984156d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:06:42.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:42.802+0000 7f6983e00640 1 -- 172.21.15.1:0/3970614249 >> v1:172.21.15.1:6789/0 conn(0x7f69841507b0 legacy=0x7f69840728e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:06:42.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:42.802+0000 7f6983e00640 1 -- 172.21.15.1:0/3970614249 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69840c2080 con 0x7f69841572c0 2024-05-09T08:06:42.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:42.802+0000 7f6983e00640 1 -- 172.21.15.1:0/3970614249 <== mon.1 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4179846320 0 0) 0x7f6970001670 con 0x7f69841572c0 2024-05-09T08:06:42.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:42.803+0000 7f6988e00640 1 --2- 172.21.15.1:0/3970614249 >> v2:172.21.15.1:3300/0 conn(0x7f6984159b40 0x7f6984156d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T08:06:42.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:42.803+0000 7f6983e00640 1 -- 172.21.15.1:0/3970614249 >> v1:172.21.15.7:6789/0 conn(0x7f69841572c0 legacy=0x7f698406fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:06:42.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:42.803+0000 7f6983e00640 1 --2- 172.21.15.1:0/3970614249 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f696c0039b0 0x7f696c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:06:42.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:42.803+0000 7f6983e00640 1 --2- 172.21.15.1:0/3970614249 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f696c004800 0x7f696c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:06:42.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:42.803+0000 7f6983e00640 1 --2- 172.21.15.1:0/3970614249 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f696c0052e0 0x7f696c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:06:42.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:42.803+0000 7f6983e00640 1 -- 172.21.15.1:0/3970614249 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f69841568f0 con 0x7f696c004800 2024-05-09T08:06:42.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:42.803+0000 7f6983e00640 1 -- 172.21.15.1:0/3970614249 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f696c00c1b0 con 0x7f696c0039b0 2024-05-09T08:06:42.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:42.803+0000 7f6983e00640 1 -- 172.21.15.1:0/3970614249 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f696c00c3b0 con 0x7f696c0052e0 2024-05-09T08:06:42.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:42.803+0000 7f698a200640 1 --2- 172.21.15.1:0/3970614249 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f696c0052e0 0x7f696c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:06:42.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:42.804+0000 7f698a200640 1 -- 172.21.15.1:0/3970614249 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f696c0039b0 msgr2=0x7f696c003d90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T08:06:42.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:42.804+0000 7f698a200640 1 --2- 172.21.15.1:0/3970614249 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f696c0039b0 0x7f696c003d90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:06:42.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:42.804+0000 7f698a200640 1 -- 172.21.15.1:0/3970614249 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f696c004800 msgr2=0x7f696c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:06:42.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:42.804+0000 7f6988e00640 1 --2- 172.21.15.1:0/3970614249 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f696c004800 0x7f696c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:06:42.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:42.804+0000 7f698a200640 1 --2- 172.21.15.1:0/3970614249 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f696c004800 0x7f696c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:06:42.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:42.804+0000 7f698a200640 1 -- 172.21.15.1:0/3970614249 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f696c003520 con 0x7f696c0052e0 2024-05-09T08:06:42.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:42.805+0000 7f6988e00640 1 --2- 172.21.15.1:0/3970614249 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f696c004800 0x7f696c004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T08:06:42.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:42.806+0000 7f698a200640 1 -- 172.21.15.1:0/3970614249 reap_dead start 2024-05-09T08:06:42.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:42.806+0000 7f698a200640 1 --2- 172.21.15.1:0/3970614249 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f696c0052e0 0x7f696c00b9c0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f69840c1fe0 tx=0x7f696c005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=38851d8e749b33f7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:06:42.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:42.807+0000 7f6983e00640 1 -- 172.21.15.1:0/3970614249 <== mon.2 v2:172.21.15.7:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6970004be0 con 0x7f696c0052e0 2024-05-09T08:06:42.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:42.807+0000 7f6983e00640 1 -- 172.21.15.1:0/3970614249 <== mon.2 v2:172.21.15.7:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f697002f030 con 0x7f696c0052e0 2024-05-09T08:06:42.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:42.807+0000 7f6983e00640 1 -- 172.21.15.1:0/3970614249 <== mon.2 v2:172.21.15.7:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6970005d10 con 0x7f696c0052e0 2024-05-09T08:06:42.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:42.808+0000 7f698c200640 1 -- 172.21.15.1:0/3970614249 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f696c0052e0 msgr2=0x7f696c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:06:42.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:42.808+0000 7f698c200640 1 --2- 172.21.15.1:0/3970614249 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f696c0052e0 0x7f696c00b9c0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f69840c1fe0 tx=0x7f696c005830 comp rx=0 tx=0).stop 2024-05-09T08:06:42.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:42.808+0000 7f698c200640 1 -- 172.21.15.1:0/3970614249 shutdown_connections 2024-05-09T08:06:42.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:42.808+0000 7f698c200640 1 --2- 172.21.15.1:0/3970614249 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f696c0052e0 0x7f696c00b9c0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:06:42.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:42.808+0000 7f698c200640 1 -- 172.21.15.1:0/3970614249 >> 172.21.15.1:0/3970614249 conn(0x7f698406dad0 msgr2=0x7f698406deb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:06:42.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:42.808+0000 7f698c200640 1 -- 172.21.15.1:0/3970614249 shutdown_connections 2024-05-09T08:06:42.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:42.808+0000 7f698c200640 1 -- 172.21.15.1:0/3970614249 wait complete. 2024-05-09T08:06:42.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:42.810+0000 7f698c200640 1 Processor -- start 2024-05-09T08:06:42.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:42.810+0000 7f698c200640 1 -- start start 2024-05-09T08:06:42.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:42.812+0000 7f698c200640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f696c0052e0 0x7f698414bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:06:42.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:42.812+0000 7f698c200640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f6984147640 0x7f69841ef630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:06:42.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:42.812+0000 7f698c200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6984157760 0x7f6984145970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:06:42.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:42.812+0000 7f698c200640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f698415a690 con 0x7f696c0052e0 2024-05-09T08:06:42.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:42.812+0000 7f698a200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6984157760 0x7f6984145970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:06:42.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:42.813+0000 7f698c200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f698415a3b0 con 0x7f6984157760 2024-05-09T08:06:42.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:42.813+0000 7f698a200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6984157760 0x7f6984145970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.1:50686/0 (socket says 172.21.15.1:50686) 2024-05-09T08:06:42.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:42.813+0000 7f698c200640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f698415a970 con 0x7f6984147640 2024-05-09T08:06:42.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:42.813+0000 7f698a200640 1 -- 172.21.15.1:0/1944360915 learned_addr learned my addr 172.21.15.1:0/1944360915 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:06:42.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:42.813+0000 7f6989800640 1 --2- 172.21.15.1:0/1944360915 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f696c0052e0 0x7f698414bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:06:42.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:42.813+0000 7f6988e00640 1 --2- 172.21.15.1:0/1944360915 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f6984147640 0x7f69841ef630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:06:42.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:42.813+0000 7f6989800640 1 -- 172.21.15.1:0/1944360915 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f6984147640 msgr2=0x7f69841ef630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:06:42.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:42.813+0000 7f6989800640 1 --2- 172.21.15.1:0/1944360915 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f6984147640 0x7f69841ef630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:06:42.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:42.813+0000 7f6989800640 1 -- 172.21.15.1:0/1944360915 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6984157760 msgr2=0x7f6984145970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:06:42.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:42.813+0000 7f6989800640 1 --2- 172.21.15.1:0/1944360915 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6984157760 0x7f6984145970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:06:42.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:42.813+0000 7f6989800640 1 -- 172.21.15.1:0/1944360915 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f698414c760 con 0x7f696c0052e0 2024-05-09T08:06:42.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:42.813+0000 7f698a200640 1 --2- 172.21.15.1:0/1944360915 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6984157760 0x7f6984145970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:06:42.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:42.814+0000 7f6988e00640 1 --2- 172.21.15.1:0/1944360915 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f6984147640 0x7f69841ef630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T08:06:42.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:42.814+0000 7f6989800640 1 --2- 172.21.15.1:0/1944360915 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f696c0052e0 0x7f698414bfe0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f697c00c3c0 tx=0x7f697c00c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:06:42.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:42.814+0000 7f6981600640 1 -- 172.21.15.1:0/1944360915 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f697c007020 con 0x7f696c0052e0 2024-05-09T08:06:42.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:42.814+0000 7f6981600640 1 -- 172.21.15.1:0/1944360915 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f697c0071c0 con 0x7f696c0052e0 2024-05-09T08:06:42.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:42.814+0000 7f6981600640 1 -- 172.21.15.1:0/1944360915 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f697c007570 con 0x7f696c0052e0 2024-05-09T08:06:42.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:42.814+0000 7f698c200640 1 -- 172.21.15.1:0/1944360915 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f698414ca40 con 0x7f696c0052e0 2024-05-09T08:06:42.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:42.815+0000 7f698c200640 1 -- 172.21.15.1:0/1944360915 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6984200950 con 0x7f696c0052e0 2024-05-09T08:06:42.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:42.816+0000 7f698c200640 1 --2- 172.21.15.1:0/1944360915 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6984200b50 0x7f6984145eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:06:42.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:42.816+0000 7f698c200640 1 -- 172.21.15.1:0/1944360915 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f698414bd20 con 0x7f6984200b50 2024-05-09T08:06:42.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:42.817+0000 7f6981600640 1 -- 172.21.15.1:0/1944360915 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f697c019020 con 0x7f696c0052e0 2024-05-09T08:06:42.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:42.817+0000 7f6988e00640 1 --2- 172.21.15.1:0/1944360915 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6984200b50 0x7f6984145eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:06:42.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:42.818+0000 7f6988e00640 1 -- 172.21.15.1:0/1944360915 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6984150b90 con 0x7f6984200b50 2024-05-09T08:06:42.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:42.818+0000 7f6988e00640 1 --2- 172.21.15.1:0/1944360915 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6984200b50 0x7f6984145eb0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f6978009fa0 tx=0x7f6978007810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:06:42.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:42.818+0000 7f6981600640 1 --2- 172.21.15.1:0/1944360915 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f696406bed0 0x7f696406e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:06:42.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:42.819+0000 7f6981600640 1 -- 172.21.15.1:0/1944360915 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7f697c08a230 con 0x7f696c0052e0 2024-05-09T08:06:42.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:42.819+0000 7f6981600640 1 -- 172.21.15.1:0/1944360915 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f697801a070 con 0x7f6984200b50 2024-05-09T08:06:42.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:42.819+0000 7f698a200640 1 --2- 172.21.15.1:0/1944360915 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f696406bed0 0x7f696406e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:06:42.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:42.819+0000 7f6981600640 1 -- 172.21.15.1:0/1944360915 <== mon.0 v2:172.21.15.1:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f697800a420 con 0x7f6984200b50 2024-05-09T08:06:42.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:42.819+0000 7f6981600640 1 -- 172.21.15.1:0/1944360915 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6984200b50 msgr2=0x7f6984145eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:06:42.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:42.819+0000 7f6981600640 1 --2- 172.21.15.1:0/1944360915 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6984200b50 0x7f6984145eb0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f6978009fa0 tx=0x7f6978007810 comp rx=0 tx=0).stop 2024-05-09T08:06:42.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:42.819+0000 7f6981600640 1 -- 172.21.15.1:0/1944360915 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6984200b50 msgr2=0x7f6984145eb0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-05-09T08:06:42.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:42.819+0000 7f6981600640 1 --2- 172.21.15.1:0/1944360915 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6984200b50 0x7f6984145eb0 secure :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f6978009fa0 tx=0x7f6978007810 comp rx=0 tx=0).stop 2024-05-09T08:06:42.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:42.820+0000 7f698a200640 1 --2- 172.21.15.1:0/1944360915 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f696406bed0 0x7f696406e390 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f69700050a0 tx=0x7f6970038040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:06:42.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:42.830+0000 7f698c200640 1 --2- 172.21.15.1:0/1944360915 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f69840726c0 0x7f6984158aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:06:42.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:42.830+0000 7f698c200640 1 -- 172.21.15.1:0/1944360915 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f698414c520 con 0x7f69840726c0 2024-05-09T08:06:42.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:42.830+0000 7f6988e00640 1 --2- 172.21.15.1:0/1944360915 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f69840726c0 0x7f6984158aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:06:42.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:42.831+0000 7f6988e00640 1 -- 172.21.15.1:0/1944360915 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f6984074d70 con 0x7f69840726c0 2024-05-09T08:06:42.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:42.831+0000 7f6988e00640 1 --2- 172.21.15.1:0/1944360915 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f69840726c0 0x7f6984158aa0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f6984200f50 tx=0x7f6978004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:06:42.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:42.831+0000 7f6981600640 1 -- 172.21.15.1:0/1944360915 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f697801a040 con 0x7f69840726c0 2024-05-09T08:06:42.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:42.832+0000 7f6981600640 1 -- 172.21.15.1:0/1944360915 <== mon.0 v2:172.21.15.1:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4876 (secure 0 0 0) 0x7f697800a5d0 con 0x7f69840726c0 2024-05-09T08:06:42.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:42.832+0000 7f6981600640 1 -- 172.21.15.1:0/1944360915 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f69840726c0 msgr2=0x7f6984158aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:06:42.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:42.832+0000 7f6981600640 1 --2- 172.21.15.1:0/1944360915 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f69840726c0 0x7f6984158aa0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f6984200f50 tx=0x7f6978004ab0 comp rx=0 tx=0).stop 2024-05-09T08:06:42.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:42.832+0000 7f6981600640 1 -- 172.21.15.1:0/1944360915 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f69840726c0 msgr2=0x7f6984158aa0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-05-09T08:06:42.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:42.832+0000 7f6981600640 1 --2- 172.21.15.1:0/1944360915 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f69840726c0 0x7f6984158aa0 secure :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f6984200f50 tx=0x7f6978004ab0 comp rx=0 tx=0).stop 2024-05-09T08:06:42.844 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-05-09T08:06:42.844 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "a", 2024-05-09T08:06:42.844 INFO:teuthology.orchestra.run.smithi001.stdout: "rank": 0, 2024-05-09T08:06:42.844 INFO:teuthology.orchestra.run.smithi001.stdout: "state": "leader", 2024-05-09T08:06:42.845 INFO:teuthology.orchestra.run.smithi001.stdout: "election_epoch": 36, 2024-05-09T08:06:42.845 INFO:teuthology.orchestra.run.smithi001.stdout: "uptime": 65448, 2024-05-09T08:06:42.845 INFO:teuthology.orchestra.run.smithi001.stdout: "quorum": [ 2024-05-09T08:06:42.845 INFO:teuthology.orchestra.run.smithi001.stdout: 0, 2024-05-09T08:06:42.845 INFO:teuthology.orchestra.run.smithi001.stdout: 1, 2024-05-09T08:06:42.845 INFO:teuthology.orchestra.run.smithi001.stdout: 2 2024-05-09T08:06:42.845 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:06:42.845 INFO:teuthology.orchestra.run.smithi001.stdout: "quorum_age": 3, 2024-05-09T08:06:42.845 INFO:teuthology.orchestra.run.smithi001.stdout: "features": { 2024-05-09T08:06:42.845 INFO:teuthology.orchestra.run.smithi001.stdout: "required_con": "2450521708007866372", 2024-05-09T08:06:42.845 INFO:teuthology.orchestra.run.smithi001.stdout: "required_mon": [ 2024-05-09T08:06:42.845 INFO:teuthology.orchestra.run.smithi001.stdout: "kraken", 2024-05-09T08:06:42.845 INFO:teuthology.orchestra.run.smithi001.stdout: "luminous", 2024-05-09T08:06:42.845 INFO:teuthology.orchestra.run.smithi001.stdout: "mimic", 2024-05-09T08:06:42.846 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap-prune", 2024-05-09T08:06:42.846 INFO:teuthology.orchestra.run.smithi001.stdout: "nautilus", 2024-05-09T08:06:42.846 INFO:teuthology.orchestra.run.smithi001.stdout: "octopus", 2024-05-09T08:06:42.846 INFO:teuthology.orchestra.run.smithi001.stdout: "pacific", 2024-05-09T08:06:42.846 INFO:teuthology.orchestra.run.smithi001.stdout: "elector-pinging", 2024-05-09T08:06:42.846 INFO:teuthology.orchestra.run.smithi001.stdout: "quincy", 2024-05-09T08:06:42.846 INFO:teuthology.orchestra.run.smithi001.stdout: "reef", 2024-05-09T08:06:42.846 INFO:teuthology.orchestra.run.smithi001.stdout: "squid" 2024-05-09T08:06:42.846 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:06:42.846 INFO:teuthology.orchestra.run.smithi001.stdout: "quorum_con": "4540701547738038271", 2024-05-09T08:06:42.846 INFO:teuthology.orchestra.run.smithi001.stdout: "quorum_mon": [ 2024-05-09T08:06:42.846 INFO:teuthology.orchestra.run.smithi001.stdout: "kraken", 2024-05-09T08:06:42.846 INFO:teuthology.orchestra.run.smithi001.stdout: "luminous", 2024-05-09T08:06:42.847 INFO:teuthology.orchestra.run.smithi001.stdout: "mimic", 2024-05-09T08:06:42.847 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap-prune", 2024-05-09T08:06:42.847 INFO:teuthology.orchestra.run.smithi001.stdout: "nautilus", 2024-05-09T08:06:42.847 INFO:teuthology.orchestra.run.smithi001.stdout: "octopus", 2024-05-09T08:06:42.847 INFO:teuthology.orchestra.run.smithi001.stdout: "pacific", 2024-05-09T08:06:42.847 INFO:teuthology.orchestra.run.smithi001.stdout: "elector-pinging", 2024-05-09T08:06:42.847 INFO:teuthology.orchestra.run.smithi001.stdout: "quincy", 2024-05-09T08:06:42.847 INFO:teuthology.orchestra.run.smithi001.stdout: "reef", 2024-05-09T08:06:42.847 INFO:teuthology.orchestra.run.smithi001.stdout: "squid" 2024-05-09T08:06:42.847 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:06:42.847 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:06:42.847 INFO:teuthology.orchestra.run.smithi001.stdout: "outside_quorum": [], 2024-05-09T08:06:42.847 INFO:teuthology.orchestra.run.smithi001.stdout: "extra_probe_peers": [], 2024-05-09T08:06:42.847 INFO:teuthology.orchestra.run.smithi001.stdout: "sync_provider": [], 2024-05-09T08:06:42.848 INFO:teuthology.orchestra.run.smithi001.stdout: "monmap": { 2024-05-09T08:06:42.848 INFO:teuthology.orchestra.run.smithi001.stdout: "epoch": 1, 2024-05-09T08:06:42.848 INFO:teuthology.orchestra.run.smithi001.stdout: "fsid": "54329ec8-3270-4043-9a82-c6cd7420aa7f", 2024-05-09T08:06:42.848 INFO:teuthology.orchestra.run.smithi001.stdout: "modified": "2024-05-09T08:01:21.819270Z", 2024-05-09T08:06:42.848 INFO:teuthology.orchestra.run.smithi001.stdout: "created": "2024-05-09T08:01:21.819270Z", 2024-05-09T08:06:42.848 INFO:teuthology.orchestra.run.smithi001.stdout: "min_mon_release": 19, 2024-05-09T08:06:42.848 INFO:teuthology.orchestra.run.smithi001.stdout: "min_mon_release_name": "squid", 2024-05-09T08:06:42.848 INFO:teuthology.orchestra.run.smithi001.stdout: "election_strategy": 1, 2024-05-09T08:06:42.848 INFO:teuthology.orchestra.run.smithi001.stdout: "disallowed_leaders: ": "", 2024-05-09T08:06:42.848 INFO:teuthology.orchestra.run.smithi001.stdout: "stretch_mode": false, 2024-05-09T08:06:42.848 INFO:teuthology.orchestra.run.smithi001.stdout: "tiebreaker_mon": "", 2024-05-09T08:06:42.848 INFO:teuthology.orchestra.run.smithi001.stdout: "removed_ranks: ": "", 2024-05-09T08:06:42.848 INFO:teuthology.orchestra.run.smithi001.stdout: "features": { 2024-05-09T08:06:42.848 INFO:teuthology.orchestra.run.smithi001.stdout: "persistent": [ 2024-05-09T08:06:42.849 INFO:teuthology.orchestra.run.smithi001.stdout: "kraken", 2024-05-09T08:06:42.849 INFO:teuthology.orchestra.run.smithi001.stdout: "luminous", 2024-05-09T08:06:42.849 INFO:teuthology.orchestra.run.smithi001.stdout: "mimic", 2024-05-09T08:06:42.849 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap-prune", 2024-05-09T08:06:42.849 INFO:teuthology.orchestra.run.smithi001.stdout: "nautilus", 2024-05-09T08:06:42.849 INFO:teuthology.orchestra.run.smithi001.stdout: "octopus", 2024-05-09T08:06:42.849 INFO:teuthology.orchestra.run.smithi001.stdout: "pacific", 2024-05-09T08:06:42.849 INFO:teuthology.orchestra.run.smithi001.stdout: "elector-pinging", 2024-05-09T08:06:42.849 INFO:teuthology.orchestra.run.smithi001.stdout: "quincy", 2024-05-09T08:06:42.849 INFO:teuthology.orchestra.run.smithi001.stdout: "reef", 2024-05-09T08:06:42.849 INFO:teuthology.orchestra.run.smithi001.stdout: "squid" 2024-05-09T08:06:42.849 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:06:42.849 INFO:teuthology.orchestra.run.smithi001.stdout: "optional": [] 2024-05-09T08:06:42.849 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:06:42.850 INFO:teuthology.orchestra.run.smithi001.stdout: "mons": [ 2024-05-09T08:06:42.850 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:06:42.850 INFO:teuthology.orchestra.run.smithi001.stdout: "rank": 0, 2024-05-09T08:06:42.850 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "a", 2024-05-09T08:06:42.850 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-05-09T08:06:42.850 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-05-09T08:06:42.850 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:06:42.850 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-05-09T08:06:42.850 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:3300", 2024-05-09T08:06:42.850 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:06:42.850 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:06:42.850 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:06:42.850 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-05-09T08:06:42.851 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6789", 2024-05-09T08:06:42.851 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:06:42.851 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:06:42.851 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:06:42.851 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:06:42.851 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6789/0", 2024-05-09T08:06:42.851 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.1:6789/0", 2024-05-09T08:06:42.851 INFO:teuthology.orchestra.run.smithi001.stdout: "priority": 0, 2024-05-09T08:06:42.851 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 0, 2024-05-09T08:06:42.851 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_location": "{}" 2024-05-09T08:06:42.851 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:06:42.851 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:06:42.851 INFO:teuthology.orchestra.run.smithi001.stdout: "rank": 1, 2024-05-09T08:06:42.851 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "b", 2024-05-09T08:06:42.852 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-05-09T08:06:42.852 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-05-09T08:06:42.852 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:06:42.852 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-05-09T08:06:42.852 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:3300", 2024-05-09T08:06:42.852 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:06:42.852 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:06:42.852 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:06:42.852 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-05-09T08:06:42.852 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:6789", 2024-05-09T08:06:42.852 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:06:42.852 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:06:42.852 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:06:42.852 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:06:42.853 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:6789/0", 2024-05-09T08:06:42.853 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.7:6789/0", 2024-05-09T08:06:42.853 INFO:teuthology.orchestra.run.smithi001.stdout: "priority": 0, 2024-05-09T08:06:42.853 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 0, 2024-05-09T08:06:42.853 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_location": "{}" 2024-05-09T08:06:42.853 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:06:42.853 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:06:42.853 INFO:teuthology.orchestra.run.smithi001.stdout: "rank": 2, 2024-05-09T08:06:42.853 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "c", 2024-05-09T08:06:42.853 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-05-09T08:06:42.853 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-05-09T08:06:42.853 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:06:42.853 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-05-09T08:06:42.853 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:3301", 2024-05-09T08:06:42.854 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:06:42.854 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:06:42.854 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:06:42.854 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-05-09T08:06:42.854 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:6790", 2024-05-09T08:06:42.854 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:06:42.854 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:06:42.854 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:06:42.854 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:06:42.854 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:6790/0", 2024-05-09T08:06:42.854 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.7:6790/0", 2024-05-09T08:06:42.854 INFO:teuthology.orchestra.run.smithi001.stdout: "priority": 0, 2024-05-09T08:06:42.854 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 0, 2024-05-09T08:06:42.855 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_location": "{}" 2024-05-09T08:06:42.855 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:06:42.855 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:06:42.855 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:06:42.855 INFO:teuthology.orchestra.run.smithi001.stdout: "feature_map": { 2024-05-09T08:06:42.855 INFO:teuthology.orchestra.run.smithi001.stdout: "mon": [ 2024-05-09T08:06:42.855 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:06:42.855 INFO:teuthology.orchestra.run.smithi001.stdout: "features": "0x3f03cffffffdffff", 2024-05-09T08:06:42.855 INFO:teuthology.orchestra.run.smithi001.stdout: "release": "luminous", 2024-05-09T08:06:42.855 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 1 2024-05-09T08:06:42.855 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:06:42.855 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:06:42.855 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": [ 2024-05-09T08:06:42.855 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:06:42.856 INFO:teuthology.orchestra.run.smithi001.stdout: "features": "0x3f03cffffffdffff", 2024-05-09T08:06:42.856 INFO:teuthology.orchestra.run.smithi001.stdout: "release": "luminous", 2024-05-09T08:06:42.856 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 1 2024-05-09T08:06:42.856 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:06:42.856 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:06:42.856 INFO:teuthology.orchestra.run.smithi001.stdout: "client": [ 2024-05-09T08:06:42.856 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:06:42.856 INFO:teuthology.orchestra.run.smithi001.stdout: "features": "0x3f03cffffffdffff", 2024-05-09T08:06:42.856 INFO:teuthology.orchestra.run.smithi001.stdout: "release": "luminous", 2024-05-09T08:06:42.856 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 2 2024-05-09T08:06:42.856 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:06:42.856 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:06:42.856 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr": [ 2024-05-09T08:06:42.856 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:06:42.857 INFO:teuthology.orchestra.run.smithi001.stdout: "features": "0x3f03cffffffdffff", 2024-05-09T08:06:42.857 INFO:teuthology.orchestra.run.smithi001.stdout: "release": "luminous", 2024-05-09T08:06:42.857 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 1 2024-05-09T08:06:42.857 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:06:42.857 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:06:42.857 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:06:42.857 INFO:teuthology.orchestra.run.smithi001.stdout: "stretch_mode": false 2024-05-09T08:06:42.857 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-05-09T08:06:42.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:42.832+0000 7f698c200640 1 -- 172.21.15.1:0/1944360915 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f696406bed0 msgr2=0x7f696406e390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:06:42.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:42.832+0000 7f698c200640 1 --2- 172.21.15.1:0/1944360915 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f696406bed0 0x7f696406e390 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f69700050a0 tx=0x7f6970038040 comp rx=0 tx=0).stop 2024-05-09T08:06:42.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:42.832+0000 7f698c200640 1 -- 172.21.15.1:0/1944360915 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f696c0052e0 msgr2=0x7f698414bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:06:42.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:42.832+0000 7f698c200640 1 --2- 172.21.15.1:0/1944360915 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f696c0052e0 0x7f698414bfe0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f697c00c3c0 tx=0x7f697c00c7c0 comp rx=0 tx=0).stop 2024-05-09T08:06:42.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:42.832+0000 7f698a200640 1 -- 172.21.15.1:0/1944360915 reap_dead start 2024-05-09T08:06:42.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:42.832+0000 7f698a200640 1 -- 172.21.15.1:0/1944360915 reap_dead start 2024-05-09T08:06:42.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:42.832+0000 7f698c200640 1 -- 172.21.15.1:0/1944360915 shutdown_connections 2024-05-09T08:06:42.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:42.832+0000 7f698c200640 1 -- 172.21.15.1:0/1944360915 >> 172.21.15.1:0/1944360915 conn(0x7f698406dad0 msgr2=0x7f698414a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:06:42.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:42.833+0000 7f698c200640 1 -- 172.21.15.1:0/1944360915 shutdown_connections 2024-05-09T08:06:42.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:42.833+0000 7f698c200640 1 -- 172.21.15.1:0/1944360915 wait complete. 2024-05-09T08:06:42.865 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-05-09T08:06:42.865 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-05-09T08:06:42.865 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-05-09T08:06:43.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.163+0000 7f6c12600640 1 Processor -- start 2024-05-09T08:06:43.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.163+0000 7f6c12600640 1 -- start start 2024-05-09T08:06:43.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.168+0000 7f6c12600640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f6c0c0724b0 0x7f6c0c161f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:06:43.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.168+0000 7f6c12600640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f6c0c0c7e00 0x7f6c0c06f720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:06:43.181 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.168+0000 7f6c12600640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6c0c1625d0 con 0x7f6c0c158e20 2024-05-09T08:06:43.181 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.169+0000 7f6c0be00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f6c0c0c7e00 0x7f6c0c06f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:06:43.181 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.169+0000 7f6c0aa00640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f6c0c0724b0 0x7f6c0c161f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:06:43.181 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.169+0000 7f6c0be00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f6c0c0c7e00 0x7f6c0c06f720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3301/0 says I am v2:172.21.15.1:33746/0 (socket says 172.21.15.1:33746) 2024-05-09T08:06:43.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.169+0000 7f6c0aa00640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f6c0c0724b0 0x7f6c0c161f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:47394/0 (socket says 172.21.15.1:47394) 2024-05-09T08:06:43.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.169+0000 7f6c0be00640 1 -- 172.21.15.1:0/1750805680 learned_addr learned my addr 172.21.15.1:0/1750805680 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:06:43.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.169+0000 7f6c0b400640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f6c0c158e20 0x7f6c0c15f3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:45248/0 (socket says 172.21.15.1:45248) 2024-05-09T08:06:43.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.173+0000 7f6c12600640 1 -- 172.21.15.1:0/1750805680 --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f6c0c1628b0 con 0x7f6c0c0724b0 2024-05-09T08:06:43.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.173+0000 7f6c12600640 1 -- 172.21.15.1:0/1750805680 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f6c0c162b90 con 0x7f6c0c0c7e00 2024-05-09T08:06:43.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.173+0000 7f6c0be00640 1 -- 172.21.15.1:0/1750805680 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f6c0c0c7e00 msgr2=0x7f6c0c06f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-05-09T08:06:43.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.173+0000 7f6c0be00640 1 -- 172.21.15.1:0/1750805680 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f6c0c0c7e00 msgr2=0x7f6c0c06f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-09T08:06:43.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.174+0000 7f6c0be00640 1 --2- 172.21.15.1:0/1750805680 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f6c0c0c7e00 0x7f6c0c06f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-09T08:06:43.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.174+0000 7f6c0be00640 1 --2- 172.21.15.1:0/1750805680 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f6c0c0c7e00 0x7f6c0c06f720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T08:06:43.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.177+0000 7f6c0aa00640 1 -- 172.21.15.1:0/1750805680 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f6c0c0c7e00 msgr2=0x7f6c0c06f720 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T08:06:43.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.177+0000 7f6c0aa00640 1 --2- 172.21.15.1:0/1750805680 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f6c0c0c7e00 0x7f6c0c06f720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:06:43.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.177+0000 7f6c0aa00640 1 -- 172.21.15.1:0/1750805680 >> v1:172.21.15.1:6789/0 conn(0x7f6c0c158e20 legacy=0x7f6c0c15f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:06:43.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.177+0000 7f6c0aa00640 1 -- 172.21.15.1:0/1750805680 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c0c15ebc0 con 0x7f6c0c0724b0 2024-05-09T08:06:43.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.182+0000 7f6c0aa00640 1 --2- 172.21.15.1:0/1750805680 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6c0c0724b0 0x7f6c0c161f20 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f6bf80097a0 tx=0x7f6bf802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3dc6df6a9ffa206 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:06:43.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.182+0000 7f6c0a000640 1 -- 172.21.15.1:0/1750805680 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bf8037070 con 0x7f6c0c0724b0 2024-05-09T08:06:43.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.183+0000 7f6c0a000640 1 -- 172.21.15.1:0/1750805680 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6bf8032440 con 0x7f6c0c0724b0 2024-05-09T08:06:43.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.183+0000 7f6c0a000640 1 -- 172.21.15.1:0/1750805680 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bf8037070 con 0x7f6c0c0724b0 2024-05-09T08:06:43.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.184+0000 7f6c12600640 1 -- 172.21.15.1:0/1750805680 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6c0c0724b0 msgr2=0x7f6c0c161f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:06:43.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.184+0000 7f6c12600640 1 --2- 172.21.15.1:0/1750805680 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6c0c0724b0 0x7f6c0c161f20 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f6bf80097a0 tx=0x7f6bf802a230 comp rx=0 tx=0).stop 2024-05-09T08:06:43.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.185+0000 7f6c12600640 1 -- 172.21.15.1:0/1750805680 shutdown_connections 2024-05-09T08:06:43.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.185+0000 7f6c12600640 1 --2- 172.21.15.1:0/1750805680 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f6c0c0c7e00 0x7f6c0c06f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:06:43.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.185+0000 7f6c12600640 1 --2- 172.21.15.1:0/1750805680 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6c0c0724b0 0x7f6c0c161f20 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:06:43.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.185+0000 7f6c12600640 1 -- 172.21.15.1:0/1750805680 >> 172.21.15.1:0/1750805680 conn(0x7f6c0c06dad0 msgr2=0x7f6c0c06deb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:06:43.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.189+0000 7f6c12600640 1 -- 172.21.15.1:0/1750805680 shutdown_connections 2024-05-09T08:06:43.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.189+0000 7f6c12600640 1 -- 172.21.15.1:0/1750805680 wait complete. 2024-05-09T08:06:43.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.192+0000 7f6c12600640 1 Processor -- start 2024-05-09T08:06:43.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.197+0000 7f6c12600640 1 -- start start 2024-05-09T08:06:43.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.199+0000 7f6c12600640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f6c0c0724b0 0x7f6c0c160e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:06:43.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.199+0000 7f6c12600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c0c0c7e00 0x7f6c0c15b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:06:43.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.199+0000 7f6c12600640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6c0c158e20 0x7f6c0c15e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:06:43.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.199+0000 7f6c12600640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f6c0c071e00 con 0x7f6c0c158e20 2024-05-09T08:06:43.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.199+0000 7f6c12600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f6c0c071c80 con 0x7f6c0c0c7e00 2024-05-09T08:06:43.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.199+0000 7f6c12600640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f6c0c071f80 con 0x7f6c0c0724b0 2024-05-09T08:06:43.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.202+0000 7f6c0aa00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c0c0c7e00 0x7f6c0c15b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:06:43.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.203+0000 7f6c0aa00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c0c0c7e00 0x7f6c0c15b1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.1:50690/0 (socket says 172.21.15.1:50690) 2024-05-09T08:06:43.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.203+0000 7f6c0aa00640 1 -- 172.21.15.1:0/915007560 learned_addr learned my addr 172.21.15.1:0/915007560 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:06:43.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.203+0000 7f6c0b400640 1 --2- 172.21.15.1:0/915007560 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f6c0c0724b0 0x7f6c0c160e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:06:43.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.204+0000 7f6c0aa00640 1 -- 172.21.15.1:0/915007560 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f6c0c0724b0 msgr2=0x7f6c0c160e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:06:43.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.204+0000 7f6c0aa00640 1 --2- 172.21.15.1:0/915007560 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f6c0c0724b0 0x7f6c0c160e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:06:43.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.204+0000 7f6c0aa00640 1 -- 172.21.15.1:0/915007560 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6c0c158e20 msgr2=0x7f6c0c15e1f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T08:06:43.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.204+0000 7f6c0aa00640 1 --2- 172.21.15.1:0/915007560 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6c0c158e20 0x7f6c0c15e1f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:06:43.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.204+0000 7f6c0aa00640 1 -- 172.21.15.1:0/915007560 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c0c159810 con 0x7f6c0c0c7e00 2024-05-09T08:06:43.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.207+0000 7f6c0aa00640 1 --2- 172.21.15.1:0/915007560 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c0c0c7e00 0x7f6c0c15b1f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f6bf802a820 tx=0x7f6bf80046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:06:43.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.207+0000 7f6c03400640 1 -- 172.21.15.1:0/915007560 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bf8037040 con 0x7f6c0c0c7e00 2024-05-09T08:06:43.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.208+0000 7f6c03400640 1 -- 172.21.15.1:0/915007560 <== mon.1 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6bf8003d40 con 0x7f6c0c0c7e00 2024-05-09T08:06:43.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.208+0000 7f6c03400640 1 -- 172.21.15.1:0/915007560 <== mon.1 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bf8006e10 con 0x7f6c0c0c7e00 2024-05-09T08:06:43.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.214+0000 7f6c12600640 1 -- 172.21.15.1:0/915007560 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c0c161880 con 0x7f6c0c0c7e00 2024-05-09T08:06:43.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.215+0000 7f6c12600640 1 -- 172.21.15.1:0/915007560 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c0c161e40 con 0x7f6c0c0c7e00 2024-05-09T08:06:43.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.217+0000 7f6c12600640 1 --2- 172.21.15.1:0/915007560 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c0c159320 0x7f6c0c15c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:06:43.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.217+0000 7f6c12600640 1 -- 172.21.15.1:0/915007560 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f6c0c161630 con 0x7f6c0c159320 2024-05-09T08:06:43.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.228+0000 7f6c03400640 1 -- 172.21.15.1:0/915007560 <== mon.1 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6bf8044020 con 0x7f6c0c0c7e00 2024-05-09T08:06:43.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.230+0000 7f6c03400640 1 --2- 172.21.15.1:0/915007560 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f6bec06be80 0x7f6bec06e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:06:43.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.230+0000 7f6c03400640 1 -- 172.21.15.1:0/915007560 <== mon.1 v2:172.21.15.7:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7f6bf80ab5a0 con 0x7f6c0c0c7e00 2024-05-09T08:06:43.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.233+0000 7f6c0be00640 1 --2- 172.21.15.1:0/915007560 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f6bec06be80 0x7f6bec06e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:06:43.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.236+0000 7f6c0be00640 1 --2- 172.21.15.1:0/915007560 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f6bec06be80 0x7f6bec06e340 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f6c0c1f8f40 tx=0x7f6bfc008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:06:43.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.253+0000 7f6c0b400640 1 --2- 172.21.15.1:0/915007560 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c0c159320 0x7f6c0c15c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:06:43.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.254+0000 7f6c0b400640 1 -- 172.21.15.1:0/915007560 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6c0c162040 con 0x7f6c0c159320 2024-05-09T08:06:43.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.255+0000 7f6c0b400640 1 --2- 172.21.15.1:0/915007560 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c0c159320 0x7f6c0c15c820 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f6c04001ab0 tx=0x7f6c04001f70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:06:43.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.256+0000 7f6c03400640 1 -- 172.21.15.1:0/915007560 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c0400e970 con 0x7f6c0c159320 2024-05-09T08:06:43.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.256+0000 7f6c03400640 1 -- 172.21.15.1:0/915007560 <== mon.1 v2:172.21.15.7:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f6c04018070 con 0x7f6c0c159320 2024-05-09T08:06:43.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.256+0000 7f6c03400640 1 -- 172.21.15.1:0/915007560 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c0c159320 msgr2=0x7f6c0c15c820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:06:43.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.256+0000 7f6c03400640 1 --2- 172.21.15.1:0/915007560 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c0c159320 0x7f6c0c15c820 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f6c04001ab0 tx=0x7f6c04001f70 comp rx=0 tx=0).stop 2024-05-09T08:06:43.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.256+0000 7f6c03400640 1 -- 172.21.15.1:0/915007560 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c0c159320 msgr2=0x7f6c0c15c820 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-09T08:06:43.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.256+0000 7f6c03400640 1 --2- 172.21.15.1:0/915007560 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c0c159320 0x7f6c0c15c820 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:06:43.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.272+0000 7f6c12600640 1 --2- 172.21.15.1:0/915007560 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c0c15cd60 0x7f6c0c15d170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:06:43.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.272+0000 7f6c12600640 1 -- 172.21.15.1:0/915007560 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f6c0c15d6b0 con 0x7f6c0c15cd60 2024-05-09T08:06:43.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.273+0000 7f6c0b400640 1 --2- 172.21.15.1:0/915007560 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c0c15cd60 0x7f6c0c15d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:06:43.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.274+0000 7f6c0b400640 1 -- 172.21.15.1:0/915007560 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f6c0c156410 con 0x7f6c0c15cd60 2024-05-09T08:06:43.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.275+0000 7f6c0b400640 1 --2- 172.21.15.1:0/915007560 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c0c15cd60 0x7f6c0c15d170 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f6c04001ab0 tx=0x7f6c040014a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:06:43.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.275+0000 7f6c03400640 1 -- 172.21.15.1:0/915007560 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c0400e970 con 0x7f6c0c15cd60 2024-05-09T08:06:43.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.276+0000 7f6c03400640 1 -- 172.21.15.1:0/915007560 <== mon.1 v2:172.21.15.7:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5188 (secure 0 0 0) 0x7f6c04013630 con 0x7f6c0c15cd60 2024-05-09T08:06:43.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.276+0000 7f6c03400640 1 -- 172.21.15.1:0/915007560 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c0c15cd60 msgr2=0x7f6c0c15d170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:06:43.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.276+0000 7f6c03400640 1 --2- 172.21.15.1:0/915007560 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c0c15cd60 0x7f6c0c15d170 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f6c04001ab0 tx=0x7f6c040014a0 comp rx=0 tx=0).stop 2024-05-09T08:06:43.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.276+0000 7f6c03400640 1 -- 172.21.15.1:0/915007560 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c0c15cd60 msgr2=0x7f6c0c15d170 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-05-09T08:06:43.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.276+0000 7f6c03400640 1 --2- 172.21.15.1:0/915007560 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c0c15cd60 0x7f6c0c15d170 secure :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f6c04001ab0 tx=0x7f6c040014a0 comp rx=0 tx=0).stop 2024-05-09T08:06:43.288 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-05-09T08:06:43.288 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "b", 2024-05-09T08:06:43.288 INFO:teuthology.orchestra.run.smithi001.stdout: "rank": 1, 2024-05-09T08:06:43.288 INFO:teuthology.orchestra.run.smithi001.stdout: "state": "peon", 2024-05-09T08:06:43.288 INFO:teuthology.orchestra.run.smithi001.stdout: "election_epoch": 36, 2024-05-09T08:06:43.289 INFO:teuthology.orchestra.run.smithi001.stdout: "uptime": 170937, 2024-05-09T08:06:43.289 INFO:teuthology.orchestra.run.smithi001.stdout: "quorum": [ 2024-05-09T08:06:43.289 INFO:teuthology.orchestra.run.smithi001.stdout: 0, 2024-05-09T08:06:43.289 INFO:teuthology.orchestra.run.smithi001.stdout: 1, 2024-05-09T08:06:43.289 INFO:teuthology.orchestra.run.smithi001.stdout: 2 2024-05-09T08:06:43.289 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:06:43.289 INFO:teuthology.orchestra.run.smithi001.stdout: "quorum_age": 4, 2024-05-09T08:06:43.289 INFO:teuthology.orchestra.run.smithi001.stdout: "features": { 2024-05-09T08:06:43.289 INFO:teuthology.orchestra.run.smithi001.stdout: "required_con": "2450521708007866372", 2024-05-09T08:06:43.289 INFO:teuthology.orchestra.run.smithi001.stdout: "required_mon": [ 2024-05-09T08:06:43.289 INFO:teuthology.orchestra.run.smithi001.stdout: "kraken", 2024-05-09T08:06:43.289 INFO:teuthology.orchestra.run.smithi001.stdout: "luminous", 2024-05-09T08:06:43.289 INFO:teuthology.orchestra.run.smithi001.stdout: "mimic", 2024-05-09T08:06:43.290 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap-prune", 2024-05-09T08:06:43.290 INFO:teuthology.orchestra.run.smithi001.stdout: "nautilus", 2024-05-09T08:06:43.290 INFO:teuthology.orchestra.run.smithi001.stdout: "octopus", 2024-05-09T08:06:43.290 INFO:teuthology.orchestra.run.smithi001.stdout: "pacific", 2024-05-09T08:06:43.290 INFO:teuthology.orchestra.run.smithi001.stdout: "elector-pinging", 2024-05-09T08:06:43.290 INFO:teuthology.orchestra.run.smithi001.stdout: "quincy", 2024-05-09T08:06:43.290 INFO:teuthology.orchestra.run.smithi001.stdout: "reef", 2024-05-09T08:06:43.290 INFO:teuthology.orchestra.run.smithi001.stdout: "squid" 2024-05-09T08:06:43.290 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:06:43.290 INFO:teuthology.orchestra.run.smithi001.stdout: "quorum_con": "4540701547738038271", 2024-05-09T08:06:43.290 INFO:teuthology.orchestra.run.smithi001.stdout: "quorum_mon": [ 2024-05-09T08:06:43.290 INFO:teuthology.orchestra.run.smithi001.stdout: "kraken", 2024-05-09T08:06:43.290 INFO:teuthology.orchestra.run.smithi001.stdout: "luminous", 2024-05-09T08:06:43.290 INFO:teuthology.orchestra.run.smithi001.stdout: "mimic", 2024-05-09T08:06:43.291 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap-prune", 2024-05-09T08:06:43.291 INFO:teuthology.orchestra.run.smithi001.stdout: "nautilus", 2024-05-09T08:06:43.291 INFO:teuthology.orchestra.run.smithi001.stdout: "octopus", 2024-05-09T08:06:43.291 INFO:teuthology.orchestra.run.smithi001.stdout: "pacific", 2024-05-09T08:06:43.291 INFO:teuthology.orchestra.run.smithi001.stdout: "elector-pinging", 2024-05-09T08:06:43.291 INFO:teuthology.orchestra.run.smithi001.stdout: "quincy", 2024-05-09T08:06:43.291 INFO:teuthology.orchestra.run.smithi001.stdout: "reef", 2024-05-09T08:06:43.291 INFO:teuthology.orchestra.run.smithi001.stdout: "squid" 2024-05-09T08:06:43.291 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:06:43.291 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:06:43.291 INFO:teuthology.orchestra.run.smithi001.stdout: "outside_quorum": [], 2024-05-09T08:06:43.291 INFO:teuthology.orchestra.run.smithi001.stdout: "extra_probe_peers": [], 2024-05-09T08:06:43.291 INFO:teuthology.orchestra.run.smithi001.stdout: "sync_provider": [], 2024-05-09T08:06:43.292 INFO:teuthology.orchestra.run.smithi001.stdout: "monmap": { 2024-05-09T08:06:43.292 INFO:teuthology.orchestra.run.smithi001.stdout: "epoch": 1, 2024-05-09T08:06:43.292 INFO:teuthology.orchestra.run.smithi001.stdout: "fsid": "54329ec8-3270-4043-9a82-c6cd7420aa7f", 2024-05-09T08:06:43.292 INFO:teuthology.orchestra.run.smithi001.stdout: "modified": "2024-05-09T08:01:21.819270Z", 2024-05-09T08:06:43.292 INFO:teuthology.orchestra.run.smithi001.stdout: "created": "2024-05-09T08:01:21.819270Z", 2024-05-09T08:06:43.292 INFO:teuthology.orchestra.run.smithi001.stdout: "min_mon_release": 19, 2024-05-09T08:06:43.292 INFO:teuthology.orchestra.run.smithi001.stdout: "min_mon_release_name": "squid", 2024-05-09T08:06:43.292 INFO:teuthology.orchestra.run.smithi001.stdout: "election_strategy": 1, 2024-05-09T08:06:43.292 INFO:teuthology.orchestra.run.smithi001.stdout: "disallowed_leaders: ": "", 2024-05-09T08:06:43.292 INFO:teuthology.orchestra.run.smithi001.stdout: "stretch_mode": false, 2024-05-09T08:06:43.292 INFO:teuthology.orchestra.run.smithi001.stdout: "tiebreaker_mon": "", 2024-05-09T08:06:43.292 INFO:teuthology.orchestra.run.smithi001.stdout: "removed_ranks: ": "", 2024-05-09T08:06:43.292 INFO:teuthology.orchestra.run.smithi001.stdout: "features": { 2024-05-09T08:06:43.292 INFO:teuthology.orchestra.run.smithi001.stdout: "persistent": [ 2024-05-09T08:06:43.293 INFO:teuthology.orchestra.run.smithi001.stdout: "kraken", 2024-05-09T08:06:43.293 INFO:teuthology.orchestra.run.smithi001.stdout: "luminous", 2024-05-09T08:06:43.293 INFO:teuthology.orchestra.run.smithi001.stdout: "mimic", 2024-05-09T08:06:43.293 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap-prune", 2024-05-09T08:06:43.293 INFO:teuthology.orchestra.run.smithi001.stdout: "nautilus", 2024-05-09T08:06:43.293 INFO:teuthology.orchestra.run.smithi001.stdout: "octopus", 2024-05-09T08:06:43.293 INFO:teuthology.orchestra.run.smithi001.stdout: "pacific", 2024-05-09T08:06:43.293 INFO:teuthology.orchestra.run.smithi001.stdout: "elector-pinging", 2024-05-09T08:06:43.293 INFO:teuthology.orchestra.run.smithi001.stdout: "quincy", 2024-05-09T08:06:43.293 INFO:teuthology.orchestra.run.smithi001.stdout: "reef", 2024-05-09T08:06:43.293 INFO:teuthology.orchestra.run.smithi001.stdout: "squid" 2024-05-09T08:06:43.293 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:06:43.293 INFO:teuthology.orchestra.run.smithi001.stdout: "optional": [] 2024-05-09T08:06:43.293 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:06:43.294 INFO:teuthology.orchestra.run.smithi001.stdout: "mons": [ 2024-05-09T08:06:43.294 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:06:43.294 INFO:teuthology.orchestra.run.smithi001.stdout: "rank": 0, 2024-05-09T08:06:43.294 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "a", 2024-05-09T08:06:43.294 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-05-09T08:06:43.294 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-05-09T08:06:43.294 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:06:43.294 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-05-09T08:06:43.294 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:3300", 2024-05-09T08:06:43.294 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:06:43.294 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:06:43.294 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:06:43.294 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-05-09T08:06:43.295 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6789", 2024-05-09T08:06:43.295 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:06:43.295 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:06:43.295 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:06:43.295 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:06:43.295 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6789/0", 2024-05-09T08:06:43.295 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.1:6789/0", 2024-05-09T08:06:43.295 INFO:teuthology.orchestra.run.smithi001.stdout: "priority": 0, 2024-05-09T08:06:43.295 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 0, 2024-05-09T08:06:43.295 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_location": "{}" 2024-05-09T08:06:43.295 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:06:43.295 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:06:43.295 INFO:teuthology.orchestra.run.smithi001.stdout: "rank": 1, 2024-05-09T08:06:43.296 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "b", 2024-05-09T08:06:43.296 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-05-09T08:06:43.296 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-05-09T08:06:43.296 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:06:43.296 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-05-09T08:06:43.296 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:3300", 2024-05-09T08:06:43.296 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:06:43.296 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:06:43.296 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:06:43.296 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-05-09T08:06:43.296 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:6789", 2024-05-09T08:06:43.296 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:06:43.296 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:06:43.297 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:06:43.297 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:06:43.297 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:6789/0", 2024-05-09T08:06:43.297 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.7:6789/0", 2024-05-09T08:06:43.297 INFO:teuthology.orchestra.run.smithi001.stdout: "priority": 0, 2024-05-09T08:06:43.297 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 0, 2024-05-09T08:06:43.297 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_location": "{}" 2024-05-09T08:06:43.297 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:06:43.297 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:06:43.297 INFO:teuthology.orchestra.run.smithi001.stdout: "rank": 2, 2024-05-09T08:06:43.297 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "c", 2024-05-09T08:06:43.297 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-05-09T08:06:43.297 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-05-09T08:06:43.298 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:06:43.298 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-05-09T08:06:43.298 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:3301", 2024-05-09T08:06:43.298 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:06:43.298 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:06:43.298 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:06:43.298 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-05-09T08:06:43.298 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:6790", 2024-05-09T08:06:43.298 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:06:43.298 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:06:43.298 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:06:43.298 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:06:43.298 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:6790/0", 2024-05-09T08:06:43.298 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.7:6790/0", 2024-05-09T08:06:43.299 INFO:teuthology.orchestra.run.smithi001.stdout: "priority": 0, 2024-05-09T08:06:43.299 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 0, 2024-05-09T08:06:43.299 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_location": "{}" 2024-05-09T08:06:43.299 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:06:43.299 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:06:43.299 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:06:43.299 INFO:teuthology.orchestra.run.smithi001.stdout: "feature_map": { 2024-05-09T08:06:43.299 INFO:teuthology.orchestra.run.smithi001.stdout: "mon": [ 2024-05-09T08:06:43.299 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:06:43.299 INFO:teuthology.orchestra.run.smithi001.stdout: "features": "0x3f03cffffffdffff", 2024-05-09T08:06:43.299 INFO:teuthology.orchestra.run.smithi001.stdout: "release": "luminous", 2024-05-09T08:06:43.299 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 1 2024-05-09T08:06:43.299 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:06:43.300 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:06:43.300 INFO:teuthology.orchestra.run.smithi001.stdout: "mds": [ 2024-05-09T08:06:43.300 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:06:43.300 INFO:teuthology.orchestra.run.smithi001.stdout: "features": "0x3f03cffffffdffff", 2024-05-09T08:06:43.300 INFO:teuthology.orchestra.run.smithi001.stdout: "release": "luminous", 2024-05-09T08:06:43.300 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 6 2024-05-09T08:06:43.300 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:06:43.300 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:06:43.300 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": [ 2024-05-09T08:06:43.300 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:06:43.300 INFO:teuthology.orchestra.run.smithi001.stdout: "features": "0x3f03cffffffdffff", 2024-05-09T08:06:43.300 INFO:teuthology.orchestra.run.smithi001.stdout: "release": "luminous", 2024-05-09T08:06:43.300 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 7 2024-05-09T08:06:43.301 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:06:43.301 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:06:43.301 INFO:teuthology.orchestra.run.smithi001.stdout: "client": [ 2024-05-09T08:06:43.301 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:06:43.301 INFO:teuthology.orchestra.run.smithi001.stdout: "features": "0x2f018fb87aa4aafe", 2024-05-09T08:06:43.301 INFO:teuthology.orchestra.run.smithi001.stdout: "release": "luminous", 2024-05-09T08:06:43.301 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 1 2024-05-09T08:06:43.301 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:06:43.301 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:06:43.301 INFO:teuthology.orchestra.run.smithi001.stdout: "features": "0x3f03cffffffdffff", 2024-05-09T08:06:43.301 INFO:teuthology.orchestra.run.smithi001.stdout: "release": "luminous", 2024-05-09T08:06:43.301 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 6 2024-05-09T08:06:43.301 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:06:43.301 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:06:43.302 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr": [ 2024-05-09T08:06:43.302 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:06:43.302 INFO:teuthology.orchestra.run.smithi001.stdout: "features": "0x3f03cffffffdffff", 2024-05-09T08:06:43.302 INFO:teuthology.orchestra.run.smithi001.stdout: "release": "luminous", 2024-05-09T08:06:43.302 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 1 2024-05-09T08:06:43.302 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:06:43.302 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:06:43.302 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:06:43.302 INFO:teuthology.orchestra.run.smithi001.stdout: "stretch_mode": false 2024-05-09T08:06:43.302 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-05-09T08:06:43.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.277+0000 7f6c12600640 1 -- 172.21.15.1:0/915007560 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f6bec06be80 msgr2=0x7f6bec06e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:06:43.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.277+0000 7f6c12600640 1 --2- 172.21.15.1:0/915007560 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f6bec06be80 0x7f6bec06e340 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f6c0c1f8f40 tx=0x7f6bfc008040 comp rx=0 tx=0).stop 2024-05-09T08:06:43.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.277+0000 7f6c12600640 1 -- 172.21.15.1:0/915007560 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c0c0c7e00 msgr2=0x7f6c0c15b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:06:43.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.277+0000 7f6c12600640 1 --2- 172.21.15.1:0/915007560 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c0c0c7e00 0x7f6c0c15b1f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f6bf802a820 tx=0x7f6bf80046c0 comp rx=0 tx=0).stop 2024-05-09T08:06:43.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.277+0000 7f6c0be00640 1 -- 172.21.15.1:0/915007560 reap_dead start 2024-05-09T08:06:43.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.277+0000 7f6c0be00640 1 -- 172.21.15.1:0/915007560 reap_dead start 2024-05-09T08:06:43.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.278+0000 7f6c12600640 1 -- 172.21.15.1:0/915007560 shutdown_connections 2024-05-09T08:06:43.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.278+0000 7f6c12600640 1 -- 172.21.15.1:0/915007560 >> 172.21.15.1:0/915007560 conn(0x7f6c0c06dad0 msgr2=0x7f6c0c0c2c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:06:43.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.278+0000 7f6c12600640 1 -- 172.21.15.1:0/915007560 shutdown_connections 2024-05-09T08:06:43.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.279+0000 7f6c12600640 1 -- 172.21.15.1:0/915007560 wait complete. 2024-05-09T08:06:43.337 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-05-09T08:06:43.337 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-05-09T08:06:43.337 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-05-09T08:06:43.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.627+0000 7f40f5e00640 1 Processor -- start 2024-05-09T08:06:43.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.628+0000 7f40f5e00640 1 -- start start 2024-05-09T08:06:43.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.630+0000 7f40f5e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f40f00cc000 0x7f40f01547c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:06:43.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.630+0000 7f40f5e00640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f40f015b860 con 0x7f40f006c900 2024-05-09T08:06:43.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.630+0000 7f40f5e00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f40f00b5de0 con 0x7f40f006cce0 2024-05-09T08:06:43.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.630+0000 7f40f5e00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f40f00c7f30 con 0x7f40f00cc000 2024-05-09T08:06:43.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.630+0000 7f40ee000640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f40f006cce0 0x7f40f00c81f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.1:42784/0 (socket says 172.21.15.1:42784) 2024-05-09T08:06:43.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.630+0000 7f40ee000640 1 -- 172.21.15.1:0/1198951786 learned_addr learned my addr 172.21.15.1:0/1198951786 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T08:06:43.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.630+0000 7f40ed600640 1 -- 172.21.15.1:0/1198951786 <== mon.1 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1095044325 0 0) 0x7f40f00b5de0 con 0x7f40f006cce0 2024-05-09T08:06:43.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.630+0000 7f40ed600640 1 -- 172.21.15.1:0/1198951786 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f40d4003520 con 0x7f40f006cce0 2024-05-09T08:06:43.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.630+0000 7f40ed600640 1 -- 172.21.15.1:0/1198951786 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2644019894 0 0) 0x7f40f015b860 con 0x7f40f006c900 2024-05-09T08:06:43.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.631+0000 7f40ed600640 1 -- 172.21.15.1:0/1198951786 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f40f00b5de0 con 0x7f40f006c900 2024-05-09T08:06:43.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.631+0000 7f40ed600640 1 -- 172.21.15.1:0/1198951786 <== mon.1 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2138233799 0 0) 0x7f40d4003520 con 0x7f40f006cce0 2024-05-09T08:06:43.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.631+0000 7f40ed600640 1 -- 172.21.15.1:0/1198951786 >> v2:172.21.15.7:3300/0 conn(0x7f40f00cc000 msgr2=0x7f40f01547c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:06:43.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.631+0000 7f40ef400640 1 --2- 172.21.15.1:0/1198951786 >> v2:172.21.15.7:3300/0 conn(0x7f40f00cc000 0x7f40f01547c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:06:43.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.632+0000 7f40ed600640 1 --2- 172.21.15.1:0/1198951786 >> v2:172.21.15.7:3300/0 conn(0x7f40f00cc000 0x7f40f01547c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:06:43.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.632+0000 7f40ed600640 1 -- 172.21.15.1:0/1198951786 >> v1:172.21.15.1:6789/0 conn(0x7f40f006c900 legacy=0x7f40f00c7280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:06:43.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.632+0000 7f40ed600640 1 -- 172.21.15.1:0/1198951786 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40f00c7990 con 0x7f40f006cce0 2024-05-09T08:06:43.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.632+0000 7f40ed600640 1 -- 172.21.15.1:0/1198951786 <== mon.1 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4179846320 0 0) 0x7f40d8001670 con 0x7f40f006cce0 2024-05-09T08:06:43.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.632+0000 7f40ed600640 1 -- 172.21.15.1:0/1198951786 >> v1:172.21.15.7:6789/0 conn(0x7f40f006cce0 legacy=0x7f40f00c81f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:06:43.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.632+0000 7f40ed600640 1 --2- 172.21.15.1:0/1198951786 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f40d4003b40 0x7f40d4003f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:06:43.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.632+0000 7f40ed600640 1 --2- 172.21.15.1:0/1198951786 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f40d4004800 0x7f40d4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:06:43.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.632+0000 7f40ed600640 1 --2- 172.21.15.1:0/1198951786 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f40d4005210 0x7f40d400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:06:43.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.632+0000 7f40ed600640 1 -- 172.21.15.1:0/1198951786 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f40f00c7f30 con 0x7f40d4003b40 2024-05-09T08:06:43.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.632+0000 7f40ed600640 1 -- 172.21.15.1:0/1198951786 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f40d400c140 con 0x7f40d4004800 2024-05-09T08:06:43.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.632+0000 7f40ed600640 1 -- 172.21.15.1:0/1198951786 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f40d400c340 con 0x7f40d4005210 2024-05-09T08:06:43.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.633+0000 7f40ef400640 1 --2- 172.21.15.1:0/1198951786 >> v2:172.21.15.7:3300/0 conn(0x7f40f00cc000 0x7f40f01547c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T08:06:43.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.633+0000 7f40eea00640 1 --2- 172.21.15.1:0/1198951786 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f40d4003b40 0x7f40d4003f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:06:43.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.633+0000 7f40ef400640 1 --2- 172.21.15.1:0/1198951786 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f40d4005210 0x7f40d400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:06:43.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.633+0000 7f40ee000640 1 --2- 172.21.15.1:0/1198951786 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f40d4004800 0x7f40d4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:06:43.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.634+0000 7f40ee000640 1 -- 172.21.15.1:0/1198951786 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f40d4005210 msgr2=0x7f40d400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:06:43.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.634+0000 7f40ee000640 1 --2- 172.21.15.1:0/1198951786 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f40d4005210 0x7f40d400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:06:43.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.634+0000 7f40ee000640 1 -- 172.21.15.1:0/1198951786 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f40d4003b40 msgr2=0x7f40d4003f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:06:43.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.634+0000 7f40ee000640 1 --2- 172.21.15.1:0/1198951786 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f40d4003b40 0x7f40d4003f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:06:43.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.634+0000 7f40ee000640 1 -- 172.21.15.1:0/1198951786 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40d4003520 con 0x7f40d4004800 2024-05-09T08:06:43.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.634+0000 7f40ef400640 1 --2- 172.21.15.1:0/1198951786 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f40d4005210 0x7f40d400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T08:06:43.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.634+0000 7f40eea00640 1 --2- 172.21.15.1:0/1198951786 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f40d4003b40 0x7f40d4003f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:06:43.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.634+0000 7f40ef400640 1 -- 172.21.15.1:0/1198951786 reap_dead start 2024-05-09T08:06:43.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.635+0000 7f40ee000640 1 --2- 172.21.15.1:0/1198951786 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f40d4004800 0x7f40d4004c80 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f40d8004f70 tx=0x7f40d4005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b67d90490123d87b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:06:43.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.635+0000 7f40ed600640 1 -- 172.21.15.1:0/1198951786 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40d8004b70 con 0x7f40d4004800 2024-05-09T08:06:43.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.635+0000 7f40ed600640 1 -- 172.21.15.1:0/1198951786 <== mon.1 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40d802f030 con 0x7f40d4004800 2024-05-09T08:06:43.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.636+0000 7f40ed600640 1 -- 172.21.15.1:0/1198951786 <== mon.1 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40d8005d10 con 0x7f40d4004800 2024-05-09T08:06:43.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.636+0000 7f40f5e00640 1 -- 172.21.15.1:0/1198951786 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f40d4004800 msgr2=0x7f40d4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:06:43.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.636+0000 7f40f5e00640 1 --2- 172.21.15.1:0/1198951786 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f40d4004800 0x7f40d4004c80 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f40d8004f70 tx=0x7f40d4005830 comp rx=0 tx=0).stop 2024-05-09T08:06:43.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.638+0000 7f40f5e00640 1 -- 172.21.15.1:0/1198951786 shutdown_connections 2024-05-09T08:06:43.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.638+0000 7f40f5e00640 1 --2- 172.21.15.1:0/1198951786 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f40d4004800 0x7f40d4004c80 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:06:43.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.638+0000 7f40f5e00640 1 -- 172.21.15.1:0/1198951786 >> 172.21.15.1:0/1198951786 conn(0x7f40f00cfe80 msgr2=0x7f40f00d0290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:06:43.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.638+0000 7f40f5e00640 1 -- 172.21.15.1:0/1198951786 shutdown_connections 2024-05-09T08:06:43.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.638+0000 7f40f5e00640 1 -- 172.21.15.1:0/1198951786 wait complete. 2024-05-09T08:06:43.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.640+0000 7f40f5e00640 1 Processor -- start 2024-05-09T08:06:43.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.640+0000 7f40f5e00640 1 -- start start 2024-05-09T08:06:43.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.641+0000 7f40f5e00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f40d4004800 0x7f40f0154bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:06:43.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.641+0000 7f40f5e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f40f01508a0 0x7f40f00c6550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:06:43.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.641+0000 7f40f5e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f40f014f6b0 0x7f40f01f3570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:06:43.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.641+0000 7f40f5e00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f40f015ad80 con 0x7f40f01508a0 2024-05-09T08:06:43.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.641+0000 7f40f5e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f40f015aaa0 con 0x7f40f014f6b0 2024-05-09T08:06:43.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.641+0000 7f40f5e00640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f40f015b060 con 0x7f40d4004800 2024-05-09T08:06:43.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.641+0000 7f40ef400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f40f014f6b0 0x7f40f01f3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:06:43.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.641+0000 7f40ef400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f40f014f6b0 0x7f40f01f3570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.1:50730/0 (socket says 172.21.15.1:50730) 2024-05-09T08:06:43.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.642+0000 7f40ef400640 1 -- 172.21.15.1:0/4136855511 learned_addr learned my addr 172.21.15.1:0/4136855511 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:06:43.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.642+0000 7f40eea00640 1 --2- 172.21.15.1:0/4136855511 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f40d4004800 0x7f40f0154bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:06:43.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.642+0000 7f40ef400640 1 -- 172.21.15.1:0/4136855511 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f40d4004800 msgr2=0x7f40f0154bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:06:43.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.642+0000 7f40ef400640 1 --2- 172.21.15.1:0/4136855511 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f40d4004800 0x7f40f0154bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:06:43.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.642+0000 7f40ef400640 1 -- 172.21.15.1:0/4136855511 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f40f01508a0 msgr2=0x7f40f00c6550 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T08:06:43.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.642+0000 7f40ee000640 1 --2- 172.21.15.1:0/4136855511 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f40f01508a0 0x7f40f00c6550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:06:43.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.642+0000 7f40ef400640 1 --2- 172.21.15.1:0/4136855511 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f40f01508a0 0x7f40f00c6550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:06:43.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.642+0000 7f40ef400640 1 -- 172.21.15.1:0/4136855511 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40f0155200 con 0x7f40f014f6b0 2024-05-09T08:06:43.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.644+0000 7f40ef400640 1 --2- 172.21.15.1:0/4136855511 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f40f014f6b0 0x7f40f01f3570 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f40f008d500 tx=0x7f40e800b7a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:06:43.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.646+0000 7f40e7400640 1 -- 172.21.15.1:0/4136855511 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40e801a070 con 0x7f40f014f6b0 2024-05-09T08:06:43.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.647+0000 7f40e7400640 1 -- 172.21.15.1:0/4136855511 <== mon.1 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40e800be80 con 0x7f40f014f6b0 2024-05-09T08:06:43.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.647+0000 7f40e7400640 1 -- 172.21.15.1:0/4136855511 <== mon.1 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40e8009800 con 0x7f40f014f6b0 2024-05-09T08:06:43.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.647+0000 7f40f5e00640 1 -- 172.21.15.1:0/4136855511 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40f01554e0 con 0x7f40f014f6b0 2024-05-09T08:06:43.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.647+0000 7f40f5e00640 1 -- 172.21.15.1:0/4136855511 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40f01f0350 con 0x7f40f014f6b0 2024-05-09T08:06:43.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.649+0000 7f40f5e00640 1 --2- 172.21.15.1:0/4136855511 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f40f014fb30 0x7f40f01f3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:06:43.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.649+0000 7f40f5e00640 1 -- 172.21.15.1:0/4136855511 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f40f01531a0 con 0x7f40f014fb30 2024-05-09T08:06:43.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.650+0000 7f40eea00640 1 --2- 172.21.15.1:0/4136855511 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f40f014fb30 0x7f40f01f3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:06:43.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.651+0000 7f40e7400640 1 -- 172.21.15.1:0/4136855511 <== mon.1 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f40e801d070 con 0x7f40f014f6b0 2024-05-09T08:06:43.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.652+0000 7f40eea00640 1 -- 172.21.15.1:0/4136855511 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f40f014fb30 msgr2=0x7f40f01f3ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 20 2024-05-09T08:06:43.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.652+0000 7f40eea00640 1 -- 172.21.15.1:0/4136855511 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f40f014fb30 msgr2=0x7f40f01f3ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-09T08:06:43.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.652+0000 7f40eea00640 1 --2- 172.21.15.1:0/4136855511 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f40f014fb30 0x7f40f01f3ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-09T08:06:43.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.652+0000 7f40eea00640 1 --2- 172.21.15.1:0/4136855511 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f40f014fb30 0x7f40f01f3ab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T08:06:43.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.652+0000 7f40e7400640 1 --2- 172.21.15.1:0/4136855511 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f40cc06bf30 0x7f40cc06e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:06:43.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.653+0000 7f40e7400640 1 -- 172.21.15.1:0/4136855511 <== mon.1 v2:172.21.15.7:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7f40e808b4b0 con 0x7f40f014f6b0 2024-05-09T08:06:43.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.653+0000 7f40ee000640 1 --2- 172.21.15.1:0/4136855511 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f40cc06bf30 0x7f40cc06e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:06:43.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.654+0000 7f40ee000640 1 --2- 172.21.15.1:0/4136855511 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f40cc06bf30 0x7f40cc06e3f0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f40d8033000 tx=0x7f40d800a070 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:06:43.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.853+0000 7f40eea00640 1 --2- 172.21.15.1:0/4136855511 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f40f014fb30 0x7f40f01f3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:06:43.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.853+0000 7f40eea00640 1 -- 172.21.15.1:0/4136855511 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f40f014fb30 msgr2=0x7f40f01f3ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 21 2024-05-09T08:06:43.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.853+0000 7f40eea00640 1 -- 172.21.15.1:0/4136855511 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f40f014fb30 msgr2=0x7f40f01f3ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-09T08:06:43.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.853+0000 7f40eea00640 1 --2- 172.21.15.1:0/4136855511 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f40f014fb30 0x7f40f01f3ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-09T08:06:43.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:43.853+0000 7f40eea00640 1 --2- 172.21.15.1:0/4136855511 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f40f014fb30 0x7f40f01f3ab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-09T08:06:44.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:44.254+0000 7f40eea00640 1 --2- 172.21.15.1:0/4136855511 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f40f014fb30 0x7f40f01f3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:06:44.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:44.254+0000 7f40eea00640 1 -- 172.21.15.1:0/4136855511 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f40f0074d70 con 0x7f40f014fb30 2024-05-09T08:06:44.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:44.255+0000 7f40eea00640 1 --2- 172.21.15.1:0/4136855511 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f40f014fb30 0x7f40f01f3ab0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f40f0050ef0 tx=0x7f40e0003d20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:06:44.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:44.255+0000 7f40e7400640 1 -- 172.21.15.1:0/4136855511 <== mon.2 v2:172.21.15.7:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40e0007b10 con 0x7f40f014fb30 2024-05-09T08:06:44.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:44.255+0000 7f40e7400640 1 -- 172.21.15.1:0/4136855511 <== mon.2 v2:172.21.15.7:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f40e00079e0 con 0x7f40f014fb30 2024-05-09T08:06:44.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:44.255+0000 7f40e7400640 1 -- 172.21.15.1:0/4136855511 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f40f014fb30 msgr2=0x7f40f01f3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:06:44.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:44.255+0000 7f40e7400640 1 --2- 172.21.15.1:0/4136855511 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f40f014fb30 0x7f40f01f3ab0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f40f0050ef0 tx=0x7f40e0003d20 comp rx=0 tx=0).stop 2024-05-09T08:06:44.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:44.255+0000 7f40e7400640 1 -- 172.21.15.1:0/4136855511 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f40f014fb30 msgr2=0x7f40f01f3ab0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-05-09T08:06:44.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:44.255+0000 7f40e7400640 1 --2- 172.21.15.1:0/4136855511 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f40f014fb30 0x7f40f01f3ab0 secure :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f40f0050ef0 tx=0x7f40e0003d20 comp rx=0 tx=0).stop 2024-05-09T08:06:44.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:44.266+0000 7f40f5e00640 1 --2- 172.21.15.1:0/4136855511 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f40f01f0520 0x7f40f01f3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:06:44.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:44.266+0000 7f40f5e00640 1 -- 172.21.15.1:0/4136855511 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f40f01527a0 con 0x7f40f01f0520 2024-05-09T08:06:44.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:44.267+0000 7f40eea00640 1 --2- 172.21.15.1:0/4136855511 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f40f01f0520 0x7f40f01f3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:06:44.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:44.267+0000 7f40eea00640 1 -- 172.21.15.1:0/4136855511 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f40f006b570 con 0x7f40f01f0520 2024-05-09T08:06:44.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:44.267+0000 7f40eea00640 1 --2- 172.21.15.1:0/4136855511 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f40f01f0520 0x7f40f01f3ff0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f40e0001760 tx=0x7f40e0001df0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:06:44.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:44.268+0000 7f40e7400640 1 -- 172.21.15.1:0/4136855511 <== mon.2 v2:172.21.15.7:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40e00169a0 con 0x7f40f01f0520 2024-05-09T08:06:44.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:44.268+0000 7f40e7400640 1 -- 172.21.15.1:0/4136855511 <== mon.2 v2:172.21.15.7:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4534 (secure 0 0 0) 0x7f40e00169a0 con 0x7f40f01f0520 2024-05-09T08:06:44.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:44.268+0000 7f40e7400640 1 -- 172.21.15.1:0/4136855511 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f40f01f0520 msgr2=0x7f40f01f3ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:06:44.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:44.268+0000 7f40e7400640 1 --2- 172.21.15.1:0/4136855511 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f40f01f0520 0x7f40f01f3ff0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f40e0001760 tx=0x7f40e0001df0 comp rx=0 tx=0).stop 2024-05-09T08:06:44.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:44.268+0000 7f40e7400640 1 -- 172.21.15.1:0/4136855511 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f40f01f0520 msgr2=0x7f40f01f3ff0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-05-09T08:06:44.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:44.268+0000 7f40e7400640 1 --2- 172.21.15.1:0/4136855511 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f40f01f0520 0x7f40f01f3ff0 secure :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f40e0001760 tx=0x7f40e0001df0 comp rx=0 tx=0).stop 2024-05-09T08:06:44.280 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-05-09T08:06:44.280 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "c", 2024-05-09T08:06:44.280 INFO:teuthology.orchestra.run.smithi001.stdout: "rank": 2, 2024-05-09T08:06:44.281 INFO:teuthology.orchestra.run.smithi001.stdout: "state": "peon", 2024-05-09T08:06:44.281 INFO:teuthology.orchestra.run.smithi001.stdout: "election_epoch": 36, 2024-05-09T08:06:44.281 INFO:teuthology.orchestra.run.smithi001.stdout: "uptime": 10194, 2024-05-09T08:06:44.281 INFO:teuthology.orchestra.run.smithi001.stdout: "quorum": [ 2024-05-09T08:06:44.281 INFO:teuthology.orchestra.run.smithi001.stdout: 0, 2024-05-09T08:06:44.281 INFO:teuthology.orchestra.run.smithi001.stdout: 1, 2024-05-09T08:06:44.281 INFO:teuthology.orchestra.run.smithi001.stdout: 2 2024-05-09T08:06:44.281 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:06:44.281 INFO:teuthology.orchestra.run.smithi001.stdout: "quorum_age": 5, 2024-05-09T08:06:44.281 INFO:teuthology.orchestra.run.smithi001.stdout: "features": { 2024-05-09T08:06:44.281 INFO:teuthology.orchestra.run.smithi001.stdout: "required_con": "2450521708007866372", 2024-05-09T08:06:44.281 INFO:teuthology.orchestra.run.smithi001.stdout: "required_mon": [ 2024-05-09T08:06:44.282 INFO:teuthology.orchestra.run.smithi001.stdout: "kraken", 2024-05-09T08:06:44.282 INFO:teuthology.orchestra.run.smithi001.stdout: "luminous", 2024-05-09T08:06:44.282 INFO:teuthology.orchestra.run.smithi001.stdout: "mimic", 2024-05-09T08:06:44.282 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap-prune", 2024-05-09T08:06:44.282 INFO:teuthology.orchestra.run.smithi001.stdout: "nautilus", 2024-05-09T08:06:44.282 INFO:teuthology.orchestra.run.smithi001.stdout: "octopus", 2024-05-09T08:06:44.282 INFO:teuthology.orchestra.run.smithi001.stdout: "pacific", 2024-05-09T08:06:44.282 INFO:teuthology.orchestra.run.smithi001.stdout: "elector-pinging", 2024-05-09T08:06:44.282 INFO:teuthology.orchestra.run.smithi001.stdout: "quincy", 2024-05-09T08:06:44.282 INFO:teuthology.orchestra.run.smithi001.stdout: "reef", 2024-05-09T08:06:44.282 INFO:teuthology.orchestra.run.smithi001.stdout: "squid" 2024-05-09T08:06:44.282 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:06:44.282 INFO:teuthology.orchestra.run.smithi001.stdout: "quorum_con": "4540701547738038271", 2024-05-09T08:06:44.283 INFO:teuthology.orchestra.run.smithi001.stdout: "quorum_mon": [ 2024-05-09T08:06:44.283 INFO:teuthology.orchestra.run.smithi001.stdout: "kraken", 2024-05-09T08:06:44.283 INFO:teuthology.orchestra.run.smithi001.stdout: "luminous", 2024-05-09T08:06:44.283 INFO:teuthology.orchestra.run.smithi001.stdout: "mimic", 2024-05-09T08:06:44.283 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap-prune", 2024-05-09T08:06:44.283 INFO:teuthology.orchestra.run.smithi001.stdout: "nautilus", 2024-05-09T08:06:44.283 INFO:teuthology.orchestra.run.smithi001.stdout: "octopus", 2024-05-09T08:06:44.283 INFO:teuthology.orchestra.run.smithi001.stdout: "pacific", 2024-05-09T08:06:44.283 INFO:teuthology.orchestra.run.smithi001.stdout: "elector-pinging", 2024-05-09T08:06:44.283 INFO:teuthology.orchestra.run.smithi001.stdout: "quincy", 2024-05-09T08:06:44.283 INFO:teuthology.orchestra.run.smithi001.stdout: "reef", 2024-05-09T08:06:44.283 INFO:teuthology.orchestra.run.smithi001.stdout: "squid" 2024-05-09T08:06:44.283 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:06:44.284 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:06:44.284 INFO:teuthology.orchestra.run.smithi001.stdout: "outside_quorum": [], 2024-05-09T08:06:44.284 INFO:teuthology.orchestra.run.smithi001.stdout: "extra_probe_peers": [], 2024-05-09T08:06:44.284 INFO:teuthology.orchestra.run.smithi001.stdout: "sync_provider": [], 2024-05-09T08:06:44.284 INFO:teuthology.orchestra.run.smithi001.stdout: "monmap": { 2024-05-09T08:06:44.284 INFO:teuthology.orchestra.run.smithi001.stdout: "epoch": 1, 2024-05-09T08:06:44.284 INFO:teuthology.orchestra.run.smithi001.stdout: "fsid": "54329ec8-3270-4043-9a82-c6cd7420aa7f", 2024-05-09T08:06:44.284 INFO:teuthology.orchestra.run.smithi001.stdout: "modified": "2024-05-09T08:01:21.819270Z", 2024-05-09T08:06:44.284 INFO:teuthology.orchestra.run.smithi001.stdout: "created": "2024-05-09T08:01:21.819270Z", 2024-05-09T08:06:44.284 INFO:teuthology.orchestra.run.smithi001.stdout: "min_mon_release": 19, 2024-05-09T08:06:44.284 INFO:teuthology.orchestra.run.smithi001.stdout: "min_mon_release_name": "squid", 2024-05-09T08:06:44.284 INFO:teuthology.orchestra.run.smithi001.stdout: "election_strategy": 1, 2024-05-09T08:06:44.284 INFO:teuthology.orchestra.run.smithi001.stdout: "disallowed_leaders: ": "", 2024-05-09T08:06:44.285 INFO:teuthology.orchestra.run.smithi001.stdout: "stretch_mode": false, 2024-05-09T08:06:44.285 INFO:teuthology.orchestra.run.smithi001.stdout: "tiebreaker_mon": "", 2024-05-09T08:06:44.285 INFO:teuthology.orchestra.run.smithi001.stdout: "removed_ranks: ": "", 2024-05-09T08:06:44.285 INFO:teuthology.orchestra.run.smithi001.stdout: "features": { 2024-05-09T08:06:44.285 INFO:teuthology.orchestra.run.smithi001.stdout: "persistent": [ 2024-05-09T08:06:44.285 INFO:teuthology.orchestra.run.smithi001.stdout: "kraken", 2024-05-09T08:06:44.285 INFO:teuthology.orchestra.run.smithi001.stdout: "luminous", 2024-05-09T08:06:44.285 INFO:teuthology.orchestra.run.smithi001.stdout: "mimic", 2024-05-09T08:06:44.285 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap-prune", 2024-05-09T08:06:44.285 INFO:teuthology.orchestra.run.smithi001.stdout: "nautilus", 2024-05-09T08:06:44.285 INFO:teuthology.orchestra.run.smithi001.stdout: "octopus", 2024-05-09T08:06:44.285 INFO:teuthology.orchestra.run.smithi001.stdout: "pacific", 2024-05-09T08:06:44.285 INFO:teuthology.orchestra.run.smithi001.stdout: "elector-pinging", 2024-05-09T08:06:44.286 INFO:teuthology.orchestra.run.smithi001.stdout: "quincy", 2024-05-09T08:06:44.286 INFO:teuthology.orchestra.run.smithi001.stdout: "reef", 2024-05-09T08:06:44.286 INFO:teuthology.orchestra.run.smithi001.stdout: "squid" 2024-05-09T08:06:44.286 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:06:44.286 INFO:teuthology.orchestra.run.smithi001.stdout: "optional": [] 2024-05-09T08:06:44.286 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:06:44.286 INFO:teuthology.orchestra.run.smithi001.stdout: "mons": [ 2024-05-09T08:06:44.286 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:06:44.286 INFO:teuthology.orchestra.run.smithi001.stdout: "rank": 0, 2024-05-09T08:06:44.286 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "a", 2024-05-09T08:06:44.286 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-05-09T08:06:44.286 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-05-09T08:06:44.286 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:06:44.287 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-05-09T08:06:44.287 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:3300", 2024-05-09T08:06:44.287 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:06:44.287 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:06:44.287 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:06:44.287 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-05-09T08:06:44.287 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6789", 2024-05-09T08:06:44.287 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:06:44.287 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:06:44.287 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:06:44.287 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:06:44.287 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6789/0", 2024-05-09T08:06:44.287 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.1:6789/0", 2024-05-09T08:06:44.287 INFO:teuthology.orchestra.run.smithi001.stdout: "priority": 0, 2024-05-09T08:06:44.288 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 0, 2024-05-09T08:06:44.288 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_location": "{}" 2024-05-09T08:06:44.288 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:06:44.288 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:06:44.288 INFO:teuthology.orchestra.run.smithi001.stdout: "rank": 1, 2024-05-09T08:06:44.288 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "b", 2024-05-09T08:06:44.288 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-05-09T08:06:44.288 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-05-09T08:06:44.288 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:06:44.288 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-05-09T08:06:44.288 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:3300", 2024-05-09T08:06:44.288 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:06:44.288 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:06:44.289 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:06:44.289 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-05-09T08:06:44.289 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:6789", 2024-05-09T08:06:44.289 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:06:44.289 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:06:44.289 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:06:44.289 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:06:44.289 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:6789/0", 2024-05-09T08:06:44.289 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.7:6789/0", 2024-05-09T08:06:44.289 INFO:teuthology.orchestra.run.smithi001.stdout: "priority": 0, 2024-05-09T08:06:44.289 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 0, 2024-05-09T08:06:44.289 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_location": "{}" 2024-05-09T08:06:44.289 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:06:44.289 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:06:44.290 INFO:teuthology.orchestra.run.smithi001.stdout: "rank": 2, 2024-05-09T08:06:44.290 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "c", 2024-05-09T08:06:44.290 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-05-09T08:06:44.290 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-05-09T08:06:44.290 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:06:44.290 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-05-09T08:06:44.290 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:3301", 2024-05-09T08:06:44.290 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:06:44.290 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:06:44.290 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:06:44.290 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-05-09T08:06:44.290 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:6790", 2024-05-09T08:06:44.290 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:06:44.291 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:06:44.291 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:06:44.291 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:06:44.291 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:6790/0", 2024-05-09T08:06:44.291 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.7:6790/0", 2024-05-09T08:06:44.291 INFO:teuthology.orchestra.run.smithi001.stdout: "priority": 0, 2024-05-09T08:06:44.291 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 0, 2024-05-09T08:06:44.291 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_location": "{}" 2024-05-09T08:06:44.291 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:06:44.291 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:06:44.291 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:06:44.291 INFO:teuthology.orchestra.run.smithi001.stdout: "feature_map": { 2024-05-09T08:06:44.291 INFO:teuthology.orchestra.run.smithi001.stdout: "mon": [ 2024-05-09T08:06:44.292 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:06:44.292 INFO:teuthology.orchestra.run.smithi001.stdout: "features": "0x3f03cffffffdffff", 2024-05-09T08:06:44.292 INFO:teuthology.orchestra.run.smithi001.stdout: "release": "luminous", 2024-05-09T08:06:44.292 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 1 2024-05-09T08:06:44.292 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:06:44.292 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:06:44.292 INFO:teuthology.orchestra.run.smithi001.stdout: "client": [ 2024-05-09T08:06:44.292 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:06:44.292 INFO:teuthology.orchestra.run.smithi001.stdout: "features": "0x3f03cffffffdffff", 2024-05-09T08:06:44.292 INFO:teuthology.orchestra.run.smithi001.stdout: "release": "luminous", 2024-05-09T08:06:44.292 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 1 2024-05-09T08:06:44.292 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:06:44.292 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:06:44.292 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:06:44.293 INFO:teuthology.orchestra.run.smithi001.stdout: "stretch_mode": false 2024-05-09T08:06:44.293 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-05-09T08:06:44.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:44.269+0000 7f40f5e00640 1 -- 172.21.15.1:0/4136855511 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f40cc06bf30 msgr2=0x7f40cc06e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:06:44.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:44.269+0000 7f40f5e00640 1 --2- 172.21.15.1:0/4136855511 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f40cc06bf30 0x7f40cc06e3f0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f40d8033000 tx=0x7f40d800a070 comp rx=0 tx=0).stop 2024-05-09T08:06:44.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:44.269+0000 7f40f5e00640 1 -- 172.21.15.1:0/4136855511 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f40f014f6b0 msgr2=0x7f40f01f3570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:06:44.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:44.269+0000 7f40f5e00640 1 --2- 172.21.15.1:0/4136855511 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f40f014f6b0 0x7f40f01f3570 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f40f008d500 tx=0x7f40e800b7a0 comp rx=0 tx=0).stop 2024-05-09T08:06:44.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:44.269+0000 7f40ef400640 1 -- 172.21.15.1:0/4136855511 reap_dead start 2024-05-09T08:06:44.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:44.269+0000 7f40ef400640 1 -- 172.21.15.1:0/4136855511 reap_dead start 2024-05-09T08:06:44.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:44.269+0000 7f40f5e00640 1 -- 172.21.15.1:0/4136855511 shutdown_connections 2024-05-09T08:06:44.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:44.269+0000 7f40f5e00640 1 -- 172.21.15.1:0/4136855511 >> 172.21.15.1:0/4136855511 conn(0x7f40f00cfe80 msgr2=0x7f40f00d0260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:06:44.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:44.269+0000 7f40f5e00640 1 -- 172.21.15.1:0/4136855511 shutdown_connections 2024-05-09T08:06:44.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:44.269+0000 7f40f5e00640 1 -- 172.21.15.1:0/4136855511 wait complete. 2024-05-09T08:06:44.306 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-05-09T08:06:44.306 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-05-09T08:06:44.306 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-05-09T08:06:44.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:44.616+0000 7ff23d000640 1 Processor -- start 2024-05-09T08:06:44.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:44.617+0000 7ff23d000640 1 -- start start 2024-05-09T08:06:44.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:44.618+0000 7ff23d000640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff2381548e0 0x7ff2380cc400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:06:44.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:44.618+0000 7ff23d000640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7ff2380cb760 0x7ff23815ab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:06:44.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:44.618+0000 7ff23d000640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff2380cbb70 con 0x7ff2380cbf20 2024-05-09T08:06:44.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:44.618+0000 7ff23d000640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7ff23815ec60 con 0x7ff2380cb760 2024-05-09T08:06:44.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:44.618+0000 7ff23d000640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7ff23815e6f0 con 0x7ff2381548e0 2024-05-09T08:06:44.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:44.619+0000 7ff235400640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7ff2380cbf20 0x7ff23808bff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:58954/0 (socket says 172.21.15.1:58954) 2024-05-09T08:06:44.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:44.619+0000 7ff235400640 1 -- 172.21.15.1:0/178537970 learned_addr learned my addr 172.21.15.1:0/178537970 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T08:06:44.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:44.619+0000 7ff234a00640 1 -- 172.21.15.1:0/178537970 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 414108628 0 0) 0x7ff224001110 con 0x7ff2380cbf20 2024-05-09T08:06:44.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:44.619+0000 7ff234a00640 1 -- 172.21.15.1:0/178537970 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff21c003520 con 0x7ff2380cbf20 2024-05-09T08:06:44.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:44.620+0000 7ff234a00640 1 -- 172.21.15.1:0/178537970 <== mon.0 v1:172.21.15.1:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2330317009 0 0) 0x7ff21c003520 con 0x7ff2380cbf20 2024-05-09T08:06:44.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:44.620+0000 7ff234a00640 1 -- 172.21.15.1:0/178537970 >> v2:172.21.15.7:3300/0 conn(0x7ff2381548e0 msgr2=0x7ff2380cc400 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T08:06:44.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:44.620+0000 7ff234a00640 1 --2- 172.21.15.1:0/178537970 >> v2:172.21.15.7:3300/0 conn(0x7ff2381548e0 0x7ff2380cc400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:06:44.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:44.620+0000 7ff234a00640 1 -- 172.21.15.1:0/178537970 >> v2:172.21.15.1:3300/0 conn(0x7ff2380cb760 msgr2=0x7ff23815ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:06:44.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:44.620+0000 7ff234a00640 1 --2- 172.21.15.1:0/178537970 >> v2:172.21.15.1:3300/0 conn(0x7ff2380cb760 0x7ff23815ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:06:44.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:44.620+0000 7ff234a00640 1 -- 172.21.15.1:0/178537970 --> v1:172.21.15.1:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff23815e980 con 0x7ff2380cbf20 2024-05-09T08:06:44.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:44.620+0000 7ff234a00640 1 -- 172.21.15.1:0/178537970 <== mon.0 v1:172.21.15.1:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4179846320 0 0) 0x7ff224001840 con 0x7ff2380cbf20 2024-05-09T08:06:44.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:44.620+0000 7ff234a00640 1 -- 172.21.15.1:0/178537970 >> v1:172.21.15.1:6789/0 conn(0x7ff2380cbf20 legacy=0x7ff23808bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:06:44.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:44.620+0000 7ff234a00640 1 --2- 172.21.15.1:0/178537970 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ff21c003a00 0x7ff21c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:06:44.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:44.620+0000 7ff234a00640 1 --2- 172.21.15.1:0/178537970 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff21c004800 0x7ff21c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:06:44.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:44.620+0000 7ff234a00640 1 --2- 172.21.15.1:0/178537970 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff21c0053c0 0x7ff21c014210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:06:44.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:44.620+0000 7ff234a00640 1 -- 172.21.15.1:0/178537970 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ff23815ec60 con 0x7ff21c004800 2024-05-09T08:06:44.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:44.620+0000 7ff234a00640 1 -- 172.21.15.1:0/178537970 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff23815e6f0 con 0x7ff21c0053c0 2024-05-09T08:06:44.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:44.620+0000 7ff234a00640 1 -- 172.21.15.1:0/178537970 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7ff21c005a40 con 0x7ff21c003a00 2024-05-09T08:06:44.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:44.620+0000 7ff235400640 1 --2- 172.21.15.1:0/178537970 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff21c004800 0x7ff21c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:06:44.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:44.621+0000 7ff236800640 1 --2- 172.21.15.1:0/178537970 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff21c0053c0 0x7ff21c014210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:06:44.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:44.621+0000 7ff235400640 1 -- 172.21.15.1:0/178537970 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ff21c003a00 msgr2=0x7ff21c003e20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T08:06:44.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:44.621+0000 7ff235400640 1 --2- 172.21.15.1:0/178537970 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ff21c003a00 0x7ff21c003e20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:06:44.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:44.621+0000 7ff235400640 1 -- 172.21.15.1:0/178537970 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff21c0053c0 msgr2=0x7ff21c014210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:06:44.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:44.621+0000 7ff235400640 1 --2- 172.21.15.1:0/178537970 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff21c0053c0 0x7ff21c014210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:06:44.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:44.622+0000 7ff235400640 1 -- 172.21.15.1:0/178537970 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff21c003520 con 0x7ff21c004800 2024-05-09T08:06:44.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:44.622+0000 7ff236800640 1 -- 172.21.15.1:0/178537970 reap_dead start 2024-05-09T08:06:44.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:44.623+0000 7ff235400640 1 --2- 172.21.15.1:0/178537970 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff21c004800 0x7ff21c004c80 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff22400c320 tx=0x7ff21c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6025090abe8f956 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:06:44.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:44.623+0000 7ff234a00640 1 -- 172.21.15.1:0/178537970 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff224035070 con 0x7ff21c004800 2024-05-09T08:06:44.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:44.623+0000 7ff234a00640 1 -- 172.21.15.1:0/178537970 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff22402c040 con 0x7ff21c004800 2024-05-09T08:06:44.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:44.623+0000 7ff234a00640 1 -- 172.21.15.1:0/178537970 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff224007e60 con 0x7ff21c004800 2024-05-09T08:06:44.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:44.624+0000 7ff23d000640 1 -- 172.21.15.1:0/178537970 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff21c004800 msgr2=0x7ff21c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:06:44.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:44.624+0000 7ff23d000640 1 --2- 172.21.15.1:0/178537970 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff21c004800 0x7ff21c004c80 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff22400c320 tx=0x7ff21c005830 comp rx=0 tx=0).stop 2024-05-09T08:06:44.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:44.624+0000 7ff23d000640 1 -- 172.21.15.1:0/178537970 shutdown_connections 2024-05-09T08:06:44.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:44.624+0000 7ff23d000640 1 --2- 172.21.15.1:0/178537970 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff21c004800 0x7ff21c004c80 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:06:44.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:44.624+0000 7ff23d000640 1 -- 172.21.15.1:0/178537970 >> 172.21.15.1:0/178537970 conn(0x7ff238073010 msgr2=0x7ff23806c880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:06:44.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:44.624+0000 7ff23d000640 1 -- 172.21.15.1:0/178537970 shutdown_connections 2024-05-09T08:06:44.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:44.624+0000 7ff23d000640 1 -- 172.21.15.1:0/178537970 wait complete. 2024-05-09T08:06:44.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:44.626+0000 7ff23d000640 1 Processor -- start 2024-05-09T08:06:44.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:44.626+0000 7ff23d000640 1 -- start start 2024-05-09T08:06:44.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:44.627+0000 7ff23d000640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff21c004800 0x7ff2380c84f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:06:44.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:44.627+0000 7ff23d000640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ff2381efec0 0x7ff238147c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:06:44.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:44.627+0000 7ff23d000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff2380b4690 0x7ff2380b4b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:06:44.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:44.627+0000 7ff23d000640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ff238154cc0 con 0x7ff21c004800 2024-05-09T08:06:44.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:44.627+0000 7ff23d000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff2380cacf0 con 0x7ff2380b4690 2024-05-09T08:06:44.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:44.627+0000 7ff23d000640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7ff238154e40 con 0x7ff2381efec0 2024-05-09T08:06:44.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:44.627+0000 7ff235400640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ff2381efec0 0x7ff238147c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:06:44.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:44.628+0000 7ff236800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff2380b4690 0x7ff2380b4b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:06:44.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:44.628+0000 7ff235400640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ff2381efec0 0x7ff238147c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3301/0 says I am v2:172.21.15.1:43768/0 (socket says 172.21.15.1:43768) 2024-05-09T08:06:44.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:44.628+0000 7ff235400640 1 -- 172.21.15.1:0/1792475684 learned_addr learned my addr 172.21.15.1:0/1792475684 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:06:44.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:44.628+0000 7ff235e00640 1 --2- 172.21.15.1:0/1792475684 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff21c004800 0x7ff2380c84f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:06:44.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:44.629+0000 7ff236800640 1 -- 172.21.15.1:0/1792475684 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ff2381efec0 msgr2=0x7ff238147c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:06:44.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:44.629+0000 7ff236800640 1 --2- 172.21.15.1:0/1792475684 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ff2381efec0 0x7ff238147c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:06:44.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:44.629+0000 7ff236800640 1 -- 172.21.15.1:0/1792475684 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff21c004800 msgr2=0x7ff2380c84f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:06:44.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:44.629+0000 7ff236800640 1 --2- 172.21.15.1:0/1792475684 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff21c004800 0x7ff2380c84f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:06:44.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:44.629+0000 7ff236800640 1 -- 172.21.15.1:0/1792475684 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2380c7fe0 con 0x7ff2380b4690 2024-05-09T08:06:44.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:44.629+0000 7ff235e00640 1 --2- 172.21.15.1:0/1792475684 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff21c004800 0x7ff2380c84f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:06:44.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:44.629+0000 7ff236800640 1 --2- 172.21.15.1:0/1792475684 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff2380b4690 0x7ff2380b4b10 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7ff23808ca80 tx=0x7ff23000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:06:44.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:44.630+0000 7ff22e000640 1 -- 172.21.15.1:0/1792475684 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff230014070 con 0x7ff2380b4690 2024-05-09T08:06:44.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:44.630+0000 7ff22e000640 1 -- 172.21.15.1:0/1792475684 <== mon.1 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff230006e50 con 0x7ff2380b4690 2024-05-09T08:06:44.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:44.630+0000 7ff22e000640 1 -- 172.21.15.1:0/1792475684 <== mon.1 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff230008040 con 0x7ff2380b4690 2024-05-09T08:06:44.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:44.630+0000 7ff23d000640 1 -- 172.21.15.1:0/1792475684 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2380c82a0 con 0x7ff2380b4690 2024-05-09T08:06:44.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:44.630+0000 7ff23d000640 1 -- 172.21.15.1:0/1792475684 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2380d09c0 con 0x7ff2380b4690 2024-05-09T08:06:44.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:44.631+0000 7ff23d000640 1 -- 172.21.15.1:0/1792475684 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff238074d70 con 0x7ff2380b4690 2024-05-09T08:06:44.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:44.633+0000 7ff22e000640 1 -- 172.21.15.1:0/1792475684 <== mon.1 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff23001f060 con 0x7ff2380b4690 2024-05-09T08:06:44.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:44.634+0000 7ff22e000640 1 --2- 172.21.15.1:0/1792475684 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7ff21806be80 0x7ff21806e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:06:44.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:44.634+0000 7ff22e000640 1 -- 172.21.15.1:0/1792475684 <== mon.1 v2:172.21.15.7:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7ff23008b0b0 con 0x7ff2380b4690 2024-05-09T08:06:44.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:44.635+0000 7ff235e00640 1 --2- 172.21.15.1:0/1792475684 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7ff21806be80 0x7ff21806e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:06:44.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:44.635+0000 7ff235e00640 1 --2- 172.21.15.1:0/1792475684 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7ff21806be80 0x7ff21806e340 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7ff228006fd0 tx=0x7ff228009060 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:06:44.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:44.638+0000 7ff22e000640 1 -- 172.21.15.1:0/1792475684 <== mon.1 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ff2300595c0 con 0x7ff2380b4690 2024-05-09T08:06:45.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:45.222+0000 7ff23d000640 1 -- 172.21.15.1:0/1792475684 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "mon scrub"} v 0) -- 0x7ff2380b5350 con 0x7ff2380b4690 2024-05-09T08:06:45.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:45.226+0000 7ff22e000640 1 -- 172.21.15.1:0/1792475684 <== mon.1 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) ==== 57+0+0 (secure 0 0 0) 0x7ff230058ec0 con 0x7ff2380b4690 2024-05-09T08:06:45.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:45.232+0000 7ff23d000640 1 -- 172.21.15.1:0/1792475684 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7ff21806be80 msgr2=0x7ff21806e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:06:45.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:45.232+0000 7ff23d000640 1 --2- 172.21.15.1:0/1792475684 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7ff21806be80 0x7ff21806e340 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7ff228006fd0 tx=0x7ff228009060 comp rx=0 tx=0).stop 2024-05-09T08:06:45.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:45.232+0000 7ff23d000640 1 -- 172.21.15.1:0/1792475684 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff2380b4690 msgr2=0x7ff2380b4b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:06:45.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:45.232+0000 7ff23d000640 1 --2- 172.21.15.1:0/1792475684 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff2380b4690 0x7ff2380b4b10 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7ff23808ca80 tx=0x7ff23000b330 comp rx=0 tx=0).stop 2024-05-09T08:06:45.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:45.232+0000 7ff23d000640 1 -- 172.21.15.1:0/1792475684 shutdown_connections 2024-05-09T08:06:45.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:45.232+0000 7ff23d000640 1 --2- 172.21.15.1:0/1792475684 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7ff21806be80 0x7ff21806e340 secure :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7ff228006fd0 tx=0x7ff228009060 comp rx=0 tx=0).stop 2024-05-09T08:06:45.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:45.232+0000 7ff23d000640 1 --2- 172.21.15.1:0/1792475684 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff2380b4690 0x7ff2380b4b10 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:06:45.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:45.232+0000 7ff23d000640 1 --2- 172.21.15.1:0/1792475684 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ff2381efec0 0x7ff238147c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:06:45.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:45.232+0000 7ff23d000640 1 --2- 172.21.15.1:0/1792475684 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff21c004800 0x7ff2380c84f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:06:45.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:45.232+0000 7ff23d000640 1 -- 172.21.15.1:0/1792475684 >> 172.21.15.1:0/1792475684 conn(0x7ff238073010 msgr2=0x7ff2380b37c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:06:45.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:45.232+0000 7ff23d000640 1 -- 172.21.15.1:0/1792475684 shutdown_connections 2024-05-09T08:06:45.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:45.233+0000 7ff23d000640 1 -- 172.21.15.1:0/1792475684 wait complete. 2024-05-09T08:06:45.278 INFO:tasks.mon_thrash.mon_thrasher:waiting for 10.0 secs before continuing thrashing 2024-05-09T08:06:51.437 DEBUG:teuthology.orchestra.run.smithi001:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T08:06:51.441 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T08:06:53.027 INFO:tasks.workunit.client.0.smithi001.stdout: 2024-05-09T08:06:53.028 INFO:tasks.workunit.client.0.smithi001.stdout: Children see throughput for 1 rewriters = 253030.38 kB/sec 2024-05-09T08:06:53.028 INFO:tasks.workunit.client.0.smithi001.stdout: Parent sees throughput for 1 rewriters = 253023.77 kB/sec 2024-05-09T08:06:53.028 INFO:tasks.workunit.client.0.smithi001.stdout: Min throughput per process = 253030.38 kB/sec 2024-05-09T08:06:53.028 INFO:tasks.workunit.client.0.smithi001.stdout: Max throughput per process = 253030.38 kB/sec 2024-05-09T08:06:53.028 INFO:tasks.workunit.client.0.smithi001.stdout: Avg throughput per process = 253030.38 kB/sec 2024-05-09T08:06:53.028 INFO:tasks.workunit.client.0.smithi001.stdout: Min xfer = 10485760.00 kB 2024-05-09T08:06:55.281 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-05-09T08:06:55.282 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-05-09T08:06:55.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:55.489+0000 7f65c3000640 1 Processor -- start 2024-05-09T08:06:55.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:55.489+0000 7f65c3000640 1 -- start start 2024-05-09T08:06:55.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:55.491+0000 7f65c3000640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f65bc151660 0x7f65bc0c6c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:06:55.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:55.491+0000 7f65c3000640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f65bc0c1ba0 0x7f65bc072d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:06:55.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:55.491+0000 7f65c3000640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f65bc0c72f0 con 0x7f65bc0b1d30 2024-05-09T08:06:55.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:55.491+0000 7f65c3000640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f65bc151a40 con 0x7f65bc151660 2024-05-09T08:06:55.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:55.491+0000 7f65c3000640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f65bc157620 con 0x7f65bc0c1ba0 2024-05-09T08:06:55.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:55.492+0000 7f65bbe00640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f65bc151660 0x7f65bc0c6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:06:55.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:55.492+0000 7f65bb400640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f65bc0c1ba0 0x7f65bc072d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:06:55.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:55.492+0000 7f65bbe00640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f65bc151660 0x7f65bc0c6c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:36502/0 (socket says 172.21.15.1:36502) 2024-05-09T08:06:55.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:55.492+0000 7f65bbe00640 1 -- 172.21.15.1:0/2395468901 learned_addr learned my addr 172.21.15.1:0/2395468901 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:06:55.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:55.492+0000 7f65bb400640 1 -- 172.21.15.1:0/2395468901 >> v2:172.21.15.1:3300/0 conn(0x7f65bc151660 msgr2=0x7f65bc0c6c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:06:55.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:55.492+0000 7f65bb400640 1 --2- 172.21.15.1:0/2395468901 >> v2:172.21.15.1:3300/0 conn(0x7f65bc151660 0x7f65bc0c6c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:06:55.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:55.492+0000 7f65bb400640 1 -- 172.21.15.1:0/2395468901 >> v1:172.21.15.1:6789/0 conn(0x7f65bc0b1d30 legacy=0x7f65bc0c7a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:06:55.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:55.492+0000 7f65bb400640 1 -- 172.21.15.1:0/2395468901 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65bc157900 con 0x7f65bc0c1ba0 2024-05-09T08:06:55.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:55.493+0000 7f65bb400640 1 --2- 172.21.15.1:0/2395468901 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f65bc0c1ba0 0x7f65bc072d90 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f65a80097a0 tx=0x7f65a802a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ac1b0a4996344dc server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:06:55.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:55.494+0000 7f65baa00640 1 -- 172.21.15.1:0/2395468901 <== mon.2 v2:172.21.15.7:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65a8037070 con 0x7f65bc0c1ba0 2024-05-09T08:06:55.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:55.494+0000 7f65baa00640 1 -- 172.21.15.1:0/2395468901 <== mon.2 v2:172.21.15.7:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65a8032440 con 0x7f65bc0c1ba0 2024-05-09T08:06:55.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:55.494+0000 7f65baa00640 1 -- 172.21.15.1:0/2395468901 <== mon.2 v2:172.21.15.7:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65a8032810 con 0x7f65bc0c1ba0 2024-05-09T08:06:55.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:55.494+0000 7f65c3000640 1 -- 172.21.15.1:0/2395468901 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f65bc0c1ba0 msgr2=0x7f65bc072d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:06:55.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:55.494+0000 7f65c3000640 1 --2- 172.21.15.1:0/2395468901 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f65bc0c1ba0 0x7f65bc072d90 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f65a80097a0 tx=0x7f65a802a180 comp rx=0 tx=0).stop 2024-05-09T08:06:55.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:55.494+0000 7f65c3000640 1 -- 172.21.15.1:0/2395468901 shutdown_connections 2024-05-09T08:06:55.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:55.494+0000 7f65c3000640 1 --2- 172.21.15.1:0/2395468901 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f65bc0c1ba0 0x7f65bc072d90 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:06:55.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:55.494+0000 7f65c3000640 1 --2- 172.21.15.1:0/2395468901 >> v2:172.21.15.1:3300/0 conn(0x7f65bc151660 0x7f65bc0c6c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:06:55.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:55.494+0000 7f65c3000640 1 -- 172.21.15.1:0/2395468901 >> 172.21.15.1:0/2395468901 conn(0x7f65bc0cc070 msgr2=0x7f65bc0cc480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:06:55.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:55.495+0000 7f65c3000640 1 -- 172.21.15.1:0/2395468901 shutdown_connections 2024-05-09T08:06:55.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:55.495+0000 7f65c3000640 1 -- 172.21.15.1:0/2395468901 wait complete. 2024-05-09T08:06:55.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:55.496+0000 7f65c3000640 1 Processor -- start 2024-05-09T08:06:55.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:55.496+0000 7f65c3000640 1 -- start start 2024-05-09T08:06:55.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:55.497+0000 7f65c3000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65bc0b1d30 0x7f65bc14b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:06:55.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:55.497+0000 7f65c3000640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f65bc0c1ba0 0x7f65bc14fe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:06:55.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:55.497+0000 7f65c3000640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f65bc151660 0x7f65bc14ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:06:55.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:55.497+0000 7f65c3000640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f65bc1581a0 con 0x7f65bc0c1ba0 2024-05-09T08:06:55.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:55.497+0000 7f65c3000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f65bc157ec0 con 0x7f65bc0b1d30 2024-05-09T08:06:55.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:55.497+0000 7f65c3000640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f65bc1572d0 con 0x7f65bc151660 2024-05-09T08:06:55.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:55.498+0000 7f65bbe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65bc0b1d30 0x7f65bc14b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:06:55.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:55.498+0000 7f65bbe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65bc0b1d30 0x7f65bc14b8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.1:36838/0 (socket says 172.21.15.1:36838) 2024-05-09T08:06:55.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:55.498+0000 7f65bbe00640 1 -- 172.21.15.1:0/2608452230 learned_addr learned my addr 172.21.15.1:0/2608452230 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:06:55.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:55.498+0000 7f65c1000640 1 --2- 172.21.15.1:0/2608452230 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f65bc151660 0x7f65bc14ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:06:55.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:55.498+0000 7f65bb400640 1 --2- 172.21.15.1:0/2608452230 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f65bc0c1ba0 0x7f65bc14fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:06:55.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:55.498+0000 7f65bbe00640 1 -- 172.21.15.1:0/2608452230 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f65bc151660 msgr2=0x7f65bc14ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:06:55.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:55.498+0000 7f65bbe00640 1 --2- 172.21.15.1:0/2608452230 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f65bc151660 0x7f65bc14ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:06:55.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:55.498+0000 7f65bbe00640 1 -- 172.21.15.1:0/2608452230 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f65bc0c1ba0 msgr2=0x7f65bc14fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:06:55.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:55.498+0000 7f65bbe00640 1 --2- 172.21.15.1:0/2608452230 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f65bc0c1ba0 0x7f65bc14fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:06:55.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:55.498+0000 7f65bbe00640 1 -- 172.21.15.1:0/2608452230 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65bc1f4040 con 0x7f65bc0b1d30 2024-05-09T08:06:55.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:55.499+0000 7f65bbe00640 1 --2- 172.21.15.1:0/2608452230 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65bc0b1d30 0x7f65bc14b8a0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f65ac00e8d0 tx=0x7f65ac00ed90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:06:55.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:55.499+0000 7f65b3e00640 1 -- 172.21.15.1:0/2608452230 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65ac017070 con 0x7f65bc0b1d30 2024-05-09T08:06:55.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:55.499+0000 7f65b3e00640 1 -- 172.21.15.1:0/2608452230 <== mon.1 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65ac00c940 con 0x7f65bc0b1d30 2024-05-09T08:06:55.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:55.499+0000 7f65b3e00640 1 -- 172.21.15.1:0/2608452230 <== mon.1 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65ac00cd30 con 0x7f65bc0b1d30 2024-05-09T08:06:55.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:55.499+0000 7f65c3000640 1 -- 172.21.15.1:0/2608452230 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65bc158530 con 0x7f65bc0b1d30 2024-05-09T08:06:55.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:55.499+0000 7f65c3000640 1 -- 172.21.15.1:0/2608452230 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65bc1f42e0 con 0x7f65bc0b1d30 2024-05-09T08:06:55.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:55.500+0000 7f65c3000640 1 -- 172.21.15.1:0/2608452230 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65bc074d70 con 0x7f65bc0b1d30 2024-05-09T08:06:55.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:55.501+0000 7f65b3e00640 1 -- 172.21.15.1:0/2608452230 <== mon.1 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f65ac01a070 con 0x7f65bc0b1d30 2024-05-09T08:06:55.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:55.502+0000 7f65b3e00640 1 --2- 172.21.15.1:0/2608452230 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f659c06be30 0x7f659c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:06:55.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:55.502+0000 7f65b3e00640 1 -- 172.21.15.1:0/2608452230 <== mon.1 v2:172.21.15.7:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7f65ac08b0f0 con 0x7f65bc0b1d30 2024-05-09T08:06:55.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:55.502+0000 7f65bb400640 1 --2- 172.21.15.1:0/2608452230 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f659c06be30 0x7f659c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:06:55.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:55.503+0000 7f65bb400640 1 --2- 172.21.15.1:0/2608452230 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f659c06be30 0x7f659c06e2f0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f65a8034000 tx=0x7f65a802e000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:06:55.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:55.507+0000 7f65b3e00640 1 -- 172.21.15.1:0/2608452230 <== mon.1 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f65ac059050 con 0x7f65bc0b1d30 2024-05-09T08:06:55.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:55.787+0000 7f65c3000640 1 -- 172.21.15.1:0/2608452230 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f65bc0b1900 con 0x7f65bc0b1d30 2024-05-09T08:06:55.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:55.788+0000 7f65b3e00640 1 -- 172.21.15.1:0/2608452230 <== mon.1 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1545 (secure 0 0 0) 0x7f65ac05d350 con 0x7f65bc0b1d30 2024-05-09T08:06:55.800 INFO:teuthology.orchestra.run.smithi001.stdout:{"election_epoch":36,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":16,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"54329ec8-3270-4043-9a82-c6cd7420aa7f","modified":"2024-05-09T08:01:21.819270Z","created":"2024-05-09T08:01:21.819270Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:3300","nonce":0},{"type":"v1","addr":"172.21.15.1:6789","nonce":0}]},"addr":"172.21.15.1:6789/0","public_addr":"172.21.15.1:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:3300","nonce":0},{"type":"v1","addr":"172.21.15.7:6789","nonce":0}]},"addr":"172.21.15.7:6789/0","public_addr":"172.21.15.7:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:3301","nonce":0},{"type":"v1","addr":"172.21.15.7:6790","nonce":0}]},"addr":"172.21.15.7:6790/0","public_addr":"172.21.15.7:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-05-09T08:06:55.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:55.791+0000 7f65c3000640 1 -- 172.21.15.1:0/2608452230 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f659c06be30 msgr2=0x7f659c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:06:55.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:55.791+0000 7f65c3000640 1 --2- 172.21.15.1:0/2608452230 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f659c06be30 0x7f659c06e2f0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f65a8034000 tx=0x7f65a802e000 comp rx=0 tx=0).stop 2024-05-09T08:06:55.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:55.792+0000 7f65c3000640 1 -- 172.21.15.1:0/2608452230 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65bc0b1d30 msgr2=0x7f65bc14b8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:06:55.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:55.792+0000 7f65c3000640 1 --2- 172.21.15.1:0/2608452230 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65bc0b1d30 0x7f65bc14b8a0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f65ac00e8d0 tx=0x7f65ac00ed90 comp rx=0 tx=0).stop 2024-05-09T08:06:55.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:55.792+0000 7f65c3000640 1 -- 172.21.15.1:0/2608452230 shutdown_connections 2024-05-09T08:06:55.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:55.792+0000 7f65c3000640 1 --2- 172.21.15.1:0/2608452230 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f659c06be30 0x7f659c06e2f0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:06:55.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:55.792+0000 7f65c3000640 1 --2- 172.21.15.1:0/2608452230 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f65bc151660 0x7f65bc14ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:06:55.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:55.792+0000 7f65c3000640 1 --2- 172.21.15.1:0/2608452230 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f65bc0c1ba0 0x7f65bc14fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:06:55.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:55.792+0000 7f65c3000640 1 --2- 172.21.15.1:0/2608452230 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65bc0b1d30 0x7f65bc14b8a0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:06:55.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:55.792+0000 7f65c3000640 1 -- 172.21.15.1:0/2608452230 >> 172.21.15.1:0/2608452230 conn(0x7f65bc0cc070 msgr2=0x7f65bc14ada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:06:55.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:55.792+0000 7f65c3000640 1 -- 172.21.15.1:0/2608452230 shutdown_connections 2024-05-09T08:06:55.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:55.792+0000 7f65c3000640 1 -- 172.21.15.1:0/2608452230 wait complete. 2024-05-09T08:06:55.822 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-05-09T08:06:55.822 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-05-09T08:06:55.822 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-05-09T08:06:55.822 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-05-09T08:06:56.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.016+0000 7f12f2a00640 1 Processor -- start 2024-05-09T08:06:56.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.016+0000 7f12f2a00640 1 -- start start 2024-05-09T08:06:56.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.018+0000 7f12f2a00640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f12ec154950 0x7f12ec15e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:06:56.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.018+0000 7f12f2a00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f12ec0c5660 0x7f12ec0c5af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:06:56.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.018+0000 7f12f2a00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f12ec0c51b0 con 0x7f12ec06b930 2024-05-09T08:06:56.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.018+0000 7f12f2a00640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f12ec15ea10 con 0x7f12ec154950 2024-05-09T08:06:56.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.018+0000 7f12f2a00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f12ec15a4b0 con 0x7f12ec0c5660 2024-05-09T08:06:56.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.018+0000 7f12f1600640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f12ec154950 0x7f12ec15e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:06:56.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.018+0000 7f12f1600640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f12ec154950 0x7f12ec15e4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:36512/0 (socket says 172.21.15.1:36512) 2024-05-09T08:06:56.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.018+0000 7f12f1600640 1 -- 172.21.15.1:0/701961674 learned_addr learned my addr 172.21.15.1:0/701961674 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:06:56.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.018+0000 7f12f2000640 1 --2- 172.21.15.1:0/701961674 >> v2:172.21.15.7:3300/0 conn(0x7f12ec0c5660 0x7f12ec0c5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:06:56.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.019+0000 7f12f1600640 1 -- 172.21.15.1:0/701961674 >> v2:172.21.15.7:3300/0 conn(0x7f12ec0c5660 msgr2=0x7f12ec0c5af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:06:56.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.019+0000 7f12f1600640 1 --2- 172.21.15.1:0/701961674 >> v2:172.21.15.7:3300/0 conn(0x7f12ec0c5660 0x7f12ec0c5af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:06:56.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.019+0000 7f12f1600640 1 -- 172.21.15.1:0/701961674 >> v1:172.21.15.7:6789/0 conn(0x7f12ec06b930 legacy=0x7f12ec0cc940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:06:56.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.019+0000 7f12f1600640 1 -- 172.21.15.1:0/701961674 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12ec0c6030 con 0x7f12ec154950 2024-05-09T08:06:56.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.020+0000 7f12f1600640 1 --2- 172.21.15.1:0/701961674 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f12ec154950 0x7f12ec15e4d0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f12ec0c50d0 tx=0x7f12dc02a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=172031212872b18f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:06:56.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.020+0000 7f12ebe00640 1 -- 172.21.15.1:0/701961674 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12dc036070 con 0x7f12ec154950 2024-05-09T08:06:56.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.020+0000 7f12ebe00640 1 -- 172.21.15.1:0/701961674 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f12dc031600 con 0x7f12ec154950 2024-05-09T08:06:56.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.020+0000 7f12ebe00640 1 -- 172.21.15.1:0/701961674 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12dc0319d0 con 0x7f12ec154950 2024-05-09T08:06:56.033 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.021+0000 7f12f2a00640 1 -- 172.21.15.1:0/701961674 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f12ec154950 msgr2=0x7f12ec15e4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:06:56.033 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.021+0000 7f12f2a00640 1 --2- 172.21.15.1:0/701961674 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f12ec154950 0x7f12ec15e4d0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f12ec0c50d0 tx=0x7f12dc02a610 comp rx=0 tx=0).stop 2024-05-09T08:06:56.033 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.021+0000 7f12f2a00640 1 -- 172.21.15.1:0/701961674 shutdown_connections 2024-05-09T08:06:56.033 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.021+0000 7f12f2a00640 1 --2- 172.21.15.1:0/701961674 >> v2:172.21.15.7:3300/0 conn(0x7f12ec0c5660 0x7f12ec0c5af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:06:56.033 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.021+0000 7f12f2a00640 1 --2- 172.21.15.1:0/701961674 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f12ec154950 0x7f12ec15e4d0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:06:56.033 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.021+0000 7f12f2a00640 1 -- 172.21.15.1:0/701961674 >> 172.21.15.1:0/701961674 conn(0x7f12ec06c360 msgr2=0x7f12ec06c740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:06:56.033 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.021+0000 7f12f2a00640 1 -- 172.21.15.1:0/701961674 shutdown_connections 2024-05-09T08:06:56.033 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.021+0000 7f12f2a00640 1 -- 172.21.15.1:0/701961674 wait complete. 2024-05-09T08:06:56.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.023+0000 7f12f2a00640 1 Processor -- start 2024-05-09T08:06:56.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.023+0000 7f12f2a00640 1 -- start start 2024-05-09T08:06:56.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.024+0000 7f12f2a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f12ec06b930 0x7f12ec155ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:06:56.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.024+0000 7f12f2a00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f12ec0c5660 0x7f12ec1e4c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:06:56.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.024+0000 7f12f2a00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f12ec154950 0x7f12ec14ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:06:56.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.024+0000 7f12f2a00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f12ec15ad30 con 0x7f12ec0c5660 2024-05-09T08:06:56.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.024+0000 7f12f2a00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f12ec15aa50 con 0x7f12ec06b930 2024-05-09T08:06:56.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.024+0000 7f12f2a00640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f12ec15b010 con 0x7f12ec154950 2024-05-09T08:06:56.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.025+0000 7f12f1600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f12ec06b930 0x7f12ec155ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:06:56.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.025+0000 7f12f0c00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f12ec0c5660 0x7f12ec1e4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:06:56.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.025+0000 7f12f2000640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f12ec154950 0x7f12ec14ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:06:56.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.025+0000 7f12f1600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f12ec06b930 0x7f12ec155ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.1:36860/0 (socket says 172.21.15.1:36860) 2024-05-09T08:06:56.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.025+0000 7f12f1600640 1 -- 172.21.15.1:0/2877894894 learned_addr learned my addr 172.21.15.1:0/2877894894 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:06:56.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.025+0000 7f12f1600640 1 -- 172.21.15.1:0/2877894894 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f12ec154950 msgr2=0x7f12ec14ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:06:56.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.025+0000 7f12f1600640 1 --2- 172.21.15.1:0/2877894894 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f12ec154950 0x7f12ec14ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:06:56.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.025+0000 7f12f1600640 1 -- 172.21.15.1:0/2877894894 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f12ec0c5660 msgr2=0x7f12ec1e4c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:06:56.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.025+0000 7f12f1600640 1 --2- 172.21.15.1:0/2877894894 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f12ec0c5660 0x7f12ec1e4c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:06:56.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.025+0000 7f12f1600640 1 -- 172.21.15.1:0/2877894894 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12ec15b7a0 con 0x7f12ec06b930 2024-05-09T08:06:56.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.025+0000 7f12f1600640 1 --2- 172.21.15.1:0/2877894894 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f12ec06b930 0x7f12ec155ae0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f12dc02ac00 tx=0x7f12dc004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:06:56.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.026+0000 7f12e9600640 1 -- 172.21.15.1:0/2877894894 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12dc036030 con 0x7f12ec06b930 2024-05-09T08:06:56.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.026+0000 7f12e9600640 1 -- 172.21.15.1:0/2877894894 <== mon.1 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f12dc040070 con 0x7f12ec06b930 2024-05-09T08:06:56.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.026+0000 7f12e9600640 1 -- 172.21.15.1:0/2877894894 <== mon.1 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12dc009510 con 0x7f12ec06b930 2024-05-09T08:06:56.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.026+0000 7f12f2a00640 1 -- 172.21.15.1:0/2877894894 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12ec15ba80 con 0x7f12ec06b930 2024-05-09T08:06:56.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.026+0000 7f12f2a00640 1 -- 172.21.15.1:0/2877894894 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12ec1e5470 con 0x7f12ec06b930 2024-05-09T08:06:56.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.027+0000 7f12f2a00640 1 --2- 172.21.15.1:0/2877894894 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f12ec1e6110 0x7f12ec14f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:06:56.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.027+0000 7f12f2a00640 1 -- 172.21.15.1:0/2877894894 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f12ec1e5190 con 0x7f12ec1e6110 2024-05-09T08:06:56.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.027+0000 7f12f0c00640 1 --2- 172.21.15.1:0/2877894894 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f12ec1e6110 0x7f12ec14f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:06:56.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.028+0000 7f12f0c00640 1 -- 172.21.15.1:0/2877894894 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f12ec1e6670 con 0x7f12ec1e6110 2024-05-09T08:06:56.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.028+0000 7f12f0c00640 1 --2- 172.21.15.1:0/2877894894 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f12ec1e6110 0x7f12ec14f1a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f12d8007fd0 tx=0x7f12d800f9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:06:56.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.028+0000 7f12e9600640 1 -- 172.21.15.1:0/2877894894 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12d80052e0 con 0x7f12ec1e6110 2024-05-09T08:06:56.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.028+0000 7f12e9600640 1 -- 172.21.15.1:0/2877894894 <== mon.1 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f12dc007c50 con 0x7f12ec06b930 2024-05-09T08:06:56.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.029+0000 7f12e9600640 1 --2- 172.21.15.1:0/2877894894 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f12cc06bd30 0x7f12cc06e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:06:56.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.029+0000 7f12e9600640 1 -- 172.21.15.1:0/2877894894 <== mon.1 v2:172.21.15.7:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7f12dc0ab860 con 0x7f12ec06b930 2024-05-09T08:06:56.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.029+0000 7f12e9600640 1 -- 172.21.15.1:0/2877894894 <== mon.0 v2:172.21.15.1:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f12d8018070 con 0x7f12ec1e6110 2024-05-09T08:06:56.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.029+0000 7f12f2000640 1 --2- 172.21.15.1:0/2877894894 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f12cc06bd30 0x7f12cc06e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:06:56.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.029+0000 7f12e9600640 1 -- 172.21.15.1:0/2877894894 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f12ec1e6110 msgr2=0x7f12ec14f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:06:56.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.029+0000 7f12e9600640 1 --2- 172.21.15.1:0/2877894894 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f12ec1e6110 0x7f12ec14f1a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f12d8007fd0 tx=0x7f12d800f9b0 comp rx=0 tx=0).stop 2024-05-09T08:06:56.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.029+0000 7f12e9600640 1 -- 172.21.15.1:0/2877894894 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f12ec1e6110 msgr2=0x7f12ec14f1a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-05-09T08:06:56.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.029+0000 7f12e9600640 1 --2- 172.21.15.1:0/2877894894 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f12ec1e6110 0x7f12ec14f1a0 secure :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f12d8007fd0 tx=0x7f12d800f9b0 comp rx=0 tx=0).stop 2024-05-09T08:06:56.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.030+0000 7f12f2000640 1 --2- 172.21.15.1:0/2877894894 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f12cc06bd30 0x7f12cc06e1f0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f12e4009b40 tx=0x7f12e4009190 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:06:56.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.037+0000 7f12f2a00640 1 --2- 172.21.15.1:0/2877894894 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f12ec156250 0x7f12ec1f4670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:06:56.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.037+0000 7f12f2a00640 1 -- 172.21.15.1:0/2877894894 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f12ec1e42b0 con 0x7f12ec156250 2024-05-09T08:06:56.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.037+0000 7f12f0c00640 1 --2- 172.21.15.1:0/2877894894 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f12ec156250 0x7f12ec1f4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:06:56.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.037+0000 7f12f0c00640 1 -- 172.21.15.1:0/2877894894 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f12ec0c49d0 con 0x7f12ec156250 2024-05-09T08:06:56.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.038+0000 7f12f0c00640 1 --2- 172.21.15.1:0/2877894894 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f12ec156250 0x7f12ec1f4670 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f12d8007fd0 tx=0x7f12d8000f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:06:56.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.038+0000 7f12e9600640 1 -- 172.21.15.1:0/2877894894 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12d8003b20 con 0x7f12ec156250 2024-05-09T08:06:56.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.038+0000 7f12e9600640 1 -- 172.21.15.1:0/2877894894 <== mon.0 v2:172.21.15.1:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4877 (secure 0 0 0) 0x7f12d8003b20 con 0x7f12ec156250 2024-05-09T08:06:56.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.038+0000 7f12e9600640 1 -- 172.21.15.1:0/2877894894 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f12ec156250 msgr2=0x7f12ec1f4670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:06:56.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.038+0000 7f12e9600640 1 --2- 172.21.15.1:0/2877894894 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f12ec156250 0x7f12ec1f4670 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f12d8007fd0 tx=0x7f12d8000f20 comp rx=0 tx=0).stop 2024-05-09T08:06:56.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.038+0000 7f12e9600640 1 -- 172.21.15.1:0/2877894894 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f12ec156250 msgr2=0x7f12ec1f4670 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-05-09T08:06:56.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.038+0000 7f12e9600640 1 --2- 172.21.15.1:0/2877894894 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f12ec156250 0x7f12ec1f4670 secure :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f12d8007fd0 tx=0x7f12d8000f20 comp rx=0 tx=0).stop 2024-05-09T08:06:56.050 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-05-09T08:06:56.051 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "a", 2024-05-09T08:06:56.051 INFO:teuthology.orchestra.run.smithi001.stdout: "rank": 0, 2024-05-09T08:06:56.051 INFO:teuthology.orchestra.run.smithi001.stdout: "state": "leader", 2024-05-09T08:06:56.051 INFO:teuthology.orchestra.run.smithi001.stdout: "election_epoch": 36, 2024-05-09T08:06:56.051 INFO:teuthology.orchestra.run.smithi001.stdout: "uptime": 78654, 2024-05-09T08:06:56.051 INFO:teuthology.orchestra.run.smithi001.stdout: "quorum": [ 2024-05-09T08:06:56.051 INFO:teuthology.orchestra.run.smithi001.stdout: 0, 2024-05-09T08:06:56.051 INFO:teuthology.orchestra.run.smithi001.stdout: 1, 2024-05-09T08:06:56.051 INFO:teuthology.orchestra.run.smithi001.stdout: 2 2024-05-09T08:06:56.051 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:06:56.051 INFO:teuthology.orchestra.run.smithi001.stdout: "quorum_age": 16, 2024-05-09T08:06:56.051 INFO:teuthology.orchestra.run.smithi001.stdout: "features": { 2024-05-09T08:06:56.052 INFO:teuthology.orchestra.run.smithi001.stdout: "required_con": "2450521708007866372", 2024-05-09T08:06:56.052 INFO:teuthology.orchestra.run.smithi001.stdout: "required_mon": [ 2024-05-09T08:06:56.052 INFO:teuthology.orchestra.run.smithi001.stdout: "kraken", 2024-05-09T08:06:56.052 INFO:teuthology.orchestra.run.smithi001.stdout: "luminous", 2024-05-09T08:06:56.052 INFO:teuthology.orchestra.run.smithi001.stdout: "mimic", 2024-05-09T08:06:56.052 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap-prune", 2024-05-09T08:06:56.052 INFO:teuthology.orchestra.run.smithi001.stdout: "nautilus", 2024-05-09T08:06:56.052 INFO:teuthology.orchestra.run.smithi001.stdout: "octopus", 2024-05-09T08:06:56.052 INFO:teuthology.orchestra.run.smithi001.stdout: "pacific", 2024-05-09T08:06:56.052 INFO:teuthology.orchestra.run.smithi001.stdout: "elector-pinging", 2024-05-09T08:06:56.052 INFO:teuthology.orchestra.run.smithi001.stdout: "quincy", 2024-05-09T08:06:56.052 INFO:teuthology.orchestra.run.smithi001.stdout: "reef", 2024-05-09T08:06:56.052 INFO:teuthology.orchestra.run.smithi001.stdout: "squid" 2024-05-09T08:06:56.052 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:06:56.053 INFO:teuthology.orchestra.run.smithi001.stdout: "quorum_con": "4540701547738038271", 2024-05-09T08:06:56.053 INFO:teuthology.orchestra.run.smithi001.stdout: "quorum_mon": [ 2024-05-09T08:06:56.053 INFO:teuthology.orchestra.run.smithi001.stdout: "kraken", 2024-05-09T08:06:56.053 INFO:teuthology.orchestra.run.smithi001.stdout: "luminous", 2024-05-09T08:06:56.053 INFO:teuthology.orchestra.run.smithi001.stdout: "mimic", 2024-05-09T08:06:56.053 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap-prune", 2024-05-09T08:06:56.053 INFO:teuthology.orchestra.run.smithi001.stdout: "nautilus", 2024-05-09T08:06:56.053 INFO:teuthology.orchestra.run.smithi001.stdout: "octopus", 2024-05-09T08:06:56.053 INFO:teuthology.orchestra.run.smithi001.stdout: "pacific", 2024-05-09T08:06:56.053 INFO:teuthology.orchestra.run.smithi001.stdout: "elector-pinging", 2024-05-09T08:06:56.053 INFO:teuthology.orchestra.run.smithi001.stdout: "quincy", 2024-05-09T08:06:56.053 INFO:teuthology.orchestra.run.smithi001.stdout: "reef", 2024-05-09T08:06:56.053 INFO:teuthology.orchestra.run.smithi001.stdout: "squid" 2024-05-09T08:06:56.054 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:06:56.054 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:06:56.054 INFO:teuthology.orchestra.run.smithi001.stdout: "outside_quorum": [], 2024-05-09T08:06:56.054 INFO:teuthology.orchestra.run.smithi001.stdout: "extra_probe_peers": [], 2024-05-09T08:06:56.054 INFO:teuthology.orchestra.run.smithi001.stdout: "sync_provider": [], 2024-05-09T08:06:56.054 INFO:teuthology.orchestra.run.smithi001.stdout: "monmap": { 2024-05-09T08:06:56.054 INFO:teuthology.orchestra.run.smithi001.stdout: "epoch": 1, 2024-05-09T08:06:56.054 INFO:teuthology.orchestra.run.smithi001.stdout: "fsid": "54329ec8-3270-4043-9a82-c6cd7420aa7f", 2024-05-09T08:06:56.054 INFO:teuthology.orchestra.run.smithi001.stdout: "modified": "2024-05-09T08:01:21.819270Z", 2024-05-09T08:06:56.054 INFO:teuthology.orchestra.run.smithi001.stdout: "created": "2024-05-09T08:01:21.819270Z", 2024-05-09T08:06:56.054 INFO:teuthology.orchestra.run.smithi001.stdout: "min_mon_release": 19, 2024-05-09T08:06:56.054 INFO:teuthology.orchestra.run.smithi001.stdout: "min_mon_release_name": "squid", 2024-05-09T08:06:56.054 INFO:teuthology.orchestra.run.smithi001.stdout: "election_strategy": 1, 2024-05-09T08:06:56.055 INFO:teuthology.orchestra.run.smithi001.stdout: "disallowed_leaders: ": "", 2024-05-09T08:06:56.055 INFO:teuthology.orchestra.run.smithi001.stdout: "stretch_mode": false, 2024-05-09T08:06:56.055 INFO:teuthology.orchestra.run.smithi001.stdout: "tiebreaker_mon": "", 2024-05-09T08:06:56.055 INFO:teuthology.orchestra.run.smithi001.stdout: "removed_ranks: ": "", 2024-05-09T08:06:56.055 INFO:teuthology.orchestra.run.smithi001.stdout: "features": { 2024-05-09T08:06:56.055 INFO:teuthology.orchestra.run.smithi001.stdout: "persistent": [ 2024-05-09T08:06:56.055 INFO:teuthology.orchestra.run.smithi001.stdout: "kraken", 2024-05-09T08:06:56.055 INFO:teuthology.orchestra.run.smithi001.stdout: "luminous", 2024-05-09T08:06:56.055 INFO:teuthology.orchestra.run.smithi001.stdout: "mimic", 2024-05-09T08:06:56.056 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap-prune", 2024-05-09T08:06:56.056 INFO:teuthology.orchestra.run.smithi001.stdout: "nautilus", 2024-05-09T08:06:56.056 INFO:teuthology.orchestra.run.smithi001.stdout: "octopus", 2024-05-09T08:06:56.056 INFO:teuthology.orchestra.run.smithi001.stdout: "pacific", 2024-05-09T08:06:56.056 INFO:teuthology.orchestra.run.smithi001.stdout: "elector-pinging", 2024-05-09T08:06:56.056 INFO:teuthology.orchestra.run.smithi001.stdout: "quincy", 2024-05-09T08:06:56.056 INFO:teuthology.orchestra.run.smithi001.stdout: "reef", 2024-05-09T08:06:56.056 INFO:teuthology.orchestra.run.smithi001.stdout: "squid" 2024-05-09T08:06:56.056 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:06:56.056 INFO:teuthology.orchestra.run.smithi001.stdout: "optional": [] 2024-05-09T08:06:56.056 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:06:56.056 INFO:teuthology.orchestra.run.smithi001.stdout: "mons": [ 2024-05-09T08:06:56.056 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:06:56.057 INFO:teuthology.orchestra.run.smithi001.stdout: "rank": 0, 2024-05-09T08:06:56.057 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "a", 2024-05-09T08:06:56.057 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-05-09T08:06:56.057 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-05-09T08:06:56.057 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:06:56.057 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-05-09T08:06:56.057 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:3300", 2024-05-09T08:06:56.057 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:06:56.057 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:06:56.057 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:06:56.057 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-05-09T08:06:56.057 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6789", 2024-05-09T08:06:56.057 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:06:56.057 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:06:56.058 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:06:56.058 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:06:56.058 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6789/0", 2024-05-09T08:06:56.058 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.1:6789/0", 2024-05-09T08:06:56.058 INFO:teuthology.orchestra.run.smithi001.stdout: "priority": 0, 2024-05-09T08:06:56.058 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 0, 2024-05-09T08:06:56.058 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_location": "{}" 2024-05-09T08:06:56.058 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:06:56.058 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:06:56.058 INFO:teuthology.orchestra.run.smithi001.stdout: "rank": 1, 2024-05-09T08:06:56.058 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "b", 2024-05-09T08:06:56.058 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-05-09T08:06:56.058 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-05-09T08:06:56.059 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:06:56.059 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-05-09T08:06:56.059 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:3300", 2024-05-09T08:06:56.059 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:06:56.059 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:06:56.059 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:06:56.059 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-05-09T08:06:56.059 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:6789", 2024-05-09T08:06:56.059 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:06:56.059 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:06:56.059 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:06:56.059 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:06:56.059 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:6789/0", 2024-05-09T08:06:56.060 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.7:6789/0", 2024-05-09T08:06:56.060 INFO:teuthology.orchestra.run.smithi001.stdout: "priority": 0, 2024-05-09T08:06:56.060 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 0, 2024-05-09T08:06:56.060 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_location": "{}" 2024-05-09T08:06:56.060 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:06:56.060 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:06:56.060 INFO:teuthology.orchestra.run.smithi001.stdout: "rank": 2, 2024-05-09T08:06:56.060 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "c", 2024-05-09T08:06:56.060 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-05-09T08:06:56.060 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-05-09T08:06:56.060 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:06:56.060 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-05-09T08:06:56.060 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:3301", 2024-05-09T08:06:56.060 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:06:56.061 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:06:56.061 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:06:56.061 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-05-09T08:06:56.061 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:6790", 2024-05-09T08:06:56.061 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:06:56.061 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:06:56.061 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:06:56.061 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:06:56.061 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:6790/0", 2024-05-09T08:06:56.061 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.7:6790/0", 2024-05-09T08:06:56.061 INFO:teuthology.orchestra.run.smithi001.stdout: "priority": 0, 2024-05-09T08:06:56.061 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 0, 2024-05-09T08:06:56.061 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_location": "{}" 2024-05-09T08:06:56.062 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:06:56.062 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:06:56.062 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:06:56.062 INFO:teuthology.orchestra.run.smithi001.stdout: "feature_map": { 2024-05-09T08:06:56.062 INFO:teuthology.orchestra.run.smithi001.stdout: "mon": [ 2024-05-09T08:06:56.062 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:06:56.062 INFO:teuthology.orchestra.run.smithi001.stdout: "features": "0x3f03cffffffdffff", 2024-05-09T08:06:56.062 INFO:teuthology.orchestra.run.smithi001.stdout: "release": "luminous", 2024-05-09T08:06:56.062 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 1 2024-05-09T08:06:56.062 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:06:56.062 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:06:56.062 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": [ 2024-05-09T08:06:56.062 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:06:56.062 INFO:teuthology.orchestra.run.smithi001.stdout: "features": "0x3f03cffffffdffff", 2024-05-09T08:06:56.063 INFO:teuthology.orchestra.run.smithi001.stdout: "release": "luminous", 2024-05-09T08:06:56.063 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 1 2024-05-09T08:06:56.063 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:06:56.063 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:06:56.063 INFO:teuthology.orchestra.run.smithi001.stdout: "client": [ 2024-05-09T08:06:56.063 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:06:56.063 INFO:teuthology.orchestra.run.smithi001.stdout: "features": "0x3f03cffffffdffff", 2024-05-09T08:06:56.063 INFO:teuthology.orchestra.run.smithi001.stdout: "release": "luminous", 2024-05-09T08:06:56.063 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 1 2024-05-09T08:06:56.063 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:06:56.063 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:06:56.063 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr": [ 2024-05-09T08:06:56.063 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:06:56.064 INFO:teuthology.orchestra.run.smithi001.stdout: "features": "0x3f03cffffffdffff", 2024-05-09T08:06:56.064 INFO:teuthology.orchestra.run.smithi001.stdout: "release": "luminous", 2024-05-09T08:06:56.064 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 1 2024-05-09T08:06:56.064 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:06:56.064 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:06:56.064 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:06:56.064 INFO:teuthology.orchestra.run.smithi001.stdout: "stretch_mode": false 2024-05-09T08:06:56.064 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-05-09T08:06:56.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.039+0000 7f12f2a00640 1 -- 172.21.15.1:0/2877894894 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f12cc06bd30 msgr2=0x7f12cc06e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:06:56.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.039+0000 7f12f2a00640 1 --2- 172.21.15.1:0/2877894894 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f12cc06bd30 0x7f12cc06e1f0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f12e4009b40 tx=0x7f12e4009190 comp rx=0 tx=0).stop 2024-05-09T08:06:56.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.039+0000 7f12f2a00640 1 -- 172.21.15.1:0/2877894894 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f12ec06b930 msgr2=0x7f12ec155ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:06:56.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.039+0000 7f12f2a00640 1 --2- 172.21.15.1:0/2877894894 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f12ec06b930 0x7f12ec155ae0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f12dc02ac00 tx=0x7f12dc004a40 comp rx=0 tx=0).stop 2024-05-09T08:06:56.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.039+0000 7f12f2000640 1 -- 172.21.15.1:0/2877894894 reap_dead start 2024-05-09T08:06:56.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.039+0000 7f12f2000640 1 -- 172.21.15.1:0/2877894894 reap_dead start 2024-05-09T08:06:56.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.039+0000 7f12f2a00640 1 -- 172.21.15.1:0/2877894894 shutdown_connections 2024-05-09T08:06:56.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.039+0000 7f12f2a00640 1 -- 172.21.15.1:0/2877894894 >> 172.21.15.1:0/2877894894 conn(0x7f12ec06c360 msgr2=0x7f12ec1524b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:06:56.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.039+0000 7f12f2a00640 1 -- 172.21.15.1:0/2877894894 shutdown_connections 2024-05-09T08:06:56.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.039+0000 7f12f2a00640 1 -- 172.21.15.1:0/2877894894 wait complete. 2024-05-09T08:06:56.069 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-05-09T08:06:56.069 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-05-09T08:06:56.069 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-05-09T08:06:56.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.254+0000 7f7412200640 1 Processor -- start 2024-05-09T08:06:56.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.255+0000 7f7412200640 1 -- start start 2024-05-09T08:06:56.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.256+0000 7f7412200640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f740c0c58d0 0x7f740c1589e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:06:56.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.256+0000 7f7412200640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f740c0c6470 0x7f740c158f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:06:56.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.256+0000 7f7412200640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f740c0718e0 con 0x7f740c0c54f0 2024-05-09T08:06:56.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.256+0000 7f7412200640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f740c0726e0 con 0x7f740c0c6470 2024-05-09T08:06:56.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.256+0000 7f7412200640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f740c070040 con 0x7f740c0c58d0 2024-05-09T08:06:56.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.257+0000 7f7410e00640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f740c0c54f0 0x7f740c06f930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:55376/0 (socket says 172.21.15.1:55376) 2024-05-09T08:06:56.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.257+0000 7f7410e00640 1 -- 172.21.15.1:0/2452461635 learned_addr learned my addr 172.21.15.1:0/2452461635 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T08:06:56.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.257+0000 7f7411800640 1 --2- 172.21.15.1:0/2452461635 >> v2:172.21.15.1:3300/0 conn(0x7f740c0c6470 0x7f740c158f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:06:56.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.257+0000 7f740be00640 1 --2- 172.21.15.1:0/2452461635 >> v2:172.21.15.7:3300/0 conn(0x7f740c0c58d0 0x7f740c1589e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:06:56.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.257+0000 7f7411800640 1 -- 172.21.15.1:0/2452461635 >> v2:172.21.15.7:3300/0 conn(0x7f740c0c58d0 msgr2=0x7f740c1589e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:06:56.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.257+0000 7f7411800640 1 --2- 172.21.15.1:0/2452461635 >> v2:172.21.15.7:3300/0 conn(0x7f740c0c58d0 0x7f740c1589e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:06:56.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.257+0000 7f7411800640 1 -- 172.21.15.1:0/2452461635 >> v1:172.21.15.1:6789/0 conn(0x7f740c0c54f0 legacy=0x7f740c06f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:06:56.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.257+0000 7f7411800640 1 -- 172.21.15.1:0/2452461635 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f740c071b60 con 0x7f740c0c6470 2024-05-09T08:06:56.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.259+0000 7f7411800640 1 --2- 172.21.15.1:0/2452461635 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f740c0c6470 0x7f740c158f20 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f740c0b2200 tx=0x7f740402a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f26e92a6a9056270 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:06:56.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.259+0000 7f740b400640 1 -- 172.21.15.1:0/2452461635 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7404007740 con 0x7f740c0c6470 2024-05-09T08:06:56.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.259+0000 7f740b400640 1 -- 172.21.15.1:0/2452461635 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74040078e0 con 0x7f740c0c6470 2024-05-09T08:06:56.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.259+0000 7f740b400640 1 -- 172.21.15.1:0/2452461635 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7404007cb0 con 0x7f740c0c6470 2024-05-09T08:06:56.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.260+0000 7f7412200640 1 -- 172.21.15.1:0/2452461635 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f740c0c6470 msgr2=0x7f740c158f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:06:56.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.260+0000 7f7412200640 1 --2- 172.21.15.1:0/2452461635 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f740c0c6470 0x7f740c158f20 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f740c0b2200 tx=0x7f740402a680 comp rx=0 tx=0).stop 2024-05-09T08:06:56.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.260+0000 7f7412200640 1 -- 172.21.15.1:0/2452461635 shutdown_connections 2024-05-09T08:06:56.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.260+0000 7f7412200640 1 --2- 172.21.15.1:0/2452461635 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f740c0c6470 0x7f740c158f20 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:06:56.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.260+0000 7f7412200640 1 --2- 172.21.15.1:0/2452461635 >> v2:172.21.15.7:3300/0 conn(0x7f740c0c58d0 0x7f740c1589e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:06:56.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.260+0000 7f7412200640 1 -- 172.21.15.1:0/2452461635 >> 172.21.15.1:0/2452461635 conn(0x7f740c06b250 msgr2=0x7f740c06b630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:06:56.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.260+0000 7f7412200640 1 -- 172.21.15.1:0/2452461635 shutdown_connections 2024-05-09T08:06:56.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.260+0000 7f7412200640 1 -- 172.21.15.1:0/2452461635 wait complete. 2024-05-09T08:06:56.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.262+0000 7f7412200640 1 Processor -- start 2024-05-09T08:06:56.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.262+0000 7f7412200640 1 -- start start 2024-05-09T08:06:56.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.263+0000 7f7412200640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f740c0c54f0 0x7f740c14d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:06:56.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.263+0000 7f7412200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f740c0c58d0 0x7f740c154360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:06:56.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.263+0000 7f7412200640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f740c0c6470 0x7f740c1f3960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:06:56.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.263+0000 7f7412200640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f740c072400 con 0x7f740c0c6470 2024-05-09T08:06:56.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.263+0000 7f7412200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f740c072120 con 0x7f740c0c58d0 2024-05-09T08:06:56.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.263+0000 7f7412200640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f740c1584c0 con 0x7f740c0c54f0 2024-05-09T08:06:56.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.263+0000 7f740be00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f740c0c58d0 0x7f740c154360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:06:56.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.264+0000 7f7410e00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f740c0c54f0 0x7f740c14d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:06:56.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.264+0000 7f740be00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f740c0c58d0 0x7f740c154360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.1:36878/0 (socket says 172.21.15.1:36878) 2024-05-09T08:06:56.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.264+0000 7f740be00640 1 -- 172.21.15.1:0/2854177051 learned_addr learned my addr 172.21.15.1:0/2854177051 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:06:56.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.264+0000 7f7411800640 1 --2- 172.21.15.1:0/2854177051 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f740c0c6470 0x7f740c1f3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:06:56.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.264+0000 7f740be00640 1 -- 172.21.15.1:0/2854177051 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f740c0c54f0 msgr2=0x7f740c14d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:06:56.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.264+0000 7f740be00640 1 --2- 172.21.15.1:0/2854177051 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f740c0c54f0 0x7f740c14d790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:06:56.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.264+0000 7f740be00640 1 -- 172.21.15.1:0/2854177051 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f740c0c6470 msgr2=0x7f740c1f3960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:06:56.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.264+0000 7f740be00640 1 --2- 172.21.15.1:0/2854177051 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f740c0c6470 0x7f740c1f3960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:06:56.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.264+0000 7f740be00640 1 -- 172.21.15.1:0/2854177051 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f740c14e020 con 0x7f740c0c58d0 2024-05-09T08:06:56.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.264+0000 7f7410e00640 1 --2- 172.21.15.1:0/2854177051 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f740c0c54f0 0x7f740c14d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:06:56.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.264+0000 7f740be00640 1 --2- 172.21.15.1:0/2854177051 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f740c0c58d0 0x7f740c154360 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f73f800e980 tx=0x7f73f800ee40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:06:56.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.265+0000 7f7408c00640 1 -- 172.21.15.1:0/2854177051 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73f8017070 con 0x7f740c0c58d0 2024-05-09T08:06:56.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.265+0000 7f7408c00640 1 -- 172.21.15.1:0/2854177051 <== mon.1 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73f800ca00 con 0x7f740c0c58d0 2024-05-09T08:06:56.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.265+0000 7f7408c00640 1 -- 172.21.15.1:0/2854177051 <== mon.1 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73f800cdd0 con 0x7f740c0c58d0 2024-05-09T08:06:56.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.265+0000 7f7412200640 1 -- 172.21.15.1:0/2854177051 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f740c1f4460 con 0x7f740c0c58d0 2024-05-09T08:06:56.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.265+0000 7f7412200640 1 -- 172.21.15.1:0/2854177051 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f740c153e40 con 0x7f740c0c58d0 2024-05-09T08:06:56.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.266+0000 7f7412200640 1 --2- 172.21.15.1:0/2854177051 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f740c14ea40 0x7f740c0cbea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:06:56.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.266+0000 7f7412200640 1 -- 172.21.15.1:0/2854177051 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f740c14c960 con 0x7f740c14ea40 2024-05-09T08:06:56.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.266+0000 7f7410e00640 1 --2- 172.21.15.1:0/2854177051 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f740c14ea40 0x7f740c0cbea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:06:56.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.267+0000 7f7410e00640 1 -- 172.21.15.1:0/2854177051 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f740c04f660 con 0x7f740c14ea40 2024-05-09T08:06:56.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.267+0000 7f7410e00640 1 --2- 172.21.15.1:0/2854177051 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f740c14ea40 0x7f740c0cbea0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f7400000f60 tx=0x7f7400001cd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:06:56.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.267+0000 7f7408c00640 1 -- 172.21.15.1:0/2854177051 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f740000f9e0 con 0x7f740c14ea40 2024-05-09T08:06:56.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.267+0000 7f7408c00640 1 -- 172.21.15.1:0/2854177051 <== mon.1 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f73f801a070 con 0x7f740c0c58d0 2024-05-09T08:06:56.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.268+0000 7f7408c00640 1 --2- 172.21.15.1:0/2854177051 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f73ec06bd30 0x7f73ec06e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:06:56.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.268+0000 7f7408c00640 1 -- 172.21.15.1:0/2854177051 <== mon.1 v2:172.21.15.7:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7f73f808b6a0 con 0x7f740c0c58d0 2024-05-09T08:06:56.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.269+0000 7f7411800640 1 --2- 172.21.15.1:0/2854177051 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f73ec06bd30 0x7f73ec06e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:06:56.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.269+0000 7f7408c00640 1 -- 172.21.15.1:0/2854177051 <== mon.1 v2:172.21.15.7:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f7400018070 con 0x7f740c14ea40 2024-05-09T08:06:56.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.269+0000 7f7408c00640 1 -- 172.21.15.1:0/2854177051 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f740c14ea40 msgr2=0x7f740c0cbea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:06:56.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.269+0000 7f7408c00640 1 --2- 172.21.15.1:0/2854177051 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f740c14ea40 0x7f740c0cbea0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f7400000f60 tx=0x7f7400001cd0 comp rx=0 tx=0).stop 2024-05-09T08:06:56.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.269+0000 7f7408c00640 1 -- 172.21.15.1:0/2854177051 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f740c14ea40 msgr2=0x7f740c0cbea0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-05-09T08:06:56.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.269+0000 7f7408c00640 1 --2- 172.21.15.1:0/2854177051 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f740c14ea40 0x7f740c0cbea0 secure :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f7400000f60 tx=0x7f7400001cd0 comp rx=0 tx=0).stop 2024-05-09T08:06:56.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.269+0000 7f7411800640 1 --2- 172.21.15.1:0/2854177051 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f73ec06bd30 0x7f73ec06e1f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f7404034000 tx=0x7f740402f000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:06:56.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.276+0000 7f7412200640 1 --2- 172.21.15.1:0/2854177051 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f740c1f3ea0 0x7f740c0cc3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:06:56.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.276+0000 7f7412200640 1 -- 172.21.15.1:0/2854177051 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f740c072e90 con 0x7f740c1f3ea0 2024-05-09T08:06:56.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.277+0000 7f7410e00640 1 --2- 172.21.15.1:0/2854177051 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f740c1f3ea0 0x7f740c0cc3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:06:56.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.277+0000 7f7410e00640 1 -- 172.21.15.1:0/2854177051 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f740c0bf850 con 0x7f740c1f3ea0 2024-05-09T08:06:56.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.278+0000 7f7410e00640 1 --2- 172.21.15.1:0/2854177051 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f740c1f3ea0 0x7f740c0cc3e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f7400000f60 tx=0x7f7400000ed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:06:56.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.278+0000 7f7408c00640 1 -- 172.21.15.1:0/2854177051 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7400013600 con 0x7f740c1f3ea0 2024-05-09T08:06:56.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.278+0000 7f7408c00640 1 -- 172.21.15.1:0/2854177051 <== mon.1 v2:172.21.15.7:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5189 (secure 0 0 0) 0x7f7400013600 con 0x7f740c1f3ea0 2024-05-09T08:06:56.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.278+0000 7f7408c00640 1 -- 172.21.15.1:0/2854177051 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f740c1f3ea0 msgr2=0x7f740c0cc3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:06:56.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.278+0000 7f7408c00640 1 --2- 172.21.15.1:0/2854177051 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f740c1f3ea0 0x7f740c0cc3e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f7400000f60 tx=0x7f7400000ed0 comp rx=0 tx=0).stop 2024-05-09T08:06:56.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.278+0000 7f7408c00640 1 -- 172.21.15.1:0/2854177051 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f740c1f3ea0 msgr2=0x7f740c0cc3e0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-05-09T08:06:56.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.278+0000 7f7408c00640 1 --2- 172.21.15.1:0/2854177051 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f740c1f3ea0 0x7f740c0cc3e0 secure :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f7400000f60 tx=0x7f7400000ed0 comp rx=0 tx=0).stop 2024-05-09T08:06:56.290 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-05-09T08:06:56.291 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "b", 2024-05-09T08:06:56.291 INFO:teuthology.orchestra.run.smithi001.stdout: "rank": 1, 2024-05-09T08:06:56.291 INFO:teuthology.orchestra.run.smithi001.stdout: "state": "peon", 2024-05-09T08:06:56.291 INFO:teuthology.orchestra.run.smithi001.stdout: "election_epoch": 36, 2024-05-09T08:06:56.291 INFO:teuthology.orchestra.run.smithi001.stdout: "uptime": 183940, 2024-05-09T08:06:56.291 INFO:teuthology.orchestra.run.smithi001.stdout: "quorum": [ 2024-05-09T08:06:56.291 INFO:teuthology.orchestra.run.smithi001.stdout: 0, 2024-05-09T08:06:56.291 INFO:teuthology.orchestra.run.smithi001.stdout: 1, 2024-05-09T08:06:56.291 INFO:teuthology.orchestra.run.smithi001.stdout: 2 2024-05-09T08:06:56.291 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:06:56.291 INFO:teuthology.orchestra.run.smithi001.stdout: "quorum_age": 17, 2024-05-09T08:06:56.291 INFO:teuthology.orchestra.run.smithi001.stdout: "features": { 2024-05-09T08:06:56.291 INFO:teuthology.orchestra.run.smithi001.stdout: "required_con": "2450521708007866372", 2024-05-09T08:06:56.292 INFO:teuthology.orchestra.run.smithi001.stdout: "required_mon": [ 2024-05-09T08:06:56.292 INFO:teuthology.orchestra.run.smithi001.stdout: "kraken", 2024-05-09T08:06:56.292 INFO:teuthology.orchestra.run.smithi001.stdout: "luminous", 2024-05-09T08:06:56.292 INFO:teuthology.orchestra.run.smithi001.stdout: "mimic", 2024-05-09T08:06:56.292 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap-prune", 2024-05-09T08:06:56.292 INFO:teuthology.orchestra.run.smithi001.stdout: "nautilus", 2024-05-09T08:06:56.292 INFO:teuthology.orchestra.run.smithi001.stdout: "octopus", 2024-05-09T08:06:56.292 INFO:teuthology.orchestra.run.smithi001.stdout: "pacific", 2024-05-09T08:06:56.292 INFO:teuthology.orchestra.run.smithi001.stdout: "elector-pinging", 2024-05-09T08:06:56.292 INFO:teuthology.orchestra.run.smithi001.stdout: "quincy", 2024-05-09T08:06:56.292 INFO:teuthology.orchestra.run.smithi001.stdout: "reef", 2024-05-09T08:06:56.292 INFO:teuthology.orchestra.run.smithi001.stdout: "squid" 2024-05-09T08:06:56.292 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:06:56.292 INFO:teuthology.orchestra.run.smithi001.stdout: "quorum_con": "4540701547738038271", 2024-05-09T08:06:56.293 INFO:teuthology.orchestra.run.smithi001.stdout: "quorum_mon": [ 2024-05-09T08:06:56.293 INFO:teuthology.orchestra.run.smithi001.stdout: "kraken", 2024-05-09T08:06:56.293 INFO:teuthology.orchestra.run.smithi001.stdout: "luminous", 2024-05-09T08:06:56.293 INFO:teuthology.orchestra.run.smithi001.stdout: "mimic", 2024-05-09T08:06:56.293 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap-prune", 2024-05-09T08:06:56.293 INFO:teuthology.orchestra.run.smithi001.stdout: "nautilus", 2024-05-09T08:06:56.293 INFO:teuthology.orchestra.run.smithi001.stdout: "octopus", 2024-05-09T08:06:56.293 INFO:teuthology.orchestra.run.smithi001.stdout: "pacific", 2024-05-09T08:06:56.293 INFO:teuthology.orchestra.run.smithi001.stdout: "elector-pinging", 2024-05-09T08:06:56.293 INFO:teuthology.orchestra.run.smithi001.stdout: "quincy", 2024-05-09T08:06:56.293 INFO:teuthology.orchestra.run.smithi001.stdout: "reef", 2024-05-09T08:06:56.293 INFO:teuthology.orchestra.run.smithi001.stdout: "squid" 2024-05-09T08:06:56.293 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:06:56.294 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:06:56.294 INFO:teuthology.orchestra.run.smithi001.stdout: "outside_quorum": [], 2024-05-09T08:06:56.294 INFO:teuthology.orchestra.run.smithi001.stdout: "extra_probe_peers": [], 2024-05-09T08:06:56.294 INFO:teuthology.orchestra.run.smithi001.stdout: "sync_provider": [], 2024-05-09T08:06:56.294 INFO:teuthology.orchestra.run.smithi001.stdout: "monmap": { 2024-05-09T08:06:56.294 INFO:teuthology.orchestra.run.smithi001.stdout: "epoch": 1, 2024-05-09T08:06:56.294 INFO:teuthology.orchestra.run.smithi001.stdout: "fsid": "54329ec8-3270-4043-9a82-c6cd7420aa7f", 2024-05-09T08:06:56.294 INFO:teuthology.orchestra.run.smithi001.stdout: "modified": "2024-05-09T08:01:21.819270Z", 2024-05-09T08:06:56.294 INFO:teuthology.orchestra.run.smithi001.stdout: "created": "2024-05-09T08:01:21.819270Z", 2024-05-09T08:06:56.294 INFO:teuthology.orchestra.run.smithi001.stdout: "min_mon_release": 19, 2024-05-09T08:06:56.294 INFO:teuthology.orchestra.run.smithi001.stdout: "min_mon_release_name": "squid", 2024-05-09T08:06:56.294 INFO:teuthology.orchestra.run.smithi001.stdout: "election_strategy": 1, 2024-05-09T08:06:56.294 INFO:teuthology.orchestra.run.smithi001.stdout: "disallowed_leaders: ": "", 2024-05-09T08:06:56.295 INFO:teuthology.orchestra.run.smithi001.stdout: "stretch_mode": false, 2024-05-09T08:06:56.295 INFO:teuthology.orchestra.run.smithi001.stdout: "tiebreaker_mon": "", 2024-05-09T08:06:56.295 INFO:teuthology.orchestra.run.smithi001.stdout: "removed_ranks: ": "", 2024-05-09T08:06:56.295 INFO:teuthology.orchestra.run.smithi001.stdout: "features": { 2024-05-09T08:06:56.295 INFO:teuthology.orchestra.run.smithi001.stdout: "persistent": [ 2024-05-09T08:06:56.295 INFO:teuthology.orchestra.run.smithi001.stdout: "kraken", 2024-05-09T08:06:56.295 INFO:teuthology.orchestra.run.smithi001.stdout: "luminous", 2024-05-09T08:06:56.295 INFO:teuthology.orchestra.run.smithi001.stdout: "mimic", 2024-05-09T08:06:56.295 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap-prune", 2024-05-09T08:06:56.295 INFO:teuthology.orchestra.run.smithi001.stdout: "nautilus", 2024-05-09T08:06:56.295 INFO:teuthology.orchestra.run.smithi001.stdout: "octopus", 2024-05-09T08:06:56.295 INFO:teuthology.orchestra.run.smithi001.stdout: "pacific", 2024-05-09T08:06:56.295 INFO:teuthology.orchestra.run.smithi001.stdout: "elector-pinging", 2024-05-09T08:06:56.295 INFO:teuthology.orchestra.run.smithi001.stdout: "quincy", 2024-05-09T08:06:56.296 INFO:teuthology.orchestra.run.smithi001.stdout: "reef", 2024-05-09T08:06:56.296 INFO:teuthology.orchestra.run.smithi001.stdout: "squid" 2024-05-09T08:06:56.296 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:06:56.296 INFO:teuthology.orchestra.run.smithi001.stdout: "optional": [] 2024-05-09T08:06:56.296 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:06:56.296 INFO:teuthology.orchestra.run.smithi001.stdout: "mons": [ 2024-05-09T08:06:56.296 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:06:56.296 INFO:teuthology.orchestra.run.smithi001.stdout: "rank": 0, 2024-05-09T08:06:56.296 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "a", 2024-05-09T08:06:56.296 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-05-09T08:06:56.296 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-05-09T08:06:56.296 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:06:56.296 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-05-09T08:06:56.297 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:3300", 2024-05-09T08:06:56.297 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:06:56.297 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:06:56.297 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:06:56.297 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-05-09T08:06:56.297 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6789", 2024-05-09T08:06:56.297 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:06:56.297 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:06:56.297 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:06:56.297 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:06:56.297 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6789/0", 2024-05-09T08:06:56.297 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.1:6789/0", 2024-05-09T08:06:56.297 INFO:teuthology.orchestra.run.smithi001.stdout: "priority": 0, 2024-05-09T08:06:56.297 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 0, 2024-05-09T08:06:56.298 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_location": "{}" 2024-05-09T08:06:56.298 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:06:56.298 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:06:56.298 INFO:teuthology.orchestra.run.smithi001.stdout: "rank": 1, 2024-05-09T08:06:56.298 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "b", 2024-05-09T08:06:56.298 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-05-09T08:06:56.298 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-05-09T08:06:56.298 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:06:56.298 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-05-09T08:06:56.299 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:3300", 2024-05-09T08:06:56.299 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:06:56.299 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:06:56.299 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:06:56.299 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-05-09T08:06:56.299 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:6789", 2024-05-09T08:06:56.299 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:06:56.299 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:06:56.299 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:06:56.299 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:06:56.299 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:6789/0", 2024-05-09T08:06:56.299 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.7:6789/0", 2024-05-09T08:06:56.299 INFO:teuthology.orchestra.run.smithi001.stdout: "priority": 0, 2024-05-09T08:06:56.299 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 0, 2024-05-09T08:06:56.300 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_location": "{}" 2024-05-09T08:06:56.300 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:06:56.300 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:06:56.300 INFO:teuthology.orchestra.run.smithi001.stdout: "rank": 2, 2024-05-09T08:06:56.300 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "c", 2024-05-09T08:06:56.300 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-05-09T08:06:56.300 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-05-09T08:06:56.300 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:06:56.300 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-05-09T08:06:56.300 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:3301", 2024-05-09T08:06:56.300 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:06:56.300 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:06:56.300 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:06:56.301 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-05-09T08:06:56.301 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:6790", 2024-05-09T08:06:56.301 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:06:56.301 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:06:56.301 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:06:56.301 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:06:56.301 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:6790/0", 2024-05-09T08:06:56.301 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.7:6790/0", 2024-05-09T08:06:56.301 INFO:teuthology.orchestra.run.smithi001.stdout: "priority": 0, 2024-05-09T08:06:56.301 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 0, 2024-05-09T08:06:56.301 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_location": "{}" 2024-05-09T08:06:56.301 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:06:56.301 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:06:56.301 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:06:56.302 INFO:teuthology.orchestra.run.smithi001.stdout: "feature_map": { 2024-05-09T08:06:56.302 INFO:teuthology.orchestra.run.smithi001.stdout: "mon": [ 2024-05-09T08:06:56.302 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:06:56.302 INFO:teuthology.orchestra.run.smithi001.stdout: "features": "0x3f03cffffffdffff", 2024-05-09T08:06:56.302 INFO:teuthology.orchestra.run.smithi001.stdout: "release": "luminous", 2024-05-09T08:06:56.302 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 1 2024-05-09T08:06:56.302 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:06:56.302 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:06:56.302 INFO:teuthology.orchestra.run.smithi001.stdout: "mds": [ 2024-05-09T08:06:56.302 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:06:56.302 INFO:teuthology.orchestra.run.smithi001.stdout: "features": "0x3f03cffffffdffff", 2024-05-09T08:06:56.302 INFO:teuthology.orchestra.run.smithi001.stdout: "release": "luminous", 2024-05-09T08:06:56.302 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 6 2024-05-09T08:06:56.303 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:06:56.303 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:06:56.303 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": [ 2024-05-09T08:06:56.303 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:06:56.303 INFO:teuthology.orchestra.run.smithi001.stdout: "features": "0x3f03cffffffdffff", 2024-05-09T08:06:56.303 INFO:teuthology.orchestra.run.smithi001.stdout: "release": "luminous", 2024-05-09T08:06:56.303 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 7 2024-05-09T08:06:56.303 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:06:56.303 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:06:56.303 INFO:teuthology.orchestra.run.smithi001.stdout: "client": [ 2024-05-09T08:06:56.303 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:06:56.303 INFO:teuthology.orchestra.run.smithi001.stdout: "features": "0x2f018fb87aa4aafe", 2024-05-09T08:06:56.303 INFO:teuthology.orchestra.run.smithi001.stdout: "release": "luminous", 2024-05-09T08:06:56.303 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 1 2024-05-09T08:06:56.304 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:06:56.304 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:06:56.304 INFO:teuthology.orchestra.run.smithi001.stdout: "features": "0x3f03cffffffdffff", 2024-05-09T08:06:56.304 INFO:teuthology.orchestra.run.smithi001.stdout: "release": "luminous", 2024-05-09T08:06:56.304 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 6 2024-05-09T08:06:56.304 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:06:56.304 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:06:56.304 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr": [ 2024-05-09T08:06:56.304 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:06:56.304 INFO:teuthology.orchestra.run.smithi001.stdout: "features": "0x3f03cffffffdffff", 2024-05-09T08:06:56.304 INFO:teuthology.orchestra.run.smithi001.stdout: "release": "luminous", 2024-05-09T08:06:56.304 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 1 2024-05-09T08:06:56.304 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:06:56.305 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:06:56.305 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:06:56.305 INFO:teuthology.orchestra.run.smithi001.stdout: "stretch_mode": false 2024-05-09T08:06:56.305 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-05-09T08:06:56.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.279+0000 7f7412200640 1 -- 172.21.15.1:0/2854177051 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f73ec06bd30 msgr2=0x7f73ec06e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:06:56.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.279+0000 7f7412200640 1 --2- 172.21.15.1:0/2854177051 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f73ec06bd30 0x7f73ec06e1f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f7404034000 tx=0x7f740402f000 comp rx=0 tx=0).stop 2024-05-09T08:06:56.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.279+0000 7f7412200640 1 -- 172.21.15.1:0/2854177051 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f740c0c58d0 msgr2=0x7f740c154360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:06:56.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.279+0000 7f7412200640 1 --2- 172.21.15.1:0/2854177051 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f740c0c58d0 0x7f740c154360 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f73f800e980 tx=0x7f73f800ee40 comp rx=0 tx=0).stop 2024-05-09T08:06:56.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.279+0000 7f7411800640 1 -- 172.21.15.1:0/2854177051 reap_dead start 2024-05-09T08:06:56.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.279+0000 7f7411800640 1 -- 172.21.15.1:0/2854177051 reap_dead start 2024-05-09T08:06:56.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.279+0000 7f7412200640 1 -- 172.21.15.1:0/2854177051 shutdown_connections 2024-05-09T08:06:56.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.279+0000 7f7412200640 1 -- 172.21.15.1:0/2854177051 >> 172.21.15.1:0/2854177051 conn(0x7f740c06b250 msgr2=0x7f740c06b630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:06:56.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.279+0000 7f7412200640 1 -- 172.21.15.1:0/2854177051 shutdown_connections 2024-05-09T08:06:56.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.279+0000 7f7412200640 1 -- 172.21.15.1:0/2854177051 wait complete. 2024-05-09T08:06:56.310 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-05-09T08:06:56.310 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-05-09T08:06:56.310 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-05-09T08:06:56.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.515+0000 7f2737a00640 1 Processor -- start 2024-05-09T08:06:56.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.515+0000 7f2737a00640 1 -- start start 2024-05-09T08:06:56.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.517+0000 7f2737a00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f2730071dc0 0x7f27300c2d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:06:56.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.517+0000 7f2737a00640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f273015c5d0 0x7f2730152170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:06:56.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.517+0000 7f2737a00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2730158dc0 con 0x7f27300719e0 2024-05-09T08:06:56.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.517+0000 7f2737a00640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f27300b3360 con 0x7f273015c5d0 2024-05-09T08:06:56.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.517+0000 7f2737a00640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f27300c32a0 con 0x7f2730071dc0 2024-05-09T08:06:56.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.517+0000 7f2735a00640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f273015c5d0 0x7f2730152170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:06:56.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.517+0000 7f2735a00640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f273015c5d0 0x7f2730152170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:36568/0 (socket says 172.21.15.1:36568) 2024-05-09T08:06:56.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.517+0000 7f2735a00640 1 -- 172.21.15.1:0/3146770448 learned_addr learned my addr 172.21.15.1:0/3146770448 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:06:56.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.517+0000 7f272fe00640 1 --2- 172.21.15.1:0/3146770448 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f2730071dc0 0x7f27300c2d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:06:56.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.518+0000 7f2735a00640 1 -- 172.21.15.1:0/3146770448 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f2730071dc0 msgr2=0x7f27300c2d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:06:56.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.518+0000 7f2735a00640 1 --2- 172.21.15.1:0/3146770448 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f2730071dc0 0x7f27300c2d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:06:56.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.518+0000 7f2735a00640 1 -- 172.21.15.1:0/3146770448 >> v1:172.21.15.7:6789/0 conn(0x7f27300719e0 legacy=0x7f2730158380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:06:56.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.518+0000 7f2735a00640 1 -- 172.21.15.1:0/3146770448 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27301590a0 con 0x7f273015c5d0 2024-05-09T08:06:56.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.518+0000 7f272f400640 1 -- 172.21.15.1:0/3146770448 <== mon.1 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2858851835 0 0) 0x7f2730158dc0 con 0x7f27300719e0 2024-05-09T08:06:56.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.519+0000 7f2735a00640 1 --2- 172.21.15.1:0/3146770448 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f273015c5d0 0x7f2730152170 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f2728009e80 tx=0x7f272802a750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa260068213411c8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:06:56.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.519+0000 7f272f400640 1 -- 172.21.15.1:0/3146770448 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2728036070 con 0x7f273015c5d0 2024-05-09T08:06:56.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.519+0000 7f272f400640 1 -- 172.21.15.1:0/3146770448 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2728031600 con 0x7f273015c5d0 2024-05-09T08:06:56.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.519+0000 7f272f400640 1 -- 172.21.15.1:0/3146770448 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27280319d0 con 0x7f273015c5d0 2024-05-09T08:06:56.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.520+0000 7f2737a00640 1 -- 172.21.15.1:0/3146770448 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f273015c5d0 msgr2=0x7f2730152170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:06:56.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.520+0000 7f2737a00640 1 --2- 172.21.15.1:0/3146770448 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f273015c5d0 0x7f2730152170 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f2728009e80 tx=0x7f272802a750 comp rx=0 tx=0).stop 2024-05-09T08:06:56.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.520+0000 7f2737a00640 1 -- 172.21.15.1:0/3146770448 shutdown_connections 2024-05-09T08:06:56.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.520+0000 7f2737a00640 1 --2- 172.21.15.1:0/3146770448 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f273015c5d0 0x7f2730152170 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:06:56.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.520+0000 7f2737a00640 1 --2- 172.21.15.1:0/3146770448 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f2730071dc0 0x7f27300c2d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:06:56.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.520+0000 7f2737a00640 1 -- 172.21.15.1:0/3146770448 >> 172.21.15.1:0/3146770448 conn(0x7f273006b880 msgr2=0x7f273006fb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:06:56.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.520+0000 7f2737a00640 1 -- 172.21.15.1:0/3146770448 shutdown_connections 2024-05-09T08:06:56.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.520+0000 7f2737a00640 1 -- 172.21.15.1:0/3146770448 wait complete. 2024-05-09T08:06:56.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.522+0000 7f2737a00640 1 Processor -- start 2024-05-09T08:06:56.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.522+0000 7f2737a00640 1 -- start start 2024-05-09T08:06:56.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.523+0000 7f2737a00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f27300719e0 0x7f2730157320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:06:56.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.523+0000 7f2737a00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2730071dc0 0x7f273014e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:06:56.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.523+0000 7f2737a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f273015c5d0 0x7f273014e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:06:56.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.523+0000 7f2737a00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f273015c0b0 con 0x7f2730071dc0 2024-05-09T08:06:56.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.523+0000 7f2737a00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f273015bdd0 con 0x7f273015c5d0 2024-05-09T08:06:56.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.523+0000 7f2737a00640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f2730158a90 con 0x7f27300719e0 2024-05-09T08:06:56.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.523+0000 7f272fe00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2730071dc0 0x7f273014e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:06:56.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.524+0000 7f272fe00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2730071dc0 0x7f273014e300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:36574/0 (socket says 172.21.15.1:36574) 2024-05-09T08:06:56.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.524+0000 7f272fe00640 1 -- 172.21.15.1:0/4130020177 learned_addr learned my addr 172.21.15.1:0/4130020177 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:06:56.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.524+0000 7f2735a00640 1 --2- 172.21.15.1:0/4130020177 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f273015c5d0 0x7f273014e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:06:56.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.524+0000 7f2735000640 1 --2- 172.21.15.1:0/4130020177 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f27300719e0 0x7f2730157320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:06:56.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.524+0000 7f272fe00640 1 -- 172.21.15.1:0/4130020177 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f27300719e0 msgr2=0x7f2730157320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:06:56.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.524+0000 7f272fe00640 1 --2- 172.21.15.1:0/4130020177 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f27300719e0 0x7f2730157320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:06:56.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.524+0000 7f272fe00640 1 -- 172.21.15.1:0/4130020177 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f273015c5d0 msgr2=0x7f273014e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:06:56.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.524+0000 7f272fe00640 1 --2- 172.21.15.1:0/4130020177 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f273015c5d0 0x7f273014e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:06:56.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.524+0000 7f272fe00640 1 -- 172.21.15.1:0/4130020177 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2730158430 con 0x7f2730071dc0 2024-05-09T08:06:56.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.524+0000 7f2735a00640 1 --2- 172.21.15.1:0/4130020177 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f273015c5d0 0x7f273014e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:06:56.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.524+0000 7f272fe00640 1 --2- 172.21.15.1:0/4130020177 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2730071dc0 0x7f273014e300 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f271800e9b0 tx=0x7f271800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:06:56.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.525+0000 7f272cc00640 1 -- 172.21.15.1:0/4130020177 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2718017070 con 0x7f2730071dc0 2024-05-09T08:06:56.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.525+0000 7f272cc00640 1 -- 172.21.15.1:0/4130020177 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f271800c9a0 con 0x7f2730071dc0 2024-05-09T08:06:56.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.525+0000 7f272cc00640 1 -- 172.21.15.1:0/4130020177 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f271800cd90 con 0x7f2730071dc0 2024-05-09T08:06:56.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.525+0000 7f2737a00640 1 -- 172.21.15.1:0/4130020177 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2730157eb0 con 0x7f2730071dc0 2024-05-09T08:06:56.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.525+0000 7f2737a00640 1 -- 172.21.15.1:0/4130020177 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2730158150 con 0x7f2730071dc0 2024-05-09T08:06:56.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.526+0000 7f2737a00640 1 --2- 172.21.15.1:0/4130020177 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f27300c5520 0x7f2730156c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:06:56.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.526+0000 7f2737a00640 1 -- 172.21.15.1:0/4130020177 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f2730150070 con 0x7f27300c5520 2024-05-09T08:06:56.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.526+0000 7f2735000640 1 --2- 172.21.15.1:0/4130020177 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f27300c5520 0x7f2730156c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:06:56.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.527+0000 7f2735000640 1 -- 172.21.15.1:0/4130020177 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2730074d70 con 0x7f27300c5520 2024-05-09T08:06:56.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.527+0000 7f2735000640 1 --2- 172.21.15.1:0/4130020177 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f27300c5520 0x7f2730156c10 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f27200011e0 tx=0x7f272000fc00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:06:56.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.527+0000 7f272cc00640 1 -- 172.21.15.1:0/4130020177 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f271801a070 con 0x7f2730071dc0 2024-05-09T08:06:56.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.528+0000 7f272cc00640 1 --2- 172.21.15.1:0/4130020177 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f270c06be30 0x7f270c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:06:56.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.528+0000 7f272cc00640 1 -- 172.21.15.1:0/4130020177 <== mon.2 v2:172.21.15.7:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27200050d0 con 0x7f27300c5520 2024-05-09T08:06:56.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.528+0000 7f272cc00640 1 -- 172.21.15.1:0/4130020177 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7f271808b2a0 con 0x7f2730071dc0 2024-05-09T08:06:56.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.528+0000 7f272cc00640 1 -- 172.21.15.1:0/4130020177 <== mon.2 v2:172.21.15.7:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f2720018070 con 0x7f27300c5520 2024-05-09T08:06:56.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.528+0000 7f272cc00640 1 -- 172.21.15.1:0/4130020177 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f27300c5520 msgr2=0x7f2730156c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:06:56.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.528+0000 7f272cc00640 1 --2- 172.21.15.1:0/4130020177 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f27300c5520 0x7f2730156c10 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f27200011e0 tx=0x7f272000fc00 comp rx=0 tx=0).stop 2024-05-09T08:06:56.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.528+0000 7f272cc00640 1 -- 172.21.15.1:0/4130020177 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f27300c5520 msgr2=0x7f2730156c10 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-05-09T08:06:56.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.528+0000 7f272cc00640 1 --2- 172.21.15.1:0/4130020177 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f27300c5520 0x7f2730156c10 secure :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f27200011e0 tx=0x7f272000fc00 comp rx=0 tx=0).stop 2024-05-09T08:06:56.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.529+0000 7f2735a00640 1 --2- 172.21.15.1:0/4130020177 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f270c06be30 0x7f270c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:06:56.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.529+0000 7f2735a00640 1 --2- 172.21.15.1:0/4130020177 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f270c06be30 0x7f270c06e2f0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f2728033000 tx=0x7f2728009000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:06:56.547 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.536+0000 7f2737a00640 1 --2- 172.21.15.1:0/4130020177 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f273014ca40 0x7f2730150460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:06:56.547 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.536+0000 7f2737a00640 1 -- 172.21.15.1:0/4130020177 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f2730151610 con 0x7f273014ca40 2024-05-09T08:06:56.547 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.536+0000 7f2735000640 1 --2- 172.21.15.1:0/4130020177 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f273014ca40 0x7f2730150460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:06:56.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.537+0000 7f2735000640 1 -- 172.21.15.1:0/4130020177 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f2730151330 con 0x7f273014ca40 2024-05-09T08:06:56.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.537+0000 7f2735000640 1 --2- 172.21.15.1:0/4130020177 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f273014ca40 0x7f2730150460 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f27200011e0 tx=0x7f2720001080 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:06:56.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.537+0000 7f272cc00640 1 -- 172.21.15.1:0/4130020177 <== mon.2 v2:172.21.15.7:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f272000d7a0 con 0x7f273014ca40 2024-05-09T08:06:56.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.538+0000 7f272cc00640 1 -- 172.21.15.1:0/4130020177 <== mon.2 v2:172.21.15.7:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4535 (secure 0 0 0) 0x7f272000d7a0 con 0x7f273014ca40 2024-05-09T08:06:56.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.538+0000 7f272cc00640 1 -- 172.21.15.1:0/4130020177 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f273014ca40 msgr2=0x7f2730150460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:06:56.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.538+0000 7f272cc00640 1 --2- 172.21.15.1:0/4130020177 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f273014ca40 0x7f2730150460 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f27200011e0 tx=0x7f2720001080 comp rx=0 tx=0).stop 2024-05-09T08:06:56.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.538+0000 7f272cc00640 1 -- 172.21.15.1:0/4130020177 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f273014ca40 msgr2=0x7f2730150460 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-05-09T08:06:56.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.538+0000 7f272cc00640 1 --2- 172.21.15.1:0/4130020177 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f273014ca40 0x7f2730150460 secure :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f27200011e0 tx=0x7f2720001080 comp rx=0 tx=0).stop 2024-05-09T08:06:56.550 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-05-09T08:06:56.550 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "c", 2024-05-09T08:06:56.551 INFO:teuthology.orchestra.run.smithi001.stdout: "rank": 2, 2024-05-09T08:06:56.551 INFO:teuthology.orchestra.run.smithi001.stdout: "state": "peon", 2024-05-09T08:06:56.551 INFO:teuthology.orchestra.run.smithi001.stdout: "election_epoch": 36, 2024-05-09T08:06:56.551 INFO:teuthology.orchestra.run.smithi001.stdout: "uptime": 22464, 2024-05-09T08:06:56.551 INFO:teuthology.orchestra.run.smithi001.stdout: "quorum": [ 2024-05-09T08:06:56.551 INFO:teuthology.orchestra.run.smithi001.stdout: 0, 2024-05-09T08:06:56.551 INFO:teuthology.orchestra.run.smithi001.stdout: 1, 2024-05-09T08:06:56.551 INFO:teuthology.orchestra.run.smithi001.stdout: 2 2024-05-09T08:06:56.551 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:06:56.551 INFO:teuthology.orchestra.run.smithi001.stdout: "quorum_age": 17, 2024-05-09T08:06:56.551 INFO:teuthology.orchestra.run.smithi001.stdout: "features": { 2024-05-09T08:06:56.552 INFO:teuthology.orchestra.run.smithi001.stdout: "required_con": "2450521708007866372", 2024-05-09T08:06:56.552 INFO:teuthology.orchestra.run.smithi001.stdout: "required_mon": [ 2024-05-09T08:06:56.552 INFO:teuthology.orchestra.run.smithi001.stdout: "kraken", 2024-05-09T08:06:56.552 INFO:teuthology.orchestra.run.smithi001.stdout: "luminous", 2024-05-09T08:06:56.552 INFO:teuthology.orchestra.run.smithi001.stdout: "mimic", 2024-05-09T08:06:56.552 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap-prune", 2024-05-09T08:06:56.552 INFO:teuthology.orchestra.run.smithi001.stdout: "nautilus", 2024-05-09T08:06:56.552 INFO:teuthology.orchestra.run.smithi001.stdout: "octopus", 2024-05-09T08:06:56.552 INFO:teuthology.orchestra.run.smithi001.stdout: "pacific", 2024-05-09T08:06:56.552 INFO:teuthology.orchestra.run.smithi001.stdout: "elector-pinging", 2024-05-09T08:06:56.552 INFO:teuthology.orchestra.run.smithi001.stdout: "quincy", 2024-05-09T08:06:56.552 INFO:teuthology.orchestra.run.smithi001.stdout: "reef", 2024-05-09T08:06:56.553 INFO:teuthology.orchestra.run.smithi001.stdout: "squid" 2024-05-09T08:06:56.553 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:06:56.553 INFO:teuthology.orchestra.run.smithi001.stdout: "quorum_con": "4540701547738038271", 2024-05-09T08:06:56.553 INFO:teuthology.orchestra.run.smithi001.stdout: "quorum_mon": [ 2024-05-09T08:06:56.553 INFO:teuthology.orchestra.run.smithi001.stdout: "kraken", 2024-05-09T08:06:56.553 INFO:teuthology.orchestra.run.smithi001.stdout: "luminous", 2024-05-09T08:06:56.553 INFO:teuthology.orchestra.run.smithi001.stdout: "mimic", 2024-05-09T08:06:56.553 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap-prune", 2024-05-09T08:06:56.553 INFO:teuthology.orchestra.run.smithi001.stdout: "nautilus", 2024-05-09T08:06:56.553 INFO:teuthology.orchestra.run.smithi001.stdout: "octopus", 2024-05-09T08:06:56.553 INFO:teuthology.orchestra.run.smithi001.stdout: "pacific", 2024-05-09T08:06:56.553 INFO:teuthology.orchestra.run.smithi001.stdout: "elector-pinging", 2024-05-09T08:06:56.553 INFO:teuthology.orchestra.run.smithi001.stdout: "quincy", 2024-05-09T08:06:56.554 INFO:teuthology.orchestra.run.smithi001.stdout: "reef", 2024-05-09T08:06:56.554 INFO:teuthology.orchestra.run.smithi001.stdout: "squid" 2024-05-09T08:06:56.554 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:06:56.554 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:06:56.554 INFO:teuthology.orchestra.run.smithi001.stdout: "outside_quorum": [], 2024-05-09T08:06:56.554 INFO:teuthology.orchestra.run.smithi001.stdout: "extra_probe_peers": [], 2024-05-09T08:06:56.554 INFO:teuthology.orchestra.run.smithi001.stdout: "sync_provider": [], 2024-05-09T08:06:56.554 INFO:teuthology.orchestra.run.smithi001.stdout: "monmap": { 2024-05-09T08:06:56.554 INFO:teuthology.orchestra.run.smithi001.stdout: "epoch": 1, 2024-05-09T08:06:56.554 INFO:teuthology.orchestra.run.smithi001.stdout: "fsid": "54329ec8-3270-4043-9a82-c6cd7420aa7f", 2024-05-09T08:06:56.554 INFO:teuthology.orchestra.run.smithi001.stdout: "modified": "2024-05-09T08:01:21.819270Z", 2024-05-09T08:06:56.554 INFO:teuthology.orchestra.run.smithi001.stdout: "created": "2024-05-09T08:01:21.819270Z", 2024-05-09T08:06:56.554 INFO:teuthology.orchestra.run.smithi001.stdout: "min_mon_release": 19, 2024-05-09T08:06:56.555 INFO:teuthology.orchestra.run.smithi001.stdout: "min_mon_release_name": "squid", 2024-05-09T08:06:56.555 INFO:teuthology.orchestra.run.smithi001.stdout: "election_strategy": 1, 2024-05-09T08:06:56.555 INFO:teuthology.orchestra.run.smithi001.stdout: "disallowed_leaders: ": "", 2024-05-09T08:06:56.555 INFO:teuthology.orchestra.run.smithi001.stdout: "stretch_mode": false, 2024-05-09T08:06:56.555 INFO:teuthology.orchestra.run.smithi001.stdout: "tiebreaker_mon": "", 2024-05-09T08:06:56.555 INFO:teuthology.orchestra.run.smithi001.stdout: "removed_ranks: ": "", 2024-05-09T08:06:56.555 INFO:teuthology.orchestra.run.smithi001.stdout: "features": { 2024-05-09T08:06:56.555 INFO:teuthology.orchestra.run.smithi001.stdout: "persistent": [ 2024-05-09T08:06:56.555 INFO:teuthology.orchestra.run.smithi001.stdout: "kraken", 2024-05-09T08:06:56.555 INFO:teuthology.orchestra.run.smithi001.stdout: "luminous", 2024-05-09T08:06:56.555 INFO:teuthology.orchestra.run.smithi001.stdout: "mimic", 2024-05-09T08:06:56.555 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap-prune", 2024-05-09T08:06:56.555 INFO:teuthology.orchestra.run.smithi001.stdout: "nautilus", 2024-05-09T08:06:56.555 INFO:teuthology.orchestra.run.smithi001.stdout: "octopus", 2024-05-09T08:06:56.556 INFO:teuthology.orchestra.run.smithi001.stdout: "pacific", 2024-05-09T08:06:56.556 INFO:teuthology.orchestra.run.smithi001.stdout: "elector-pinging", 2024-05-09T08:06:56.556 INFO:teuthology.orchestra.run.smithi001.stdout: "quincy", 2024-05-09T08:06:56.556 INFO:teuthology.orchestra.run.smithi001.stdout: "reef", 2024-05-09T08:06:56.556 INFO:teuthology.orchestra.run.smithi001.stdout: "squid" 2024-05-09T08:06:56.556 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:06:56.556 INFO:teuthology.orchestra.run.smithi001.stdout: "optional": [] 2024-05-09T08:06:56.556 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:06:56.556 INFO:teuthology.orchestra.run.smithi001.stdout: "mons": [ 2024-05-09T08:06:56.556 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:06:56.556 INFO:teuthology.orchestra.run.smithi001.stdout: "rank": 0, 2024-05-09T08:06:56.556 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "a", 2024-05-09T08:06:56.556 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-05-09T08:06:56.557 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-05-09T08:06:56.557 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:06:56.557 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-05-09T08:06:56.557 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:3300", 2024-05-09T08:06:56.557 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:06:56.557 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:06:56.557 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:06:56.557 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-05-09T08:06:56.557 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6789", 2024-05-09T08:06:56.557 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:06:56.557 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:06:56.557 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:06:56.557 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:06:56.557 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6789/0", 2024-05-09T08:06:56.557 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.1:6789/0", 2024-05-09T08:06:56.558 INFO:teuthology.orchestra.run.smithi001.stdout: "priority": 0, 2024-05-09T08:06:56.558 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 0, 2024-05-09T08:06:56.558 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_location": "{}" 2024-05-09T08:06:56.558 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:06:56.558 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:06:56.558 INFO:teuthology.orchestra.run.smithi001.stdout: "rank": 1, 2024-05-09T08:06:56.558 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "b", 2024-05-09T08:06:56.558 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-05-09T08:06:56.558 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-05-09T08:06:56.558 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:06:56.558 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-05-09T08:06:56.558 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:3300", 2024-05-09T08:06:56.558 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:06:56.559 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:06:56.559 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:06:56.559 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-05-09T08:06:56.559 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:6789", 2024-05-09T08:06:56.559 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:06:56.559 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:06:56.559 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:06:56.559 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:06:56.559 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:6789/0", 2024-05-09T08:06:56.559 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.7:6789/0", 2024-05-09T08:06:56.559 INFO:teuthology.orchestra.run.smithi001.stdout: "priority": 0, 2024-05-09T08:06:56.559 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 0, 2024-05-09T08:06:56.559 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_location": "{}" 2024-05-09T08:06:56.560 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:06:56.560 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:06:56.560 INFO:teuthology.orchestra.run.smithi001.stdout: "rank": 2, 2024-05-09T08:06:56.560 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "c", 2024-05-09T08:06:56.560 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-05-09T08:06:56.560 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-05-09T08:06:56.560 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:06:56.560 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-05-09T08:06:56.560 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:3301", 2024-05-09T08:06:56.560 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:06:56.560 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:06:56.560 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:06:56.560 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-05-09T08:06:56.560 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:6790", 2024-05-09T08:06:56.561 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:06:56.561 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:06:56.561 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:06:56.561 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:06:56.561 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:6790/0", 2024-05-09T08:06:56.561 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.7:6790/0", 2024-05-09T08:06:56.561 INFO:teuthology.orchestra.run.smithi001.stdout: "priority": 0, 2024-05-09T08:06:56.561 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 0, 2024-05-09T08:06:56.561 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_location": "{}" 2024-05-09T08:06:56.561 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:06:56.561 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:06:56.561 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:06:56.561 INFO:teuthology.orchestra.run.smithi001.stdout: "feature_map": { 2024-05-09T08:06:56.562 INFO:teuthology.orchestra.run.smithi001.stdout: "mon": [ 2024-05-09T08:06:56.562 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:06:56.562 INFO:teuthology.orchestra.run.smithi001.stdout: "features": "0x3f03cffffffdffff", 2024-05-09T08:06:56.562 INFO:teuthology.orchestra.run.smithi001.stdout: "release": "luminous", 2024-05-09T08:06:56.562 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 1 2024-05-09T08:06:56.562 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:06:56.562 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:06:56.562 INFO:teuthology.orchestra.run.smithi001.stdout: "client": [ 2024-05-09T08:06:56.562 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:06:56.562 INFO:teuthology.orchestra.run.smithi001.stdout: "features": "0x3f03cffffffdffff", 2024-05-09T08:06:56.562 INFO:teuthology.orchestra.run.smithi001.stdout: "release": "luminous", 2024-05-09T08:06:56.562 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 1 2024-05-09T08:06:56.562 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:06:56.562 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:06:56.563 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:06:56.563 INFO:teuthology.orchestra.run.smithi001.stdout: "stretch_mode": false 2024-05-09T08:06:56.563 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-05-09T08:06:56.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.539+0000 7f2737a00640 1 -- 172.21.15.1:0/4130020177 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f270c06be30 msgr2=0x7f270c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:06:56.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.539+0000 7f2737a00640 1 --2- 172.21.15.1:0/4130020177 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f270c06be30 0x7f270c06e2f0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f2728033000 tx=0x7f2728009000 comp rx=0 tx=0).stop 2024-05-09T08:06:56.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.539+0000 7f2737a00640 1 -- 172.21.15.1:0/4130020177 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2730071dc0 msgr2=0x7f273014e300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:06:56.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.539+0000 7f2737a00640 1 --2- 172.21.15.1:0/4130020177 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2730071dc0 0x7f273014e300 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f271800e9b0 tx=0x7f271800ee70 comp rx=0 tx=0).stop 2024-05-09T08:06:56.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.539+0000 7f2735a00640 1 -- 172.21.15.1:0/4130020177 reap_dead start 2024-05-09T08:06:56.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.539+0000 7f2737a00640 1 -- 172.21.15.1:0/4130020177 shutdown_connections 2024-05-09T08:06:56.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.539+0000 7f2737a00640 1 --2- 172.21.15.1:0/4130020177 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2730071dc0 0x7f273014e300 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:06:56.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.539+0000 7f2737a00640 1 -- 172.21.15.1:0/4130020177 >> 172.21.15.1:0/4130020177 conn(0x7f273006b880 msgr2=0x7f27300c87b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:06:56.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.539+0000 7f2737a00640 1 -- 172.21.15.1:0/4130020177 shutdown_connections 2024-05-09T08:06:56.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.539+0000 7f2737a00640 1 -- 172.21.15.1:0/4130020177 wait complete. 2024-05-09T08:06:56.567 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-05-09T08:06:56.567 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['c'] 2024-05-09T08:06:56.568 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2024-05-09T08:06:56.568 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.c 2024-05-09T08:06:56.568 INFO:tasks.mon_thrash.mon_thrasher:killing mon.c 2024-05-09T08:06:56.568 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-05-09T08:06:56.568 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T08:06:56.580 INFO:tasks.ceph.mon.c:Stopped 2024-05-09T08:06:56.581 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2024-05-09T08:06:56.581 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-05-09T08:06:56.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.781+0000 7f0c86800640 1 Processor -- start 2024-05-09T08:06:56.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.781+0000 7f0c86800640 1 -- start start 2024-05-09T08:06:56.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.783+0000 7f0c86800640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f0c800c5910 0x7f0c800c7720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:06:56.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.783+0000 7f0c86800640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f0c800c3110 0x7f0c80140200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:06:56.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.783+0000 7f0c86800640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0c80157430 con 0x7f0c800c5cf0 2024-05-09T08:06:56.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.783+0000 7f0c86800640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f0c800c7d60 con 0x7f0c800c3110 2024-05-09T08:06:56.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.783+0000 7f0c86800640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f0c800c8040 con 0x7f0c800c5910 2024-05-09T08:06:56.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.783+0000 7f0c85400640 1 -- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f0c800c5910 msgr2=0x7f0c800c7720 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.7:3301/0 2024-05-09T08:06:56.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.783+0000 7f0c85400640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f0c800c5910 0x7f0c800c7720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T08:06:56.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.783+0000 7f0c85e00640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f0c800c3110 0x7f0c80140200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:06:56.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.783+0000 7f0c84a00640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f0c800c5cf0 0x7f0c8015b380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:55384/0 (socket says 172.21.15.1:55384) 2024-05-09T08:06:56.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.783+0000 7f0c84a00640 1 -- 172.21.15.1:0/2097484053 learned_addr learned my addr 172.21.15.1:0/2097484053 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T08:06:56.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.784+0000 7f0c85e00640 1 -- 172.21.15.1:0/2097484053 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f0c800c5910 msgr2=0x7f0c800c7720 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T08:06:56.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.784+0000 7f0c85e00640 1 --2- 172.21.15.1:0/2097484053 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f0c800c5910 0x7f0c800c7720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:06:56.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.784+0000 7f0c85e00640 1 -- 172.21.15.1:0/2097484053 >> v1:172.21.15.1:6789/0 conn(0x7f0c800c5cf0 legacy=0x7f0c8015b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:06:56.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.784+0000 7f0c85e00640 1 -- 172.21.15.1:0/2097484053 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c801576f0 con 0x7f0c800c3110 2024-05-09T08:06:56.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.785+0000 7f0c85e00640 1 --2- 172.21.15.1:0/2097484053 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0c800c3110 0x7f0c80140200 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f0c780097a0 tx=0x7f0c7802a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9867e2f937313a7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:06:56.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.786+0000 7f0c7fe00640 1 -- 172.21.15.1:0/2097484053 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c78037070 con 0x7f0c800c3110 2024-05-09T08:06:56.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.786+0000 7f0c7fe00640 1 -- 172.21.15.1:0/2097484053 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c78032440 con 0x7f0c800c3110 2024-05-09T08:06:56.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.786+0000 7f0c7fe00640 1 -- 172.21.15.1:0/2097484053 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c7802ae60 con 0x7f0c800c3110 2024-05-09T08:06:56.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.786+0000 7f0c86800640 1 -- 172.21.15.1:0/2097484053 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0c800c3110 msgr2=0x7f0c80140200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:06:56.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.786+0000 7f0c86800640 1 --2- 172.21.15.1:0/2097484053 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0c800c3110 0x7f0c80140200 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f0c780097a0 tx=0x7f0c7802a110 comp rx=0 tx=0).stop 2024-05-09T08:06:56.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.787+0000 7f0c86800640 1 -- 172.21.15.1:0/2097484053 shutdown_connections 2024-05-09T08:06:56.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.787+0000 7f0c86800640 1 --2- 172.21.15.1:0/2097484053 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0c800c3110 0x7f0c80140200 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:06:56.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.787+0000 7f0c86800640 1 --2- 172.21.15.1:0/2097484053 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f0c800c5910 0x7f0c800c7720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:06:56.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.787+0000 7f0c86800640 1 -- 172.21.15.1:0/2097484053 >> 172.21.15.1:0/2097484053 conn(0x7f0c800cbc10 msgr2=0x7f0c800cc020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:06:56.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.787+0000 7f0c86800640 1 -- 172.21.15.1:0/2097484053 shutdown_connections 2024-05-09T08:06:56.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.787+0000 7f0c86800640 1 -- 172.21.15.1:0/2097484053 wait complete. 2024-05-09T08:06:56.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.788+0000 7f0c86800640 1 Processor -- start 2024-05-09T08:06:56.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.789+0000 7f0c86800640 1 -- start start 2024-05-09T08:06:56.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.789+0000 7f0c86800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c800c3110 0x7f0c80147b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:06:56.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.789+0000 7f0c86800640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0c800c5910 0x7f0c800caa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:06:56.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.790+0000 7f0c86800640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f0c800c5cf0 0x7f0c800caf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:06:56.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.790+0000 7f0c86800640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f0c801581d0 con 0x7f0c800c5910 2024-05-09T08:06:56.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.790+0000 7f0c86800640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0c80157cb0 con 0x7f0c800c3110 2024-05-09T08:06:56.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.790+0000 7f0c86800640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f0c80158fa0 con 0x7f0c800c5cf0 2024-05-09T08:06:56.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.790+0000 7f0c85e00640 1 -- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f0c800c5cf0 msgr2=0x7f0c800caf60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.7:3301/0 2024-05-09T08:06:56.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.790+0000 7f0c84a00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0c800c5910 0x7f0c800caa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:06:56.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.790+0000 7f0c85e00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f0c800c5cf0 0x7f0c800caf60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T08:06:56.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.790+0000 7f0c84a00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0c800c5910 0x7f0c800caa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:36590/0 (socket says 172.21.15.1:36590) 2024-05-09T08:06:56.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.790+0000 7f0c84a00640 1 -- 172.21.15.1:0/2431761565 learned_addr learned my addr 172.21.15.1:0/2431761565 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:06:56.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.790+0000 7f0c85400640 1 --2- 172.21.15.1:0/2431761565 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c800c3110 0x7f0c80147b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:06:56.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.790+0000 7f0c84a00640 1 -- 172.21.15.1:0/2431761565 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f0c800c5cf0 msgr2=0x7f0c800caf60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T08:06:56.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.790+0000 7f0c84a00640 1 --2- 172.21.15.1:0/2431761565 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f0c800c5cf0 0x7f0c800caf60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:06:56.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.790+0000 7f0c84a00640 1 -- 172.21.15.1:0/2431761565 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c800c3110 msgr2=0x7f0c80147b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:06:56.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.790+0000 7f0c84a00640 1 --2- 172.21.15.1:0/2431761565 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c800c3110 0x7f0c80147b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:06:56.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.790+0000 7f0c84a00640 1 -- 172.21.15.1:0/2431761565 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c801480d0 con 0x7f0c800c5910 2024-05-09T08:06:56.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.791+0000 7f0c84a00640 1 --2- 172.21.15.1:0/2431761565 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0c800c5910 0x7f0c800caa20 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f0c6c00c3f0 tx=0x7f0c6c00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:06:56.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.791+0000 7f0c7d600640 1 -- 172.21.15.1:0/2431761565 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c6c017070 con 0x7f0c800c5910 2024-05-09T08:06:56.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.791+0000 7f0c7d600640 1 -- 172.21.15.1:0/2431761565 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c6c005ce0 con 0x7f0c800c5910 2024-05-09T08:06:56.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.791+0000 7f0c7d600640 1 -- 172.21.15.1:0/2431761565 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c6c0072f0 con 0x7f0c800c5910 2024-05-09T08:06:56.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.791+0000 7f0c86800640 1 -- 172.21.15.1:0/2431761565 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c801591f0 con 0x7f0c800c5910 2024-05-09T08:06:56.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.791+0000 7f0c86800640 1 -- 172.21.15.1:0/2431761565 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c801e9ca0 con 0x7f0c800c5910 2024-05-09T08:06:56.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.792+0000 7f0c86800640 1 -- 172.21.15.1:0/2431761565 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c8004f660 con 0x7f0c800c5910 2024-05-09T08:06:56.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.793+0000 7f0c7d600640 1 -- 172.21.15.1:0/2431761565 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0c6c01a070 con 0x7f0c800c5910 2024-05-09T08:06:56.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.794+0000 7f0c7d600640 1 --2- 172.21.15.1:0/2431761565 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f0c6006bde0 0x7f0c6006e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:06:56.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.794+0000 7f0c7d600640 1 -- 172.21.15.1:0/2431761565 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7f0c6c08a630 con 0x7f0c800c5910 2024-05-09T08:06:56.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.795+0000 7f0c85400640 1 --2- 172.21.15.1:0/2431761565 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f0c6006bde0 0x7f0c6006e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:06:56.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.795+0000 7f0c85400640 1 --2- 172.21.15.1:0/2431761565 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f0c6006bde0 0x7f0c6006e2a0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f0c70009420 tx=0x7f0c7000d040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:06:56.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:56.799+0000 7f0c7d600640 1 -- 172.21.15.1:0/2431761565 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f0c6c058470 con 0x7f0c800c5910 2024-05-09T08:06:57.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:57.075+0000 7f0c86800640 1 -- 172.21.15.1:0/2431761565 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f0c8014a340 con 0x7f0c800c5910 2024-05-09T08:06:57.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:57.076+0000 7f0c7d600640 1 -- 172.21.15.1:0/2431761565 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1545 (secure 0 0 0) 0x7f0c6c05c770 con 0x7f0c800c5910 2024-05-09T08:06:57.088 INFO:teuthology.orchestra.run.smithi001.stdout:{"election_epoch":36,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":17,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"54329ec8-3270-4043-9a82-c6cd7420aa7f","modified":"2024-05-09T08:01:21.819270Z","created":"2024-05-09T08:01:21.819270Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:3300","nonce":0},{"type":"v1","addr":"172.21.15.1:6789","nonce":0}]},"addr":"172.21.15.1:6789/0","public_addr":"172.21.15.1:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:3300","nonce":0},{"type":"v1","addr":"172.21.15.7:6789","nonce":0}]},"addr":"172.21.15.7:6789/0","public_addr":"172.21.15.7:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:3301","nonce":0},{"type":"v1","addr":"172.21.15.7:6790","nonce":0}]},"addr":"172.21.15.7:6790/0","public_addr":"172.21.15.7:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-05-09T08:06:57.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:57.079+0000 7f0c86800640 1 -- 172.21.15.1:0/2431761565 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f0c6006bde0 msgr2=0x7f0c6006e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:06:57.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:57.079+0000 7f0c86800640 1 --2- 172.21.15.1:0/2431761565 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f0c6006bde0 0x7f0c6006e2a0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f0c70009420 tx=0x7f0c7000d040 comp rx=0 tx=0).stop 2024-05-09T08:06:57.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:57.079+0000 7f0c86800640 1 -- 172.21.15.1:0/2431761565 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0c800c5910 msgr2=0x7f0c800caa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:06:57.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:57.079+0000 7f0c86800640 1 --2- 172.21.15.1:0/2431761565 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0c800c5910 0x7f0c800caa20 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f0c6c00c3f0 tx=0x7f0c6c00c8b0 comp rx=0 tx=0).stop 2024-05-09T08:06:57.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:57.079+0000 7f0c86800640 1 -- 172.21.15.1:0/2431761565 shutdown_connections 2024-05-09T08:06:57.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:57.079+0000 7f0c86800640 1 --2- 172.21.15.1:0/2431761565 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f0c6006bde0 0x7f0c6006e2a0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:06:57.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:57.079+0000 7f0c86800640 1 --2- 172.21.15.1:0/2431761565 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f0c800c5cf0 0x7f0c800caf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:06:57.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:57.079+0000 7f0c86800640 1 --2- 172.21.15.1:0/2431761565 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0c800c5910 0x7f0c800caa20 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:06:57.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:57.079+0000 7f0c86800640 1 --2- 172.21.15.1:0/2431761565 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c800c3110 0x7f0c80147b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:06:57.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:57.079+0000 7f0c86800640 1 -- 172.21.15.1:0/2431761565 >> 172.21.15.1:0/2431761565 conn(0x7f0c800cbc10 msgr2=0x7f0c80149690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:06:57.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:57.079+0000 7f0c86800640 1 -- 172.21.15.1:0/2431761565 shutdown_connections 2024-05-09T08:06:57.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:06:57.079+0000 7f0c86800640 1 -- 172.21.15.1:0/2431761565 wait complete. 2024-05-09T08:06:58.899 INFO:tasks.workunit.client.0.smithi001.stdout: 2024-05-09T08:06:58.899 INFO:tasks.workunit.client.0.smithi001.stdout: Children see throughput for 1 readers = 5815961.50 kB/sec 2024-05-09T08:06:58.900 INFO:tasks.workunit.client.0.smithi001.stdout: Parent sees throughput for 1 readers = 5813307.34 kB/sec 2024-05-09T08:06:58.900 INFO:tasks.workunit.client.0.smithi001.stdout: Min throughput per process = 5815961.50 kB/sec 2024-05-09T08:06:58.900 INFO:tasks.workunit.client.0.smithi001.stdout: Max throughput per process = 5815961.50 kB/sec 2024-05-09T08:06:58.900 INFO:tasks.workunit.client.0.smithi001.stdout: Avg throughput per process = 5815961.50 kB/sec 2024-05-09T08:06:58.900 INFO:tasks.workunit.client.0.smithi001.stdout: Min xfer = 10485760.00 kB 2024-05-09T08:07:00.112 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-05-09T08:07:00.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:00.315+0000 7f26a4e00640 1 Processor -- start 2024-05-09T08:07:00.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:00.315+0000 7f26a4e00640 1 -- start start 2024-05-09T08:07:00.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:00.317+0000 7f26a4e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f26a00c6ea0 0x7f26a0072900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:07:00.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:00.317+0000 7f26a4e00640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f26a0073200 con 0x7f26a006ece0 2024-05-09T08:07:00.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:00.317+0000 7f26a4e00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f26a00734e0 con 0x7f26a006dca0 2024-05-09T08:07:00.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:00.317+0000 7f26a4e00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f26a0072e40 con 0x7f26a00c6ea0 2024-05-09T08:07:00.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:00.317+0000 7f269e800640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f26a006ece0 0x7f26a0152410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:55396/0 (socket says 172.21.15.1:55396) 2024-05-09T08:07:00.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:00.317+0000 7f269e800640 1 -- 172.21.15.1:0/2516990021 learned_addr learned my addr 172.21.15.1:0/2516990021 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T08:07:00.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:00.317+0000 7f269d400640 1 --2- 172.21.15.1:0/2516990021 >> v2:172.21.15.7:3300/0 conn(0x7f26a00c6ea0 0x7f26a0072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:07:00.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:00.318+0000 7f269d400640 1 -- 172.21.15.1:0/2516990021 >> v1:172.21.15.7:6789/0 conn(0x7f26a006dca0 legacy=0x7f26a006e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:07:00.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:00.318+0000 7f269d400640 1 -- 172.21.15.1:0/2516990021 >> v1:172.21.15.1:6789/0 conn(0x7f26a006ece0 legacy=0x7f26a0152410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:07:00.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:00.318+0000 7f269d400640 1 -- 172.21.15.1:0/2516990021 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26a0071e20 con 0x7f26a00c6ea0 2024-05-09T08:07:00.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:00.319+0000 7f269d400640 1 --2- 172.21.15.1:0/2516990021 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26a00c6ea0 0x7f26a0072900 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f268c0097a0 tx=0x7f268c02a2c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5a9130f27a05a118 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:07:00.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:00.319+0000 7f269ca00640 1 -- 172.21.15.1:0/2516990021 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f268c037070 con 0x7f26a00c6ea0 2024-05-09T08:07:00.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:00.319+0000 7f269ca00640 1 -- 172.21.15.1:0/2516990021 <== mon.1 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f268c032640 con 0x7f26a00c6ea0 2024-05-09T08:07:00.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:00.319+0000 7f269ca00640 1 -- 172.21.15.1:0/2516990021 <== mon.1 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f268c032a10 con 0x7f26a00c6ea0 2024-05-09T08:07:00.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:00.320+0000 7f26a4e00640 1 -- 172.21.15.1:0/2516990021 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26a00c6ea0 msgr2=0x7f26a0072900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:07:00.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:00.320+0000 7f26a4e00640 1 --2- 172.21.15.1:0/2516990021 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26a00c6ea0 0x7f26a0072900 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f268c0097a0 tx=0x7f268c02a2c0 comp rx=0 tx=0).stop 2024-05-09T08:07:00.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:00.320+0000 7f26a4e00640 1 -- 172.21.15.1:0/2516990021 shutdown_connections 2024-05-09T08:07:00.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:00.320+0000 7f26a4e00640 1 --2- 172.21.15.1:0/2516990021 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26a00c6ea0 0x7f26a0072900 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:00.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:00.320+0000 7f26a4e00640 1 -- 172.21.15.1:0/2516990021 >> 172.21.15.1:0/2516990021 conn(0x7f26a00cc070 msgr2=0x7f26a00cc480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:07:00.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:00.320+0000 7f26a4e00640 1 -- 172.21.15.1:0/2516990021 shutdown_connections 2024-05-09T08:07:00.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:00.321+0000 7f26a4e00640 1 -- 172.21.15.1:0/2516990021 wait complete. 2024-05-09T08:07:00.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:00.323+0000 7f26a4e00640 1 Processor -- start 2024-05-09T08:07:00.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:00.323+0000 7f26a4e00640 1 -- start start 2024-05-09T08:07:00.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:00.324+0000 7f26a4e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f26a006dca0 0x7f26a01e4130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:07:00.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:00.324+0000 7f26a4e00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f26a006ece0 0x7f26a014dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:07:00.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:00.324+0000 7f26a4e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26a00c6ea0 0x7f26a01f0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:07:00.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:00.324+0000 7f26a4e00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f26a0072470 con 0x7f26a006dca0 2024-05-09T08:07:00.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:00.324+0000 7f26a4e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f26a0152b20 con 0x7f26a00c6ea0 2024-05-09T08:07:00.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:00.324+0000 7f26a4e00640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f26a0152ca0 con 0x7f26a006ece0 2024-05-09T08:07:00.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:00.324+0000 7f269d400640 1 -- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f26a006ece0 msgr2=0x7f26a014dbc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.7:3301/0 2024-05-09T08:07:00.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:00.324+0000 7f269d400640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f26a006ece0 0x7f26a014dbc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T08:07:00.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:00.325+0000 7f269de00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f26a006dca0 0x7f26a01e4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:07:00.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:00.325+0000 7f269de00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f26a006dca0 0x7f26a01e4130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:36592/0 (socket says 172.21.15.1:36592) 2024-05-09T08:07:00.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:00.325+0000 7f269de00640 1 -- 172.21.15.1:0/2723391103 learned_addr learned my addr 172.21.15.1:0/2723391103 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:07:00.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:00.325+0000 7f269e800640 1 --2- 172.21.15.1:0/2723391103 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26a00c6ea0 0x7f26a01f0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:07:00.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:00.325+0000 7f269e800640 1 -- 172.21.15.1:0/2723391103 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f26a006ece0 msgr2=0x7f26a014dbc0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T08:07:00.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:00.325+0000 7f269e800640 1 --2- 172.21.15.1:0/2723391103 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f26a006ece0 0x7f26a014dbc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:00.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:00.325+0000 7f269e800640 1 -- 172.21.15.1:0/2723391103 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f26a006dca0 msgr2=0x7f26a01e4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:07:00.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:00.325+0000 7f269e800640 1 --2- 172.21.15.1:0/2723391103 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f26a006dca0 0x7f26a01e4130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:00.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:00.325+0000 7f269e800640 1 -- 172.21.15.1:0/2723391103 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26a01f3b60 con 0x7f26a00c6ea0 2024-05-09T08:07:00.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:00.325+0000 7f269de00640 1 --2- 172.21.15.1:0/2723391103 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f26a006dca0 0x7f26a01e4130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:07:00.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:00.325+0000 7f269e800640 1 --2- 172.21.15.1:0/2723391103 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26a00c6ea0 0x7f26a01f0670 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f269800cb00 tx=0x7f269800a060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:07:00.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:00.326+0000 7f2696000640 1 -- 172.21.15.1:0/2723391103 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2698017070 con 0x7f26a00c6ea0 2024-05-09T08:07:00.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:00.326+0000 7f2696000640 1 -- 172.21.15.1:0/2723391103 <== mon.1 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f269800aca0 con 0x7f26a00c6ea0 2024-05-09T08:07:00.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:00.326+0000 7f26a4e00640 1 -- 172.21.15.1:0/2723391103 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26a01f3e40 con 0x7f26a00c6ea0 2024-05-09T08:07:00.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:00.326+0000 7f2696000640 1 -- 172.21.15.1:0/2723391103 <== mon.1 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2698012600 con 0x7f26a00c6ea0 2024-05-09T08:07:00.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:00.326+0000 7f26a4e00640 1 -- 172.21.15.1:0/2723391103 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26a01f4400 con 0x7f26a00c6ea0 2024-05-09T08:07:00.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:00.327+0000 7f26a4e00640 1 -- 172.21.15.1:0/2723391103 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26a0074d70 con 0x7f26a00c6ea0 2024-05-09T08:07:00.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:00.328+0000 7f2696000640 1 -- 172.21.15.1:0/2723391103 <== mon.1 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f269801a070 con 0x7f26a00c6ea0 2024-05-09T08:07:00.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:00.329+0000 7f2696000640 1 --2- 172.21.15.1:0/2723391103 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f268006bde0 0x7f268006e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:07:00.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:00.329+0000 7f2696000640 1 -- 172.21.15.1:0/2723391103 <== mon.1 v2:172.21.15.7:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7f269808aee0 con 0x7f26a00c6ea0 2024-05-09T08:07:00.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:00.330+0000 7f269de00640 1 --2- 172.21.15.1:0/2723391103 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f268006bde0 0x7f268006e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:07:00.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:00.330+0000 7f269de00640 1 --2- 172.21.15.1:0/2723391103 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f268006bde0 0x7f268006e2a0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f2690007fd0 tx=0x7f2690009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:07:00.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:00.334+0000 7f2696000640 1 -- 172.21.15.1:0/2723391103 <== mon.1 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f2698058db0 con 0x7f26a00c6ea0 2024-05-09T08:07:00.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:00.604+0000 7f26a4e00640 1 -- 172.21.15.1:0/2723391103 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f26a00ccd20 con 0x7f26a00c6ea0 2024-05-09T08:07:00.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:00.604+0000 7f2696000640 1 -- 172.21.15.1:0/2723391103 <== mon.1 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1545 (secure 0 0 0) 0x7f269805d0b0 con 0x7f26a00c6ea0 2024-05-09T08:07:00.616 INFO:teuthology.orchestra.run.smithi001.stdout:{"election_epoch":36,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":21,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"54329ec8-3270-4043-9a82-c6cd7420aa7f","modified":"2024-05-09T08:01:21.819270Z","created":"2024-05-09T08:01:21.819270Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:3300","nonce":0},{"type":"v1","addr":"172.21.15.1:6789","nonce":0}]},"addr":"172.21.15.1:6789/0","public_addr":"172.21.15.1:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:3300","nonce":0},{"type":"v1","addr":"172.21.15.7:6789","nonce":0}]},"addr":"172.21.15.7:6789/0","public_addr":"172.21.15.7:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:3301","nonce":0},{"type":"v1","addr":"172.21.15.7:6790","nonce":0}]},"addr":"172.21.15.7:6790/0","public_addr":"172.21.15.7:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-05-09T08:07:00.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:00.607+0000 7f26a4e00640 1 -- 172.21.15.1:0/2723391103 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f268006bde0 msgr2=0x7f268006e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:07:00.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:00.607+0000 7f26a4e00640 1 --2- 172.21.15.1:0/2723391103 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f268006bde0 0x7f268006e2a0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f2690007fd0 tx=0x7f2690009040 comp rx=0 tx=0).stop 2024-05-09T08:07:00.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:00.607+0000 7f26a4e00640 1 -- 172.21.15.1:0/2723391103 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26a00c6ea0 msgr2=0x7f26a01f0670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:07:00.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:00.607+0000 7f26a4e00640 1 --2- 172.21.15.1:0/2723391103 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26a00c6ea0 0x7f26a01f0670 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f269800cb00 tx=0x7f269800a060 comp rx=0 tx=0).stop 2024-05-09T08:07:00.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:00.607+0000 7f26a4e00640 1 -- 172.21.15.1:0/2723391103 shutdown_connections 2024-05-09T08:07:00.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:00.607+0000 7f26a4e00640 1 --2- 172.21.15.1:0/2723391103 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f268006bde0 0x7f268006e2a0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:00.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:00.607+0000 7f26a4e00640 1 --2- 172.21.15.1:0/2723391103 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26a00c6ea0 0x7f26a01f0670 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:00.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:00.607+0000 7f26a4e00640 1 --2- 172.21.15.1:0/2723391103 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f26a006ece0 0x7f26a014dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:00.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:00.607+0000 7f26a4e00640 1 --2- 172.21.15.1:0/2723391103 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f26a006dca0 0x7f26a01e4130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:00.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:00.607+0000 7f26a4e00640 1 -- 172.21.15.1:0/2723391103 >> 172.21.15.1:0/2723391103 conn(0x7f26a00cc070 msgr2=0x7f26a0152350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:07:00.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:00.607+0000 7f26a4e00640 1 -- 172.21.15.1:0/2723391103 shutdown_connections 2024-05-09T08:07:00.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:00.607+0000 7f26a4e00640 1 -- 172.21.15.1:0/2723391103 wait complete. 2024-05-09T08:07:03.639 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-05-09T08:07:03.727 INFO:tasks.workunit.client.0.smithi001.stdout: 2024-05-09T08:07:03.727 INFO:tasks.workunit.client.0.smithi001.stdout: Children see throughput for 1 re-readers = 3840223.00 kB/sec 2024-05-09T08:07:03.727 INFO:tasks.workunit.client.0.smithi001.stdout: Parent sees throughput for 1 re-readers = 3839191.26 kB/sec 2024-05-09T08:07:03.727 INFO:tasks.workunit.client.0.smithi001.stdout: Min throughput per process = 3840223.00 kB/sec 2024-05-09T08:07:03.727 INFO:tasks.workunit.client.0.smithi001.stdout: Max throughput per process = 3840223.00 kB/sec 2024-05-09T08:07:03.727 INFO:tasks.workunit.client.0.smithi001.stdout: Avg throughput per process = 3840223.00 kB/sec 2024-05-09T08:07:03.728 INFO:tasks.workunit.client.0.smithi001.stdout: Min xfer = 10485760.00 kB 2024-05-09T08:07:03.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:03.829+0000 7f234fe00640 1 Processor -- start 2024-05-09T08:07:03.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:03.829+0000 7f234fe00640 1 -- start start 2024-05-09T08:07:03.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:03.830+0000 7f234fe00640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f235015a440 0x7f23500c7dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:07:03.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:03.830+0000 7f234fe00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f23500c44b0 0x7f235006b960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:07:03.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:03.830+0000 7f234fe00640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f23500c48c0 con 0x7f2350154790 2024-05-09T08:07:03.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:03.830+0000 7f234fe00640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f2350154b70 con 0x7f235015a440 2024-05-09T08:07:03.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:03.830+0000 7f234fe00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f23500c6a50 con 0x7f23500c44b0 2024-05-09T08:07:03.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:03.831+0000 7f2347e00640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f235015a440 0x7f23500c7dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:07:03.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:03.831+0000 7f2347e00640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f235015a440 0x7f23500c7dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:36600/0 (socket says 172.21.15.1:36600) 2024-05-09T08:07:03.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:03.831+0000 7f2347e00640 1 -- 172.21.15.1:0/2713009906 learned_addr learned my addr 172.21.15.1:0/2713009906 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:07:03.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:03.831+0000 7f234f400640 1 --2- 172.21.15.1:0/2713009906 >> v2:172.21.15.7:3300/0 conn(0x7f23500c44b0 0x7f235006b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:07:03.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:03.831+0000 7f2347e00640 1 -- 172.21.15.1:0/2713009906 >> v2:172.21.15.7:3300/0 conn(0x7f23500c44b0 msgr2=0x7f235006b960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:07:03.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:03.831+0000 7f2347e00640 1 --2- 172.21.15.1:0/2713009906 >> v2:172.21.15.7:3300/0 conn(0x7f23500c44b0 0x7f235006b960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:03.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:03.831+0000 7f2347e00640 1 -- 172.21.15.1:0/2713009906 >> v1:172.21.15.1:6789/0 conn(0x7f2350154790 legacy=0x7f235006b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:07:03.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:03.831+0000 7f2347e00640 1 -- 172.21.15.1:0/2713009906 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23500c6d30 con 0x7f235015a440 2024-05-09T08:07:03.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:03.833+0000 7f2347e00640 1 --2- 172.21.15.1:0/2713009906 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f235015a440 0x7f23500c7dd0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f23500c86c0 tx=0x7f233802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3edcf6153dcdab8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:07:03.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:03.833+0000 7f234e000640 1 -- 172.21.15.1:0/2713009906 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2338037070 con 0x7f235015a440 2024-05-09T08:07:03.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:03.833+0000 7f234e000640 1 -- 172.21.15.1:0/2713009906 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2338032440 con 0x7f235015a440 2024-05-09T08:07:03.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:03.833+0000 7f234e000640 1 -- 172.21.15.1:0/2713009906 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2338032810 con 0x7f235015a440 2024-05-09T08:07:03.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:03.833+0000 7f234fe00640 1 -- 172.21.15.1:0/2713009906 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f235015a440 msgr2=0x7f23500c7dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:07:03.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:03.833+0000 7f234fe00640 1 --2- 172.21.15.1:0/2713009906 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f235015a440 0x7f23500c7dd0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f23500c86c0 tx=0x7f233802a270 comp rx=0 tx=0).stop 2024-05-09T08:07:03.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:03.834+0000 7f234fe00640 1 -- 172.21.15.1:0/2713009906 shutdown_connections 2024-05-09T08:07:03.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:03.834+0000 7f234fe00640 1 --2- 172.21.15.1:0/2713009906 >> v2:172.21.15.7:3300/0 conn(0x7f23500c44b0 0x7f235006b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:03.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:03.834+0000 7f234fe00640 1 --2- 172.21.15.1:0/2713009906 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f235015a440 0x7f23500c7dd0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:03.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:03.834+0000 7f234fe00640 1 -- 172.21.15.1:0/2713009906 >> 172.21.15.1:0/2713009906 conn(0x7f23500cbc10 msgr2=0x7f23500cc020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:07:03.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:03.834+0000 7f234fe00640 1 -- 172.21.15.1:0/2713009906 shutdown_connections 2024-05-09T08:07:03.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:03.834+0000 7f234fe00640 1 -- 172.21.15.1:0/2713009906 wait complete. 2024-05-09T08:07:03.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:03.836+0000 7f234fe00640 1 Processor -- start 2024-05-09T08:07:03.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:03.836+0000 7f234fe00640 1 -- start start 2024-05-09T08:07:03.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:03.837+0000 7f234fe00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f23500c44b0 0x7f23501444b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:07:03.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:03.837+0000 7f234fe00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f2350154790 0x7f23500c9db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:07:03.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:03.837+0000 7f234fe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f235015a440 0x7f23500ca2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:07:03.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:03.837+0000 7f234fe00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f23500c75d0 con 0x7f23500c44b0 2024-05-09T08:07:03.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:03.837+0000 7f234fe00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f23500c72f0 con 0x7f235015a440 2024-05-09T08:07:03.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:03.837+0000 7f234fe00640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f23500c78b0 con 0x7f2350154790 2024-05-09T08:07:03.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:03.837+0000 7f2347e00640 1 -- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f2350154790 msgr2=0x7f23500c9db0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.7:3301/0 2024-05-09T08:07:03.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:03.837+0000 7f2347e00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f2350154790 0x7f23500c9db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T08:07:03.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:03.837+0000 7f234ea00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f23500c44b0 0x7f23501444b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:07:03.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:03.837+0000 7f234f400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f235015a440 0x7f23500ca2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:07:03.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:03.837+0000 7f234ea00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f23500c44b0 0x7f23501444b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:51944/0 (socket says 172.21.15.1:51944) 2024-05-09T08:07:03.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:03.837+0000 7f234ea00640 1 -- 172.21.15.1:0/159306662 learned_addr learned my addr 172.21.15.1:0/159306662 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:07:03.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:03.838+0000 7f234ea00640 1 -- 172.21.15.1:0/159306662 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f2350154790 msgr2=0x7f23500c9db0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T08:07:03.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:03.838+0000 7f234ea00640 1 --2- 172.21.15.1:0/159306662 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f2350154790 0x7f23500c9db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:03.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:03.838+0000 7f234ea00640 1 -- 172.21.15.1:0/159306662 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f235015a440 msgr2=0x7f23500ca2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:07:03.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:03.838+0000 7f234ea00640 1 --2- 172.21.15.1:0/159306662 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f235015a440 0x7f23500ca2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:03.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:03.838+0000 7f234ea00640 1 -- 172.21.15.1:0/159306662 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2350141460 con 0x7f23500c44b0 2024-05-09T08:07:03.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:03.838+0000 7f234f400640 1 --2- 172.21.15.1:0/159306662 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f235015a440 0x7f23500ca2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:07:03.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:03.838+0000 7f234ea00640 1 --2- 172.21.15.1:0/159306662 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f23500c44b0 0x7f23501444b0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f234000c400 tx=0x7f234000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:07:03.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:03.838+0000 7f2346a00640 1 -- 172.21.15.1:0/159306662 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23400052d0 con 0x7f23500c44b0 2024-05-09T08:07:03.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:03.839+0000 7f2346a00640 1 -- 172.21.15.1:0/159306662 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2340005490 con 0x7f23500c44b0 2024-05-09T08:07:03.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:03.839+0000 7f2346a00640 1 -- 172.21.15.1:0/159306662 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2340007260 con 0x7f23500c44b0 2024-05-09T08:07:03.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:03.839+0000 7f234fe00640 1 -- 172.21.15.1:0/159306662 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23500cb1b0 con 0x7f23500c44b0 2024-05-09T08:07:03.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:03.839+0000 7f234fe00640 1 -- 172.21.15.1:0/159306662 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23501f1850 con 0x7f23500c44b0 2024-05-09T08:07:03.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:03.840+0000 7f234fe00640 1 -- 172.21.15.1:0/159306662 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f235004f660 con 0x7f23500c44b0 2024-05-09T08:07:03.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:03.841+0000 7f2346a00640 1 -- 172.21.15.1:0/159306662 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2340007400 con 0x7f23500c44b0 2024-05-09T08:07:03.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:03.842+0000 7f2346a00640 1 --2- 172.21.15.1:0/159306662 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f232c06bde0 0x7f232c06e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:07:03.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:03.842+0000 7f2346a00640 1 -- 172.21.15.1:0/159306662 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7f234008a810 con 0x7f23500c44b0 2024-05-09T08:07:03.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:03.842+0000 7f2347e00640 1 --2- 172.21.15.1:0/159306662 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f232c06bde0 0x7f232c06e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:07:03.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:03.843+0000 7f2347e00640 1 --2- 172.21.15.1:0/159306662 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f232c06bde0 0x7f232c06e2a0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f2338034000 tx=0x7f233802e000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:07:03.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:03.847+0000 7f2346a00640 1 -- 172.21.15.1:0/159306662 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f2340058650 con 0x7f23500c44b0 2024-05-09T08:07:04.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:04.119+0000 7f234fe00640 1 -- 172.21.15.1:0/159306662 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f2350158340 con 0x7f23500c44b0 2024-05-09T08:07:04.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:04.120+0000 7f2346a00640 1 -- 172.21.15.1:0/159306662 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1545 (secure 0 0 0) 0x7f234005c950 con 0x7f23500c44b0 2024-05-09T08:07:04.131 INFO:teuthology.orchestra.run.smithi001.stdout:{"election_epoch":36,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":24,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"54329ec8-3270-4043-9a82-c6cd7420aa7f","modified":"2024-05-09T08:01:21.819270Z","created":"2024-05-09T08:01:21.819270Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:3300","nonce":0},{"type":"v1","addr":"172.21.15.1:6789","nonce":0}]},"addr":"172.21.15.1:6789/0","public_addr":"172.21.15.1:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:3300","nonce":0},{"type":"v1","addr":"172.21.15.7:6789","nonce":0}]},"addr":"172.21.15.7:6789/0","public_addr":"172.21.15.7:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:3301","nonce":0},{"type":"v1","addr":"172.21.15.7:6790","nonce":0}]},"addr":"172.21.15.7:6790/0","public_addr":"172.21.15.7:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-05-09T08:07:04.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:04.122+0000 7f234fe00640 1 -- 172.21.15.1:0/159306662 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f232c06bde0 msgr2=0x7f232c06e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:07:04.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:04.122+0000 7f234fe00640 1 --2- 172.21.15.1:0/159306662 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f232c06bde0 0x7f232c06e2a0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f2338034000 tx=0x7f233802e000 comp rx=0 tx=0).stop 2024-05-09T08:07:04.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:04.122+0000 7f234fe00640 1 -- 172.21.15.1:0/159306662 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f23500c44b0 msgr2=0x7f23501444b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:07:04.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:04.122+0000 7f234fe00640 1 --2- 172.21.15.1:0/159306662 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f23500c44b0 0x7f23501444b0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f234000c400 tx=0x7f234000c8c0 comp rx=0 tx=0).stop 2024-05-09T08:07:04.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:04.122+0000 7f234fe00640 1 -- 172.21.15.1:0/159306662 shutdown_connections 2024-05-09T08:07:04.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:04.122+0000 7f234fe00640 1 --2- 172.21.15.1:0/159306662 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f232c06bde0 0x7f232c06e2a0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:04.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:04.122+0000 7f234fe00640 1 --2- 172.21.15.1:0/159306662 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f235015a440 0x7f23500ca2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:04.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:04.122+0000 7f234fe00640 1 --2- 172.21.15.1:0/159306662 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f2350154790 0x7f23500c9db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:04.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:04.122+0000 7f234fe00640 1 --2- 172.21.15.1:0/159306662 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f23500c44b0 0x7f23501444b0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:04.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:04.122+0000 7f234fe00640 1 -- 172.21.15.1:0/159306662 >> 172.21.15.1:0/159306662 conn(0x7f23500cbc10 msgr2=0x7f2350146b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:07:04.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:04.123+0000 7f234fe00640 1 -- 172.21.15.1:0/159306662 shutdown_connections 2024-05-09T08:07:04.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:04.123+0000 7f234fe00640 1 -- 172.21.15.1:0/159306662 wait complete. 2024-05-09T08:07:07.153 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-05-09T08:07:07.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:07.365+0000 7f8e08a00640 1 Processor -- start 2024-05-09T08:07:07.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:07.365+0000 7f8e08a00640 1 -- start start 2024-05-09T08:07:07.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:07.366+0000 7f8e08a00640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f8e040c5910 0x7f8e040c7720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:07:07.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:07.366+0000 7f8e08a00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f8e040c5cf0 0x7f8e0415b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:07:07.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:07.367+0000 7f8e08a00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8e0415b8c0 con 0x7f8e040c3110 2024-05-09T08:07:07.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:07.367+0000 7f8e08a00640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f8e04157430 con 0x7f8e040c5910 2024-05-09T08:07:07.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:07.367+0000 7f8e08a00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f8e040c7d60 con 0x7f8e040c5cf0 2024-05-09T08:07:07.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:07.367+0000 7f8e02800640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f8e040c5910 0x7f8e040c7720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:07:07.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:07.367+0000 7f8dfbe00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f8e040c5cf0 0x7f8e0415b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:07:07.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:07.367+0000 7f8e03200640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f8e040c3110 0x7f8e04140200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.1:57816/0 (socket says 172.21.15.1:57816) 2024-05-09T08:07:07.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:07.367+0000 7f8e03200640 1 -- 172.21.15.1:0/3801943622 learned_addr learned my addr 172.21.15.1:0/3801943622 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T08:07:07.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:07.367+0000 7f8e02800640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f8e040c5910 0x7f8e040c7720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:51952/0 (socket says 172.21.15.1:51952) 2024-05-09T08:07:07.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:07.367+0000 7f8e02800640 1 -- 172.21.15.1:0/3801943622 >> v2:172.21.15.1:3300/0 conn(0x7f8e040c5910 msgr2=0x7f8e040c7720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-05-09T08:07:07.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:07.367+0000 7f8e02800640 1 -- 172.21.15.1:0/3801943622 >> v2:172.21.15.1:3300/0 conn(0x7f8e040c5910 msgr2=0x7f8e040c7720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-09T08:07:07.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:07.367+0000 7f8e02800640 1 --2- 172.21.15.1:0/3801943622 >> v2:172.21.15.1:3300/0 conn(0x7f8e040c5910 0x7f8e040c7720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-09T08:07:07.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:07.367+0000 7f8dfbe00640 1 -- 172.21.15.1:0/3801943622 >> v2:172.21.15.7:3300/0 conn(0x7f8e040c5cf0 msgr2=0x7f8e0415b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-09T08:07:07.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:07.367+0000 7f8dfbe00640 1 -- 172.21.15.1:0/3801943622 >> v2:172.21.15.7:3300/0 conn(0x7f8e040c5cf0 msgr2=0x7f8e0415b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-09T08:07:07.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:07.367+0000 7f8e02800640 1 --2- 172.21.15.1:0/3801943622 >> v2:172.21.15.1:3300/0 conn(0x7f8e040c5910 0x7f8e040c7720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T08:07:07.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:07.367+0000 7f8dfbe00640 1 --2- 172.21.15.1:0/3801943622 >> v2:172.21.15.7:3300/0 conn(0x7f8e040c5cf0 0x7f8e0415b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-09T08:07:07.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:07.367+0000 7f8dfbe00640 1 --2- 172.21.15.1:0/3801943622 >> v2:172.21.15.7:3300/0 conn(0x7f8e040c5cf0 0x7f8e0415b380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T08:07:07.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:07.568+0000 7f8dfbe00640 1 --2- 172.21.15.1:0/3801943622 >> v2:172.21.15.7:3300/0 conn(0x7f8e040c5cf0 0x7f8e0415b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:07:07.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:07.568+0000 7f8e02800640 1 --2- 172.21.15.1:0/3801943622 >> v2:172.21.15.1:3300/0 conn(0x7f8e040c5910 0x7f8e040c7720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:07:07.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:07.568+0000 7f8dfbe00640 1 -- 172.21.15.1:0/3801943622 >> v2:172.21.15.7:3300/0 conn(0x7f8e040c5cf0 msgr2=0x7f8e0415b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-09T08:07:07.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:07.568+0000 7f8dfbe00640 1 -- 172.21.15.1:0/3801943622 >> v2:172.21.15.7:3300/0 conn(0x7f8e040c5cf0 msgr2=0x7f8e0415b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-09T08:07:07.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:07.568+0000 7f8dfbe00640 1 --2- 172.21.15.1:0/3801943622 >> v2:172.21.15.7:3300/0 conn(0x7f8e040c5cf0 0x7f8e0415b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-09T08:07:07.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:07.568+0000 7f8dfbe00640 1 --2- 172.21.15.1:0/3801943622 >> v2:172.21.15.7:3300/0 conn(0x7f8e040c5cf0 0x7f8e0415b380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-09T08:07:07.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:07.569+0000 7f8e02800640 1 -- 172.21.15.1:0/3801943622 >> v2:172.21.15.1:3300/0 conn(0x7f8e040c5910 msgr2=0x7f8e040c7720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-05-09T08:07:07.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:07.569+0000 7f8e02800640 1 -- 172.21.15.1:0/3801943622 >> v2:172.21.15.1:3300/0 conn(0x7f8e040c5910 msgr2=0x7f8e040c7720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-09T08:07:07.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:07.569+0000 7f8e02800640 1 --2- 172.21.15.1:0/3801943622 >> v2:172.21.15.1:3300/0 conn(0x7f8e040c5910 0x7f8e040c7720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-09T08:07:07.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:07.569+0000 7f8e02800640 1 --2- 172.21.15.1:0/3801943622 >> v2:172.21.15.1:3300/0 conn(0x7f8e040c5910 0x7f8e040c7720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-09T08:07:07.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:07.970+0000 7f8dfbe00640 1 --2- 172.21.15.1:0/3801943622 >> v2:172.21.15.7:3300/0 conn(0x7f8e040c5cf0 0x7f8e0415b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:07:07.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:07.970+0000 7f8dfbe00640 1 -- 172.21.15.1:0/3801943622 >> v2:172.21.15.7:3300/0 conn(0x7f8e040c5cf0 msgr2=0x7f8e0415b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-09T08:07:07.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:07.970+0000 7f8dfbe00640 1 -- 172.21.15.1:0/3801943622 >> v2:172.21.15.7:3300/0 conn(0x7f8e040c5cf0 msgr2=0x7f8e0415b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-09T08:07:07.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:07.970+0000 7f8dfbe00640 1 --2- 172.21.15.1:0/3801943622 >> v2:172.21.15.7:3300/0 conn(0x7f8e040c5cf0 0x7f8e0415b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-09T08:07:07.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:07.970+0000 7f8dfbe00640 1 --2- 172.21.15.1:0/3801943622 >> v2:172.21.15.7:3300/0 conn(0x7f8e040c5cf0 0x7f8e0415b380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-09T08:07:07.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:07.972+0000 7f8e02800640 1 --2- 172.21.15.1:0/3801943622 >> v2:172.21.15.1:3300/0 conn(0x7f8e040c5910 0x7f8e040c7720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:07:07.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:07.972+0000 7f8e02800640 1 -- 172.21.15.1:0/3801943622 >> v2:172.21.15.1:3300/0 conn(0x7f8e040c5910 msgr2=0x7f8e040c7720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-05-09T08:07:07.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:07.972+0000 7f8e02800640 1 -- 172.21.15.1:0/3801943622 >> v2:172.21.15.1:3300/0 conn(0x7f8e040c5910 msgr2=0x7f8e040c7720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-09T08:07:07.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:07.972+0000 7f8e02800640 1 --2- 172.21.15.1:0/3801943622 >> v2:172.21.15.1:3300/0 conn(0x7f8e040c5910 0x7f8e040c7720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-09T08:07:07.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:07.972+0000 7f8e02800640 1 --2- 172.21.15.1:0/3801943622 >> v2:172.21.15.1:3300/0 conn(0x7f8e040c5910 0x7f8e040c7720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-09T08:07:08.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:08.771+0000 7f8dfbe00640 1 --2- 172.21.15.1:0/3801943622 >> v2:172.21.15.7:3300/0 conn(0x7f8e040c5cf0 0x7f8e0415b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:07:08.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:08.772+0000 7f8dfbe00640 1 -- 172.21.15.1:0/3801943622 >> v2:172.21.15.7:3300/0 conn(0x7f8e040c5cf0 msgr2=0x7f8e0415b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-09T08:07:08.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:08.772+0000 7f8dfbe00640 1 -- 172.21.15.1:0/3801943622 >> v2:172.21.15.7:3300/0 conn(0x7f8e040c5cf0 msgr2=0x7f8e0415b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-09T08:07:08.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:08.772+0000 7f8dfbe00640 1 --2- 172.21.15.1:0/3801943622 >> v2:172.21.15.7:3300/0 conn(0x7f8e040c5cf0 0x7f8e0415b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-09T08:07:08.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:08.772+0000 7f8dfbe00640 1 --2- 172.21.15.1:0/3801943622 >> v2:172.21.15.7:3300/0 conn(0x7f8e040c5cf0 0x7f8e0415b380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-09T08:07:08.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:08.773+0000 7f8e02800640 1 --2- 172.21.15.1:0/3801943622 >> v2:172.21.15.1:3300/0 conn(0x7f8e040c5910 0x7f8e040c7720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:07:08.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:08.775+0000 7f8e02800640 1 -- 172.21.15.1:0/3801943622 >> v2:172.21.15.1:3300/0 conn(0x7f8e040c5910 msgr2=0x7f8e040c7720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-09T08:07:08.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:08.775+0000 7f8e02800640 1 -- 172.21.15.1:0/3801943622 >> v2:172.21.15.1:3300/0 conn(0x7f8e040c5910 msgr2=0x7f8e040c7720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-09T08:07:08.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:08.775+0000 7f8e02800640 1 --2- 172.21.15.1:0/3801943622 >> v2:172.21.15.1:3300/0 conn(0x7f8e040c5910 0x7f8e040c7720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-09T08:07:08.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:08.775+0000 7f8e02800640 1 --2- 172.21.15.1:0/3801943622 >> v2:172.21.15.1:3300/0 conn(0x7f8e040c5910 0x7f8e040c7720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-09T08:07:10.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:10.368+0000 7f8e00a00640 1 -- 172.21.15.1:0/3801943622 >> v2:172.21.15.7:3300/0 conn(0x7f8e040c5cf0 msgr2=0x7f8e0415b380 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T08:07:10.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:10.368+0000 7f8e00a00640 1 --2- 172.21.15.1:0/3801943622 >> v2:172.21.15.7:3300/0 conn(0x7f8e040c5cf0 0x7f8e0415b380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:10.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:10.368+0000 7f8e00a00640 1 -- 172.21.15.1:0/3801943622 >> v2:172.21.15.1:3300/0 conn(0x7f8e040c5910 msgr2=0x7f8e040c7720 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T08:07:10.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:10.368+0000 7f8e00a00640 1 --2- 172.21.15.1:0/3801943622 >> v2:172.21.15.1:3300/0 conn(0x7f8e040c5910 0x7f8e040c7720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:10.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:10.368+0000 7f8e00a00640 1 -- 172.21.15.1:0/3801943622 >> v1:172.21.15.7:6789/0 conn(0x7f8e040c3110 legacy=0x7f8e04140200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:07:10.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:10.368+0000 7f8e00a00640 1 --2- 172.21.15.1:0/3801943622 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f8de0003670 0x7f8de0005ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:07:10.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:10.368+0000 7f8e00a00640 1 -- 172.21.15.1:0/3801943622 --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8e040c8040 con 0x7f8de0006150 2024-05-09T08:07:10.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:10.368+0000 7f8e00a00640 1 -- 172.21.15.1:0/3801943622 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f8e04157430 con 0x7f8de0003670 2024-05-09T08:07:10.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:10.369+0000 7f8e02800640 1 -- 172.21.15.1:0/3801943622 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f8de0003670 msgr2=0x7f8de0005ac0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.7:3301/0 2024-05-09T08:07:10.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:10.369+0000 7f8e02800640 1 --2- 172.21.15.1:0/3801943622 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f8de0003670 0x7f8de0005ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T08:07:10.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:10.570+0000 7f8e02800640 1 -- 172.21.15.1:0/3801943622 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f8de0003670 msgr2=0x7f8de0005ac0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.7:3301/0 2024-05-09T08:07:10.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:10.570+0000 7f8e02800640 1 --2- 172.21.15.1:0/3801943622 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f8de0003670 0x7f8de0005ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-09T08:07:10.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:10.971+0000 7f8e02800640 1 -- 172.21.15.1:0/3801943622 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f8de0003670 msgr2=0x7f8de0005ac0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.7:3301/0 2024-05-09T08:07:10.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:10.971+0000 7f8e02800640 1 --2- 172.21.15.1:0/3801943622 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f8de0003670 0x7f8de0005ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-09T08:07:11.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:11.326+0000 7f8e01e00640 1 -- 172.21.15.1:0/3801943622 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1153180785 0 0) 0x7f8e040c8040 con 0x7f8de0006150 2024-05-09T08:07:11.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:11.326+0000 7f8e01e00640 1 -- 172.21.15.1:0/3801943622 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8de8005730 con 0x7f8de0006150 2024-05-09T08:07:11.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:11.330+0000 7f8e01e00640 1 -- 172.21.15.1:0/3801943622 <== mon.0 v1:172.21.15.1:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2146280869 0 0) 0x7f8de8005730 con 0x7f8de0006150 2024-05-09T08:07:11.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:11.330+0000 7f8e01e00640 1 -- 172.21.15.1:0/3801943622 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f8de0003670 msgr2=0x7f8de0005ac0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T08:07:11.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:11.330+0000 7f8e01e00640 1 --2- 172.21.15.1:0/3801943622 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f8de0003670 0x7f8de0005ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:11.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:11.330+0000 7f8e01e00640 1 -- 172.21.15.1:0/3801943622 --> v1:172.21.15.1:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8de000af30 con 0x7f8de0006150 2024-05-09T08:07:11.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:11.330+0000 7f8e01e00640 1 -- 172.21.15.1:0/3801943622 <== mon.0 v1:172.21.15.1:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4179846320 0 0) 0x7f8dec00d580 con 0x7f8de0006150 2024-05-09T08:07:11.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:11.330+0000 7f8e01e00640 1 -- 172.21.15.1:0/3801943622 >> v1:172.21.15.1:6789/0 conn(0x7f8de0006150 legacy=0x7f8de00085f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:07:11.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:11.330+0000 7f8e03200640 1 -- 172.21.15.1:0/3801943622 reap_dead start 2024-05-09T08:07:11.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:11.330+0000 7f8e01e00640 1 --2- 172.21.15.1:0/3801943622 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8de8005bc0 0x7f8de8005fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:07:11.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:11.330+0000 7f8e01e00640 1 --2- 172.21.15.1:0/3801943622 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8de8006a10 0x7f8de8006e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:07:11.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:11.330+0000 7f8e01e00640 1 --2- 172.21.15.1:0/3801943622 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f8de80074f0 0x7f8de800dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:07:11.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:11.331+0000 7f8e01e00640 1 -- 172.21.15.1:0/3801943622 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f8e04157430 con 0x7f8de8006a10 2024-05-09T08:07:11.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:11.331+0000 7f8e01e00640 1 -- 172.21.15.1:0/3801943622 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f8de800e440 con 0x7f8de8005bc0 2024-05-09T08:07:11.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:11.331+0000 7f8e01e00640 1 -- 172.21.15.1:0/3801943622 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f8de800e640 con 0x7f8de80074f0 2024-05-09T08:07:11.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:11.331+0000 7f8e03200640 1 -- 172.21.15.1:0/3801943622 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f8de80074f0 msgr2=0x7f8de800dbd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.7:3301/0 2024-05-09T08:07:11.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:11.331+0000 7f8e03200640 1 --2- 172.21.15.1:0/3801943622 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f8de80074f0 0x7f8de800dbd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T08:07:11.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:11.331+0000 7f8e02800640 1 --2- 172.21.15.1:0/3801943622 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8de8005bc0 0x7f8de8005fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:07:11.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:11.332+0000 7f8e02800640 1 -- 172.21.15.1:0/3801943622 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f8de80074f0 msgr2=0x7f8de800dbd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T08:07:11.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:11.332+0000 7f8e02800640 1 --2- 172.21.15.1:0/3801943622 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f8de80074f0 0x7f8de800dbd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:11.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:11.332+0000 7f8e02800640 1 -- 172.21.15.1:0/3801943622 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8de8006a10 msgr2=0x7f8de8006e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:07:11.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:11.332+0000 7f8e02800640 1 --2- 172.21.15.1:0/3801943622 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8de8006a10 0x7f8de8006e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:11.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:11.332+0000 7f8e02800640 1 -- 172.21.15.1:0/3801943622 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8de8005730 con 0x7f8de8005bc0 2024-05-09T08:07:11.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:11.337+0000 7f8e02800640 1 --2- 172.21.15.1:0/3801943622 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8de8005bc0 0x7f8de8005fc0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f8df4006fd0 tx=0x7f8de8007a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=da5e247a4be4ab64 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:07:11.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:11.339+0000 7f8e01e00640 1 -- 172.21.15.1:0/3801943622 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8df4031070 con 0x7f8de8005bc0 2024-05-09T08:07:11.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:11.339+0000 7f8e01e00640 1 -- 172.21.15.1:0/3801943622 <== mon.1 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8df402b960 con 0x7f8de8005bc0 2024-05-09T08:07:11.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:11.339+0000 7f8e01e00640 1 -- 172.21.15.1:0/3801943622 <== mon.1 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8df4003610 con 0x7f8de8005bc0 2024-05-09T08:07:11.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:11.340+0000 7f8e08a00640 1 -- 172.21.15.1:0/3801943622 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8de8005bc0 msgr2=0x7f8de8005fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:07:11.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:11.340+0000 7f8e08a00640 1 --2- 172.21.15.1:0/3801943622 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8de8005bc0 0x7f8de8005fc0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f8df4006fd0 tx=0x7f8de8007a40 comp rx=0 tx=0).stop 2024-05-09T08:07:11.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:11.340+0000 7f8e08a00640 1 -- 172.21.15.1:0/3801943622 shutdown_connections 2024-05-09T08:07:11.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:11.340+0000 7f8e08a00640 1 --2- 172.21.15.1:0/3801943622 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f8de80074f0 0x7f8de800dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:11.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:11.340+0000 7f8e08a00640 1 --2- 172.21.15.1:0/3801943622 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8de8006a10 0x7f8de8006e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:11.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:11.340+0000 7f8e08a00640 1 --2- 172.21.15.1:0/3801943622 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8de8005bc0 0x7f8de8005fc0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:11.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:11.340+0000 7f8e08a00640 1 -- 172.21.15.1:0/3801943622 >> 172.21.15.1:0/3801943622 conn(0x7f8e040cbc10 msgr2=0x7f8e040cc020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:07:11.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:11.340+0000 7f8e08a00640 1 -- 172.21.15.1:0/3801943622 shutdown_connections 2024-05-09T08:07:11.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:11.341+0000 7f8e08a00640 1 -- 172.21.15.1:0/3801943622 wait complete. 2024-05-09T08:07:11.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:11.344+0000 7f8e08a00640 1 Processor -- start 2024-05-09T08:07:11.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:11.344+0000 7f8e08a00640 1 -- start start 2024-05-09T08:07:11.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:11.345+0000 7f8e08a00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8de8005bc0 0x7f8e041e1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:07:11.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:11.345+0000 7f8e08a00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f8de8006a10 0x7f8e0414ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:07:11.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:11.345+0000 7f8e08a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8de80074f0 0x7f8e041f3470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:07:11.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:11.345+0000 7f8e08a00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f8e04157cb0 con 0x7f8de8005bc0 2024-05-09T08:07:11.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:11.345+0000 7f8e08a00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f8e041579d0 con 0x7f8de80074f0 2024-05-09T08:07:11.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:11.345+0000 7f8e08a00640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f8e041581d0 con 0x7f8de8006a10 2024-05-09T08:07:11.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:11.346+0000 7f8dfbe00640 1 -- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f8de8006a10 msgr2=0x7f8e0414ece0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.7:3301/0 2024-05-09T08:07:11.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:11.346+0000 7f8e02800640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8de8005bc0 0x7f8e041e1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:07:11.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:11.346+0000 7f8dfbe00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f8de8006a10 0x7f8e0414ece0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T08:07:11.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:11.346+0000 7f8e02800640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8de8005bc0 0x7f8e041e1fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:51996/0 (socket says 172.21.15.1:51996) 2024-05-09T08:07:11.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:11.346+0000 7f8e02800640 1 -- 172.21.15.1:0/910144472 learned_addr learned my addr 172.21.15.1:0/910144472 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:07:11.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:11.346+0000 7f8e03200640 1 --2- 172.21.15.1:0/910144472 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8de80074f0 0x7f8e041f3470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:07:11.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:11.347+0000 7f8e02800640 1 -- 172.21.15.1:0/910144472 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f8de8006a10 msgr2=0x7f8e0414ece0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T08:07:11.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:11.347+0000 7f8e02800640 1 --2- 172.21.15.1:0/910144472 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f8de8006a10 0x7f8e0414ece0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:11.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:11.347+0000 7f8e02800640 1 -- 172.21.15.1:0/910144472 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8de80074f0 msgr2=0x7f8e041f3470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:07:11.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:11.347+0000 7f8e02800640 1 --2- 172.21.15.1:0/910144472 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8de80074f0 0x7f8e041f3470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:11.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:11.347+0000 7f8e02800640 1 -- 172.21.15.1:0/910144472 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e041e24f0 con 0x7f8de8005bc0 2024-05-09T08:07:11.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:11.347+0000 7f8e03200640 1 --2- 172.21.15.1:0/910144472 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8de80074f0 0x7f8e041f3470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:07:11.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:11.347+0000 7f8e02800640 1 --2- 172.21.15.1:0/910144472 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8de8005bc0 0x7f8e041e1fb0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f8df4007010 tx=0x7f8df40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:07:11.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:11.347+0000 7f8dfaa00640 1 -- 172.21.15.1:0/910144472 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8df4044070 con 0x7f8de8005bc0 2024-05-09T08:07:11.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:11.347+0000 7f8dfaa00640 1 -- 172.21.15.1:0/910144472 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8df4040650 con 0x7f8de8005bc0 2024-05-09T08:07:11.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:11.347+0000 7f8dfaa00640 1 -- 172.21.15.1:0/910144472 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8df4040a40 con 0x7f8de8005bc0 2024-05-09T08:07:11.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:11.347+0000 7f8e08a00640 1 -- 172.21.15.1:0/910144472 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e04158ad0 con 0x7f8de8005bc0 2024-05-09T08:07:11.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:11.348+0000 7f8e08a00640 1 -- 172.21.15.1:0/910144472 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e04159070 con 0x7f8de8005bc0 2024-05-09T08:07:11.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:11.349+0000 7f8e08a00640 1 -- 172.21.15.1:0/910144472 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e0404f660 con 0x7f8de8005bc0 2024-05-09T08:07:11.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:11.351+0000 7f8dfaa00640 1 -- 172.21.15.1:0/910144472 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8df4047070 con 0x7f8de8005bc0 2024-05-09T08:07:11.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:11.352+0000 7f8dfaa00640 1 --2- 172.21.15.1:0/910144472 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f8de006c670 0x7f8de006eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:07:11.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:11.352+0000 7f8dfaa00640 1 -- 172.21.15.1:0/910144472 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7f8df40ac450 con 0x7f8de8005bc0 2024-05-09T08:07:11.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:11.352+0000 7f8dfbe00640 1 --2- 172.21.15.1:0/910144472 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f8de006c670 0x7f8de006eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:07:11.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:11.353+0000 7f8dfbe00640 1 --2- 172.21.15.1:0/910144472 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f8de006c670 0x7f8de006eb30 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f8e0415b080 tx=0x7f8dec00d550 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:07:11.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:11.358+0000 7f8dfaa00640 1 -- 172.21.15.1:0/910144472 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f8df407a3a0 con 0x7f8de8005bc0 2024-05-09T08:07:11.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:11.949+0000 7f8e08a00640 1 -- 172.21.15.1:0/910144472 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f8e040c7cc0 con 0x7f8de8005bc0 2024-05-09T08:07:11.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:11.953+0000 7f8dfaa00640 1 -- 172.21.15.1:0/910144472 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1538 (secure 0 0 0) 0x7f8df407e6a0 con 0x7f8de8005bc0 2024-05-09T08:07:11.964 INFO:teuthology.orchestra.run.smithi001.stdout:{"election_epoch":40,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"54329ec8-3270-4043-9a82-c6cd7420aa7f","modified":"2024-05-09T08:01:21.819270Z","created":"2024-05-09T08:01:21.819270Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:3300","nonce":0},{"type":"v1","addr":"172.21.15.1:6789","nonce":0}]},"addr":"172.21.15.1:6789/0","public_addr":"172.21.15.1:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:3300","nonce":0},{"type":"v1","addr":"172.21.15.7:6789","nonce":0}]},"addr":"172.21.15.7:6789/0","public_addr":"172.21.15.7:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:3301","nonce":0},{"type":"v1","addr":"172.21.15.7:6790","nonce":0}]},"addr":"172.21.15.7:6790/0","public_addr":"172.21.15.7:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-05-09T08:07:11.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:11.957+0000 7f8e08a00640 1 -- 172.21.15.1:0/910144472 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f8de006c670 msgr2=0x7f8de006eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:07:11.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:11.957+0000 7f8e08a00640 1 --2- 172.21.15.1:0/910144472 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f8de006c670 0x7f8de006eb30 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f8e0415b080 tx=0x7f8dec00d550 comp rx=0 tx=0).stop 2024-05-09T08:07:11.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:11.957+0000 7f8e08a00640 1 -- 172.21.15.1:0/910144472 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8de8005bc0 msgr2=0x7f8e041e1fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:07:11.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:11.957+0000 7f8e08a00640 1 --2- 172.21.15.1:0/910144472 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8de8005bc0 0x7f8e041e1fb0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f8df4007010 tx=0x7f8df40046c0 comp rx=0 tx=0).stop 2024-05-09T08:07:11.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:11.957+0000 7f8e08a00640 1 -- 172.21.15.1:0/910144472 shutdown_connections 2024-05-09T08:07:11.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:11.957+0000 7f8e08a00640 1 --2- 172.21.15.1:0/910144472 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f8de006c670 0x7f8de006eb30 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:11.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:11.957+0000 7f8e08a00640 1 --2- 172.21.15.1:0/910144472 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8de80074f0 0x7f8e041f3470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:11.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:11.957+0000 7f8e08a00640 1 --2- 172.21.15.1:0/910144472 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f8de8006a10 0x7f8e0414ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:11.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:11.957+0000 7f8e08a00640 1 --2- 172.21.15.1:0/910144472 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8de8005bc0 0x7f8e041e1fb0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:11.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:11.957+0000 7f8e08a00640 1 -- 172.21.15.1:0/910144472 >> 172.21.15.1:0/910144472 conn(0x7f8e040cbc10 msgr2=0x7f8e04155560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:07:11.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:11.958+0000 7f8e08a00640 1 -- 172.21.15.1:0/910144472 shutdown_connections 2024-05-09T08:07:11.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:11.961+0000 7f8e08a00640 1 -- 172.21.15.1:0/910144472 wait complete. 2024-05-09T08:07:12.008 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2024-05-09T08:07:12.008 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon a 2024-05-09T08:07:12.008 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-05-09T08:07:12.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.280+0000 7f0ee3a00640 1 Processor -- start 2024-05-09T08:07:12.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.280+0000 7f0ee3a00640 1 -- start start 2024-05-09T08:07:12.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.282+0000 7f0ee3a00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f0edc15c360 0x7f0edc0c9570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:07:12.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.282+0000 7f0ee3a00640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0edc071c80 con 0x7f0edc072c20 2024-05-09T08:07:12.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.282+0000 7f0ee3a00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0edc071f20 con 0x7f0edc0c5e40 2024-05-09T08:07:12.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.282+0000 7f0ee3a00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f0edc15b980 con 0x7f0edc15c360 2024-05-09T08:07:12.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.282+0000 7f0edbe00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f0edc0c5e40 0x7f0edc0cc5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.1:57830/0 (socket says 172.21.15.1:57830) 2024-05-09T08:07:12.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.282+0000 7f0ee1a00640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f0edc072c20 0x7f0edc073050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:60574/0 (socket says 172.21.15.1:60574) 2024-05-09T08:07:12.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.283+0000 7f0edbe00640 1 -- 172.21.15.1:0/2173830225 learned_addr learned my addr 172.21.15.1:0/2173830225 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T08:07:12.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.283+0000 7f0edb400640 1 -- 172.21.15.1:0/2173830225 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4187153199 0 0) 0x7f0edc071c80 con 0x7f0edc072c20 2024-05-09T08:07:12.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.283+0000 7f0edb400640 1 -- 172.21.15.1:0/2173830225 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0ec0003520 con 0x7f0edc072c20 2024-05-09T08:07:12.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.283+0000 7f0edb400640 1 -- 172.21.15.1:0/2173830225 <== mon.1 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 461176703 0 0) 0x7f0edc071f20 con 0x7f0edc0c5e40 2024-05-09T08:07:12.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.283+0000 7f0edb400640 1 -- 172.21.15.1:0/2173830225 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0edc071c80 con 0x7f0edc0c5e40 2024-05-09T08:07:12.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.284+0000 7f0ee1000640 1 --2- 172.21.15.1:0/2173830225 >> v2:172.21.15.7:3300/0 conn(0x7f0edc15c360 0x7f0edc0c9570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:07:12.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.284+0000 7f0edb400640 1 -- 172.21.15.1:0/2173830225 <== mon.0 v1:172.21.15.1:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4217202491 0 0) 0x7f0ec0003520 con 0x7f0edc072c20 2024-05-09T08:07:12.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.284+0000 7f0edb400640 1 -- 172.21.15.1:0/2173830225 >> v2:172.21.15.7:3300/0 conn(0x7f0edc15c360 msgr2=0x7f0edc0c9570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:07:12.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.284+0000 7f0edb400640 1 --2- 172.21.15.1:0/2173830225 >> v2:172.21.15.7:3300/0 conn(0x7f0edc15c360 0x7f0edc0c9570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:12.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.284+0000 7f0edb400640 1 -- 172.21.15.1:0/2173830225 >> v1:172.21.15.7:6789/0 conn(0x7f0edc0c5e40 legacy=0x7f0edc0cc5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:07:12.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.284+0000 7f0edb400640 1 -- 172.21.15.1:0/2173830225 --> v1:172.21.15.1:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0edc0721e0 con 0x7f0edc072c20 2024-05-09T08:07:12.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.284+0000 7f0edb400640 1 -- 172.21.15.1:0/2173830225 <== mon.0 v1:172.21.15.1:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4179846320 0 0) 0x7f0ec8001690 con 0x7f0edc072c20 2024-05-09T08:07:12.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.284+0000 7f0edb400640 1 -- 172.21.15.1:0/2173830225 >> v1:172.21.15.1:6789/0 conn(0x7f0edc072c20 legacy=0x7f0edc073050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:07:12.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.284+0000 7f0edb400640 1 --2- 172.21.15.1:0/2173830225 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ec0004bd0 0x7f0ec0004fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:07:12.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.284+0000 7f0edb400640 1 --2- 172.21.15.1:0/2173830225 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0ec00037a0 0x7f0ec0003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:07:12.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.284+0000 7f0edb400640 1 --2- 172.21.15.1:0/2173830225 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f0ec00042d0 0x7f0ec000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:07:12.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.284+0000 7f0edb400640 1 -- 172.21.15.1:0/2173830225 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f0ec000c220 con 0x7f0ec00037a0 2024-05-09T08:07:12.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.284+0000 7f0edb400640 1 -- 172.21.15.1:0/2173830225 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0ec000c3d0 con 0x7f0ec0004bd0 2024-05-09T08:07:12.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.284+0000 7f0edb400640 1 -- 172.21.15.1:0/2173830225 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f0ec000c5d0 con 0x7f0ec00042d0 2024-05-09T08:07:12.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.285+0000 7f0ee1a00640 1 -- 172.21.15.1:0/2173830225 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f0ec00042d0 msgr2=0x7f0ec000b9c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.7:3301/0 2024-05-09T08:07:12.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.285+0000 7f0ee1a00640 1 --2- 172.21.15.1:0/2173830225 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f0ec00042d0 0x7f0ec000b9c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T08:07:12.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.285+0000 7f0ee1000640 1 --2- 172.21.15.1:0/2173830225 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ec0004bd0 0x7f0ec0004fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:07:12.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.286+0000 7f0ee1000640 1 -- 172.21.15.1:0/2173830225 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f0ec00042d0 msgr2=0x7f0ec000b9c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T08:07:12.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.286+0000 7f0ee1000640 1 --2- 172.21.15.1:0/2173830225 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f0ec00042d0 0x7f0ec000b9c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:12.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.286+0000 7f0ee1000640 1 -- 172.21.15.1:0/2173830225 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0ec00037a0 msgr2=0x7f0ec0003c20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T08:07:12.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.286+0000 7f0ee1000640 1 --2- 172.21.15.1:0/2173830225 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0ec00037a0 0x7f0ec0003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:12.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.286+0000 7f0ee1000640 1 -- 172.21.15.1:0/2173830225 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ec0003520 con 0x7f0ec0004bd0 2024-05-09T08:07:12.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.286+0000 7f0ee1a00640 1 -- 172.21.15.1:0/2173830225 reap_dead start 2024-05-09T08:07:12.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.288+0000 7f0ee1000640 1 --2- 172.21.15.1:0/2173830225 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ec0004bd0 0x7f0ec0004fb0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f0ed4007360 tx=0x7f0ec0005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aa5d2902eaed1fcf server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:07:12.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.289+0000 7f0edb400640 1 -- 172.21.15.1:0/2173830225 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ed4009be0 con 0x7f0ec0004bd0 2024-05-09T08:07:12.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.289+0000 7f0edb400640 1 -- 172.21.15.1:0/2173830225 <== mon.1 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ed4036600 con 0x7f0ec0004bd0 2024-05-09T08:07:12.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.289+0000 7f0edb400640 1 -- 172.21.15.1:0/2173830225 <== mon.1 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ed40369d0 con 0x7f0ec0004bd0 2024-05-09T08:07:12.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.290+0000 7f0ee3a00640 1 -- 172.21.15.1:0/2173830225 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ec0004bd0 msgr2=0x7f0ec0004fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:07:12.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.290+0000 7f0ee3a00640 1 --2- 172.21.15.1:0/2173830225 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ec0004bd0 0x7f0ec0004fb0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f0ed4007360 tx=0x7f0ec0005830 comp rx=0 tx=0).stop 2024-05-09T08:07:12.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.290+0000 7f0ee3a00640 1 -- 172.21.15.1:0/2173830225 shutdown_connections 2024-05-09T08:07:12.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.290+0000 7f0ee3a00640 1 --2- 172.21.15.1:0/2173830225 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ec0004bd0 0x7f0ec0004fb0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:12.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.290+0000 7f0ee3a00640 1 -- 172.21.15.1:0/2173830225 >> 172.21.15.1:0/2173830225 conn(0x7f0edc06dad0 msgr2=0x7f0edc06deb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:07:12.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.291+0000 7f0ee3a00640 1 -- 172.21.15.1:0/2173830225 shutdown_connections 2024-05-09T08:07:12.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.291+0000 7f0ee3a00640 1 -- 172.21.15.1:0/2173830225 wait complete. 2024-05-09T08:07:12.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.293+0000 7f0ee3a00640 1 Processor -- start 2024-05-09T08:07:12.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.294+0000 7f0ee3a00640 1 -- start start 2024-05-09T08:07:12.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.295+0000 7f0ee3a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ec0004bd0 0x7f0edc149c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:07:12.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.295+0000 7f0ee3a00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f0edc14e350 0x7f0edc1f12a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:07:12.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.295+0000 7f0ee3a00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0edc159a60 0x7f0edc14da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:07:12.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.295+0000 7f0edbe00640 1 -- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f0edc14e350 msgr2=0x7f0edc1f12a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.7:3301/0 2024-05-09T08:07:12.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.295+0000 7f0edbe00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f0edc14e350 0x7f0edc1f12a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T08:07:12.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.296+0000 7f0ee1000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ec0004bd0 0x7f0edc149c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:07:12.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.296+0000 7f0ee1000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ec0004bd0 0x7f0edc149c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.1:43986/0 (socket says 172.21.15.1:43986) 2024-05-09T08:07:12.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.296+0000 7f0ee3a00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f0edc0cc0d0 con 0x7f0edc159a60 2024-05-09T08:07:12.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.296+0000 7f0ee3a00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0edc0727a0 con 0x7f0ec0004bd0 2024-05-09T08:07:12.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.296+0000 7f0ee3a00640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f0edc15c740 con 0x7f0edc14e350 2024-05-09T08:07:12.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.296+0000 7f0ee1a00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0edc159a60 0x7f0edc14da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:07:12.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.296+0000 7f0ee1a00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0edc159a60 0x7f0edc14da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:52026/0 (socket says 172.21.15.1:52026) 2024-05-09T08:07:12.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.296+0000 7f0ee1a00640 1 -- 172.21.15.1:0/53246099 learned_addr learned my addr 172.21.15.1:0/53246099 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:07:12.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.296+0000 7f0ee1a00640 1 -- 172.21.15.1:0/53246099 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f0edc14e350 msgr2=0x7f0edc1f12a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T08:07:12.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.296+0000 7f0ee1a00640 1 --2- 172.21.15.1:0/53246099 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f0edc14e350 0x7f0edc1f12a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:12.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.297+0000 7f0ee1a00640 1 -- 172.21.15.1:0/53246099 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ec0004bd0 msgr2=0x7f0edc149c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:07:12.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.297+0000 7f0ee1a00640 1 --2- 172.21.15.1:0/53246099 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ec0004bd0 0x7f0edc149c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:12.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.297+0000 7f0ee1a00640 1 -- 172.21.15.1:0/53246099 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0edc15b790 con 0x7f0edc159a60 2024-05-09T08:07:12.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.297+0000 7f0ee1000640 1 --2- 172.21.15.1:0/53246099 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ec0004bd0 0x7f0edc149c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2024-05-09T08:07:12.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.297+0000 7f0ee1a00640 1 --2- 172.21.15.1:0/53246099 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0edc159a60 0x7f0edc14da10 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f0ec800c390 tx=0x7f0ec800c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:07:12.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.298+0000 7f0ed8c00640 1 -- 172.21.15.1:0/53246099 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ec8004bf0 con 0x7f0edc159a60 2024-05-09T08:07:12.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.298+0000 7f0ee3a00640 1 -- 172.21.15.1:0/53246099 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0edc14a7f0 con 0x7f0edc159a60 2024-05-09T08:07:12.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.298+0000 7f0ee3a00640 1 -- 172.21.15.1:0/53246099 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0edc149140 con 0x7f0edc159a60 2024-05-09T08:07:12.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.298+0000 7f0ed8c00640 1 -- 172.21.15.1:0/53246099 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ec8004d90 con 0x7f0edc159a60 2024-05-09T08:07:12.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.298+0000 7f0ed8c00640 1 -- 172.21.15.1:0/53246099 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ec8007230 con 0x7f0edc159a60 2024-05-09T08:07:12.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.299+0000 7f0ee3a00640 1 --2- 172.21.15.1:0/53246099 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0eb4002a70 0x7f0eb4004f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:07:12.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.299+0000 7f0ee3a00640 1 -- 172.21.15.1:0/53246099 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f0eb4007ec0 con 0x7f0eb4002a70 2024-05-09T08:07:12.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.301+0000 7f0ee1000640 1 --2- 172.21.15.1:0/53246099 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0eb4002a70 0x7f0eb4004f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:07:12.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.302+0000 7f0ed8c00640 1 -- 172.21.15.1:0/53246099 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0ec80073d0 con 0x7f0edc159a60 2024-05-09T08:07:12.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.302+0000 7f0ee1000640 1 -- 172.21.15.1:0/53246099 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0eb40080c0 con 0x7f0eb4002a70 2024-05-09T08:07:12.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.302+0000 7f0ee1000640 1 --2- 172.21.15.1:0/53246099 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0eb4002a70 0x7f0eb4004f30 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f0ed4007330 tx=0x7f0ed4004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:07:12.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.303+0000 7f0ed8c00640 1 --2- 172.21.15.1:0/53246099 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f0ebc06bde0 0x7f0ebc06e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:07:12.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.303+0000 7f0ed8c00640 1 -- 172.21.15.1:0/53246099 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7f0ec808a1b0 con 0x7f0edc159a60 2024-05-09T08:07:12.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.303+0000 7f0edbe00640 1 --2- 172.21.15.1:0/53246099 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f0ebc06bde0 0x7f0ebc06e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:07:12.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.303+0000 7f0ed8c00640 1 -- 172.21.15.1:0/53246099 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ed4036cf0 con 0x7f0eb4002a70 2024-05-09T08:07:12.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.303+0000 7f0ed8c00640 1 -- 172.21.15.1:0/53246099 <== mon.0 v2:172.21.15.1:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f0ed4046070 con 0x7f0eb4002a70 2024-05-09T08:07:12.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.303+0000 7f0ed8c00640 1 -- 172.21.15.1:0/53246099 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0eb4002a70 msgr2=0x7f0eb4004f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:07:12.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.303+0000 7f0ed8c00640 1 --2- 172.21.15.1:0/53246099 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0eb4002a70 0x7f0eb4004f30 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f0ed4007330 tx=0x7f0ed4004ab0 comp rx=0 tx=0).stop 2024-05-09T08:07:12.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.303+0000 7f0ed8c00640 1 -- 172.21.15.1:0/53246099 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0eb4002a70 msgr2=0x7f0eb4004f30 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-05-09T08:07:12.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.303+0000 7f0ed8c00640 1 --2- 172.21.15.1:0/53246099 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0eb4002a70 0x7f0eb4004f30 secure :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f0ed4007330 tx=0x7f0ed4004ab0 comp rx=0 tx=0).stop 2024-05-09T08:07:12.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.306+0000 7f0edbe00640 1 --2- 172.21.15.1:0/53246099 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f0ebc06bde0 0x7f0ebc06e2a0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f0ecc0012c0 tx=0x7f0ecc001410 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:07:12.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.316+0000 7f0ee3a00640 1 --2- 172.21.15.1:0/53246099 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0eb4006ee0 0x7f0eb4007370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:07:12.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.316+0000 7f0ee3a00640 1 -- 172.21.15.1:0/53246099 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f0eb400a4a0 con 0x7f0eb4006ee0 2024-05-09T08:07:12.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.316+0000 7f0ee1000640 1 --2- 172.21.15.1:0/53246099 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0eb4006ee0 0x7f0eb4007370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:07:12.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.317+0000 7f0ee1000640 1 -- 172.21.15.1:0/53246099 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f0eb40078b0 con 0x7f0eb4006ee0 2024-05-09T08:07:12.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.317+0000 7f0ee1000640 1 --2- 172.21.15.1:0/53246099 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0eb4006ee0 0x7f0eb4007370 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f0eb4008090 tx=0x7f0ed4030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:07:12.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.318+0000 7f0ed8c00640 1 -- 172.21.15.1:0/53246099 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ed404bbd0 con 0x7f0eb4006ee0 2024-05-09T08:07:12.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.318+0000 7f0ed8c00640 1 -- 172.21.15.1:0/53246099 <== mon.0 v2:172.21.15.1:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4865 (secure 0 0 0) 0x7f0ed402e6d0 con 0x7f0eb4006ee0 2024-05-09T08:07:12.329 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-05-09T08:07:12.330 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "a", 2024-05-09T08:07:12.330 INFO:teuthology.orchestra.run.smithi001.stdout: "rank": 0, 2024-05-09T08:07:12.330 INFO:teuthology.orchestra.run.smithi001.stdout: "state": "leader", 2024-05-09T08:07:12.330 INFO:teuthology.orchestra.run.smithi001.stdout: "election_epoch": 40, 2024-05-09T08:07:12.330 INFO:teuthology.orchestra.run.smithi001.stdout: "uptime": 94935, 2024-05-09T08:07:12.330 INFO:teuthology.orchestra.run.smithi001.stdout: "quorum": [ 2024-05-09T08:07:12.330 INFO:teuthology.orchestra.run.smithi001.stdout: 0, 2024-05-09T08:07:12.330 INFO:teuthology.orchestra.run.smithi001.stdout: 1 2024-05-09T08:07:12.330 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:07:12.330 INFO:teuthology.orchestra.run.smithi001.stdout: "quorum_age": 1, 2024-05-09T08:07:12.331 INFO:teuthology.orchestra.run.smithi001.stdout: "features": { 2024-05-09T08:07:12.331 INFO:teuthology.orchestra.run.smithi001.stdout: "required_con": "2450521708007866372", 2024-05-09T08:07:12.331 INFO:teuthology.orchestra.run.smithi001.stdout: "required_mon": [ 2024-05-09T08:07:12.331 INFO:teuthology.orchestra.run.smithi001.stdout: "kraken", 2024-05-09T08:07:12.331 INFO:teuthology.orchestra.run.smithi001.stdout: "luminous", 2024-05-09T08:07:12.331 INFO:teuthology.orchestra.run.smithi001.stdout: "mimic", 2024-05-09T08:07:12.331 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap-prune", 2024-05-09T08:07:12.331 INFO:teuthology.orchestra.run.smithi001.stdout: "nautilus", 2024-05-09T08:07:12.331 INFO:teuthology.orchestra.run.smithi001.stdout: "octopus", 2024-05-09T08:07:12.332 INFO:teuthology.orchestra.run.smithi001.stdout: "pacific", 2024-05-09T08:07:12.332 INFO:teuthology.orchestra.run.smithi001.stdout: "elector-pinging", 2024-05-09T08:07:12.332 INFO:teuthology.orchestra.run.smithi001.stdout: "quincy", 2024-05-09T08:07:12.332 INFO:teuthology.orchestra.run.smithi001.stdout: "reef", 2024-05-09T08:07:12.332 INFO:teuthology.orchestra.run.smithi001.stdout: "squid" 2024-05-09T08:07:12.332 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:07:12.332 INFO:teuthology.orchestra.run.smithi001.stdout: "quorum_con": "4540701547738038271", 2024-05-09T08:07:12.332 INFO:teuthology.orchestra.run.smithi001.stdout: "quorum_mon": [ 2024-05-09T08:07:12.332 INFO:teuthology.orchestra.run.smithi001.stdout: "kraken", 2024-05-09T08:07:12.332 INFO:teuthology.orchestra.run.smithi001.stdout: "luminous", 2024-05-09T08:07:12.332 INFO:teuthology.orchestra.run.smithi001.stdout: "mimic", 2024-05-09T08:07:12.332 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap-prune", 2024-05-09T08:07:12.332 INFO:teuthology.orchestra.run.smithi001.stdout: "nautilus", 2024-05-09T08:07:12.332 INFO:teuthology.orchestra.run.smithi001.stdout: "octopus", 2024-05-09T08:07:12.333 INFO:teuthology.orchestra.run.smithi001.stdout: "pacific", 2024-05-09T08:07:12.333 INFO:teuthology.orchestra.run.smithi001.stdout: "elector-pinging", 2024-05-09T08:07:12.333 INFO:teuthology.orchestra.run.smithi001.stdout: "quincy", 2024-05-09T08:07:12.333 INFO:teuthology.orchestra.run.smithi001.stdout: "reef", 2024-05-09T08:07:12.333 INFO:teuthology.orchestra.run.smithi001.stdout: "squid" 2024-05-09T08:07:12.333 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:07:12.333 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:07:12.333 INFO:teuthology.orchestra.run.smithi001.stdout: "outside_quorum": [], 2024-05-09T08:07:12.333 INFO:teuthology.orchestra.run.smithi001.stdout: "extra_probe_peers": [], 2024-05-09T08:07:12.333 INFO:teuthology.orchestra.run.smithi001.stdout: "sync_provider": [], 2024-05-09T08:07:12.333 INFO:teuthology.orchestra.run.smithi001.stdout: "monmap": { 2024-05-09T08:07:12.334 INFO:teuthology.orchestra.run.smithi001.stdout: "epoch": 1, 2024-05-09T08:07:12.334 INFO:teuthology.orchestra.run.smithi001.stdout: "fsid": "54329ec8-3270-4043-9a82-c6cd7420aa7f", 2024-05-09T08:07:12.334 INFO:teuthology.orchestra.run.smithi001.stdout: "modified": "2024-05-09T08:01:21.819270Z", 2024-05-09T08:07:12.334 INFO:teuthology.orchestra.run.smithi001.stdout: "created": "2024-05-09T08:01:21.819270Z", 2024-05-09T08:07:12.334 INFO:teuthology.orchestra.run.smithi001.stdout: "min_mon_release": 19, 2024-05-09T08:07:12.334 INFO:teuthology.orchestra.run.smithi001.stdout: "min_mon_release_name": "squid", 2024-05-09T08:07:12.334 INFO:teuthology.orchestra.run.smithi001.stdout: "election_strategy": 1, 2024-05-09T08:07:12.334 INFO:teuthology.orchestra.run.smithi001.stdout: "disallowed_leaders: ": "", 2024-05-09T08:07:12.334 INFO:teuthology.orchestra.run.smithi001.stdout: "stretch_mode": false, 2024-05-09T08:07:12.334 INFO:teuthology.orchestra.run.smithi001.stdout: "tiebreaker_mon": "", 2024-05-09T08:07:12.334 INFO:teuthology.orchestra.run.smithi001.stdout: "removed_ranks: ": "", 2024-05-09T08:07:12.334 INFO:teuthology.orchestra.run.smithi001.stdout: "features": { 2024-05-09T08:07:12.335 INFO:teuthology.orchestra.run.smithi001.stdout: "persistent": [ 2024-05-09T08:07:12.335 INFO:teuthology.orchestra.run.smithi001.stdout: "kraken", 2024-05-09T08:07:12.335 INFO:teuthology.orchestra.run.smithi001.stdout: "luminous", 2024-05-09T08:07:12.335 INFO:teuthology.orchestra.run.smithi001.stdout: "mimic", 2024-05-09T08:07:12.335 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap-prune", 2024-05-09T08:07:12.335 INFO:teuthology.orchestra.run.smithi001.stdout: "nautilus", 2024-05-09T08:07:12.335 INFO:teuthology.orchestra.run.smithi001.stdout: "octopus", 2024-05-09T08:07:12.335 INFO:teuthology.orchestra.run.smithi001.stdout: "pacific", 2024-05-09T08:07:12.335 INFO:teuthology.orchestra.run.smithi001.stdout: "elector-pinging", 2024-05-09T08:07:12.335 INFO:teuthology.orchestra.run.smithi001.stdout: "quincy", 2024-05-09T08:07:12.335 INFO:teuthology.orchestra.run.smithi001.stdout: "reef", 2024-05-09T08:07:12.335 INFO:teuthology.orchestra.run.smithi001.stdout: "squid" 2024-05-09T08:07:12.335 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:07:12.336 INFO:teuthology.orchestra.run.smithi001.stdout: "optional": [] 2024-05-09T08:07:12.336 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:07:12.336 INFO:teuthology.orchestra.run.smithi001.stdout: "mons": [ 2024-05-09T08:07:12.336 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:07:12.336 INFO:teuthology.orchestra.run.smithi001.stdout: "rank": 0, 2024-05-09T08:07:12.336 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "a", 2024-05-09T08:07:12.336 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-05-09T08:07:12.336 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-05-09T08:07:12.336 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:07:12.336 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-05-09T08:07:12.336 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:3300", 2024-05-09T08:07:12.336 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:07:12.337 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:07:12.337 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:07:12.337 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-05-09T08:07:12.337 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6789", 2024-05-09T08:07:12.337 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:07:12.337 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:07:12.337 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:07:12.337 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:07:12.337 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6789/0", 2024-05-09T08:07:12.337 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.1:6789/0", 2024-05-09T08:07:12.337 INFO:teuthology.orchestra.run.smithi001.stdout: "priority": 0, 2024-05-09T08:07:12.337 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 0, 2024-05-09T08:07:12.337 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_location": "{}" 2024-05-09T08:07:12.338 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:07:12.338 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:07:12.338 INFO:teuthology.orchestra.run.smithi001.stdout: "rank": 1, 2024-05-09T08:07:12.338 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "b", 2024-05-09T08:07:12.338 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-05-09T08:07:12.338 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-05-09T08:07:12.338 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:07:12.338 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-05-09T08:07:12.338 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:3300", 2024-05-09T08:07:12.338 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:07:12.338 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:07:12.338 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:07:12.339 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-05-09T08:07:12.339 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:6789", 2024-05-09T08:07:12.339 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:07:12.339 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:07:12.339 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:07:12.339 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:07:12.339 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:6789/0", 2024-05-09T08:07:12.339 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.7:6789/0", 2024-05-09T08:07:12.339 INFO:teuthology.orchestra.run.smithi001.stdout: "priority": 0, 2024-05-09T08:07:12.339 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 0, 2024-05-09T08:07:12.339 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_location": "{}" 2024-05-09T08:07:12.340 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:07:12.340 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:07:12.340 INFO:teuthology.orchestra.run.smithi001.stdout: "rank": 2, 2024-05-09T08:07:12.340 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "c", 2024-05-09T08:07:12.340 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-05-09T08:07:12.340 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-05-09T08:07:12.340 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:07:12.340 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-05-09T08:07:12.340 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:3301", 2024-05-09T08:07:12.340 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:07:12.340 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:07:12.340 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:07:12.341 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-05-09T08:07:12.341 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:6790", 2024-05-09T08:07:12.341 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:07:12.341 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:07:12.341 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:07:12.341 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:07:12.341 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:6790/0", 2024-05-09T08:07:12.341 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.7:6790/0", 2024-05-09T08:07:12.341 INFO:teuthology.orchestra.run.smithi001.stdout: "priority": 0, 2024-05-09T08:07:12.341 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 0, 2024-05-09T08:07:12.341 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_location": "{}" 2024-05-09T08:07:12.341 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:07:12.341 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:07:12.342 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:07:12.342 INFO:teuthology.orchestra.run.smithi001.stdout: "feature_map": { 2024-05-09T08:07:12.342 INFO:teuthology.orchestra.run.smithi001.stdout: "mon": [ 2024-05-09T08:07:12.342 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:07:12.342 INFO:teuthology.orchestra.run.smithi001.stdout: "features": "0x3f03cffffffdffff", 2024-05-09T08:07:12.342 INFO:teuthology.orchestra.run.smithi001.stdout: "release": "luminous", 2024-05-09T08:07:12.342 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 1 2024-05-09T08:07:12.342 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:07:12.342 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:07:12.342 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": [ 2024-05-09T08:07:12.342 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:07:12.342 INFO:teuthology.orchestra.run.smithi001.stdout: "features": "0x3f03cffffffdffff", 2024-05-09T08:07:12.342 INFO:teuthology.orchestra.run.smithi001.stdout: "release": "luminous", 2024-05-09T08:07:12.343 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 1 2024-05-09T08:07:12.343 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:07:12.343 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:07:12.343 INFO:teuthology.orchestra.run.smithi001.stdout: "client": [ 2024-05-09T08:07:12.343 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:07:12.343 INFO:teuthology.orchestra.run.smithi001.stdout: "features": "0x3f03cffffffdffff", 2024-05-09T08:07:12.343 INFO:teuthology.orchestra.run.smithi001.stdout: "release": "luminous", 2024-05-09T08:07:12.343 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 2 2024-05-09T08:07:12.343 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:07:12.343 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:07:12.343 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr": [ 2024-05-09T08:07:12.344 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:07:12.344 INFO:teuthology.orchestra.run.smithi001.stdout: "features": "0x3f03cffffffdffff", 2024-05-09T08:07:12.344 INFO:teuthology.orchestra.run.smithi001.stdout: "release": "luminous", 2024-05-09T08:07:12.344 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 1 2024-05-09T08:07:12.344 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:07:12.344 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:07:12.344 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:07:12.344 INFO:teuthology.orchestra.run.smithi001.stdout: "stretch_mode": false 2024-05-09T08:07:12.344 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-05-09T08:07:12.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.318+0000 7f0ed8c00640 1 -- 172.21.15.1:0/53246099 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0eb4006ee0 msgr2=0x7f0eb4007370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:07:12.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.318+0000 7f0ed8c00640 1 --2- 172.21.15.1:0/53246099 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0eb4006ee0 0x7f0eb4007370 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f0eb4008090 tx=0x7f0ed4030000 comp rx=0 tx=0).stop 2024-05-09T08:07:12.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.319+0000 7f0ed8c00640 1 -- 172.21.15.1:0/53246099 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0eb4006ee0 msgr2=0x7f0eb4007370 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-09T08:07:12.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.319+0000 7f0ed8c00640 1 --2- 172.21.15.1:0/53246099 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0eb4006ee0 0x7f0eb4007370 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:12.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.319+0000 7f0ee3a00640 1 -- 172.21.15.1:0/53246099 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f0ebc06bde0 msgr2=0x7f0ebc06e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:07:12.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.319+0000 7f0ee3a00640 1 --2- 172.21.15.1:0/53246099 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f0ebc06bde0 0x7f0ebc06e2a0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f0ecc0012c0 tx=0x7f0ecc001410 comp rx=0 tx=0).stop 2024-05-09T08:07:12.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.319+0000 7f0ee3a00640 1 -- 172.21.15.1:0/53246099 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0edc159a60 msgr2=0x7f0edc14da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:07:12.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.319+0000 7f0ee3a00640 1 --2- 172.21.15.1:0/53246099 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0edc159a60 0x7f0edc14da10 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f0ec800c390 tx=0x7f0ec800c850 comp rx=0 tx=0).stop 2024-05-09T08:07:12.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.319+0000 7f0ee1a00640 1 -- 172.21.15.1:0/53246099 reap_dead start 2024-05-09T08:07:12.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.319+0000 7f0ee1a00640 1 -- 172.21.15.1:0/53246099 reap_dead start 2024-05-09T08:07:12.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.319+0000 7f0ee3a00640 1 -- 172.21.15.1:0/53246099 shutdown_connections 2024-05-09T08:07:12.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.319+0000 7f0ee3a00640 1 -- 172.21.15.1:0/53246099 >> 172.21.15.1:0/53246099 conn(0x7f0edc06dad0 msgr2=0x7f0edc0cc4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:07:12.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.320+0000 7f0ee3a00640 1 -- 172.21.15.1:0/53246099 shutdown_connections 2024-05-09T08:07:12.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.320+0000 7f0ee3a00640 1 -- 172.21.15.1:0/53246099 wait complete. 2024-05-09T08:07:12.360 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-05-09T08:07:12.360 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon b 2024-05-09T08:07:12.360 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-05-09T08:07:12.369 INFO:tasks.workunit.client.0.smithi001.stdout: 2024-05-09T08:07:12.369 INFO:tasks.workunit.client.0.smithi001.stdout: 2024-05-09T08:07:12.369 INFO:tasks.workunit.client.0.smithi001.stdout: 2024-05-09T08:07:12.369 INFO:tasks.workunit.client.0.smithi001.stdout:iozone test complete. 2024-05-09T08:07:12.374 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-05-09T08:07:12.374 DEBUG:teuthology.orchestra.run.smithi001:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-05-09T08:07:12.543 INFO:tasks.workunit:Stopping ['suites/iozone.sh'] on client.0... 2024-05-09T08:07:12.544 DEBUG:teuthology.orchestra.run.smithi001:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-05-09T08:07:12.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.700+0000 7f22a7200640 1 Processor -- start 2024-05-09T08:07:12.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.700+0000 7f22a7200640 1 -- start start 2024-05-09T08:07:12.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.704+0000 7f22a7200640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f22a01509b0 0x7f22a006f820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:07:12.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.704+0000 7f22a7200640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f22a0073520 con 0x7f22a0150540 2024-05-09T08:07:12.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.704+0000 7f22a7200640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f22a00c2080 con 0x7f22a0159b40 2024-05-09T08:07:12.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.704+0000 7f22a7200640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f22a01568f0 con 0x7f22a01509b0 2024-05-09T08:07:12.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.704+0000 7f22a5200640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f22a01509b0 0x7f22a006f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:07:12.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.705+0000 7f22a5200640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f22a01509b0 0x7f22a006f820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.1:43990/0 (socket says 172.21.15.1:43990) 2024-05-09T08:07:12.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.705+0000 7f22a5200640 1 -- 172.21.15.1:0/1078337700 learned_addr learned my addr 172.21.15.1:0/1078337700 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:07:12.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.706+0000 7f229ea00640 1 -- 172.21.15.1:0/1078337700 <== mon.1 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2107699484 0 0) 0x7f22a00c2080 con 0x7f22a0159b40 2024-05-09T08:07:12.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.707+0000 7f229ea00640 1 -- 172.21.15.1:0/1078337700 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2284003520 con 0x7f22a0159b40 2024-05-09T08:07:12.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.707+0000 7f229ea00640 1 -- 172.21.15.1:0/1078337700 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1554518751 0 0) 0x7f22a0073520 con 0x7f22a0150540 2024-05-09T08:07:12.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.707+0000 7f229ea00640 1 -- 172.21.15.1:0/1078337700 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f22840048e0 con 0x7f22a0150540 2024-05-09T08:07:12.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.708+0000 7f229ea00640 1 -- 172.21.15.1:0/1078337700 <== mon.1 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3915222689 0 0) 0x7f2284003520 con 0x7f22a0159b40 2024-05-09T08:07:12.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.708+0000 7f229ea00640 1 -- 172.21.15.1:0/1078337700 >> v2:172.21.15.7:3300/0 conn(0x7f22a01509b0 msgr2=0x7f22a006f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:07:12.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.708+0000 7f229ea00640 1 --2- 172.21.15.1:0/1078337700 >> v2:172.21.15.7:3300/0 conn(0x7f22a01509b0 0x7f22a006f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:12.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.708+0000 7f229ea00640 1 -- 172.21.15.1:0/1078337700 >> v1:172.21.15.1:6789/0 conn(0x7f22a0150540 legacy=0x7f22a0156d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:07:12.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.709+0000 7f229ea00640 1 -- 172.21.15.1:0/1078337700 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22a015a0d0 con 0x7f22a0159b40 2024-05-09T08:07:12.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.709+0000 7f229ea00640 1 -- 172.21.15.1:0/1078337700 <== mon.1 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 4179846320 0 0) 0x7f2290001650 con 0x7f22a0159b40 2024-05-09T08:07:12.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.711+0000 7f229ea00640 1 -- 172.21.15.1:0/1078337700 >> v1:172.21.15.7:6789/0 conn(0x7f22a0159b40 legacy=0x7f22a00728e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:07:12.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.711+0000 7f229ea00640 1 --2- 172.21.15.1:0/1078337700 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2284004d90 0x7f2284005170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:07:12.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.712+0000 7f229ea00640 1 --2- 172.21.15.1:0/1078337700 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f22840037a0 0x7f2284003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:07:12.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.712+0000 7f229ea00640 1 --2- 172.21.15.1:0/1078337700 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2284004220 0x7f228400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:07:12.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.712+0000 7f229ea00640 1 -- 172.21.15.1:0/1078337700 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f22a01568f0 con 0x7f2284004220 2024-05-09T08:07:12.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.712+0000 7f229ea00640 1 -- 172.21.15.1:0/1078337700 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f228400c1d0 con 0x7f2284004d90 2024-05-09T08:07:12.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.712+0000 7f229ea00640 1 -- 172.21.15.1:0/1078337700 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f228400c3d0 con 0x7f22840037a0 2024-05-09T08:07:12.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.712+0000 7f229f400640 1 -- 172.21.15.1:0/1078337700 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f22840037a0 msgr2=0x7f2284003c20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.7:3301/0 2024-05-09T08:07:12.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.712+0000 7f229f400640 1 --2- 172.21.15.1:0/1078337700 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f22840037a0 0x7f2284003c20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T08:07:12.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.712+0000 7f229fe00640 1 --2- 172.21.15.1:0/1078337700 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2284004d90 0x7f2284005170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:07:12.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.713+0000 7f22a5200640 1 --2- 172.21.15.1:0/1078337700 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2284004220 0x7f228400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:07:12.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.713+0000 7f229fe00640 1 -- 172.21.15.1:0/1078337700 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f22840037a0 msgr2=0x7f2284003c20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T08:07:12.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.713+0000 7f229fe00640 1 --2- 172.21.15.1:0/1078337700 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f22840037a0 0x7f2284003c20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:12.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.713+0000 7f229fe00640 1 -- 172.21.15.1:0/1078337700 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2284004220 msgr2=0x7f228400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:07:12.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.713+0000 7f229fe00640 1 --2- 172.21.15.1:0/1078337700 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2284004220 0x7f228400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:12.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.713+0000 7f229fe00640 1 -- 172.21.15.1:0/1078337700 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2284003520 con 0x7f2284004d90 2024-05-09T08:07:12.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.713+0000 7f22a5200640 1 --2- 172.21.15.1:0/1078337700 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2284004220 0x7f228400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T08:07:12.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.713+0000 7f22a5200640 1 -- 172.21.15.1:0/1078337700 reap_dead start 2024-05-09T08:07:12.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.716+0000 7f229fe00640 1 --2- 172.21.15.1:0/1078337700 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2284004d90 0x7f2284005170 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f229000c630 tx=0x7f2284005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9d3e2958c322eb90 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:07:12.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.717+0000 7f229ea00640 1 -- 172.21.15.1:0/1078337700 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f229000c200 con 0x7f2284004d90 2024-05-09T08:07:12.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.717+0000 7f229ea00640 1 -- 172.21.15.1:0/1078337700 <== mon.1 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f229002f030 con 0x7f2284004d90 2024-05-09T08:07:12.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.718+0000 7f229ea00640 1 -- 172.21.15.1:0/1078337700 <== mon.1 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22900036b0 con 0x7f2284004d90 2024-05-09T08:07:12.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.719+0000 7f22a7200640 1 -- 172.21.15.1:0/1078337700 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2284004d90 msgr2=0x7f2284005170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:07:12.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.719+0000 7f22a7200640 1 --2- 172.21.15.1:0/1078337700 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2284004d90 0x7f2284005170 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f229000c630 tx=0x7f2284005830 comp rx=0 tx=0).stop 2024-05-09T08:07:12.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.722+0000 7f22a7200640 1 -- 172.21.15.1:0/1078337700 shutdown_connections 2024-05-09T08:07:12.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.722+0000 7f22a7200640 1 --2- 172.21.15.1:0/1078337700 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2284004d90 0x7f2284005170 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:12.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.722+0000 7f22a7200640 1 -- 172.21.15.1:0/1078337700 >> 172.21.15.1:0/1078337700 conn(0x7f22a006dad0 msgr2=0x7f22a006deb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:07:12.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.723+0000 7f22a7200640 1 -- 172.21.15.1:0/1078337700 shutdown_connections 2024-05-09T08:07:12.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.723+0000 7f22a7200640 1 -- 172.21.15.1:0/1078337700 wait complete. 2024-05-09T08:07:12.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.726+0000 7f22a7200640 1 Processor -- start 2024-05-09T08:07:12.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.730+0000 7f22a7200640 1 -- start start 2024-05-09T08:07:12.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.735+0000 7f22a7200640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2284004d90 0x7f22a0140af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:07:12.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.736+0000 7f22a7200640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f22a01419d0 0x7f22a0141e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:07:12.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.736+0000 7f229fe00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2284004d90 0x7f22a0140af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:07:12.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.736+0000 7f22a7200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22a01fe230 0x7f22a01fe6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:07:12.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.736+0000 7f229fe00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2284004d90 0x7f22a0140af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:52072/0 (socket says 172.21.15.1:52072) 2024-05-09T08:07:12.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.737+0000 7f22a5200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22a01fe230 0x7f22a01fe6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:07:12.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.737+0000 7f22a5200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22a01fe230 0x7f22a01fe6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.1:43998/0 (socket says 172.21.15.1:43998) 2024-05-09T08:07:12.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.741+0000 7f22a7200640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f22a015a970 con 0x7f2284004d90 2024-05-09T08:07:12.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.741+0000 7f22a7200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f22a015a690 con 0x7f22a01fe230 2024-05-09T08:07:12.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.741+0000 7f22a7200640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f22a00728e0 con 0x7f22a01419d0 2024-05-09T08:07:12.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.741+0000 7f229fe00640 1 -- 172.21.15.1:0/1428241062 learned_addr learned my addr 172.21.15.1:0/1428241062 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:07:12.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.741+0000 7f229f400640 1 -- 172.21.15.1:0/1428241062 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f22a01419d0 msgr2=0x7f22a0141e10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.7:3301/0 2024-05-09T08:07:12.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.741+0000 7f22a5200640 1 -- 172.21.15.1:0/1428241062 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f22a01419d0 msgr2=0x7f22a0141e10 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-09T08:07:12.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.741+0000 7f229f400640 1 --2- 172.21.15.1:0/1428241062 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f22a01419d0 0x7f22a0141e10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T08:07:12.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.742+0000 7f22a5200640 1 --2- 172.21.15.1:0/1428241062 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f22a01419d0 0x7f22a0141e10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:12.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.742+0000 7f22a5200640 1 -- 172.21.15.1:0/1428241062 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2284004d90 msgr2=0x7f22a0140af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:07:12.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.742+0000 7f22a5200640 1 --2- 172.21.15.1:0/1428241062 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2284004d90 0x7f22a0140af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:12.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.742+0000 7f22a5200640 1 -- 172.21.15.1:0/1428241062 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22a01febf0 con 0x7f22a01fe230 2024-05-09T08:07:12.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.742+0000 7f229fe00640 1 --2- 172.21.15.1:0/1428241062 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2284004d90 0x7f22a0140af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:07:12.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.742+0000 7f22a5200640 1 --2- 172.21.15.1:0/1428241062 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22a01fe230 0x7f22a01fe6b0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f22a00cd070 tx=0x7f229800f520 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:07:12.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.742+0000 7f2297e00640 1 -- 172.21.15.1:0/1428241062 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2298014410 con 0x7f22a01fe230 2024-05-09T08:07:12.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.742+0000 7f2297e00640 1 -- 172.21.15.1:0/1428241062 <== mon.1 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2298019070 con 0x7f22a01fe230 2024-05-09T08:07:12.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.742+0000 7f2297e00640 1 -- 172.21.15.1:0/1428241062 <== mon.1 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22980147d0 con 0x7f22a01fe230 2024-05-09T08:07:12.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.745+0000 7f22a7200640 1 -- 172.21.15.1:0/1428241062 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22a0155f70 con 0x7f22a01fe230 2024-05-09T08:07:12.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.745+0000 7f22a7200640 1 -- 172.21.15.1:0/1428241062 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22a0158e40 con 0x7f22a01fe230 2024-05-09T08:07:12.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.747+0000 7f22a7200640 1 --2- 172.21.15.1:0/1428241062 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22a0200110 0x7f22a02004f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:07:12.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.747+0000 7f22a7200640 1 -- 172.21.15.1:0/1428241062 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f22a0074d70 con 0x7f22a0200110 2024-05-09T08:07:12.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.748+0000 7f229fe00640 1 --2- 172.21.15.1:0/1428241062 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22a0200110 0x7f22a02004f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:07:12.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.749+0000 7f229fe00640 1 -- 172.21.15.1:0/1428241062 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f22a0142520 con 0x7f22a0200110 2024-05-09T08:07:12.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.750+0000 7f229fe00640 1 --2- 172.21.15.1:0/1428241062 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22a0200110 0x7f22a02004f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f229000c190 tx=0x7f229002d070 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:07:12.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.751+0000 7f2297e00640 1 -- 172.21.15.1:0/1428241062 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2290045070 con 0x7f22a0200110 2024-05-09T08:07:12.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.754+0000 7f2297e00640 1 -- 172.21.15.1:0/1428241062 <== mon.1 v2:172.21.15.7:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f2290006ac0 con 0x7f22a0200110 2024-05-09T08:07:12.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.754+0000 7f2297e00640 1 -- 172.21.15.1:0/1428241062 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22a0200110 msgr2=0x7f22a02004f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:07:12.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.754+0000 7f2297e00640 1 --2- 172.21.15.1:0/1428241062 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22a0200110 0x7f22a02004f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f229000c190 tx=0x7f229002d070 comp rx=0 tx=0).stop 2024-05-09T08:07:12.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.754+0000 7f2297e00640 1 -- 172.21.15.1:0/1428241062 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22a0200110 msgr2=0x7f22a02004f0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-05-09T08:07:12.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.754+0000 7f2297e00640 1 --2- 172.21.15.1:0/1428241062 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22a0200110 0x7f22a02004f0 secure :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f229000c190 tx=0x7f229002d070 comp rx=0 tx=0).stop 2024-05-09T08:07:12.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.755+0000 7f2297e00640 1 -- 172.21.15.1:0/1428241062 <== mon.1 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2298014970 con 0x7f22a01fe230 2024-05-09T08:07:12.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.756+0000 7f2297e00640 1 --2- 172.21.15.1:0/1428241062 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f227c06b910 0x7f227c06ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:07:12.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.756+0000 7f2297e00640 1 -- 172.21.15.1:0/1428241062 <== mon.1 v2:172.21.15.7:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7f229808b670 con 0x7f22a01fe230 2024-05-09T08:07:12.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.758+0000 7f229fe00640 1 --2- 172.21.15.1:0/1428241062 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f227c06b910 0x7f227c06ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:07:12.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.759+0000 7f229fe00640 1 --2- 172.21.15.1:0/1428241062 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f227c06b910 0x7f227c06ddd0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f22a0074ef0 tx=0x7f2290031000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:07:12.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.770+0000 7f22a7200640 1 --2- 172.21.15.1:0/1428241062 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22a0200a30 0x7f22a01f97e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:07:12.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.770+0000 7f22a7200640 1 -- 172.21.15.1:0/1428241062 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f22a01437f0 con 0x7f22a0200a30 2024-05-09T08:07:12.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.771+0000 7f229f400640 1 --2- 172.21.15.1:0/1428241062 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22a0200a30 0x7f22a01f97e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:07:12.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.774+0000 7f229f400640 1 -- 172.21.15.1:0/1428241062 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f22a014fad0 con 0x7f22a0200a30 2024-05-09T08:07:12.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.775+0000 7f229f400640 1 --2- 172.21.15.1:0/1428241062 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22a0200a30 0x7f22a01f97e0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f22a0070740 tx=0x7f228800cb10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:07:12.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.776+0000 7f2297e00640 1 -- 172.21.15.1:0/1428241062 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2288016020 con 0x7f22a0200a30 2024-05-09T08:07:12.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.777+0000 7f2297e00640 1 -- 172.21.15.1:0/1428241062 <== mon.1 v2:172.21.15.7:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5177 (secure 0 0 0) 0x7f228800ce50 con 0x7f22a0200a30 2024-05-09T08:07:12.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.777+0000 7f2297e00640 1 -- 172.21.15.1:0/1428241062 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22a0200a30 msgr2=0x7f22a01f97e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:07:12.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.777+0000 7f2297e00640 1 --2- 172.21.15.1:0/1428241062 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22a0200a30 0x7f22a01f97e0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f22a0070740 tx=0x7f228800cb10 comp rx=0 tx=0).stop 2024-05-09T08:07:12.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.777+0000 7f2297e00640 1 -- 172.21.15.1:0/1428241062 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22a0200a30 msgr2=0x7f22a01f97e0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-05-09T08:07:12.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.777+0000 7f2297e00640 1 --2- 172.21.15.1:0/1428241062 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22a0200a30 0x7f22a01f97e0 secure :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f22a0070740 tx=0x7f228800cb10 comp rx=0 tx=0).stop 2024-05-09T08:07:12.790 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-05-09T08:07:12.790 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "b", 2024-05-09T08:07:12.790 INFO:teuthology.orchestra.run.smithi001.stdout: "rank": 1, 2024-05-09T08:07:12.790 INFO:teuthology.orchestra.run.smithi001.stdout: "state": "peon", 2024-05-09T08:07:12.790 INFO:teuthology.orchestra.run.smithi001.stdout: "election_epoch": 40, 2024-05-09T08:07:12.790 INFO:teuthology.orchestra.run.smithi001.stdout: "uptime": 200438, 2024-05-09T08:07:12.790 INFO:teuthology.orchestra.run.smithi001.stdout: "quorum": [ 2024-05-09T08:07:12.790 INFO:teuthology.orchestra.run.smithi001.stdout: 0, 2024-05-09T08:07:12.790 INFO:teuthology.orchestra.run.smithi001.stdout: 1 2024-05-09T08:07:12.790 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:07:12.790 INFO:teuthology.orchestra.run.smithi001.stdout: "quorum_age": 1, 2024-05-09T08:07:12.791 INFO:teuthology.orchestra.run.smithi001.stdout: "features": { 2024-05-09T08:07:12.791 INFO:teuthology.orchestra.run.smithi001.stdout: "required_con": "2450521708007866372", 2024-05-09T08:07:12.791 INFO:teuthology.orchestra.run.smithi001.stdout: "required_mon": [ 2024-05-09T08:07:12.791 INFO:teuthology.orchestra.run.smithi001.stdout: "kraken", 2024-05-09T08:07:12.791 INFO:teuthology.orchestra.run.smithi001.stdout: "luminous", 2024-05-09T08:07:12.791 INFO:teuthology.orchestra.run.smithi001.stdout: "mimic", 2024-05-09T08:07:12.791 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap-prune", 2024-05-09T08:07:12.791 INFO:teuthology.orchestra.run.smithi001.stdout: "nautilus", 2024-05-09T08:07:12.791 INFO:teuthology.orchestra.run.smithi001.stdout: "octopus", 2024-05-09T08:07:12.791 INFO:teuthology.orchestra.run.smithi001.stdout: "pacific", 2024-05-09T08:07:12.791 INFO:teuthology.orchestra.run.smithi001.stdout: "elector-pinging", 2024-05-09T08:07:12.791 INFO:teuthology.orchestra.run.smithi001.stdout: "quincy", 2024-05-09T08:07:12.791 INFO:teuthology.orchestra.run.smithi001.stdout: "reef", 2024-05-09T08:07:12.791 INFO:teuthology.orchestra.run.smithi001.stdout: "squid" 2024-05-09T08:07:12.792 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:07:12.792 INFO:teuthology.orchestra.run.smithi001.stdout: "quorum_con": "4540701547738038271", 2024-05-09T08:07:12.792 INFO:teuthology.orchestra.run.smithi001.stdout: "quorum_mon": [ 2024-05-09T08:07:12.792 INFO:teuthology.orchestra.run.smithi001.stdout: "kraken", 2024-05-09T08:07:12.792 INFO:teuthology.orchestra.run.smithi001.stdout: "luminous", 2024-05-09T08:07:12.792 INFO:teuthology.orchestra.run.smithi001.stdout: "mimic", 2024-05-09T08:07:12.792 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap-prune", 2024-05-09T08:07:12.792 INFO:teuthology.orchestra.run.smithi001.stdout: "nautilus", 2024-05-09T08:07:12.792 INFO:teuthology.orchestra.run.smithi001.stdout: "octopus", 2024-05-09T08:07:12.792 INFO:teuthology.orchestra.run.smithi001.stdout: "pacific", 2024-05-09T08:07:12.792 INFO:teuthology.orchestra.run.smithi001.stdout: "elector-pinging", 2024-05-09T08:07:12.792 INFO:teuthology.orchestra.run.smithi001.stdout: "quincy", 2024-05-09T08:07:12.792 INFO:teuthology.orchestra.run.smithi001.stdout: "reef", 2024-05-09T08:07:12.793 INFO:teuthology.orchestra.run.smithi001.stdout: "squid" 2024-05-09T08:07:12.793 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:07:12.793 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:07:12.793 INFO:teuthology.orchestra.run.smithi001.stdout: "outside_quorum": [], 2024-05-09T08:07:12.793 INFO:teuthology.orchestra.run.smithi001.stdout: "extra_probe_peers": [], 2024-05-09T08:07:12.793 INFO:teuthology.orchestra.run.smithi001.stdout: "sync_provider": [], 2024-05-09T08:07:12.793 INFO:teuthology.orchestra.run.smithi001.stdout: "monmap": { 2024-05-09T08:07:12.793 INFO:teuthology.orchestra.run.smithi001.stdout: "epoch": 1, 2024-05-09T08:07:12.793 INFO:teuthology.orchestra.run.smithi001.stdout: "fsid": "54329ec8-3270-4043-9a82-c6cd7420aa7f", 2024-05-09T08:07:12.793 INFO:teuthology.orchestra.run.smithi001.stdout: "modified": "2024-05-09T08:01:21.819270Z", 2024-05-09T08:07:12.793 INFO:teuthology.orchestra.run.smithi001.stdout: "created": "2024-05-09T08:01:21.819270Z", 2024-05-09T08:07:12.793 INFO:teuthology.orchestra.run.smithi001.stdout: "min_mon_release": 19, 2024-05-09T08:07:12.793 INFO:teuthology.orchestra.run.smithi001.stdout: "min_mon_release_name": "squid", 2024-05-09T08:07:12.794 INFO:teuthology.orchestra.run.smithi001.stdout: "election_strategy": 1, 2024-05-09T08:07:12.794 INFO:teuthology.orchestra.run.smithi001.stdout: "disallowed_leaders: ": "", 2024-05-09T08:07:12.794 INFO:teuthology.orchestra.run.smithi001.stdout: "stretch_mode": false, 2024-05-09T08:07:12.794 INFO:teuthology.orchestra.run.smithi001.stdout: "tiebreaker_mon": "", 2024-05-09T08:07:12.794 INFO:teuthology.orchestra.run.smithi001.stdout: "removed_ranks: ": "", 2024-05-09T08:07:12.794 INFO:teuthology.orchestra.run.smithi001.stdout: "features": { 2024-05-09T08:07:12.794 INFO:teuthology.orchestra.run.smithi001.stdout: "persistent": [ 2024-05-09T08:07:12.794 INFO:teuthology.orchestra.run.smithi001.stdout: "kraken", 2024-05-09T08:07:12.794 INFO:teuthology.orchestra.run.smithi001.stdout: "luminous", 2024-05-09T08:07:12.794 INFO:teuthology.orchestra.run.smithi001.stdout: "mimic", 2024-05-09T08:07:12.794 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap-prune", 2024-05-09T08:07:12.794 INFO:teuthology.orchestra.run.smithi001.stdout: "nautilus", 2024-05-09T08:07:12.794 INFO:teuthology.orchestra.run.smithi001.stdout: "octopus", 2024-05-09T08:07:12.795 INFO:teuthology.orchestra.run.smithi001.stdout: "pacific", 2024-05-09T08:07:12.795 INFO:teuthology.orchestra.run.smithi001.stdout: "elector-pinging", 2024-05-09T08:07:12.795 INFO:teuthology.orchestra.run.smithi001.stdout: "quincy", 2024-05-09T08:07:12.795 INFO:teuthology.orchestra.run.smithi001.stdout: "reef", 2024-05-09T08:07:12.795 INFO:teuthology.orchestra.run.smithi001.stdout: "squid" 2024-05-09T08:07:12.795 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:07:12.795 INFO:teuthology.orchestra.run.smithi001.stdout: "optional": [] 2024-05-09T08:07:12.795 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:07:12.795 INFO:teuthology.orchestra.run.smithi001.stdout: "mons": [ 2024-05-09T08:07:12.795 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:07:12.795 INFO:teuthology.orchestra.run.smithi001.stdout: "rank": 0, 2024-05-09T08:07:12.795 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "a", 2024-05-09T08:07:12.795 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-05-09T08:07:12.796 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-05-09T08:07:12.796 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:07:12.797 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-05-09T08:07:12.797 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:3300", 2024-05-09T08:07:12.797 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:07:12.797 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:07:12.797 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:07:12.797 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-05-09T08:07:12.797 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6789", 2024-05-09T08:07:12.797 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:07:12.797 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:07:12.797 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:07:12.797 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:07:12.798 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6789/0", 2024-05-09T08:07:12.798 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.1:6789/0", 2024-05-09T08:07:12.798 INFO:teuthology.orchestra.run.smithi001.stdout: "priority": 0, 2024-05-09T08:07:12.798 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 0, 2024-05-09T08:07:12.798 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_location": "{}" 2024-05-09T08:07:12.798 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:07:12.798 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:07:12.798 INFO:teuthology.orchestra.run.smithi001.stdout: "rank": 1, 2024-05-09T08:07:12.798 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "b", 2024-05-09T08:07:12.798 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-05-09T08:07:12.798 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-05-09T08:07:12.798 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:07:12.799 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-05-09T08:07:12.799 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:3300", 2024-05-09T08:07:12.799 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:07:12.799 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:07:12.799 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:07:12.799 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-05-09T08:07:12.799 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:6789", 2024-05-09T08:07:12.799 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:07:12.799 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:07:12.799 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:07:12.799 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:07:12.799 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:6789/0", 2024-05-09T08:07:12.800 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.7:6789/0", 2024-05-09T08:07:12.800 INFO:teuthology.orchestra.run.smithi001.stdout: "priority": 0, 2024-05-09T08:07:12.800 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 0, 2024-05-09T08:07:12.800 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_location": "{}" 2024-05-09T08:07:12.800 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:07:12.800 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:07:12.800 INFO:teuthology.orchestra.run.smithi001.stdout: "rank": 2, 2024-05-09T08:07:12.800 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "c", 2024-05-09T08:07:12.800 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-05-09T08:07:12.800 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-05-09T08:07:12.800 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:07:12.800 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-05-09T08:07:12.800 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:3301", 2024-05-09T08:07:12.801 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:07:12.801 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:07:12.801 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:07:12.801 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-05-09T08:07:12.801 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:6790", 2024-05-09T08:07:12.801 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:07:12.801 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:07:12.801 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:07:12.801 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:07:12.801 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:6790/0", 2024-05-09T08:07:12.801 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.7:6790/0", 2024-05-09T08:07:12.801 INFO:teuthology.orchestra.run.smithi001.stdout: "priority": 0, 2024-05-09T08:07:12.802 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 0, 2024-05-09T08:07:12.802 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_location": "{}" 2024-05-09T08:07:12.802 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:07:12.802 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:07:12.802 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:07:12.802 INFO:teuthology.orchestra.run.smithi001.stdout: "feature_map": { 2024-05-09T08:07:12.802 INFO:teuthology.orchestra.run.smithi001.stdout: "mon": [ 2024-05-09T08:07:12.802 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:07:12.802 INFO:teuthology.orchestra.run.smithi001.stdout: "features": "0x3f03cffffffdffff", 2024-05-09T08:07:12.802 INFO:teuthology.orchestra.run.smithi001.stdout: "release": "luminous", 2024-05-09T08:07:12.802 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 1 2024-05-09T08:07:12.803 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:07:12.803 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:07:12.803 INFO:teuthology.orchestra.run.smithi001.stdout: "mds": [ 2024-05-09T08:07:12.803 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:07:12.803 INFO:teuthology.orchestra.run.smithi001.stdout: "features": "0x3f03cffffffdffff", 2024-05-09T08:07:12.803 INFO:teuthology.orchestra.run.smithi001.stdout: "release": "luminous", 2024-05-09T08:07:12.803 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 6 2024-05-09T08:07:12.803 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:07:12.803 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:07:12.803 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": [ 2024-05-09T08:07:12.803 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:07:12.803 INFO:teuthology.orchestra.run.smithi001.stdout: "features": "0x3f03cffffffdffff", 2024-05-09T08:07:12.804 INFO:teuthology.orchestra.run.smithi001.stdout: "release": "luminous", 2024-05-09T08:07:12.804 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 7 2024-05-09T08:07:12.804 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:07:12.804 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:07:12.804 INFO:teuthology.orchestra.run.smithi001.stdout: "client": [ 2024-05-09T08:07:12.804 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:07:12.804 INFO:teuthology.orchestra.run.smithi001.stdout: "features": "0x2f018fb87aa4aafe", 2024-05-09T08:07:12.804 INFO:teuthology.orchestra.run.smithi001.stdout: "release": "luminous", 2024-05-09T08:07:12.804 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 1 2024-05-09T08:07:12.804 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:07:12.804 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:07:12.804 INFO:teuthology.orchestra.run.smithi001.stdout: "features": "0x3f03cffffffdffff", 2024-05-09T08:07:12.804 INFO:teuthology.orchestra.run.smithi001.stdout: "release": "luminous", 2024-05-09T08:07:12.805 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 6 2024-05-09T08:07:12.805 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:07:12.805 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:07:12.805 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr": [ 2024-05-09T08:07:12.805 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:07:12.805 INFO:teuthology.orchestra.run.smithi001.stdout: "features": "0x3f03cffffffdffff", 2024-05-09T08:07:12.805 INFO:teuthology.orchestra.run.smithi001.stdout: "release": "luminous", 2024-05-09T08:07:12.805 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 1 2024-05-09T08:07:12.805 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:07:12.805 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:07:12.805 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:07:12.805 INFO:teuthology.orchestra.run.smithi001.stdout: "stretch_mode": false 2024-05-09T08:07:12.805 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-05-09T08:07:12.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.779+0000 7f22a7200640 1 -- 172.21.15.1:0/1428241062 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f227c06b910 msgr2=0x7f227c06ddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:07:12.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.779+0000 7f22a7200640 1 --2- 172.21.15.1:0/1428241062 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f227c06b910 0x7f227c06ddd0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f22a0074ef0 tx=0x7f2290031000 comp rx=0 tx=0).stop 2024-05-09T08:07:12.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.779+0000 7f22a7200640 1 -- 172.21.15.1:0/1428241062 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22a01fe230 msgr2=0x7f22a01fe6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:07:12.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.779+0000 7f22a7200640 1 --2- 172.21.15.1:0/1428241062 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22a01fe230 0x7f22a01fe6b0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f22a00cd070 tx=0x7f229800f520 comp rx=0 tx=0).stop 2024-05-09T08:07:12.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.780+0000 7f22a5200640 1 -- 172.21.15.1:0/1428241062 reap_dead start 2024-05-09T08:07:12.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.780+0000 7f22a5200640 1 -- 172.21.15.1:0/1428241062 reap_dead start 2024-05-09T08:07:12.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.780+0000 7f22a7200640 1 -- 172.21.15.1:0/1428241062 shutdown_connections 2024-05-09T08:07:12.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.780+0000 7f22a7200640 1 -- 172.21.15.1:0/1428241062 >> 172.21.15.1:0/1428241062 conn(0x7f22a006dad0 msgr2=0x7f22a006f720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:07:12.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.780+0000 7f22a7200640 1 -- 172.21.15.1:0/1428241062 shutdown_connections 2024-05-09T08:07:12.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:12.780+0000 7f22a7200640 1 -- 172.21.15.1:0/1428241062 wait complete. 2024-05-09T08:07:12.839 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-05-09T08:07:12.839 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2024-05-09T08:07:14.612 DEBUG:teuthology.parallel:result is None 2024-05-09T08:07:14.612 DEBUG:teuthology.orchestra.run.smithi001:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-05-09T08:07:14.739 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-05-09T08:07:14.739 DEBUG:teuthology.run_tasks:Unwinding manager mon_thrash 2024-05-09T08:07:14.750 INFO:tasks.mon_thrash:joining mon_thrasher 2024-05-09T08:07:21.588 DEBUG:teuthology.orchestra.run.smithi001:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T08:07:21.596 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T08:07:32.840 INFO:tasks.mon_thrash.mon_thrasher:killing mon.c 2024-05-09T08:07:32.840 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.c 2024-05-09T08:07:32.840 INFO:tasks.ceph.mon.c:Restarting daemon 2024-05-09T08:07:32.841 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-05-09T08:07:32.844 INFO:tasks.ceph.mon.c:Started 2024-05-09T08:07:32.844 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-05-09T08:07:32.844 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-05-09T08:07:33.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:33.053+0000 7f5adbe00640 1 Processor -- start 2024-05-09T08:07:33.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:33.053+0000 7f5adbe00640 1 -- start start 2024-05-09T08:07:33.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:33.054+0000 7f5adbe00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f5ad40c1ba0 0x7f5ad40c6c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:07:33.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:33.055+0000 7f5adbe00640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f5ad40b1d30 0x7f5ad40c7a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:07:33.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:33.055+0000 7f5adbe00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ad40c72f0 con 0x7f5ad4151660 2024-05-09T08:07:33.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:33.055+0000 7f5adbe00640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f5ad4151a40 con 0x7f5ad40b1d30 2024-05-09T08:07:33.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:33.055+0000 7f5adbe00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f5ad4157620 con 0x7f5ad40c1ba0 2024-05-09T08:07:33.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:33.055+0000 7f5ad9400640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f5ad4151660 0x7f5ad4072d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.1:33136/0 (socket says 172.21.15.1:33136) 2024-05-09T08:07:33.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:33.055+0000 7f5ad9400640 1 -- 172.21.15.1:0/4097706834 learned_addr learned my addr 172.21.15.1:0/4097706834 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T08:07:33.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:33.055+0000 7f5ad9e00640 1 --2- 172.21.15.1:0/4097706834 >> v2:172.21.15.1:3300/0 conn(0x7f5ad40b1d30 0x7f5ad40c7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:07:33.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:33.055+0000 7f5ad8a00640 1 --2- 172.21.15.1:0/4097706834 >> v2:172.21.15.7:3300/0 conn(0x7f5ad40c1ba0 0x7f5ad40c6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:07:33.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:33.055+0000 7f5ad3e00640 1 -- 172.21.15.1:0/4097706834 <== mon.1 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4075408696 0 0) 0x7f5ad40c72f0 con 0x7f5ad4151660 2024-05-09T08:07:33.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:33.055+0000 7f5ad3e00640 1 -- 172.21.15.1:0/4097706834 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5abc003520 con 0x7f5ad4151660 2024-05-09T08:07:33.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:33.056+0000 7f5ad9e00640 1 -- 172.21.15.1:0/4097706834 >> v2:172.21.15.7:3300/0 conn(0x7f5ad40c1ba0 msgr2=0x7f5ad40c6c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:07:33.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:33.056+0000 7f5ad9e00640 1 --2- 172.21.15.1:0/4097706834 >> v2:172.21.15.7:3300/0 conn(0x7f5ad40c1ba0 0x7f5ad40c6c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:33.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:33.056+0000 7f5ad9e00640 1 -- 172.21.15.1:0/4097706834 >> v1:172.21.15.7:6789/0 conn(0x7f5ad4151660 legacy=0x7f5ad4072d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:07:33.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:33.056+0000 7f5ad9e00640 1 -- 172.21.15.1:0/4097706834 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ad4157900 con 0x7f5ad40b1d30 2024-05-09T08:07:33.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:33.056+0000 7f5ad8a00640 1 --2- 172.21.15.1:0/4097706834 >> v2:172.21.15.7:3300/0 conn(0x7f5ad40c1ba0 0x7f5ad40c6c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:07:33.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:33.057+0000 7f5ad9e00640 1 --2- 172.21.15.1:0/4097706834 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5ad40b1d30 0x7f5ad40c7a10 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f5ad406d3a0 tx=0x7f5acc02a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eca8432f83087f1e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:07:33.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:33.057+0000 7f5ad3e00640 1 -- 172.21.15.1:0/4097706834 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5acc007740 con 0x7f5ad40b1d30 2024-05-09T08:07:33.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:33.057+0000 7f5ad3e00640 1 -- 172.21.15.1:0/4097706834 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5acc0078e0 con 0x7f5ad40b1d30 2024-05-09T08:07:33.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:33.057+0000 7f5ad3e00640 1 -- 172.21.15.1:0/4097706834 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5acc007cb0 con 0x7f5ad40b1d30 2024-05-09T08:07:33.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:33.058+0000 7f5adbe00640 1 -- 172.21.15.1:0/4097706834 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5ad40b1d30 msgr2=0x7f5ad40c7a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:07:33.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:33.058+0000 7f5adbe00640 1 --2- 172.21.15.1:0/4097706834 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5ad40b1d30 0x7f5ad40c7a10 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f5ad406d3a0 tx=0x7f5acc02a680 comp rx=0 tx=0).stop 2024-05-09T08:07:33.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:33.058+0000 7f5adbe00640 1 -- 172.21.15.1:0/4097706834 shutdown_connections 2024-05-09T08:07:33.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:33.058+0000 7f5adbe00640 1 --2- 172.21.15.1:0/4097706834 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5ad40b1d30 0x7f5ad40c7a10 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:33.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:33.058+0000 7f5adbe00640 1 --2- 172.21.15.1:0/4097706834 >> v2:172.21.15.7:3300/0 conn(0x7f5ad40c1ba0 0x7f5ad40c6c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:33.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:33.058+0000 7f5adbe00640 1 -- 172.21.15.1:0/4097706834 >> 172.21.15.1:0/4097706834 conn(0x7f5ad40cc070 msgr2=0x7f5ad40cc480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:07:33.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:33.058+0000 7f5adbe00640 1 -- 172.21.15.1:0/4097706834 shutdown_connections 2024-05-09T08:07:33.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:33.058+0000 7f5adbe00640 1 -- 172.21.15.1:0/4097706834 wait complete. 2024-05-09T08:07:33.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:33.060+0000 7f5adbe00640 1 Processor -- start 2024-05-09T08:07:33.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:33.060+0000 7f5adbe00640 1 -- start start 2024-05-09T08:07:33.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:33.061+0000 7f5adbe00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f5ad40b1d30 0x7f5ad4142540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:07:33.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:33.061+0000 7f5adbe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5ad40c1ba0 0x7f5ad4142a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:07:33.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:33.061+0000 7f5adbe00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5ad4151660 0x7f5ad40caad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:07:33.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:33.061+0000 7f5adbe00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f5ad41581a0 con 0x7f5ad4151660 2024-05-09T08:07:33.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:33.061+0000 7f5adbe00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5ad4157ec0 con 0x7f5ad40c1ba0 2024-05-09T08:07:33.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:33.061+0000 7f5adbe00640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f5ad41572d0 con 0x7f5ad40b1d30 2024-05-09T08:07:33.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:33.062+0000 7f5ad9e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5ad4151660 0x7f5ad40caad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:07:33.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:33.062+0000 7f5ad9400640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f5ad40b1d30 0x7f5ad4142540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:07:33.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:33.062+0000 7f5ad9e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5ad4151660 0x7f5ad40caad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41382/0 (socket says 172.21.15.1:41382) 2024-05-09T08:07:33.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:33.062+0000 7f5ad9e00640 1 -- 172.21.15.1:0/1069652311 learned_addr learned my addr 172.21.15.1:0/1069652311 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:07:33.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:33.062+0000 7f5ad9400640 1 -- 172.21.15.1:0/1069652311 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f5ad40b1d30 msgr2=0x7f5ad4142540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-05-09T08:07:33.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:33.062+0000 7f5ad9400640 1 -- 172.21.15.1:0/1069652311 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f5ad40b1d30 msgr2=0x7f5ad4142540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-09T08:07:33.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:33.062+0000 7f5ad9400640 1 --2- 172.21.15.1:0/1069652311 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f5ad40b1d30 0x7f5ad4142540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-09T08:07:33.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:33.062+0000 7f5ad8a00640 1 --2- 172.21.15.1:0/1069652311 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5ad40c1ba0 0x7f5ad4142a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:07:33.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:33.062+0000 7f5ad9400640 1 --2- 172.21.15.1:0/1069652311 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f5ad40b1d30 0x7f5ad4142540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T08:07:33.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:33.062+0000 7f5ad9e00640 1 -- 172.21.15.1:0/1069652311 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f5ad40b1d30 msgr2=0x7f5ad4142540 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T08:07:33.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:33.062+0000 7f5ad9e00640 1 --2- 172.21.15.1:0/1069652311 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f5ad40b1d30 0x7f5ad4142540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:33.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:33.062+0000 7f5ad9e00640 1 -- 172.21.15.1:0/1069652311 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5ad40c1ba0 msgr2=0x7f5ad4142a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:07:33.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:33.062+0000 7f5ad9e00640 1 --2- 172.21.15.1:0/1069652311 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5ad40c1ba0 0x7f5ad4142a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:33.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:33.062+0000 7f5ad9e00640 1 -- 172.21.15.1:0/1069652311 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ad40c99c0 con 0x7f5ad4151660 2024-05-09T08:07:33.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:33.063+0000 7f5ad9e00640 1 --2- 172.21.15.1:0/1069652311 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5ad4151660 0x7f5ad40caad0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5acc009e50 tx=0x7f5acc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:07:33.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:33.063+0000 7f5ad1600640 1 -- 172.21.15.1:0/1069652311 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5acc03f070 con 0x7f5ad4151660 2024-05-09T08:07:33.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:33.063+0000 7f5ad1600640 1 -- 172.21.15.1:0/1069652311 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5acc031bf0 con 0x7f5ad4151660 2024-05-09T08:07:33.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:33.063+0000 7f5ad1600640 1 -- 172.21.15.1:0/1069652311 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5acc009450 con 0x7f5ad4151660 2024-05-09T08:07:33.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:33.063+0000 7f5adbe00640 1 -- 172.21.15.1:0/1069652311 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ad414abc0 con 0x7f5ad4151660 2024-05-09T08:07:33.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:33.063+0000 7f5adbe00640 1 -- 172.21.15.1:0/1069652311 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ad40c8e40 con 0x7f5ad4151660 2024-05-09T08:07:33.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:33.064+0000 7f5adbe00640 1 -- 172.21.15.1:0/1069652311 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ad406d760 con 0x7f5ad4151660 2024-05-09T08:07:33.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:33.065+0000 7f5ad1600640 1 -- 172.21.15.1:0/1069652311 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5acc03b410 con 0x7f5ad4151660 2024-05-09T08:07:33.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:33.066+0000 7f5ad1600640 1 --2- 172.21.15.1:0/1069652311 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f5ab406be60 0x7f5ab406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:07:33.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:33.066+0000 7f5ad1600640 1 -- 172.21.15.1:0/1069652311 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7f5acc0ac290 con 0x7f5ad4151660 2024-05-09T08:07:33.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:33.067+0000 7f5ad9400640 1 --2- 172.21.15.1:0/1069652311 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f5ab406be60 0x7f5ab406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:07:33.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:33.067+0000 7f5ad9400640 1 --2- 172.21.15.1:0/1069652311 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f5ab406be60 0x7f5ab406e320 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f5ac80011e0 tx=0x7f5ac8009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:07:33.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:33.071+0000 7f5ad1600640 1 -- 172.21.15.1:0/1069652311 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f5ad406d760 con 0x7f5ad4151660 2024-05-09T08:07:33.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:33.355+0000 7f5adbe00640 1 -- 172.21.15.1:0/1069652311 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f5ad4067960 con 0x7f5ad4151660 2024-05-09T08:07:33.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:33.356+0000 7f5ad1600640 1 -- 172.21.15.1:0/1069652311 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1539 (secure 0 0 0) 0x7f5ad4067960 con 0x7f5ad4151660 2024-05-09T08:07:33.367 INFO:teuthology.orchestra.run.smithi001.stdout:{"election_epoch":40,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":22,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"54329ec8-3270-4043-9a82-c6cd7420aa7f","modified":"2024-05-09T08:01:21.819270Z","created":"2024-05-09T08:01:21.819270Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:3300","nonce":0},{"type":"v1","addr":"172.21.15.1:6789","nonce":0}]},"addr":"172.21.15.1:6789/0","public_addr":"172.21.15.1:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:3300","nonce":0},{"type":"v1","addr":"172.21.15.7:6789","nonce":0}]},"addr":"172.21.15.7:6789/0","public_addr":"172.21.15.7:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:3301","nonce":0},{"type":"v1","addr":"172.21.15.7:6790","nonce":0}]},"addr":"172.21.15.7:6790/0","public_addr":"172.21.15.7:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-05-09T08:07:33.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:33.358+0000 7f5adbe00640 1 -- 172.21.15.1:0/1069652311 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f5ab406be60 msgr2=0x7f5ab406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:07:33.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:33.359+0000 7f5adbe00640 1 --2- 172.21.15.1:0/1069652311 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f5ab406be60 0x7f5ab406e320 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f5ac80011e0 tx=0x7f5ac8009040 comp rx=0 tx=0).stop 2024-05-09T08:07:33.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:33.359+0000 7f5adbe00640 1 -- 172.21.15.1:0/1069652311 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5ad4151660 msgr2=0x7f5ad40caad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:07:33.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:33.359+0000 7f5adbe00640 1 --2- 172.21.15.1:0/1069652311 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5ad4151660 0x7f5ad40caad0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5acc009e50 tx=0x7f5acc004a40 comp rx=0 tx=0).stop 2024-05-09T08:07:33.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:33.359+0000 7f5adbe00640 1 -- 172.21.15.1:0/1069652311 shutdown_connections 2024-05-09T08:07:33.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:33.359+0000 7f5adbe00640 1 --2- 172.21.15.1:0/1069652311 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f5ab406be60 0x7f5ab406e320 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:33.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:33.359+0000 7f5adbe00640 1 --2- 172.21.15.1:0/1069652311 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5ad40c1ba0 0x7f5ad4142a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:33.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:33.359+0000 7f5adbe00640 1 --2- 172.21.15.1:0/1069652311 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5ad4151660 0x7f5ad40caad0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:33.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:33.359+0000 7f5adbe00640 1 --2- 172.21.15.1:0/1069652311 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f5ad40b1d30 0x7f5ad4142540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:33.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:33.359+0000 7f5adbe00640 1 -- 172.21.15.1:0/1069652311 >> 172.21.15.1:0/1069652311 conn(0x7f5ad40cc070 msgr2=0x7f5ad406d9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:07:33.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:33.359+0000 7f5adbe00640 1 -- 172.21.15.1:0/1069652311 shutdown_connections 2024-05-09T08:07:33.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:33.359+0000 7f5adbe00640 1 -- 172.21.15.1:0/1069652311 wait complete. 2024-05-09T08:07:36.392 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-05-09T08:07:36.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:36.602+0000 7f91b0400640 1 Processor -- start 2024-05-09T08:07:36.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:36.602+0000 7f91b0400640 1 -- start start 2024-05-09T08:07:36.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:36.604+0000 7f91b0400640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f91a806dca0 0x7f91a8072900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:07:36.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:36.604+0000 7f91b0400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f91a80c6ea0 0x7f91a8159010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:07:36.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:36.604+0000 7f91b0400640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f91a8073200 con 0x7f91a806ece0 2024-05-09T08:07:36.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:36.604+0000 7f91b0400640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f91a8072e40 con 0x7f91a80c6ea0 2024-05-09T08:07:36.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:36.604+0000 7f91b0400640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f91a80737c0 con 0x7f91a806dca0 2024-05-09T08:07:36.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:36.604+0000 7f91ad000640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f91a80c6ea0 0x7f91a8159010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:07:36.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:36.604+0000 7f91ae400640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f91a806ece0 0x7f91a806e080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.1:47794/0 (socket says 172.21.15.1:47794) 2024-05-09T08:07:36.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:36.604+0000 7f91ada00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f91a806dca0 0x7f91a8072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:07:36.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:36.604+0000 7f91ae400640 1 -- 172.21.15.1:0/2893276604 learned_addr learned my addr 172.21.15.1:0/2893276604 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T08:07:36.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:36.605+0000 7f91ada00640 1 -- 172.21.15.1:0/2893276604 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f91a806dca0 msgr2=0x7f91a8072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-09T08:07:36.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:36.605+0000 7f91ada00640 1 -- 172.21.15.1:0/2893276604 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f91a806dca0 msgr2=0x7f91a8072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-09T08:07:36.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:36.605+0000 7f91ada00640 1 --2- 172.21.15.1:0/2893276604 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f91a806dca0 0x7f91a8072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-09T08:07:36.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:36.605+0000 7f91ada00640 1 --2- 172.21.15.1:0/2893276604 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f91a806dca0 0x7f91a8072900 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T08:07:36.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:36.605+0000 7f91ad000640 1 -- 172.21.15.1:0/2893276604 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f91a806dca0 msgr2=0x7f91a8072900 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T08:07:36.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:36.605+0000 7f91ad000640 1 --2- 172.21.15.1:0/2893276604 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f91a806dca0 0x7f91a8072900 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:36.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:36.605+0000 7f91ad000640 1 -- 172.21.15.1:0/2893276604 >> v1:172.21.15.7:6789/0 conn(0x7f91a806ece0 legacy=0x7f91a806e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:07:36.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:36.605+0000 7f91ad000640 1 -- 172.21.15.1:0/2893276604 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91a80734e0 con 0x7f91a80c6ea0 2024-05-09T08:07:36.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:36.605+0000 7f91a7e00640 1 -- 172.21.15.1:0/2893276604 <== mon.1 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3229606983 0 0) 0x7f91a8073200 con 0x7f91a806ece0 2024-05-09T08:07:36.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:36.606+0000 7f91ad000640 1 --2- 172.21.15.1:0/2893276604 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f91a80c6ea0 0x7f91a8159010 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f91940097a0 tx=0x7f919402a210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2cf6684396955f6e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:07:36.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:36.606+0000 7f91a7e00640 1 -- 172.21.15.1:0/2893276604 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9194037070 con 0x7f91a80c6ea0 2024-05-09T08:07:36.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:36.606+0000 7f91a7e00640 1 -- 172.21.15.1:0/2893276604 <== mon.1 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9194032440 con 0x7f91a80c6ea0 2024-05-09T08:07:36.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:36.606+0000 7f91a7e00640 1 -- 172.21.15.1:0/2893276604 <== mon.1 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9194032810 con 0x7f91a80c6ea0 2024-05-09T08:07:36.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:36.607+0000 7f91b0400640 1 -- 172.21.15.1:0/2893276604 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f91a80c6ea0 msgr2=0x7f91a8159010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:07:36.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:36.607+0000 7f91b0400640 1 --2- 172.21.15.1:0/2893276604 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f91a80c6ea0 0x7f91a8159010 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f91940097a0 tx=0x7f919402a210 comp rx=0 tx=0).stop 2024-05-09T08:07:36.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:36.607+0000 7f91b0400640 1 -- 172.21.15.1:0/2893276604 shutdown_connections 2024-05-09T08:07:36.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:36.607+0000 7f91b0400640 1 --2- 172.21.15.1:0/2893276604 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f91a80c6ea0 0x7f91a8159010 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:36.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:36.607+0000 7f91b0400640 1 --2- 172.21.15.1:0/2893276604 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f91a806dca0 0x7f91a8072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:36.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:36.607+0000 7f91b0400640 1 -- 172.21.15.1:0/2893276604 >> 172.21.15.1:0/2893276604 conn(0x7f91a80cc070 msgr2=0x7f91a80cc480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:07:36.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:36.607+0000 7f91b0400640 1 -- 172.21.15.1:0/2893276604 shutdown_connections 2024-05-09T08:07:36.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:36.607+0000 7f91b0400640 1 -- 172.21.15.1:0/2893276604 wait complete. 2024-05-09T08:07:36.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:36.609+0000 7f91b0400640 1 Processor -- start 2024-05-09T08:07:36.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:36.609+0000 7f91b0400640 1 -- start start 2024-05-09T08:07:36.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:36.610+0000 7f91b0400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f91a806dca0 0x7f91a81f1b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:07:36.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:36.610+0000 7f91b0400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f91a806ece0 0x7f91a8147a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:07:36.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:36.610+0000 7f91b0400640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f91a80c6ea0 0x7f91a8147f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:07:36.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:36.610+0000 7f91b0400640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f91a8072470 con 0x7f91a806dca0 2024-05-09T08:07:36.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:36.610+0000 7f91b0400640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f91a8159550 con 0x7f91a806ece0 2024-05-09T08:07:36.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:36.610+0000 7f91b0400640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f91a8072100 con 0x7f91a80c6ea0 2024-05-09T08:07:36.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:36.611+0000 7f91ada00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f91a806dca0 0x7f91a81f1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:07:36.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:36.611+0000 7f91ada00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f91a806dca0 0x7f91a81f1b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:58354/0 (socket says 172.21.15.1:58354) 2024-05-09T08:07:36.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:36.611+0000 7f91ada00640 1 -- 172.21.15.1:0/1116538996 learned_addr learned my addr 172.21.15.1:0/1116538996 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:07:36.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:36.611+0000 7f91ad000640 1 --2- 172.21.15.1:0/1116538996 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f91a806ece0 0x7f91a8147a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:07:36.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:36.611+0000 7f91ae400640 1 --2- 172.21.15.1:0/1116538996 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f91a80c6ea0 0x7f91a8147f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:07:36.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:36.611+0000 7f91ae400640 1 -- 172.21.15.1:0/1116538996 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f91a80c6ea0 msgr2=0x7f91a8147f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-05-09T08:07:36.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:36.611+0000 7f91ae400640 1 -- 172.21.15.1:0/1116538996 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f91a80c6ea0 msgr2=0x7f91a8147f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-09T08:07:36.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:36.611+0000 7f91ae400640 1 --2- 172.21.15.1:0/1116538996 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f91a80c6ea0 0x7f91a8147f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-09T08:07:36.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:36.611+0000 7f91ae400640 1 --2- 172.21.15.1:0/1116538996 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f91a80c6ea0 0x7f91a8147f60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T08:07:36.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:36.611+0000 7f91ada00640 1 -- 172.21.15.1:0/1116538996 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f91a80c6ea0 msgr2=0x7f91a8147f60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T08:07:36.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:36.611+0000 7f91ada00640 1 --2- 172.21.15.1:0/1116538996 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f91a80c6ea0 0x7f91a8147f60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:36.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:36.611+0000 7f91ada00640 1 -- 172.21.15.1:0/1116538996 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f91a806ece0 msgr2=0x7f91a8147a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:07:36.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:36.611+0000 7f91ada00640 1 --2- 172.21.15.1:0/1116538996 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f91a806ece0 0x7f91a8147a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:36.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:36.611+0000 7f91ada00640 1 -- 172.21.15.1:0/1116538996 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91a81f1400 con 0x7f91a806dca0 2024-05-09T08:07:36.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:36.612+0000 7f91ada00640 1 --2- 172.21.15.1:0/1116538996 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f91a806dca0 0x7f91a81f1b30 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f919c00d800 tx=0x7f919c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:07:36.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:36.612+0000 7f91a5600640 1 -- 172.21.15.1:0/1116538996 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f919c00b6c0 con 0x7f91a806dca0 2024-05-09T08:07:36.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:36.612+0000 7f91a5600640 1 -- 172.21.15.1:0/1116538996 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f919c019070 con 0x7f91a806dca0 2024-05-09T08:07:36.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:36.612+0000 7f91a5600640 1 -- 172.21.15.1:0/1116538996 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f919c00bac0 con 0x7f91a806dca0 2024-05-09T08:07:36.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:36.612+0000 7f91b0400640 1 -- 172.21.15.1:0/1116538996 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91a8153d00 con 0x7f91a806dca0 2024-05-09T08:07:36.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:36.612+0000 7f91b0400640 1 -- 172.21.15.1:0/1116538996 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91a81542c0 con 0x7f91a806dca0 2024-05-09T08:07:36.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:36.613+0000 7f91b0400640 1 -- 172.21.15.1:0/1116538996 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91a8073200 con 0x7f91a806dca0 2024-05-09T08:07:36.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:36.614+0000 7f91a5600640 1 -- 172.21.15.1:0/1116538996 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f919c00bc60 con 0x7f91a806dca0 2024-05-09T08:07:36.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:36.616+0000 7f91a5600640 1 --2- 172.21.15.1:0/1116538996 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f918806be60 0x7f918806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:07:36.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:36.616+0000 7f91a5600640 1 -- 172.21.15.1:0/1116538996 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7f919c08b750 con 0x7f91a806dca0 2024-05-09T08:07:36.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:36.616+0000 7f91ad000640 1 --2- 172.21.15.1:0/1116538996 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f918806be60 0x7f918806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:07:36.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:36.617+0000 7f91ad000640 1 --2- 172.21.15.1:0/1116538996 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f918806be60 0x7f918806e320 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f9194034000 tx=0x7f919402e000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:07:36.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:36.620+0000 7f91a5600640 1 -- 172.21.15.1:0/1116538996 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f91a8073200 con 0x7f91a806dca0 2024-05-09T08:07:36.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:36.899+0000 7f91b0400640 1 -- 172.21.15.1:0/1116538996 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f91a80cc450 con 0x7f91a806dca0 2024-05-09T08:07:36.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:36.900+0000 7f91a5600640 1 -- 172.21.15.1:0/1116538996 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1539 (secure 0 0 0) 0x7f91a80cc450 con 0x7f91a806dca0 2024-05-09T08:07:36.910 INFO:teuthology.orchestra.run.smithi001.stdout:{"election_epoch":40,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":25,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"54329ec8-3270-4043-9a82-c6cd7420aa7f","modified":"2024-05-09T08:01:21.819270Z","created":"2024-05-09T08:01:21.819270Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:3300","nonce":0},{"type":"v1","addr":"172.21.15.1:6789","nonce":0}]},"addr":"172.21.15.1:6789/0","public_addr":"172.21.15.1:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:3300","nonce":0},{"type":"v1","addr":"172.21.15.7:6789","nonce":0}]},"addr":"172.21.15.7:6789/0","public_addr":"172.21.15.7:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:3301","nonce":0},{"type":"v1","addr":"172.21.15.7:6790","nonce":0}]},"addr":"172.21.15.7:6790/0","public_addr":"172.21.15.7:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-05-09T08:07:36.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:36.902+0000 7f91b0400640 1 -- 172.21.15.1:0/1116538996 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f918806be60 msgr2=0x7f918806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:07:36.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:36.902+0000 7f91b0400640 1 --2- 172.21.15.1:0/1116538996 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f918806be60 0x7f918806e320 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f9194034000 tx=0x7f919402e000 comp rx=0 tx=0).stop 2024-05-09T08:07:36.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:36.902+0000 7f91b0400640 1 -- 172.21.15.1:0/1116538996 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f91a806dca0 msgr2=0x7f91a81f1b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:07:36.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:36.902+0000 7f91b0400640 1 --2- 172.21.15.1:0/1116538996 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f91a806dca0 0x7f91a81f1b30 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f919c00d800 tx=0x7f919c00dcc0 comp rx=0 tx=0).stop 2024-05-09T08:07:36.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:36.902+0000 7f91b0400640 1 -- 172.21.15.1:0/1116538996 shutdown_connections 2024-05-09T08:07:36.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:36.902+0000 7f91b0400640 1 --2- 172.21.15.1:0/1116538996 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f918806be60 0x7f918806e320 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:36.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:36.902+0000 7f91b0400640 1 --2- 172.21.15.1:0/1116538996 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f91a806ece0 0x7f91a8147a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:36.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:36.902+0000 7f91b0400640 1 --2- 172.21.15.1:0/1116538996 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f91a80c6ea0 0x7f91a8147f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:36.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:36.902+0000 7f91b0400640 1 --2- 172.21.15.1:0/1116538996 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f91a806dca0 0x7f91a81f1b30 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:36.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:36.902+0000 7f91b0400640 1 -- 172.21.15.1:0/1116538996 >> 172.21.15.1:0/1116538996 conn(0x7f91a80cc070 msgr2=0x7f91a80c7f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:07:36.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:36.903+0000 7f91b0400640 1 -- 172.21.15.1:0/1116538996 shutdown_connections 2024-05-09T08:07:36.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:36.903+0000 7f91b0400640 1 -- 172.21.15.1:0/1116538996 wait complete. 2024-05-09T08:07:39.933 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-05-09T08:07:40.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.143+0000 7ffaaf000640 1 Processor -- start 2024-05-09T08:07:40.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.143+0000 7ffaaf000640 1 -- start start 2024-05-09T08:07:40.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.145+0000 7ffaaf000640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7ffaa8151660 0x7ffaa80c6c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:07:40.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.145+0000 7ffaaf000640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffaa80c72f0 con 0x7ffaa80c1ba0 2024-05-09T08:07:40.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.145+0000 7ffaaf000640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffaa8157620 con 0x7ffaa80b1d30 2024-05-09T08:07:40.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.145+0000 7ffaaf000640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7ffaa8151a40 con 0x7ffaa8151660 2024-05-09T08:07:40.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.145+0000 7ffaa7e00640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7ffaa8151660 0x7ffaa80c6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:07:40.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.145+0000 7ffaa7e00640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7ffaa8151660 0x7ffaa80c6c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:58370/0 (socket says 172.21.15.1:58370) 2024-05-09T08:07:40.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.145+0000 7ffaa7e00640 1 -- 172.21.15.1:0/855041423 learned_addr learned my addr 172.21.15.1:0/855041423 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:07:40.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.146+0000 7ffaa6a00640 1 -- 172.21.15.1:0/855041423 <== mon.1 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3353057582 0 0) 0x7ffaa8157620 con 0x7ffaa80b1d30 2024-05-09T08:07:40.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.146+0000 7ffaa6a00640 1 -- 172.21.15.1:0/855041423 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffa8c003520 con 0x7ffaa80b1d30 2024-05-09T08:07:40.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.146+0000 7ffaa7e00640 1 -- 172.21.15.1:0/855041423 >> v1:172.21.15.7:6789/0 conn(0x7ffaa80b1d30 legacy=0x7ffaa80c7a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:07:40.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.146+0000 7ffaa7e00640 1 -- 172.21.15.1:0/855041423 >> v1:172.21.15.1:6789/0 conn(0x7ffaa80c1ba0 legacy=0x7ffaa8072d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:07:40.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.146+0000 7ffaa7e00640 1 -- 172.21.15.1:0/855041423 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffaa8157900 con 0x7ffaa8151660 2024-05-09T08:07:40.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.147+0000 7ffaa7e00640 1 --2- 172.21.15.1:0/855041423 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffaa8151660 0x7ffaa80c6c90 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ffa9800bb10 tx=0x7ffa9802c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20d26d5caaca9734 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:07:40.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.148+0000 7ffaa6a00640 1 -- 172.21.15.1:0/855041423 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa98035070 con 0x7ffaa8151660 2024-05-09T08:07:40.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.148+0000 7ffaa6a00640 1 -- 172.21.15.1:0/855041423 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa98031610 con 0x7ffaa8151660 2024-05-09T08:07:40.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.148+0000 7ffaa6a00640 1 -- 172.21.15.1:0/855041423 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa980319c0 con 0x7ffaa8151660 2024-05-09T08:07:40.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.148+0000 7ffaaf000640 1 -- 172.21.15.1:0/855041423 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffaa8151660 msgr2=0x7ffaa80c6c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:07:40.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.148+0000 7ffaaf000640 1 --2- 172.21.15.1:0/855041423 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffaa8151660 0x7ffaa80c6c90 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ffa9800bb10 tx=0x7ffa9802c9c0 comp rx=0 tx=0).stop 2024-05-09T08:07:40.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.148+0000 7ffaaf000640 1 -- 172.21.15.1:0/855041423 shutdown_connections 2024-05-09T08:07:40.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.148+0000 7ffaaf000640 1 --2- 172.21.15.1:0/855041423 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffaa8151660 0x7ffaa80c6c90 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:40.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.148+0000 7ffaaf000640 1 -- 172.21.15.1:0/855041423 >> 172.21.15.1:0/855041423 conn(0x7ffaa80cc070 msgr2=0x7ffaa80cc480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:07:40.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.148+0000 7ffaaf000640 1 -- 172.21.15.1:0/855041423 shutdown_connections 2024-05-09T08:07:40.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.148+0000 7ffaaf000640 1 -- 172.21.15.1:0/855041423 wait complete. 2024-05-09T08:07:40.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.150+0000 7ffaaf000640 1 Processor -- start 2024-05-09T08:07:40.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.150+0000 7ffaaf000640 1 -- start start 2024-05-09T08:07:40.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.151+0000 7ffaaf000640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ffaa80b1d30 0x7ffaa8148500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:07:40.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.151+0000 7ffaaf000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffaa80c1ba0 0x7ffaa80cb910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:07:40.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.151+0000 7ffaaf000640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffaa8151660 0x7ffaa8072f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:07:40.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.151+0000 7ffaaf000640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ffaa81581a0 con 0x7ffaa8151660 2024-05-09T08:07:40.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.151+0000 7ffaaf000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ffaa8157ec0 con 0x7ffaa80c1ba0 2024-05-09T08:07:40.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.151+0000 7ffaaf000640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7ffaa81572d0 con 0x7ffaa80b1d30 2024-05-09T08:07:40.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.151+0000 7ffaad000640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffaa8151660 0x7ffaa8072f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:07:40.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.152+0000 7ffaad000640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffaa8151660 0x7ffaa8072f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:58374/0 (socket says 172.21.15.1:58374) 2024-05-09T08:07:40.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.152+0000 7ffaad000640 1 -- 172.21.15.1:0/344476513 learned_addr learned my addr 172.21.15.1:0/344476513 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:07:40.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.152+0000 7ffaa7e00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ffaa80b1d30 0x7ffaa8148500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:07:40.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.152+0000 7ffaa7400640 1 --2- 172.21.15.1:0/344476513 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffaa80c1ba0 0x7ffaa80cb910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:07:40.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.152+0000 7ffaad000640 1 -- 172.21.15.1:0/344476513 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ffaa80b1d30 msgr2=0x7ffaa8148500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:07:40.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.152+0000 7ffaad000640 1 --2- 172.21.15.1:0/344476513 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ffaa80b1d30 0x7ffaa8148500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:40.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.152+0000 7ffaad000640 1 -- 172.21.15.1:0/344476513 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffaa80c1ba0 msgr2=0x7ffaa80cb910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:07:40.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.152+0000 7ffaad000640 1 --2- 172.21.15.1:0/344476513 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffaa80c1ba0 0x7ffaa80cb910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:40.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.152+0000 7ffaad000640 1 -- 172.21.15.1:0/344476513 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffaa814b870 con 0x7ffaa8151660 2024-05-09T08:07:40.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.152+0000 7ffaad000640 1 --2- 172.21.15.1:0/344476513 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffaa8151660 0x7ffaa8072f40 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7ffaa000c790 tx=0x7ffaa000cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:07:40.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.153+0000 7ffa9fe00640 1 -- 172.21.15.1:0/344476513 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaa0003b70 con 0x7ffaa8151660 2024-05-09T08:07:40.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.153+0000 7ffa9fe00640 1 -- 172.21.15.1:0/344476513 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffaa0003d10 con 0x7ffaa8151660 2024-05-09T08:07:40.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.153+0000 7ffa9fe00640 1 -- 172.21.15.1:0/344476513 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaa000aa50 con 0x7ffaa8151660 2024-05-09T08:07:40.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.153+0000 7ffaaf000640 1 -- 172.21.15.1:0/344476513 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffaa814bb10 con 0x7ffaa8151660 2024-05-09T08:07:40.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.153+0000 7ffaaf000640 1 -- 172.21.15.1:0/344476513 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffaa8147780 con 0x7ffaa8151660 2024-05-09T08:07:40.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.154+0000 7ffaaf000640 1 -- 172.21.15.1:0/344476513 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffaa8147980 con 0x7ffaa8151660 2024-05-09T08:07:40.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.155+0000 7ffa9fe00640 1 -- 172.21.15.1:0/344476513 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ffaa000abf0 con 0x7ffaa8151660 2024-05-09T08:07:40.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.156+0000 7ffa9fe00640 1 --2- 172.21.15.1:0/344476513 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7ffa8806be30 0x7ffa8806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:07:40.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.156+0000 7ffa9fe00640 1 -- 172.21.15.1:0/344476513 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7ffaa008b120 con 0x7ffaa8151660 2024-05-09T08:07:40.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.156+0000 7ffaa7e00640 1 --2- 172.21.15.1:0/344476513 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7ffa8806be30 0x7ffa8806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:07:40.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.157+0000 7ffaa7e00640 1 --2- 172.21.15.1:0/344476513 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7ffa8806be30 0x7ffa8806e2f0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7ffa98037000 tx=0x7ffa98032000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:07:40.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.160+0000 7ffa9fe00640 1 -- 172.21.15.1:0/344476513 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ffaa8147980 con 0x7ffaa8151660 2024-05-09T08:07:40.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.435+0000 7ffaaf000640 1 -- 172.21.15.1:0/344476513 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7ffaa8074d70 con 0x7ffaa8151660 2024-05-09T08:07:40.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.436+0000 7ffa9fe00640 1 -- 172.21.15.1:0/344476513 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1544 (secure 0 0 0) 0x7ffaa0058ff0 con 0x7ffaa8151660 2024-05-09T08:07:40.447 INFO:teuthology.orchestra.run.smithi001.stdout:{"election_epoch":42,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":2,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"54329ec8-3270-4043-9a82-c6cd7420aa7f","modified":"2024-05-09T08:01:21.819270Z","created":"2024-05-09T08:01:21.819270Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:3300","nonce":0},{"type":"v1","addr":"172.21.15.1:6789","nonce":0}]},"addr":"172.21.15.1:6789/0","public_addr":"172.21.15.1:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:3300","nonce":0},{"type":"v1","addr":"172.21.15.7:6789","nonce":0}]},"addr":"172.21.15.7:6789/0","public_addr":"172.21.15.7:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:3301","nonce":0},{"type":"v1","addr":"172.21.15.7:6790","nonce":0}]},"addr":"172.21.15.7:6790/0","public_addr":"172.21.15.7:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-05-09T08:07:40.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.438+0000 7ffaaf000640 1 -- 172.21.15.1:0/344476513 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7ffa8806be30 msgr2=0x7ffa8806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:07:40.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.438+0000 7ffaaf000640 1 --2- 172.21.15.1:0/344476513 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7ffa8806be30 0x7ffa8806e2f0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7ffa98037000 tx=0x7ffa98032000 comp rx=0 tx=0).stop 2024-05-09T08:07:40.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.438+0000 7ffaaf000640 1 -- 172.21.15.1:0/344476513 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffaa8151660 msgr2=0x7ffaa8072f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:07:40.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.438+0000 7ffaaf000640 1 --2- 172.21.15.1:0/344476513 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffaa8151660 0x7ffaa8072f40 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7ffaa000c790 tx=0x7ffaa000cc50 comp rx=0 tx=0).stop 2024-05-09T08:07:40.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.439+0000 7ffaaf000640 1 -- 172.21.15.1:0/344476513 shutdown_connections 2024-05-09T08:07:40.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.439+0000 7ffaaf000640 1 --2- 172.21.15.1:0/344476513 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7ffa8806be30 0x7ffa8806e2f0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:40.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.439+0000 7ffaaf000640 1 --2- 172.21.15.1:0/344476513 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffaa80c1ba0 0x7ffaa80cb910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:40.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.439+0000 7ffaaf000640 1 --2- 172.21.15.1:0/344476513 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffaa8151660 0x7ffaa8072f40 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:40.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.439+0000 7ffaaf000640 1 --2- 172.21.15.1:0/344476513 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ffaa80b1d30 0x7ffaa8148500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:40.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.439+0000 7ffaaf000640 1 -- 172.21.15.1:0/344476513 >> 172.21.15.1:0/344476513 conn(0x7ffaa80cc070 msgr2=0x7ffaa80b1180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:07:40.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.439+0000 7ffaaf000640 1 -- 172.21.15.1:0/344476513 shutdown_connections 2024-05-09T08:07:40.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.439+0000 7ffaaf000640 1 -- 172.21.15.1:0/344476513 wait complete. 2024-05-09T08:07:40.469 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-05-09T08:07:40.469 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-05-09T08:07:40.469 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-05-09T08:07:40.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.666+0000 7f2be5e00640 1 Processor -- start 2024-05-09T08:07:40.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.666+0000 7f2be5e00640 1 -- start start 2024-05-09T08:07:40.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.668+0000 7f2be5e00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f2be00c54f0 0x7f2be006f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:07:40.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.668+0000 7f2be5e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f2be00c58d0 0x7f2be01589e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:07:40.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.668+0000 7f2be5e00640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f2be00c6470 0x7f2be0158f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:07:40.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.668+0000 7f2be5e00640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f2be00726e0 con 0x7f2be00c6470 2024-05-09T08:07:40.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.668+0000 7f2be5e00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f2be00718e0 con 0x7f2be00c58d0 2024-05-09T08:07:40.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.668+0000 7f2be5e00640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f2be0071b60 con 0x7f2be00c54f0 2024-05-09T08:07:40.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.668+0000 7f2be5400640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f2be00c6470 0x7f2be0158f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:07:40.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.668+0000 7f2be5400640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f2be00c6470 0x7f2be0158f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:58388/0 (socket says 172.21.15.1:58388) 2024-05-09T08:07:40.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.668+0000 7f2be5400640 1 -- 172.21.15.1:0/1507000669 learned_addr learned my addr 172.21.15.1:0/1507000669 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:07:40.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.668+0000 7f2be4a00640 1 --2- 172.21.15.1:0/1507000669 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f2be00c54f0 0x7f2be006f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:07:40.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.669+0000 7f2bdfe00640 1 --2- 172.21.15.1:0/1507000669 >> v2:172.21.15.7:3300/0 conn(0x7f2be00c58d0 0x7f2be01589e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:07:40.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.669+0000 7f2be5400640 1 -- 172.21.15.1:0/1507000669 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f2be00c54f0 msgr2=0x7f2be006f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:07:40.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.669+0000 7f2be5400640 1 --2- 172.21.15.1:0/1507000669 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f2be00c54f0 0x7f2be006f930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:40.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.669+0000 7f2be5400640 1 -- 172.21.15.1:0/1507000669 >> v2:172.21.15.7:3300/0 conn(0x7f2be00c58d0 msgr2=0x7f2be01589e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:07:40.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.669+0000 7f2be5400640 1 --2- 172.21.15.1:0/1507000669 >> v2:172.21.15.7:3300/0 conn(0x7f2be00c58d0 0x7f2be01589e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:40.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.669+0000 7f2be5400640 1 -- 172.21.15.1:0/1507000669 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2be0071e40 con 0x7f2be00c6470 2024-05-09T08:07:40.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.670+0000 7f2be5400640 1 --2- 172.21.15.1:0/1507000669 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2be00c6470 0x7f2be0158f20 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f2be00c9730 tx=0x7f2bd802a600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=565cd507f7da0b7f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:07:40.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.670+0000 7f2bdf400640 1 -- 172.21.15.1:0/1507000669 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bd8036070 con 0x7f2be00c6470 2024-05-09T08:07:40.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.670+0000 7f2bdf400640 1 -- 172.21.15.1:0/1507000669 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2bd8031600 con 0x7f2be00c6470 2024-05-09T08:07:40.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.670+0000 7f2bdf400640 1 -- 172.21.15.1:0/1507000669 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bd80319d0 con 0x7f2be00c6470 2024-05-09T08:07:40.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.671+0000 7f2be5e00640 1 -- 172.21.15.1:0/1507000669 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2be00c6470 msgr2=0x7f2be0158f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:07:40.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.671+0000 7f2be5e00640 1 --2- 172.21.15.1:0/1507000669 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2be00c6470 0x7f2be0158f20 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f2be00c9730 tx=0x7f2bd802a600 comp rx=0 tx=0).stop 2024-05-09T08:07:40.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.671+0000 7f2be5e00640 1 -- 172.21.15.1:0/1507000669 shutdown_connections 2024-05-09T08:07:40.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.671+0000 7f2be5e00640 1 --2- 172.21.15.1:0/1507000669 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2be00c6470 0x7f2be0158f20 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:40.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.671+0000 7f2be5e00640 1 --2- 172.21.15.1:0/1507000669 >> v2:172.21.15.7:3300/0 conn(0x7f2be00c58d0 0x7f2be01589e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:40.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.671+0000 7f2be5e00640 1 --2- 172.21.15.1:0/1507000669 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f2be00c54f0 0x7f2be006f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:40.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.671+0000 7f2be5e00640 1 -- 172.21.15.1:0/1507000669 >> 172.21.15.1:0/1507000669 conn(0x7f2be006b250 msgr2=0x7f2be006b630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:07:40.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.671+0000 7f2be5e00640 1 -- 172.21.15.1:0/1507000669 shutdown_connections 2024-05-09T08:07:40.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.671+0000 7f2be5e00640 1 -- 172.21.15.1:0/1507000669 wait complete. 2024-05-09T08:07:40.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.673+0000 7f2be5e00640 1 Processor -- start 2024-05-09T08:07:40.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.673+0000 7f2be5e00640 1 -- start start 2024-05-09T08:07:40.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.674+0000 7f2be5e00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f2be00c54f0 0x7f2be0154e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:07:40.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.674+0000 7f2be5e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2be00c58d0 0x7f2be0156420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:07:40.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.674+0000 7f2be5e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2be00c6470 0x7f2be0156960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:07:40.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.674+0000 7f2be5e00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f2be01584c0 con 0x7f2be00c58d0 2024-05-09T08:07:40.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.674+0000 7f2be5e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2be0072400 con 0x7f2be00c6470 2024-05-09T08:07:40.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.674+0000 7f2be5e00640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f2be006fe70 con 0x7f2be00c54f0 2024-05-09T08:07:40.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.674+0000 7f2bdfe00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2be00c58d0 0x7f2be0156420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:07:40.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.674+0000 7f2bdfe00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2be00c58d0 0x7f2be0156420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:58398/0 (socket says 172.21.15.1:58398) 2024-05-09T08:07:40.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.674+0000 7f2bdfe00640 1 -- 172.21.15.1:0/1152479270 learned_addr learned my addr 172.21.15.1:0/1152479270 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:07:40.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.674+0000 7f2be5400640 1 --2- 172.21.15.1:0/1152479270 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2be00c6470 0x7f2be0156960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:07:40.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.674+0000 7f2be4a00640 1 --2- 172.21.15.1:0/1152479270 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f2be00c54f0 0x7f2be0154e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:07:40.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.675+0000 7f2bdfe00640 1 -- 172.21.15.1:0/1152479270 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f2be00c54f0 msgr2=0x7f2be0154e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:07:40.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.675+0000 7f2bdfe00640 1 --2- 172.21.15.1:0/1152479270 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f2be00c54f0 0x7f2be0154e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:40.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.675+0000 7f2bdfe00640 1 -- 172.21.15.1:0/1152479270 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2be00c6470 msgr2=0x7f2be0156960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:07:40.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.675+0000 7f2bdfe00640 1 --2- 172.21.15.1:0/1152479270 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2be00c6470 0x7f2be0156960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:40.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.675+0000 7f2bdfe00640 1 -- 172.21.15.1:0/1152479270 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2be00cc8e0 con 0x7f2be00c58d0 2024-05-09T08:07:40.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.675+0000 7f2bdfe00640 1 --2- 172.21.15.1:0/1152479270 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2be00c58d0 0x7f2be0156420 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f2bcc00d830 tx=0x7f2bcc00dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:07:40.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.675+0000 7f2bdcc00640 1 -- 172.21.15.1:0/1152479270 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bcc00b6c0 con 0x7f2be00c58d0 2024-05-09T08:07:40.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.675+0000 7f2bdcc00640 1 -- 172.21.15.1:0/1152479270 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2bcc019070 con 0x7f2be00c58d0 2024-05-09T08:07:40.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.675+0000 7f2be5e00640 1 -- 172.21.15.1:0/1152479270 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2be01ebf90 con 0x7f2be00c58d0 2024-05-09T08:07:40.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.675+0000 7f2bdcc00640 1 -- 172.21.15.1:0/1152479270 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bcc00ba80 con 0x7f2be00c58d0 2024-05-09T08:07:40.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.676+0000 7f2be5e00640 1 -- 172.21.15.1:0/1152479270 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2be01ec550 con 0x7f2be00c58d0 2024-05-09T08:07:40.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.677+0000 7f2be5e00640 1 --2- 172.21.15.1:0/1152479270 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2be01ec750 0x7f2be00bf530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:07:40.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.677+0000 7f2be5e00640 1 -- 172.21.15.1:0/1152479270 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f2be01544f0 con 0x7f2be01ec750 2024-05-09T08:07:40.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.677+0000 7f2be4a00640 1 --2- 172.21.15.1:0/1152479270 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2be01ec750 0x7f2be00bf530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:07:40.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.677+0000 7f2be4a00640 1 -- 172.21.15.1:0/1152479270 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2be0156140 con 0x7f2be01ec750 2024-05-09T08:07:40.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.678+0000 7f2bdcc00640 1 -- 172.21.15.1:0/1152479270 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2bcc01e460 con 0x7f2be00c58d0 2024-05-09T08:07:40.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.678+0000 7f2be4a00640 1 --2- 172.21.15.1:0/1152479270 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2be01ec750 0x7f2be00bf530 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f2bd400b750 tx=0x7f2bd400bc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:07:40.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.679+0000 7f2bdcc00640 1 --2- 172.21.15.1:0/1152479270 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f2bc006be30 0x7f2bc006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:07:40.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.679+0000 7f2bdcc00640 1 -- 172.21.15.1:0/1152479270 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7f2bcc08b7d0 con 0x7f2be00c58d0 2024-05-09T08:07:40.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.679+0000 7f2bdcc00640 1 -- 172.21.15.1:0/1152479270 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bd4018070 con 0x7f2be01ec750 2024-05-09T08:07:40.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.679+0000 7f2be5400640 1 --2- 172.21.15.1:0/1152479270 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f2bc006be30 0x7f2bc006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:07:40.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.679+0000 7f2bdcc00640 1 -- 172.21.15.1:0/1152479270 <== mon.0 v2:172.21.15.1:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f2bd400be00 con 0x7f2be01ec750 2024-05-09T08:07:40.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.679+0000 7f2bdcc00640 1 -- 172.21.15.1:0/1152479270 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2be01ec750 msgr2=0x7f2be00bf530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:07:40.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.679+0000 7f2bdcc00640 1 --2- 172.21.15.1:0/1152479270 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2be01ec750 0x7f2be00bf530 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f2bd400b750 tx=0x7f2bd400bc10 comp rx=0 tx=0).stop 2024-05-09T08:07:40.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.679+0000 7f2bdcc00640 1 -- 172.21.15.1:0/1152479270 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2be01ec750 msgr2=0x7f2be00bf530 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-05-09T08:07:40.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.679+0000 7f2bdcc00640 1 --2- 172.21.15.1:0/1152479270 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2be01ec750 0x7f2be00bf530 secure :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f2bd400b750 tx=0x7f2bd400bc10 comp rx=0 tx=0).stop 2024-05-09T08:07:40.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.680+0000 7f2be5400640 1 --2- 172.21.15.1:0/1152479270 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f2bc006be30 0x7f2bc006e2f0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f2bd8033000 tx=0x7f2bd8009000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:07:40.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.686+0000 7f2be5e00640 1 --2- 172.21.15.1:0/1152479270 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2be00cc410 0x7f2be0144c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:07:40.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.686+0000 7f2be5e00640 1 -- 172.21.15.1:0/1152479270 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f2be00c6260 con 0x7f2be00cc410 2024-05-09T08:07:40.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.687+0000 7f2be4a00640 1 --2- 172.21.15.1:0/1152479270 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2be00cc410 0x7f2be0144c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:07:40.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.687+0000 7f2be4a00640 1 -- 172.21.15.1:0/1152479270 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f2be006f680 con 0x7f2be00cc410 2024-05-09T08:07:40.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.687+0000 7f2be4a00640 1 --2- 172.21.15.1:0/1152479270 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2be00cc410 0x7f2be0144c90 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f2bd40106c0 tx=0x7f2bd4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:07:40.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.688+0000 7f2bdcc00640 1 -- 172.21.15.1:0/1152479270 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bd401de00 con 0x7f2be00cc410 2024-05-09T08:07:40.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.688+0000 7f2bdcc00640 1 -- 172.21.15.1:0/1152479270 <== mon.0 v2:172.21.15.1:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4877 (secure 0 0 0) 0x7f2bd4014430 con 0x7f2be00cc410 2024-05-09T08:07:40.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.688+0000 7f2bdcc00640 1 -- 172.21.15.1:0/1152479270 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2be00cc410 msgr2=0x7f2be0144c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:07:40.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.688+0000 7f2bdcc00640 1 --2- 172.21.15.1:0/1152479270 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2be00cc410 0x7f2be0144c90 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f2bd40106c0 tx=0x7f2bd4004730 comp rx=0 tx=0).stop 2024-05-09T08:07:40.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.688+0000 7f2bdcc00640 1 -- 172.21.15.1:0/1152479270 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2be00cc410 msgr2=0x7f2be0144c90 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-05-09T08:07:40.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.688+0000 7f2bdcc00640 1 --2- 172.21.15.1:0/1152479270 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2be00cc410 0x7f2be0144c90 secure :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f2bd40106c0 tx=0x7f2bd4004730 comp rx=0 tx=0).stop 2024-05-09T08:07:40.699 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-05-09T08:07:40.699 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "a", 2024-05-09T08:07:40.699 INFO:teuthology.orchestra.run.smithi001.stdout: "rank": 0, 2024-05-09T08:07:40.699 INFO:teuthology.orchestra.run.smithi001.stdout: "state": "leader", 2024-05-09T08:07:40.699 INFO:teuthology.orchestra.run.smithi001.stdout: "election_epoch": 42, 2024-05-09T08:07:40.699 INFO:teuthology.orchestra.run.smithi001.stdout: "uptime": 123305, 2024-05-09T08:07:40.700 INFO:teuthology.orchestra.run.smithi001.stdout: "quorum": [ 2024-05-09T08:07:40.700 INFO:teuthology.orchestra.run.smithi001.stdout: 0, 2024-05-09T08:07:40.700 INFO:teuthology.orchestra.run.smithi001.stdout: 1, 2024-05-09T08:07:40.700 INFO:teuthology.orchestra.run.smithi001.stdout: 2 2024-05-09T08:07:40.700 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:07:40.700 INFO:teuthology.orchestra.run.smithi001.stdout: "quorum_age": 3, 2024-05-09T08:07:40.700 INFO:teuthology.orchestra.run.smithi001.stdout: "features": { 2024-05-09T08:07:40.700 INFO:teuthology.orchestra.run.smithi001.stdout: "required_con": "2450521708007866372", 2024-05-09T08:07:40.700 INFO:teuthology.orchestra.run.smithi001.stdout: "required_mon": [ 2024-05-09T08:07:40.700 INFO:teuthology.orchestra.run.smithi001.stdout: "kraken", 2024-05-09T08:07:40.700 INFO:teuthology.orchestra.run.smithi001.stdout: "luminous", 2024-05-09T08:07:40.700 INFO:teuthology.orchestra.run.smithi001.stdout: "mimic", 2024-05-09T08:07:40.700 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap-prune", 2024-05-09T08:07:40.700 INFO:teuthology.orchestra.run.smithi001.stdout: "nautilus", 2024-05-09T08:07:40.701 INFO:teuthology.orchestra.run.smithi001.stdout: "octopus", 2024-05-09T08:07:40.701 INFO:teuthology.orchestra.run.smithi001.stdout: "pacific", 2024-05-09T08:07:40.701 INFO:teuthology.orchestra.run.smithi001.stdout: "elector-pinging", 2024-05-09T08:07:40.701 INFO:teuthology.orchestra.run.smithi001.stdout: "quincy", 2024-05-09T08:07:40.701 INFO:teuthology.orchestra.run.smithi001.stdout: "reef", 2024-05-09T08:07:40.701 INFO:teuthology.orchestra.run.smithi001.stdout: "squid" 2024-05-09T08:07:40.701 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:07:40.701 INFO:teuthology.orchestra.run.smithi001.stdout: "quorum_con": "4540701547738038271", 2024-05-09T08:07:40.701 INFO:teuthology.orchestra.run.smithi001.stdout: "quorum_mon": [ 2024-05-09T08:07:40.701 INFO:teuthology.orchestra.run.smithi001.stdout: "kraken", 2024-05-09T08:07:40.701 INFO:teuthology.orchestra.run.smithi001.stdout: "luminous", 2024-05-09T08:07:40.701 INFO:teuthology.orchestra.run.smithi001.stdout: "mimic", 2024-05-09T08:07:40.701 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap-prune", 2024-05-09T08:07:40.701 INFO:teuthology.orchestra.run.smithi001.stdout: "nautilus", 2024-05-09T08:07:40.702 INFO:teuthology.orchestra.run.smithi001.stdout: "octopus", 2024-05-09T08:07:40.702 INFO:teuthology.orchestra.run.smithi001.stdout: "pacific", 2024-05-09T08:07:40.702 INFO:teuthology.orchestra.run.smithi001.stdout: "elector-pinging", 2024-05-09T08:07:40.702 INFO:teuthology.orchestra.run.smithi001.stdout: "quincy", 2024-05-09T08:07:40.702 INFO:teuthology.orchestra.run.smithi001.stdout: "reef", 2024-05-09T08:07:40.702 INFO:teuthology.orchestra.run.smithi001.stdout: "squid" 2024-05-09T08:07:40.702 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:07:40.702 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:07:40.702 INFO:teuthology.orchestra.run.smithi001.stdout: "outside_quorum": [], 2024-05-09T08:07:40.702 INFO:teuthology.orchestra.run.smithi001.stdout: "extra_probe_peers": [], 2024-05-09T08:07:40.702 INFO:teuthology.orchestra.run.smithi001.stdout: "sync_provider": [], 2024-05-09T08:07:40.702 INFO:teuthology.orchestra.run.smithi001.stdout: "monmap": { 2024-05-09T08:07:40.702 INFO:teuthology.orchestra.run.smithi001.stdout: "epoch": 1, 2024-05-09T08:07:40.703 INFO:teuthology.orchestra.run.smithi001.stdout: "fsid": "54329ec8-3270-4043-9a82-c6cd7420aa7f", 2024-05-09T08:07:40.703 INFO:teuthology.orchestra.run.smithi001.stdout: "modified": "2024-05-09T08:01:21.819270Z", 2024-05-09T08:07:40.703 INFO:teuthology.orchestra.run.smithi001.stdout: "created": "2024-05-09T08:01:21.819270Z", 2024-05-09T08:07:40.703 INFO:teuthology.orchestra.run.smithi001.stdout: "min_mon_release": 19, 2024-05-09T08:07:40.703 INFO:teuthology.orchestra.run.smithi001.stdout: "min_mon_release_name": "squid", 2024-05-09T08:07:40.703 INFO:teuthology.orchestra.run.smithi001.stdout: "election_strategy": 1, 2024-05-09T08:07:40.703 INFO:teuthology.orchestra.run.smithi001.stdout: "disallowed_leaders: ": "", 2024-05-09T08:07:40.703 INFO:teuthology.orchestra.run.smithi001.stdout: "stretch_mode": false, 2024-05-09T08:07:40.703 INFO:teuthology.orchestra.run.smithi001.stdout: "tiebreaker_mon": "", 2024-05-09T08:07:40.703 INFO:teuthology.orchestra.run.smithi001.stdout: "removed_ranks: ": "", 2024-05-09T08:07:40.703 INFO:teuthology.orchestra.run.smithi001.stdout: "features": { 2024-05-09T08:07:40.703 INFO:teuthology.orchestra.run.smithi001.stdout: "persistent": [ 2024-05-09T08:07:40.703 INFO:teuthology.orchestra.run.smithi001.stdout: "kraken", 2024-05-09T08:07:40.704 INFO:teuthology.orchestra.run.smithi001.stdout: "luminous", 2024-05-09T08:07:40.704 INFO:teuthology.orchestra.run.smithi001.stdout: "mimic", 2024-05-09T08:07:40.704 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap-prune", 2024-05-09T08:07:40.704 INFO:teuthology.orchestra.run.smithi001.stdout: "nautilus", 2024-05-09T08:07:40.704 INFO:teuthology.orchestra.run.smithi001.stdout: "octopus", 2024-05-09T08:07:40.704 INFO:teuthology.orchestra.run.smithi001.stdout: "pacific", 2024-05-09T08:07:40.704 INFO:teuthology.orchestra.run.smithi001.stdout: "elector-pinging", 2024-05-09T08:07:40.704 INFO:teuthology.orchestra.run.smithi001.stdout: "quincy", 2024-05-09T08:07:40.704 INFO:teuthology.orchestra.run.smithi001.stdout: "reef", 2024-05-09T08:07:40.704 INFO:teuthology.orchestra.run.smithi001.stdout: "squid" 2024-05-09T08:07:40.704 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:07:40.704 INFO:teuthology.orchestra.run.smithi001.stdout: "optional": [] 2024-05-09T08:07:40.704 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:07:40.704 INFO:teuthology.orchestra.run.smithi001.stdout: "mons": [ 2024-05-09T08:07:40.705 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:07:40.705 INFO:teuthology.orchestra.run.smithi001.stdout: "rank": 0, 2024-05-09T08:07:40.705 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "a", 2024-05-09T08:07:40.705 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-05-09T08:07:40.705 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-05-09T08:07:40.705 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:07:40.705 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-05-09T08:07:40.705 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:3300", 2024-05-09T08:07:40.705 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:07:40.705 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:07:40.705 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:07:40.705 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-05-09T08:07:40.706 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6789", 2024-05-09T08:07:40.706 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:07:40.706 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:07:40.706 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:07:40.706 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:07:40.706 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6789/0", 2024-05-09T08:07:40.706 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.1:6789/0", 2024-05-09T08:07:40.706 INFO:teuthology.orchestra.run.smithi001.stdout: "priority": 0, 2024-05-09T08:07:40.706 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 0, 2024-05-09T08:07:40.706 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_location": "{}" 2024-05-09T08:07:40.706 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:07:40.707 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:07:40.707 INFO:teuthology.orchestra.run.smithi001.stdout: "rank": 1, 2024-05-09T08:07:40.707 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "b", 2024-05-09T08:07:40.707 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-05-09T08:07:40.707 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-05-09T08:07:40.707 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:07:40.707 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-05-09T08:07:40.707 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:3300", 2024-05-09T08:07:40.707 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:07:40.707 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:07:40.707 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:07:40.707 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-05-09T08:07:40.707 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:6789", 2024-05-09T08:07:40.708 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:07:40.708 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:07:40.708 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:07:40.708 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:07:40.708 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:6789/0", 2024-05-09T08:07:40.708 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.7:6789/0", 2024-05-09T08:07:40.708 INFO:teuthology.orchestra.run.smithi001.stdout: "priority": 0, 2024-05-09T08:07:40.708 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 0, 2024-05-09T08:07:40.708 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_location": "{}" 2024-05-09T08:07:40.708 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:07:40.708 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:07:40.708 INFO:teuthology.orchestra.run.smithi001.stdout: "rank": 2, 2024-05-09T08:07:40.708 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "c", 2024-05-09T08:07:40.708 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-05-09T08:07:40.709 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-05-09T08:07:40.709 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:07:40.709 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-05-09T08:07:40.709 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:3301", 2024-05-09T08:07:40.709 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:07:40.709 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:07:40.709 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:07:40.709 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-05-09T08:07:40.709 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:6790", 2024-05-09T08:07:40.709 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:07:40.709 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:07:40.709 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:07:40.709 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:07:40.710 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:6790/0", 2024-05-09T08:07:40.710 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.7:6790/0", 2024-05-09T08:07:40.710 INFO:teuthology.orchestra.run.smithi001.stdout: "priority": 0, 2024-05-09T08:07:40.710 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 0, 2024-05-09T08:07:40.710 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_location": "{}" 2024-05-09T08:07:40.710 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:07:40.710 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:07:40.710 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:07:40.710 INFO:teuthology.orchestra.run.smithi001.stdout: "feature_map": { 2024-05-09T08:07:40.710 INFO:teuthology.orchestra.run.smithi001.stdout: "mon": [ 2024-05-09T08:07:40.710 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:07:40.710 INFO:teuthology.orchestra.run.smithi001.stdout: "features": "0x3f03cffffffdffff", 2024-05-09T08:07:40.710 INFO:teuthology.orchestra.run.smithi001.stdout: "release": "luminous", 2024-05-09T08:07:40.711 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 1 2024-05-09T08:07:40.711 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:07:40.711 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:07:40.711 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": [ 2024-05-09T08:07:40.711 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:07:40.711 INFO:teuthology.orchestra.run.smithi001.stdout: "features": "0x3f03cffffffdffff", 2024-05-09T08:07:40.711 INFO:teuthology.orchestra.run.smithi001.stdout: "release": "luminous", 2024-05-09T08:07:40.711 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 1 2024-05-09T08:07:40.711 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:07:40.711 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:07:40.711 INFO:teuthology.orchestra.run.smithi001.stdout: "client": [ 2024-05-09T08:07:40.711 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:07:40.711 INFO:teuthology.orchestra.run.smithi001.stdout: "features": "0x3f03cffffffdffff", 2024-05-09T08:07:40.712 INFO:teuthology.orchestra.run.smithi001.stdout: "release": "luminous", 2024-05-09T08:07:40.712 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 2 2024-05-09T08:07:40.712 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:07:40.712 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:07:40.712 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr": [ 2024-05-09T08:07:40.712 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:07:40.712 INFO:teuthology.orchestra.run.smithi001.stdout: "features": "0x3f03cffffffdffff", 2024-05-09T08:07:40.712 INFO:teuthology.orchestra.run.smithi001.stdout: "release": "luminous", 2024-05-09T08:07:40.712 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 1 2024-05-09T08:07:40.712 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:07:40.712 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:07:40.712 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:07:40.712 INFO:teuthology.orchestra.run.smithi001.stdout: "stretch_mode": false 2024-05-09T08:07:40.713 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-05-09T08:07:40.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.688+0000 7f2be5e00640 1 -- 172.21.15.1:0/1152479270 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f2bc006be30 msgr2=0x7f2bc006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:07:40.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.688+0000 7f2be5e00640 1 --2- 172.21.15.1:0/1152479270 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f2bc006be30 0x7f2bc006e2f0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f2bd8033000 tx=0x7f2bd8009000 comp rx=0 tx=0).stop 2024-05-09T08:07:40.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.688+0000 7f2be5e00640 1 -- 172.21.15.1:0/1152479270 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2be00c58d0 msgr2=0x7f2be0156420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:07:40.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.688+0000 7f2be5e00640 1 --2- 172.21.15.1:0/1152479270 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2be00c58d0 0x7f2be0156420 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f2bcc00d830 tx=0x7f2bcc00dcf0 comp rx=0 tx=0).stop 2024-05-09T08:07:40.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.688+0000 7f2be5400640 1 -- 172.21.15.1:0/1152479270 reap_dead start 2024-05-09T08:07:40.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.688+0000 7f2be5400640 1 -- 172.21.15.1:0/1152479270 reap_dead start 2024-05-09T08:07:40.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.688+0000 7f2be5e00640 1 -- 172.21.15.1:0/1152479270 shutdown_connections 2024-05-09T08:07:40.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.688+0000 7f2be5e00640 1 -- 172.21.15.1:0/1152479270 >> 172.21.15.1:0/1152479270 conn(0x7f2be006b250 msgr2=0x7f2be006b630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:07:40.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.688+0000 7f2be5e00640 1 -- 172.21.15.1:0/1152479270 shutdown_connections 2024-05-09T08:07:40.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.689+0000 7f2be5e00640 1 -- 172.21.15.1:0/1152479270 wait complete. 2024-05-09T08:07:40.717 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-05-09T08:07:40.717 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-05-09T08:07:40.717 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-05-09T08:07:40.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.918+0000 7f03d2200640 1 Processor -- start 2024-05-09T08:07:40.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.918+0000 7f03d2200640 1 -- start start 2024-05-09T08:07:40.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.919+0000 7f03d2200640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f03cc1546f0 0x7f03cc0ca160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:07:40.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.919+0000 7f03d2200640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f03cc15ae00 con 0x7f03cc0c9620 2024-05-09T08:07:40.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.919+0000 7f03d2200640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f03cc15b0e0 con 0x7f03cc15a270 2024-05-09T08:07:40.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.919+0000 7f03d2200640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f03cc15db20 con 0x7f03cc1546f0 2024-05-09T08:07:40.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.920+0000 7f03d0e00640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f03cc1546f0 0x7f03cc0ca160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:07:40.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.920+0000 7f03d1800640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f03cc15a270 0x7f03cc164bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.1:47820/0 (socket says 172.21.15.1:47820) 2024-05-09T08:07:40.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.920+0000 7f03d1800640 1 -- 172.21.15.1:0/308183388 learned_addr learned my addr 172.21.15.1:0/308183388 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T08:07:40.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.920+0000 7f03cb400640 1 -- 172.21.15.1:0/308183388 <== mon.1 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1786257901 0 0) 0x7f03cc15b0e0 con 0x7f03cc15a270 2024-05-09T08:07:40.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.920+0000 7f03d0e00640 1 -- 172.21.15.1:0/308183388 >> v1:172.21.15.7:6789/0 conn(0x7f03cc15a270 legacy=0x7f03cc164bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:07:40.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.920+0000 7f03d0e00640 1 -- 172.21.15.1:0/308183388 >> v1:172.21.15.1:6789/0 conn(0x7f03cc0c9620 legacy=0x7f03cc0c8850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:07:40.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.920+0000 7f03d0e00640 1 -- 172.21.15.1:0/308183388 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03cc15de00 con 0x7f03cc1546f0 2024-05-09T08:07:40.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.921+0000 7f03d0e00640 1 --2- 172.21.15.1:0/308183388 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f03cc1546f0 0x7f03cc0ca160 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f03c00097a0 tx=0x7f03c002a2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f544edc9ab9ee81d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:07:40.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.922+0000 7f03cb400640 1 -- 172.21.15.1:0/308183388 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03c0037070 con 0x7f03cc1546f0 2024-05-09T08:07:40.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.922+0000 7f03cb400640 1 -- 172.21.15.1:0/308183388 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03c0032640 con 0x7f03cc1546f0 2024-05-09T08:07:40.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.922+0000 7f03cb400640 1 -- 172.21.15.1:0/308183388 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03c0032a10 con 0x7f03cc1546f0 2024-05-09T08:07:40.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.922+0000 7f03d2200640 1 -- 172.21.15.1:0/308183388 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f03cc1546f0 msgr2=0x7f03cc0ca160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:07:40.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.922+0000 7f03d2200640 1 --2- 172.21.15.1:0/308183388 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f03cc1546f0 0x7f03cc0ca160 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f03c00097a0 tx=0x7f03c002a2c0 comp rx=0 tx=0).stop 2024-05-09T08:07:40.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.922+0000 7f03d2200640 1 -- 172.21.15.1:0/308183388 shutdown_connections 2024-05-09T08:07:40.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.922+0000 7f03d2200640 1 --2- 172.21.15.1:0/308183388 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f03cc1546f0 0x7f03cc0ca160 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:40.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.922+0000 7f03d2200640 1 -- 172.21.15.1:0/308183388 >> 172.21.15.1:0/308183388 conn(0x7f03cc071620 msgr2=0x7f03cc071a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:07:40.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.922+0000 7f03d2200640 1 -- 172.21.15.1:0/308183388 shutdown_connections 2024-05-09T08:07:40.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.922+0000 7f03d2200640 1 -- 172.21.15.1:0/308183388 wait complete. 2024-05-09T08:07:40.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.924+0000 7f03d2200640 1 Processor -- start 2024-05-09T08:07:40.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.924+0000 7f03d2200640 1 -- start start 2024-05-09T08:07:40.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.925+0000 7f03d2200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03cc0c9620 0x7f03cc152f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:07:40.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.925+0000 7f03d2200640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f03cc1546f0 0x7f03cc06c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:07:40.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.925+0000 7f03d2200640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f03cc15a270 0x7f03cc06cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:07:40.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.925+0000 7f03d2200640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f03cc15e600 con 0x7f03cc1546f0 2024-05-09T08:07:40.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.925+0000 7f03d2200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f03cc15d7f0 con 0x7f03cc0c9620 2024-05-09T08:07:40.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.925+0000 7f03d2200640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f03cc153490 con 0x7f03cc15a270 2024-05-09T08:07:40.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.925+0000 7f03cbe00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f03cc1546f0 0x7f03cc06c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:07:40.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.926+0000 7f03d0e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03cc0c9620 0x7f03cc152f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:07:40.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.926+0000 7f03d1800640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f03cc15a270 0x7f03cc06cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:07:40.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.926+0000 7f03cbe00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f03cc1546f0 0x7f03cc06c6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:58442/0 (socket says 172.21.15.1:58442) 2024-05-09T08:07:40.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.926+0000 7f03cbe00640 1 -- 172.21.15.1:0/3594213953 learned_addr learned my addr 172.21.15.1:0/3594213953 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:07:40.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.926+0000 7f03d0e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03cc0c9620 0x7f03cc152f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.1:53908/0 (socket says 172.21.15.1:53908) 2024-05-09T08:07:40.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.926+0000 7f03d0e00640 1 -- 172.21.15.1:0/3594213953 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f03cc15a270 msgr2=0x7f03cc06cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:07:40.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.926+0000 7f03d0e00640 1 --2- 172.21.15.1:0/3594213953 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f03cc15a270 0x7f03cc06cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:40.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.926+0000 7f03d0e00640 1 -- 172.21.15.1:0/3594213953 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f03cc1546f0 msgr2=0x7f03cc06c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:07:40.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.926+0000 7f03d0e00640 1 --2- 172.21.15.1:0/3594213953 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f03cc1546f0 0x7f03cc06c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:40.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.926+0000 7f03d0e00640 1 -- 172.21.15.1:0/3594213953 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03cc1e6110 con 0x7f03cc0c9620 2024-05-09T08:07:40.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.926+0000 7f03cbe00640 1 --2- 172.21.15.1:0/3594213953 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f03cc1546f0 0x7f03cc06c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:07:40.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.926+0000 7f03d0e00640 1 --2- 172.21.15.1:0/3594213953 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03cc0c9620 0x7f03cc152f50 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f03cc072200 tx=0x7f03c00046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:07:40.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.927+0000 7f03c8c00640 1 -- 172.21.15.1:0/3594213953 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03c0037040 con 0x7f03cc0c9620 2024-05-09T08:07:40.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.927+0000 7f03c8c00640 1 -- 172.21.15.1:0/3594213953 <== mon.1 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03c0042070 con 0x7f03cc0c9620 2024-05-09T08:07:40.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.927+0000 7f03d2200640 1 -- 172.21.15.1:0/3594213953 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03cc1f0a80 con 0x7f03cc0c9620 2024-05-09T08:07:40.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.927+0000 7f03c8c00640 1 -- 172.21.15.1:0/3594213953 <== mon.1 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03c0030bb0 con 0x7f03cc0c9620 2024-05-09T08:07:40.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.927+0000 7f03d2200640 1 -- 172.21.15.1:0/3594213953 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03cc1e5b50 con 0x7f03cc0c9620 2024-05-09T08:07:40.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.928+0000 7f03d2200640 1 --2- 172.21.15.1:0/3594213953 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03cc1580c0 0x7f03cc156470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:07:40.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.928+0000 7f03d2200640 1 -- 172.21.15.1:0/3594213953 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f03cc04f660 con 0x7f03cc1580c0 2024-05-09T08:07:40.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.928+0000 7f03cbe00640 1 --2- 172.21.15.1:0/3594213953 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03cc1580c0 0x7f03cc156470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:07:40.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.929+0000 7f03cbe00640 1 -- 172.21.15.1:0/3594213953 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f03cc15e780 con 0x7f03cc1580c0 2024-05-09T08:07:40.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.929+0000 7f03cbe00640 1 --2- 172.21.15.1:0/3594213953 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03cc1580c0 0x7f03cc156470 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f03cc15c980 tx=0x7f03b800fec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:07:40.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.929+0000 7f03c8c00640 1 -- 172.21.15.1:0/3594213953 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03b80052e0 con 0x7f03cc1580c0 2024-05-09T08:07:40.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.929+0000 7f03c8c00640 1 -- 172.21.15.1:0/3594213953 <== mon.1 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f03c0047460 con 0x7f03cc0c9620 2024-05-09T08:07:40.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.930+0000 7f03c8c00640 1 --2- 172.21.15.1:0/3594213953 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f03ac06bdb0 0x7f03ac06e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:07:40.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.931+0000 7f03c8c00640 1 -- 172.21.15.1:0/3594213953 <== mon.1 v2:172.21.15.7:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7f03c00aba00 con 0x7f03cc0c9620 2024-05-09T08:07:40.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.931+0000 7f03d1800640 1 --2- 172.21.15.1:0/3594213953 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f03ac06bdb0 0x7f03ac06e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:07:40.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.931+0000 7f03c8c00640 1 -- 172.21.15.1:0/3594213953 <== mon.1 v2:172.21.15.7:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f03b8018070 con 0x7f03cc1580c0 2024-05-09T08:07:40.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.931+0000 7f03c8c00640 1 -- 172.21.15.1:0/3594213953 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03cc1580c0 msgr2=0x7f03cc156470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:07:40.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.931+0000 7f03c8c00640 1 --2- 172.21.15.1:0/3594213953 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03cc1580c0 0x7f03cc156470 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f03cc15c980 tx=0x7f03b800fec0 comp rx=0 tx=0).stop 2024-05-09T08:07:40.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.931+0000 7f03c8c00640 1 -- 172.21.15.1:0/3594213953 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03cc1580c0 msgr2=0x7f03cc156470 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-09T08:07:40.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.931+0000 7f03c8c00640 1 --2- 172.21.15.1:0/3594213953 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03cc1580c0 0x7f03cc156470 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:40.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.931+0000 7f03d1800640 1 --2- 172.21.15.1:0/3594213953 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f03ac06bdb0 0x7f03ac06e270 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f03cc155c40 tx=0x7f03c4004d40 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:07:40.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.938+0000 7f03d2200640 1 --2- 172.21.15.1:0/3594213953 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03cc155eb0 0x7f03cc1569b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:07:40.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.938+0000 7f03d2200640 1 -- 172.21.15.1:0/3594213953 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f03cc0c9400 con 0x7f03cc155eb0 2024-05-09T08:07:40.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.939+0000 7f03cbe00640 1 --2- 172.21.15.1:0/3594213953 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03cc155eb0 0x7f03cc1569b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:07:40.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.939+0000 7f03cbe00640 1 -- 172.21.15.1:0/3594213953 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f03cc0ca290 con 0x7f03cc155eb0 2024-05-09T08:07:40.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.939+0000 7f03cbe00640 1 --2- 172.21.15.1:0/3594213953 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03cc155eb0 0x7f03cc1569b0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f03b8014000 tx=0x7f03b8001120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:07:40.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.939+0000 7f03c8c00640 1 -- 172.21.15.1:0/3594213953 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03b8007030 con 0x7f03cc155eb0 2024-05-09T08:07:40.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.940+0000 7f03c8c00640 1 -- 172.21.15.1:0/3594213953 <== mon.1 v2:172.21.15.7:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5188 (secure 0 0 0) 0x7f03b8007030 con 0x7f03cc155eb0 2024-05-09T08:07:40.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.940+0000 7f03c8c00640 1 -- 172.21.15.1:0/3594213953 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03cc155eb0 msgr2=0x7f03cc1569b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:07:40.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.940+0000 7f03c8c00640 1 --2- 172.21.15.1:0/3594213953 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03cc155eb0 0x7f03cc1569b0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f03b8014000 tx=0x7f03b8001120 comp rx=0 tx=0).stop 2024-05-09T08:07:40.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.940+0000 7f03c8c00640 1 -- 172.21.15.1:0/3594213953 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03cc155eb0 msgr2=0x7f03cc1569b0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-05-09T08:07:40.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.940+0000 7f03c8c00640 1 --2- 172.21.15.1:0/3594213953 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03cc155eb0 0x7f03cc1569b0 secure :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f03b8014000 tx=0x7f03b8001120 comp rx=0 tx=0).stop 2024-05-09T08:07:40.951 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-05-09T08:07:40.952 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "b", 2024-05-09T08:07:40.952 INFO:teuthology.orchestra.run.smithi001.stdout: "rank": 1, 2024-05-09T08:07:40.952 INFO:teuthology.orchestra.run.smithi001.stdout: "state": "peon", 2024-05-09T08:07:40.952 INFO:teuthology.orchestra.run.smithi001.stdout: "election_epoch": 42, 2024-05-09T08:07:40.952 INFO:teuthology.orchestra.run.smithi001.stdout: "uptime": 228601, 2024-05-09T08:07:40.952 INFO:teuthology.orchestra.run.smithi001.stdout: "quorum": [ 2024-05-09T08:07:40.952 INFO:teuthology.orchestra.run.smithi001.stdout: 0, 2024-05-09T08:07:40.952 INFO:teuthology.orchestra.run.smithi001.stdout: 1, 2024-05-09T08:07:40.952 INFO:teuthology.orchestra.run.smithi001.stdout: 2 2024-05-09T08:07:40.952 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:07:40.952 INFO:teuthology.orchestra.run.smithi001.stdout: "quorum_age": 3, 2024-05-09T08:07:40.952 INFO:teuthology.orchestra.run.smithi001.stdout: "features": { 2024-05-09T08:07:40.952 INFO:teuthology.orchestra.run.smithi001.stdout: "required_con": "2450521708007866372", 2024-05-09T08:07:40.953 INFO:teuthology.orchestra.run.smithi001.stdout: "required_mon": [ 2024-05-09T08:07:40.953 INFO:teuthology.orchestra.run.smithi001.stdout: "kraken", 2024-05-09T08:07:40.953 INFO:teuthology.orchestra.run.smithi001.stdout: "luminous", 2024-05-09T08:07:40.953 INFO:teuthology.orchestra.run.smithi001.stdout: "mimic", 2024-05-09T08:07:40.953 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap-prune", 2024-05-09T08:07:40.953 INFO:teuthology.orchestra.run.smithi001.stdout: "nautilus", 2024-05-09T08:07:40.953 INFO:teuthology.orchestra.run.smithi001.stdout: "octopus", 2024-05-09T08:07:40.953 INFO:teuthology.orchestra.run.smithi001.stdout: "pacific", 2024-05-09T08:07:40.953 INFO:teuthology.orchestra.run.smithi001.stdout: "elector-pinging", 2024-05-09T08:07:40.953 INFO:teuthology.orchestra.run.smithi001.stdout: "quincy", 2024-05-09T08:07:40.953 INFO:teuthology.orchestra.run.smithi001.stdout: "reef", 2024-05-09T08:07:40.953 INFO:teuthology.orchestra.run.smithi001.stdout: "squid" 2024-05-09T08:07:40.954 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:07:40.954 INFO:teuthology.orchestra.run.smithi001.stdout: "quorum_con": "4540701547738038271", 2024-05-09T08:07:40.954 INFO:teuthology.orchestra.run.smithi001.stdout: "quorum_mon": [ 2024-05-09T08:07:40.954 INFO:teuthology.orchestra.run.smithi001.stdout: "kraken", 2024-05-09T08:07:40.954 INFO:teuthology.orchestra.run.smithi001.stdout: "luminous", 2024-05-09T08:07:40.954 INFO:teuthology.orchestra.run.smithi001.stdout: "mimic", 2024-05-09T08:07:40.954 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap-prune", 2024-05-09T08:07:40.954 INFO:teuthology.orchestra.run.smithi001.stdout: "nautilus", 2024-05-09T08:07:40.954 INFO:teuthology.orchestra.run.smithi001.stdout: "octopus", 2024-05-09T08:07:40.954 INFO:teuthology.orchestra.run.smithi001.stdout: "pacific", 2024-05-09T08:07:40.954 INFO:teuthology.orchestra.run.smithi001.stdout: "elector-pinging", 2024-05-09T08:07:40.955 INFO:teuthology.orchestra.run.smithi001.stdout: "quincy", 2024-05-09T08:07:40.955 INFO:teuthology.orchestra.run.smithi001.stdout: "reef", 2024-05-09T08:07:40.955 INFO:teuthology.orchestra.run.smithi001.stdout: "squid" 2024-05-09T08:07:40.955 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:07:40.955 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:07:40.955 INFO:teuthology.orchestra.run.smithi001.stdout: "outside_quorum": [], 2024-05-09T08:07:40.955 INFO:teuthology.orchestra.run.smithi001.stdout: "extra_probe_peers": [], 2024-05-09T08:07:40.955 INFO:teuthology.orchestra.run.smithi001.stdout: "sync_provider": [], 2024-05-09T08:07:40.955 INFO:teuthology.orchestra.run.smithi001.stdout: "monmap": { 2024-05-09T08:07:40.955 INFO:teuthology.orchestra.run.smithi001.stdout: "epoch": 1, 2024-05-09T08:07:40.955 INFO:teuthology.orchestra.run.smithi001.stdout: "fsid": "54329ec8-3270-4043-9a82-c6cd7420aa7f", 2024-05-09T08:07:40.955 INFO:teuthology.orchestra.run.smithi001.stdout: "modified": "2024-05-09T08:01:21.819270Z", 2024-05-09T08:07:40.955 INFO:teuthology.orchestra.run.smithi001.stdout: "created": "2024-05-09T08:01:21.819270Z", 2024-05-09T08:07:40.956 INFO:teuthology.orchestra.run.smithi001.stdout: "min_mon_release": 19, 2024-05-09T08:07:40.956 INFO:teuthology.orchestra.run.smithi001.stdout: "min_mon_release_name": "squid", 2024-05-09T08:07:40.956 INFO:teuthology.orchestra.run.smithi001.stdout: "election_strategy": 1, 2024-05-09T08:07:40.956 INFO:teuthology.orchestra.run.smithi001.stdout: "disallowed_leaders: ": "", 2024-05-09T08:07:40.956 INFO:teuthology.orchestra.run.smithi001.stdout: "stretch_mode": false, 2024-05-09T08:07:40.956 INFO:teuthology.orchestra.run.smithi001.stdout: "tiebreaker_mon": "", 2024-05-09T08:07:40.956 INFO:teuthology.orchestra.run.smithi001.stdout: "removed_ranks: ": "", 2024-05-09T08:07:40.956 INFO:teuthology.orchestra.run.smithi001.stdout: "features": { 2024-05-09T08:07:40.956 INFO:teuthology.orchestra.run.smithi001.stdout: "persistent": [ 2024-05-09T08:07:40.956 INFO:teuthology.orchestra.run.smithi001.stdout: "kraken", 2024-05-09T08:07:40.956 INFO:teuthology.orchestra.run.smithi001.stdout: "luminous", 2024-05-09T08:07:40.956 INFO:teuthology.orchestra.run.smithi001.stdout: "mimic", 2024-05-09T08:07:40.956 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap-prune", 2024-05-09T08:07:40.956 INFO:teuthology.orchestra.run.smithi001.stdout: "nautilus", 2024-05-09T08:07:40.957 INFO:teuthology.orchestra.run.smithi001.stdout: "octopus", 2024-05-09T08:07:40.957 INFO:teuthology.orchestra.run.smithi001.stdout: "pacific", 2024-05-09T08:07:40.957 INFO:teuthology.orchestra.run.smithi001.stdout: "elector-pinging", 2024-05-09T08:07:40.957 INFO:teuthology.orchestra.run.smithi001.stdout: "quincy", 2024-05-09T08:07:40.957 INFO:teuthology.orchestra.run.smithi001.stdout: "reef", 2024-05-09T08:07:40.957 INFO:teuthology.orchestra.run.smithi001.stdout: "squid" 2024-05-09T08:07:40.957 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:07:40.957 INFO:teuthology.orchestra.run.smithi001.stdout: "optional": [] 2024-05-09T08:07:40.957 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:07:40.957 INFO:teuthology.orchestra.run.smithi001.stdout: "mons": [ 2024-05-09T08:07:40.957 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:07:40.957 INFO:teuthology.orchestra.run.smithi001.stdout: "rank": 0, 2024-05-09T08:07:40.958 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "a", 2024-05-09T08:07:40.958 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-05-09T08:07:40.958 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-05-09T08:07:40.958 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:07:40.958 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-05-09T08:07:40.958 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:3300", 2024-05-09T08:07:40.958 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:07:40.958 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:07:40.959 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:07:40.959 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-05-09T08:07:40.959 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6789", 2024-05-09T08:07:40.959 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:07:40.959 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:07:40.959 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:07:40.959 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:07:40.959 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6789/0", 2024-05-09T08:07:40.959 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.1:6789/0", 2024-05-09T08:07:40.959 INFO:teuthology.orchestra.run.smithi001.stdout: "priority": 0, 2024-05-09T08:07:40.959 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 0, 2024-05-09T08:07:40.959 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_location": "{}" 2024-05-09T08:07:40.959 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:07:40.959 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:07:40.960 INFO:teuthology.orchestra.run.smithi001.stdout: "rank": 1, 2024-05-09T08:07:40.960 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "b", 2024-05-09T08:07:40.960 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-05-09T08:07:40.960 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-05-09T08:07:40.960 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:07:40.960 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-05-09T08:07:40.960 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:3300", 2024-05-09T08:07:40.960 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:07:40.960 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:07:40.960 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:07:40.960 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-05-09T08:07:40.960 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:6789", 2024-05-09T08:07:40.960 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:07:40.961 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:07:40.961 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:07:40.961 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:07:40.961 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:6789/0", 2024-05-09T08:07:40.961 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.7:6789/0", 2024-05-09T08:07:40.961 INFO:teuthology.orchestra.run.smithi001.stdout: "priority": 0, 2024-05-09T08:07:40.961 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 0, 2024-05-09T08:07:40.961 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_location": "{}" 2024-05-09T08:07:40.961 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:07:40.961 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:07:40.962 INFO:teuthology.orchestra.run.smithi001.stdout: "rank": 2, 2024-05-09T08:07:40.962 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "c", 2024-05-09T08:07:40.962 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-05-09T08:07:40.962 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-05-09T08:07:40.962 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:07:40.962 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-05-09T08:07:40.962 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:3301", 2024-05-09T08:07:40.962 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:07:40.962 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:07:40.962 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:07:40.963 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-05-09T08:07:40.963 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:6790", 2024-05-09T08:07:40.963 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:07:40.963 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:07:40.963 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:07:40.963 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:07:40.963 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:6790/0", 2024-05-09T08:07:40.963 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.7:6790/0", 2024-05-09T08:07:40.963 INFO:teuthology.orchestra.run.smithi001.stdout: "priority": 0, 2024-05-09T08:07:40.963 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 0, 2024-05-09T08:07:40.963 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_location": "{}" 2024-05-09T08:07:40.963 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:07:40.963 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:07:40.963 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:07:40.964 INFO:teuthology.orchestra.run.smithi001.stdout: "feature_map": { 2024-05-09T08:07:40.964 INFO:teuthology.orchestra.run.smithi001.stdout: "mon": [ 2024-05-09T08:07:40.964 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:07:40.964 INFO:teuthology.orchestra.run.smithi001.stdout: "features": "0x3f03cffffffdffff", 2024-05-09T08:07:40.964 INFO:teuthology.orchestra.run.smithi001.stdout: "release": "luminous", 2024-05-09T08:07:40.964 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 1 2024-05-09T08:07:40.964 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:07:40.964 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:07:40.964 INFO:teuthology.orchestra.run.smithi001.stdout: "mds": [ 2024-05-09T08:07:40.964 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:07:40.964 INFO:teuthology.orchestra.run.smithi001.stdout: "features": "0x3f03cffffffdffff", 2024-05-09T08:07:40.964 INFO:teuthology.orchestra.run.smithi001.stdout: "release": "luminous", 2024-05-09T08:07:40.964 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 6 2024-05-09T08:07:40.965 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:07:40.965 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:07:40.965 INFO:teuthology.orchestra.run.smithi001.stdout: "osd": [ 2024-05-09T08:07:40.965 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:07:40.965 INFO:teuthology.orchestra.run.smithi001.stdout: "features": "0x3f03cffffffdffff", 2024-05-09T08:07:40.965 INFO:teuthology.orchestra.run.smithi001.stdout: "release": "luminous", 2024-05-09T08:07:40.965 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 7 2024-05-09T08:07:40.965 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:07:40.965 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:07:40.965 INFO:teuthology.orchestra.run.smithi001.stdout: "client": [ 2024-05-09T08:07:40.965 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:07:40.966 INFO:teuthology.orchestra.run.smithi001.stdout: "features": "0x2f018fb87aa4aafe", 2024-05-09T08:07:40.966 INFO:teuthology.orchestra.run.smithi001.stdout: "release": "luminous", 2024-05-09T08:07:40.966 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 1 2024-05-09T08:07:40.966 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:07:40.966 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:07:40.966 INFO:teuthology.orchestra.run.smithi001.stdout: "features": "0x3f03cffffffdffff", 2024-05-09T08:07:40.966 INFO:teuthology.orchestra.run.smithi001.stdout: "release": "luminous", 2024-05-09T08:07:40.966 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 6 2024-05-09T08:07:40.966 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:07:40.967 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:07:40.967 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr": [ 2024-05-09T08:07:40.967 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:07:40.967 INFO:teuthology.orchestra.run.smithi001.stdout: "features": "0x3f03cffffffdffff", 2024-05-09T08:07:40.967 INFO:teuthology.orchestra.run.smithi001.stdout: "release": "luminous", 2024-05-09T08:07:40.967 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 1 2024-05-09T08:07:40.967 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:07:40.967 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:07:40.967 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:07:40.967 INFO:teuthology.orchestra.run.smithi001.stdout: "stretch_mode": false 2024-05-09T08:07:40.967 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-05-09T08:07:40.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.940+0000 7f03d2200640 1 -- 172.21.15.1:0/3594213953 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f03ac06bdb0 msgr2=0x7f03ac06e270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:07:40.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.940+0000 7f03d2200640 1 --2- 172.21.15.1:0/3594213953 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f03ac06bdb0 0x7f03ac06e270 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f03cc155c40 tx=0x7f03c4004d40 comp rx=0 tx=0).stop 2024-05-09T08:07:40.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.940+0000 7f03d2200640 1 -- 172.21.15.1:0/3594213953 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03cc0c9620 msgr2=0x7f03cc152f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:07:40.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.940+0000 7f03d2200640 1 --2- 172.21.15.1:0/3594213953 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03cc0c9620 0x7f03cc152f50 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f03cc072200 tx=0x7f03c00046c0 comp rx=0 tx=0).stop 2024-05-09T08:07:40.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.940+0000 7f03d1800640 1 -- 172.21.15.1:0/3594213953 reap_dead start 2024-05-09T08:07:40.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.940+0000 7f03d1800640 1 -- 172.21.15.1:0/3594213953 reap_dead start 2024-05-09T08:07:40.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.941+0000 7f03d2200640 1 -- 172.21.15.1:0/3594213953 shutdown_connections 2024-05-09T08:07:40.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.941+0000 7f03d2200640 1 -- 172.21.15.1:0/3594213953 >> 172.21.15.1:0/3594213953 conn(0x7f03cc071620 msgr2=0x7f03cc1599f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:07:40.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.941+0000 7f03d2200640 1 -- 172.21.15.1:0/3594213953 shutdown_connections 2024-05-09T08:07:40.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:40.941+0000 7f03d2200640 1 -- 172.21.15.1:0/3594213953 wait complete. 2024-05-09T08:07:40.970 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-05-09T08:07:40.970 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-05-09T08:07:40.970 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-05-09T08:07:41.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:41.159+0000 7fd429000640 1 Processor -- start 2024-05-09T08:07:41.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:41.160+0000 7fd429000640 1 -- start start 2024-05-09T08:07:41.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:41.161+0000 7fd429000640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fd4240cac40 0x7fd42406da40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:07:41.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:41.161+0000 7fd429000640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd42415a610 0x7fd42415aa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:07:41.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:41.161+0000 7fd429000640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd4240cb020 con 0x7fd4240cd2a0 2024-05-09T08:07:41.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:41.161+0000 7fd429000640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fd424154dd0 con 0x7fd42415a610 2024-05-09T08:07:41.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:41.161+0000 7fd429000640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7fd4240cce10 con 0x7fd4240cac40 2024-05-09T08:07:41.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:41.162+0000 7fd422a00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fd4240cd2a0 0x7fd42415ba80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.1:47828/0 (socket says 172.21.15.1:47828) 2024-05-09T08:07:41.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:41.162+0000 7fd422a00640 1 -- 172.21.15.1:0/30414099 learned_addr learned my addr 172.21.15.1:0/30414099 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T08:07:41.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:41.162+0000 7fd423e00640 1 --2- 172.21.15.1:0/30414099 >> v2:172.21.15.7:3300/0 conn(0x7fd42415a610 0x7fd42415aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:07:41.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:41.162+0000 7fd423400640 1 --2- 172.21.15.1:0/30414099 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fd4240cac40 0x7fd42406da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:07:41.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:41.162+0000 7fd423e00640 1 -- 172.21.15.1:0/30414099 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fd4240cac40 msgr2=0x7fd42406da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:07:41.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:41.162+0000 7fd423e00640 1 --2- 172.21.15.1:0/30414099 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fd4240cac40 0x7fd42406da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:41.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:41.162+0000 7fd423e00640 1 -- 172.21.15.1:0/30414099 >> v1:172.21.15.7:6789/0 conn(0x7fd4240cd2a0 legacy=0x7fd42415ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:07:41.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:41.162+0000 7fd423e00640 1 -- 172.21.15.1:0/30414099 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4240cc550 con 0x7fd42415a610 2024-05-09T08:07:41.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:41.163+0000 7fd423e00640 1 --2- 172.21.15.1:0/30414099 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd42415a610 0x7fd42415aa90 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fd41c0097a0 tx=0x7fd41c02a210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f2ce7d9b59faaca5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:07:41.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:41.163+0000 7fd422000640 1 -- 172.21.15.1:0/30414099 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd41c037070 con 0x7fd42415a610 2024-05-09T08:07:41.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:41.163+0000 7fd422000640 1 -- 172.21.15.1:0/30414099 <== mon.1 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd41c032440 con 0x7fd42415a610 2024-05-09T08:07:41.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:41.164+0000 7fd422000640 1 -- 172.21.15.1:0/30414099 <== mon.1 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd41c032810 con 0x7fd42415a610 2024-05-09T08:07:41.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:41.164+0000 7fd429000640 1 -- 172.21.15.1:0/30414099 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd42415a610 msgr2=0x7fd42415aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:07:41.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:41.164+0000 7fd429000640 1 --2- 172.21.15.1:0/30414099 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd42415a610 0x7fd42415aa90 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fd41c0097a0 tx=0x7fd41c02a210 comp rx=0 tx=0).stop 2024-05-09T08:07:41.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:41.164+0000 7fd429000640 1 -- 172.21.15.1:0/30414099 shutdown_connections 2024-05-09T08:07:41.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:41.164+0000 7fd429000640 1 --2- 172.21.15.1:0/30414099 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd42415a610 0x7fd42415aa90 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:41.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:41.164+0000 7fd429000640 1 --2- 172.21.15.1:0/30414099 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fd4240cac40 0x7fd42406da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:41.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:41.164+0000 7fd429000640 1 -- 172.21.15.1:0/30414099 >> 172.21.15.1:0/30414099 conn(0x7fd4240670b0 msgr2=0x7fd4240674c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:07:41.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:41.164+0000 7fd429000640 1 -- 172.21.15.1:0/30414099 shutdown_connections 2024-05-09T08:07:41.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:41.164+0000 7fd429000640 1 -- 172.21.15.1:0/30414099 wait complete. 2024-05-09T08:07:41.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:41.166+0000 7fd429000640 1 Processor -- start 2024-05-09T08:07:41.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:41.166+0000 7fd429000640 1 -- start start 2024-05-09T08:07:41.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:41.167+0000 7fd429000640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd4240cac40 0x7fd424156eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:07:41.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:41.167+0000 7fd429000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd42415a610 0x7fd4240c3730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:07:41.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:41.167+0000 7fd429000640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fd424154180 0x7fd42414b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:07:41.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:41.167+0000 7fd429000640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fd4240cc7c0 con 0x7fd4240cac40 2024-05-09T08:07:41.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:41.167+0000 7fd429000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd42415b410 con 0x7fd42415a610 2024-05-09T08:07:41.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:41.167+0000 7fd429000640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7fd4240ccaa0 con 0x7fd424154180 2024-05-09T08:07:41.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:41.168+0000 7fd423400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd4240cac40 0x7fd424156eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:07:41.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:41.168+0000 7fd423400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd4240cac40 0x7fd424156eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:58448/0 (socket says 172.21.15.1:58448) 2024-05-09T08:07:41.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:41.168+0000 7fd422a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd42415a610 0x7fd4240c3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:07:41.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:41.168+0000 7fd423400640 1 -- 172.21.15.1:0/1112640089 learned_addr learned my addr 172.21.15.1:0/1112640089 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:07:41.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:41.168+0000 7fd423e00640 1 --2- 172.21.15.1:0/1112640089 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fd424154180 0x7fd42414b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:07:41.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:41.168+0000 7fd423400640 1 -- 172.21.15.1:0/1112640089 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fd424154180 msgr2=0x7fd42414b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:07:41.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:41.168+0000 7fd423400640 1 --2- 172.21.15.1:0/1112640089 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fd424154180 0x7fd42414b0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:41.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:41.168+0000 7fd423400640 1 -- 172.21.15.1:0/1112640089 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd42415a610 msgr2=0x7fd4240c3730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:07:41.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:41.168+0000 7fd423400640 1 --2- 172.21.15.1:0/1112640089 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd42415a610 0x7fd4240c3730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:41.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:41.168+0000 7fd423400640 1 -- 172.21.15.1:0/1112640089 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4240c6060 con 0x7fd4240cac40 2024-05-09T08:07:41.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:41.168+0000 7fd423e00640 1 --2- 172.21.15.1:0/1112640089 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fd424154180 0x7fd42414b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:07:41.181 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:41.168+0000 7fd423400640 1 --2- 172.21.15.1:0/1112640089 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd4240cac40 0x7fd424156eb0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fd414009db0 tx=0x7fd41400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:07:41.181 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:41.169+0000 7fd41b400640 1 -- 172.21.15.1:0/1112640089 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd414018020 con 0x7fd4240cac40 2024-05-09T08:07:41.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:41.169+0000 7fd41b400640 1 -- 172.21.15.1:0/1112640089 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd414014480 con 0x7fd4240cac40 2024-05-09T08:07:41.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:41.169+0000 7fd429000640 1 -- 172.21.15.1:0/1112640089 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4240c6300 con 0x7fd4240cac40 2024-05-09T08:07:41.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:41.169+0000 7fd41b400640 1 -- 172.21.15.1:0/1112640089 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd414014850 con 0x7fd4240cac40 2024-05-09T08:07:41.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:41.169+0000 7fd429000640 1 -- 172.21.15.1:0/1112640089 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd424156810 con 0x7fd4240cac40 2024-05-09T08:07:41.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:41.170+0000 7fd429000640 1 --2- 172.21.15.1:0/1112640089 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fd4240cd2a0 0x7fd42414b630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:07:41.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:41.170+0000 7fd429000640 1 -- 172.21.15.1:0/1112640089 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7fd42404f660 con 0x7fd4240cd2a0 2024-05-09T08:07:41.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:41.170+0000 7fd422a00640 1 --2- 172.21.15.1:0/1112640089 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fd4240cd2a0 0x7fd42414b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:07:41.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:41.171+0000 7fd422a00640 1 -- 172.21.15.1:0/1112640089 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd42406e0d0 con 0x7fd4240cd2a0 2024-05-09T08:07:41.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:41.171+0000 7fd422a00640 1 --2- 172.21.15.1:0/1112640089 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fd4240cd2a0 0x7fd42414b630 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fd4241edfa0 tx=0x7fd40c001920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:07:41.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:41.171+0000 7fd41b400640 1 -- 172.21.15.1:0/1112640089 <== mon.2 v2:172.21.15.7:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd40c011580 con 0x7fd4240cd2a0 2024-05-09T08:07:41.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:41.171+0000 7fd41b400640 1 -- 172.21.15.1:0/1112640089 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd414005d10 con 0x7fd4240cac40 2024-05-09T08:07:41.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:41.172+0000 7fd41b400640 1 --2- 172.21.15.1:0/1112640089 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fd40006bd30 0x7fd40006e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:07:41.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:41.172+0000 7fd41b400640 1 -- 172.21.15.1:0/1112640089 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7fd41408beb0 con 0x7fd4240cac40 2024-05-09T08:07:41.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:41.172+0000 7fd41b400640 1 -- 172.21.15.1:0/1112640089 <== mon.2 v2:172.21.15.7:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fd40c011720 con 0x7fd4240cd2a0 2024-05-09T08:07:41.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:41.172+0000 7fd41b400640 1 -- 172.21.15.1:0/1112640089 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fd4240cd2a0 msgr2=0x7fd42414b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:07:41.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:41.172+0000 7fd41b400640 1 --2- 172.21.15.1:0/1112640089 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fd4240cd2a0 0x7fd42414b630 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fd4241edfa0 tx=0x7fd40c001920 comp rx=0 tx=0).stop 2024-05-09T08:07:41.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:41.172+0000 7fd41b400640 1 -- 172.21.15.1:0/1112640089 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fd4240cd2a0 msgr2=0x7fd42414b630 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-05-09T08:07:41.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:41.172+0000 7fd41b400640 1 --2- 172.21.15.1:0/1112640089 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fd4240cd2a0 0x7fd42414b630 secure :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fd4241edfa0 tx=0x7fd40c001920 comp rx=0 tx=0).stop 2024-05-09T08:07:41.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:41.172+0000 7fd423e00640 1 --2- 172.21.15.1:0/1112640089 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fd40006bd30 0x7fd40006e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:07:41.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:41.173+0000 7fd423e00640 1 --2- 172.21.15.1:0/1112640089 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fd40006bd30 0x7fd40006e1f0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fd41c034000 tx=0x7fd41c02e000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:07:41.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:41.180+0000 7fd429000640 1 --2- 172.21.15.1:0/1112640089 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fd424156a50 0x7fd4241575d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:07:41.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:41.180+0000 7fd429000640 1 -- 172.21.15.1:0/1112640089 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7fd424073840 con 0x7fd424156a50 2024-05-09T08:07:41.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:41.180+0000 7fd422a00640 1 --2- 172.21.15.1:0/1112640089 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fd424156a50 0x7fd4241575d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:07:41.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:41.181+0000 7fd422a00640 1 -- 172.21.15.1:0/1112640089 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fd42415d260 con 0x7fd424156a50 2024-05-09T08:07:41.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:41.181+0000 7fd422a00640 1 --2- 172.21.15.1:0/1112640089 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fd424156a50 0x7fd4241575d0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fd42415aeb0 tx=0x7fd40c000ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:07:41.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:41.181+0000 7fd41b400640 1 -- 172.21.15.1:0/1112640089 <== mon.2 v2:172.21.15.7:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd40c01e110 con 0x7fd424156a50 2024-05-09T08:07:41.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:41.182+0000 7fd41b400640 1 -- 172.21.15.1:0/1112640089 <== mon.2 v2:172.21.15.7:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4533 (secure 0 0 0) 0x7fd40c01e110 con 0x7fd424156a50 2024-05-09T08:07:41.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:41.182+0000 7fd41b400640 1 -- 172.21.15.1:0/1112640089 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fd424156a50 msgr2=0x7fd4241575d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:07:41.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:41.182+0000 7fd41b400640 1 --2- 172.21.15.1:0/1112640089 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fd424156a50 0x7fd4241575d0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fd42415aeb0 tx=0x7fd40c000ed0 comp rx=0 tx=0).stop 2024-05-09T08:07:41.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:41.182+0000 7fd41b400640 1 -- 172.21.15.1:0/1112640089 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fd424156a50 msgr2=0x7fd4241575d0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-09T08:07:41.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:41.182+0000 7fd41b400640 1 --2- 172.21.15.1:0/1112640089 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fd424156a50 0x7fd4241575d0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:41.193 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-05-09T08:07:41.193 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "c", 2024-05-09T08:07:41.193 INFO:teuthology.orchestra.run.smithi001.stdout: "rank": 2, 2024-05-09T08:07:41.193 INFO:teuthology.orchestra.run.smithi001.stdout: "state": "peon", 2024-05-09T08:07:41.194 INFO:teuthology.orchestra.run.smithi001.stdout: "election_epoch": 42, 2024-05-09T08:07:41.194 INFO:teuthology.orchestra.run.smithi001.stdout: "uptime": 8256, 2024-05-09T08:07:41.194 INFO:teuthology.orchestra.run.smithi001.stdout: "quorum": [ 2024-05-09T08:07:41.194 INFO:teuthology.orchestra.run.smithi001.stdout: 0, 2024-05-09T08:07:41.194 INFO:teuthology.orchestra.run.smithi001.stdout: 1, 2024-05-09T08:07:41.194 INFO:teuthology.orchestra.run.smithi001.stdout: 2 2024-05-09T08:07:41.194 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:07:41.194 INFO:teuthology.orchestra.run.smithi001.stdout: "quorum_age": 3, 2024-05-09T08:07:41.194 INFO:teuthology.orchestra.run.smithi001.stdout: "features": { 2024-05-09T08:07:41.194 INFO:teuthology.orchestra.run.smithi001.stdout: "required_con": "2450521708007866372", 2024-05-09T08:07:41.195 INFO:teuthology.orchestra.run.smithi001.stdout: "required_mon": [ 2024-05-09T08:07:41.195 INFO:teuthology.orchestra.run.smithi001.stdout: "kraken", 2024-05-09T08:07:41.195 INFO:teuthology.orchestra.run.smithi001.stdout: "luminous", 2024-05-09T08:07:41.195 INFO:teuthology.orchestra.run.smithi001.stdout: "mimic", 2024-05-09T08:07:41.195 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap-prune", 2024-05-09T08:07:41.195 INFO:teuthology.orchestra.run.smithi001.stdout: "nautilus", 2024-05-09T08:07:41.195 INFO:teuthology.orchestra.run.smithi001.stdout: "octopus", 2024-05-09T08:07:41.195 INFO:teuthology.orchestra.run.smithi001.stdout: "pacific", 2024-05-09T08:07:41.195 INFO:teuthology.orchestra.run.smithi001.stdout: "elector-pinging", 2024-05-09T08:07:41.195 INFO:teuthology.orchestra.run.smithi001.stdout: "quincy", 2024-05-09T08:07:41.195 INFO:teuthology.orchestra.run.smithi001.stdout: "reef", 2024-05-09T08:07:41.196 INFO:teuthology.orchestra.run.smithi001.stdout: "squid" 2024-05-09T08:07:41.196 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:07:41.196 INFO:teuthology.orchestra.run.smithi001.stdout: "quorum_con": "4540701547738038271", 2024-05-09T08:07:41.196 INFO:teuthology.orchestra.run.smithi001.stdout: "quorum_mon": [ 2024-05-09T08:07:41.196 INFO:teuthology.orchestra.run.smithi001.stdout: "kraken", 2024-05-09T08:07:41.196 INFO:teuthology.orchestra.run.smithi001.stdout: "luminous", 2024-05-09T08:07:41.196 INFO:teuthology.orchestra.run.smithi001.stdout: "mimic", 2024-05-09T08:07:41.196 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap-prune", 2024-05-09T08:07:41.196 INFO:teuthology.orchestra.run.smithi001.stdout: "nautilus", 2024-05-09T08:07:41.196 INFO:teuthology.orchestra.run.smithi001.stdout: "octopus", 2024-05-09T08:07:41.196 INFO:teuthology.orchestra.run.smithi001.stdout: "pacific", 2024-05-09T08:07:41.196 INFO:teuthology.orchestra.run.smithi001.stdout: "elector-pinging", 2024-05-09T08:07:41.196 INFO:teuthology.orchestra.run.smithi001.stdout: "quincy", 2024-05-09T08:07:41.197 INFO:teuthology.orchestra.run.smithi001.stdout: "reef", 2024-05-09T08:07:41.197 INFO:teuthology.orchestra.run.smithi001.stdout: "squid" 2024-05-09T08:07:41.197 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:07:41.197 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:07:41.197 INFO:teuthology.orchestra.run.smithi001.stdout: "outside_quorum": [], 2024-05-09T08:07:41.197 INFO:teuthology.orchestra.run.smithi001.stdout: "extra_probe_peers": [], 2024-05-09T08:07:41.197 INFO:teuthology.orchestra.run.smithi001.stdout: "sync_provider": [], 2024-05-09T08:07:41.197 INFO:teuthology.orchestra.run.smithi001.stdout: "monmap": { 2024-05-09T08:07:41.197 INFO:teuthology.orchestra.run.smithi001.stdout: "epoch": 1, 2024-05-09T08:07:41.197 INFO:teuthology.orchestra.run.smithi001.stdout: "fsid": "54329ec8-3270-4043-9a82-c6cd7420aa7f", 2024-05-09T08:07:41.197 INFO:teuthology.orchestra.run.smithi001.stdout: "modified": "2024-05-09T08:01:21.819270Z", 2024-05-09T08:07:41.198 INFO:teuthology.orchestra.run.smithi001.stdout: "created": "2024-05-09T08:01:21.819270Z", 2024-05-09T08:07:41.198 INFO:teuthology.orchestra.run.smithi001.stdout: "min_mon_release": 19, 2024-05-09T08:07:41.198 INFO:teuthology.orchestra.run.smithi001.stdout: "min_mon_release_name": "squid", 2024-05-09T08:07:41.198 INFO:teuthology.orchestra.run.smithi001.stdout: "election_strategy": 1, 2024-05-09T08:07:41.198 INFO:teuthology.orchestra.run.smithi001.stdout: "disallowed_leaders: ": "", 2024-05-09T08:07:41.198 INFO:teuthology.orchestra.run.smithi001.stdout: "stretch_mode": false, 2024-05-09T08:07:41.198 INFO:teuthology.orchestra.run.smithi001.stdout: "tiebreaker_mon": "", 2024-05-09T08:07:41.198 INFO:teuthology.orchestra.run.smithi001.stdout: "removed_ranks: ": "", 2024-05-09T08:07:41.198 INFO:teuthology.orchestra.run.smithi001.stdout: "features": { 2024-05-09T08:07:41.198 INFO:teuthology.orchestra.run.smithi001.stdout: "persistent": [ 2024-05-09T08:07:41.198 INFO:teuthology.orchestra.run.smithi001.stdout: "kraken", 2024-05-09T08:07:41.199 INFO:teuthology.orchestra.run.smithi001.stdout: "luminous", 2024-05-09T08:07:41.199 INFO:teuthology.orchestra.run.smithi001.stdout: "mimic", 2024-05-09T08:07:41.199 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap-prune", 2024-05-09T08:07:41.199 INFO:teuthology.orchestra.run.smithi001.stdout: "nautilus", 2024-05-09T08:07:41.199 INFO:teuthology.orchestra.run.smithi001.stdout: "octopus", 2024-05-09T08:07:41.199 INFO:teuthology.orchestra.run.smithi001.stdout: "pacific", 2024-05-09T08:07:41.199 INFO:teuthology.orchestra.run.smithi001.stdout: "elector-pinging", 2024-05-09T08:07:41.199 INFO:teuthology.orchestra.run.smithi001.stdout: "quincy", 2024-05-09T08:07:41.199 INFO:teuthology.orchestra.run.smithi001.stdout: "reef", 2024-05-09T08:07:41.199 INFO:teuthology.orchestra.run.smithi001.stdout: "squid" 2024-05-09T08:07:41.199 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:07:41.199 INFO:teuthology.orchestra.run.smithi001.stdout: "optional": [] 2024-05-09T08:07:41.199 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:07:41.199 INFO:teuthology.orchestra.run.smithi001.stdout: "mons": [ 2024-05-09T08:07:41.200 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:07:41.200 INFO:teuthology.orchestra.run.smithi001.stdout: "rank": 0, 2024-05-09T08:07:41.200 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "a", 2024-05-09T08:07:41.200 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-05-09T08:07:41.200 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-05-09T08:07:41.200 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:07:41.200 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-05-09T08:07:41.200 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:3300", 2024-05-09T08:07:41.200 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:07:41.200 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:07:41.200 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:07:41.200 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-05-09T08:07:41.200 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6789", 2024-05-09T08:07:41.200 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:07:41.201 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:07:41.201 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:07:41.201 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:07:41.201 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6789/0", 2024-05-09T08:07:41.201 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.1:6789/0", 2024-05-09T08:07:41.201 INFO:teuthology.orchestra.run.smithi001.stdout: "priority": 0, 2024-05-09T08:07:41.201 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 0, 2024-05-09T08:07:41.201 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_location": "{}" 2024-05-09T08:07:41.201 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:07:41.201 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:07:41.201 INFO:teuthology.orchestra.run.smithi001.stdout: "rank": 1, 2024-05-09T08:07:41.202 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "b", 2024-05-09T08:07:41.202 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-05-09T08:07:41.202 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-05-09T08:07:41.202 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:07:41.202 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-05-09T08:07:41.202 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:3300", 2024-05-09T08:07:41.202 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:07:41.202 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:07:41.202 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:07:41.202 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-05-09T08:07:41.202 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:6789", 2024-05-09T08:07:41.202 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:07:41.202 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:07:41.203 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:07:41.203 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:07:41.203 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:6789/0", 2024-05-09T08:07:41.203 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.7:6789/0", 2024-05-09T08:07:41.203 INFO:teuthology.orchestra.run.smithi001.stdout: "priority": 0, 2024-05-09T08:07:41.203 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 0, 2024-05-09T08:07:41.203 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_location": "{}" 2024-05-09T08:07:41.203 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:07:41.203 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:07:41.203 INFO:teuthology.orchestra.run.smithi001.stdout: "rank": 2, 2024-05-09T08:07:41.203 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "c", 2024-05-09T08:07:41.203 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addrs": { 2024-05-09T08:07:41.203 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-05-09T08:07:41.203 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:07:41.203 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-05-09T08:07:41.204 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:3301", 2024-05-09T08:07:41.204 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:07:41.204 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:07:41.204 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:07:41.204 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-05-09T08:07:41.204 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:6790", 2024-05-09T08:07:41.204 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 0 2024-05-09T08:07:41.204 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:07:41.204 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:07:41.204 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:07:41.204 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.7:6790/0", 2024-05-09T08:07:41.204 INFO:teuthology.orchestra.run.smithi001.stdout: "public_addr": "172.21.15.7:6790/0", 2024-05-09T08:07:41.204 INFO:teuthology.orchestra.run.smithi001.stdout: "priority": 0, 2024-05-09T08:07:41.204 INFO:teuthology.orchestra.run.smithi001.stdout: "weight": 0, 2024-05-09T08:07:41.204 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_location": "{}" 2024-05-09T08:07:41.205 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:07:41.205 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:07:41.205 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:07:41.205 INFO:teuthology.orchestra.run.smithi001.stdout: "feature_map": { 2024-05-09T08:07:41.205 INFO:teuthology.orchestra.run.smithi001.stdout: "mon": [ 2024-05-09T08:07:41.205 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:07:41.205 INFO:teuthology.orchestra.run.smithi001.stdout: "features": "0x3f03cffffffdffff", 2024-05-09T08:07:41.205 INFO:teuthology.orchestra.run.smithi001.stdout: "release": "luminous", 2024-05-09T08:07:41.205 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 1 2024-05-09T08:07:41.205 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:07:41.205 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-09T08:07:41.205 INFO:teuthology.orchestra.run.smithi001.stdout: "client": [ 2024-05-09T08:07:41.205 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-09T08:07:41.205 INFO:teuthology.orchestra.run.smithi001.stdout: "features": "0x3f03cffffffdffff", 2024-05-09T08:07:41.206 INFO:teuthology.orchestra.run.smithi001.stdout: "release": "luminous", 2024-05-09T08:07:41.206 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 1 2024-05-09T08:07:41.206 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-09T08:07:41.206 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-09T08:07:41.206 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-09T08:07:41.206 INFO:teuthology.orchestra.run.smithi001.stdout: "stretch_mode": false 2024-05-09T08:07:41.206 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-05-09T08:07:41.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:41.182+0000 7fd429000640 1 -- 172.21.15.1:0/1112640089 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fd40006bd30 msgr2=0x7fd40006e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:07:41.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:41.182+0000 7fd429000640 1 --2- 172.21.15.1:0/1112640089 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fd40006bd30 0x7fd40006e1f0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fd41c034000 tx=0x7fd41c02e000 comp rx=0 tx=0).stop 2024-05-09T08:07:41.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:41.182+0000 7fd429000640 1 -- 172.21.15.1:0/1112640089 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd4240cac40 msgr2=0x7fd424156eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:07:41.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:41.182+0000 7fd429000640 1 --2- 172.21.15.1:0/1112640089 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd4240cac40 0x7fd424156eb0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fd414009db0 tx=0x7fd41400c490 comp rx=0 tx=0).stop 2024-05-09T08:07:41.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:41.182+0000 7fd423e00640 1 -- 172.21.15.1:0/1112640089 reap_dead start 2024-05-09T08:07:41.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:41.182+0000 7fd423e00640 1 -- 172.21.15.1:0/1112640089 reap_dead start 2024-05-09T08:07:41.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:41.182+0000 7fd429000640 1 -- 172.21.15.1:0/1112640089 shutdown_connections 2024-05-09T08:07:41.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:41.182+0000 7fd429000640 1 -- 172.21.15.1:0/1112640089 >> 172.21.15.1:0/1112640089 conn(0x7fd4240670b0 msgr2=0x7fd42406caf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:07:41.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:41.182+0000 7fd429000640 1 -- 172.21.15.1:0/1112640089 shutdown_connections 2024-05-09T08:07:41.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:41.182+0000 7fd429000640 1 -- 172.21.15.1:0/1112640089 wait complete. 2024-05-09T08:07:41.210 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-05-09T08:07:41.210 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-05-09T08:07:41.211 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-05-09T08:07:41.420 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:41.409+0000 7ffae9200640 1 Processor -- start 2024-05-09T08:07:41.420 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:41.410+0000 7ffae9200640 1 -- start start 2024-05-09T08:07:41.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:41.411+0000 7ffae9200640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7ffae4152500 0x7ffae40c8180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:07:41.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:41.411+0000 7ffae9200640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ffae40cc730 0x7ffae415c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:07:41.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:41.411+0000 7ffae9200640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffae40c6a30 con 0x7ffae40c7660 2024-05-09T08:07:41.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:41.411+0000 7ffae9200640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7ffae40c5e50 con 0x7ffae4152500 2024-05-09T08:07:41.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:41.411+0000 7ffae9200640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7ffae40c22d0 con 0x7ffae40cc730 2024-05-09T08:07:41.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:41.412+0000 7ffae3400640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7ffae4152500 0x7ffae40c8180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:07:41.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:41.412+0000 7ffadaa00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ffae40cc730 0x7ffae415c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:07:41.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:41.412+0000 7ffae3400640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7ffae4152500 0x7ffae40c8180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:58464/0 (socket says 172.21.15.1:58464) 2024-05-09T08:07:41.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:41.412+0000 7ffae3400640 1 -- 172.21.15.1:0/2372677242 learned_addr learned my addr 172.21.15.1:0/2372677242 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:07:41.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:41.412+0000 7ffadaa00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ffae40cc730 0x7ffae415c080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3301/0 says I am v2:172.21.15.1:43172/0 (socket says 172.21.15.1:43172) 2024-05-09T08:07:41.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:41.412+0000 7ffae3400640 1 -- 172.21.15.1:0/2372677242 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ffae40cc730 msgr2=0x7ffae415c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:07:41.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:41.412+0000 7ffae3400640 1 --2- 172.21.15.1:0/2372677242 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ffae40cc730 0x7ffae415c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:41.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:41.412+0000 7ffae3400640 1 -- 172.21.15.1:0/2372677242 >> v1:172.21.15.1:6789/0 conn(0x7ffae40c7660 legacy=0x7ffae406b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:07:41.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:41.412+0000 7ffae3400640 1 -- 172.21.15.1:0/2372677242 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffae40c6cf0 con 0x7ffae4152500 2024-05-09T08:07:41.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:41.413+0000 7ffadaa00640 1 --2- 172.21.15.1:0/2372677242 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ffae40cc730 0x7ffae415c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:07:41.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:41.414+0000 7ffae3400640 1 --2- 172.21.15.1:0/2372677242 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffae4152500 0x7ffae40c8180 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ffad4009870 tx=0x7ffad402a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3844e13a032a85c8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:07:41.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:41.414+0000 7ffae2a00640 1 -- 172.21.15.1:0/2372677242 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffad4037070 con 0x7ffae4152500 2024-05-09T08:07:41.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:41.414+0000 7ffae2a00640 1 -- 172.21.15.1:0/2372677242 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffad402ae50 con 0x7ffae4152500 2024-05-09T08:07:41.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:41.414+0000 7ffae2a00640 1 -- 172.21.15.1:0/2372677242 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffad4032860 con 0x7ffae4152500 2024-05-09T08:07:41.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:41.415+0000 7ffae9200640 1 -- 172.21.15.1:0/2372677242 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffae4152500 msgr2=0x7ffae40c8180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:07:41.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:41.415+0000 7ffae9200640 1 --2- 172.21.15.1:0/2372677242 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffae4152500 0x7ffae40c8180 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ffad4009870 tx=0x7ffad402a300 comp rx=0 tx=0).stop 2024-05-09T08:07:41.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:41.415+0000 7ffae9200640 1 -- 172.21.15.1:0/2372677242 shutdown_connections 2024-05-09T08:07:41.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:41.415+0000 7ffae9200640 1 --2- 172.21.15.1:0/2372677242 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ffae40cc730 0x7ffae415c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:41.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:41.415+0000 7ffae9200640 1 --2- 172.21.15.1:0/2372677242 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffae4152500 0x7ffae40c8180 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:41.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:41.415+0000 7ffae9200640 1 -- 172.21.15.1:0/2372677242 >> 172.21.15.1:0/2372677242 conn(0x7ffae40cbc10 msgr2=0x7ffae40cc020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:07:41.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:41.415+0000 7ffae9200640 1 -- 172.21.15.1:0/2372677242 shutdown_connections 2024-05-09T08:07:41.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:41.415+0000 7ffae9200640 1 -- 172.21.15.1:0/2372677242 wait complete. 2024-05-09T08:07:41.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:41.417+0000 7ffae9200640 1 Processor -- start 2024-05-09T08:07:41.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:41.417+0000 7ffae9200640 1 -- start start 2024-05-09T08:07:41.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:41.418+0000 7ffae9200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffae40c7660 0x7ffae41ee410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:07:41.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:41.418+0000 7ffae9200640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffae40cc730 0x7ffae414f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:07:41.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:41.418+0000 7ffae9200640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ffae4152500 0x7ffae40c7af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:07:41.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:41.418+0000 7ffae9200640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ffae40c72b0 con 0x7ffae40cc730 2024-05-09T08:07:41.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:41.418+0000 7ffae9200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ffae40ccb50 con 0x7ffae40c7660 2024-05-09T08:07:41.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:41.418+0000 7ffae9200640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7ffae415b880 con 0x7ffae4152500 2024-05-09T08:07:41.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:41.418+0000 7ffadaa00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffae40cc730 0x7ffae414f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:07:41.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:41.418+0000 7ffadaa00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffae40cc730 0x7ffae414f030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:58480/0 (socket says 172.21.15.1:58480) 2024-05-09T08:07:41.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:41.418+0000 7ffadaa00640 1 -- 172.21.15.1:0/3520620894 learned_addr learned my addr 172.21.15.1:0/3520620894 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:07:41.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:41.418+0000 7ffae3e00640 1 --2- 172.21.15.1:0/3520620894 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ffae4152500 0x7ffae40c7af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:07:41.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:41.418+0000 7ffae3400640 1 --2- 172.21.15.1:0/3520620894 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffae40c7660 0x7ffae41ee410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:07:41.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:41.418+0000 7ffadaa00640 1 -- 172.21.15.1:0/3520620894 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ffae4152500 msgr2=0x7ffae40c7af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:07:41.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:41.418+0000 7ffadaa00640 1 --2- 172.21.15.1:0/3520620894 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ffae4152500 0x7ffae40c7af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:41.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:41.418+0000 7ffadaa00640 1 -- 172.21.15.1:0/3520620894 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffae40c7660 msgr2=0x7ffae41ee410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:07:41.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:41.418+0000 7ffadaa00640 1 --2- 172.21.15.1:0/3520620894 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffae40c7660 0x7ffae41ee410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:41.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:41.418+0000 7ffadaa00640 1 -- 172.21.15.1:0/3520620894 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffae41e2ab0 con 0x7ffae40cc730 2024-05-09T08:07:41.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:41.419+0000 7ffae3400640 1 --2- 172.21.15.1:0/3520620894 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffae40c7660 0x7ffae41ee410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T08:07:41.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:41.419+0000 7ffae3e00640 1 --2- 172.21.15.1:0/3520620894 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ffae4152500 0x7ffae40c7af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T08:07:41.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:41.419+0000 7ffadaa00640 1 --2- 172.21.15.1:0/3520620894 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffae40cc730 0x7ffae414f030 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ffacc00ebb0 tx=0x7ffacc00c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:07:41.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:41.419+0000 7ffadbe00640 1 -- 172.21.15.1:0/3520620894 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffacc017070 con 0x7ffae40cc730 2024-05-09T08:07:41.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:41.419+0000 7ffadbe00640 1 -- 172.21.15.1:0/3520620894 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffacc00cc30 con 0x7ffae40cc730 2024-05-09T08:07:41.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:41.419+0000 7ffadbe00640 1 -- 172.21.15.1:0/3520620894 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffacc013470 con 0x7ffae40cc730 2024-05-09T08:07:41.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:41.419+0000 7ffae9200640 1 -- 172.21.15.1:0/3520620894 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffae41e31c0 con 0x7ffae40cc730 2024-05-09T08:07:41.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:41.419+0000 7ffae9200640 1 -- 172.21.15.1:0/3520620894 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffae41e3d40 con 0x7ffae40cc730 2024-05-09T08:07:41.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:41.420+0000 7ffae9200640 1 -- 172.21.15.1:0/3520620894 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffae404f660 con 0x7ffae40cc730 2024-05-09T08:07:41.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:41.421+0000 7ffadbe00640 1 -- 172.21.15.1:0/3520620894 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ffacc01a070 con 0x7ffae40cc730 2024-05-09T08:07:41.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:41.422+0000 7ffadbe00640 1 --2- 172.21.15.1:0/3520620894 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7ffac006be30 0x7ffac006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:07:41.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:41.422+0000 7ffadbe00640 1 -- 172.21.15.1:0/3520620894 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7ffacc08b510 con 0x7ffae40cc730 2024-05-09T08:07:41.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:41.423+0000 7ffae3400640 1 --2- 172.21.15.1:0/3520620894 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7ffac006be30 0x7ffac006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:07:41.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:41.424+0000 7ffae3400640 1 --2- 172.21.15.1:0/3520620894 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7ffac006be30 0x7ffac006e2f0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7ffad4034000 tx=0x7ffad402e000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:07:41.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:41.427+0000 7ffadbe00640 1 -- 172.21.15.1:0/3520620894 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ffacc059ac0 con 0x7ffae40cc730 2024-05-09T08:07:41.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:41.703+0000 7ffae9200640 1 -- 172.21.15.1:0/3520620894 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "mon scrub"} v 0) -- 0x7ffae40c2a40 con 0x7ffae40cc730 2024-05-09T08:07:41.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:41.704+0000 7ffadbe00640 1 -- 172.21.15.1:0/3520620894 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) ==== 57+0+0 (secure 0 0 0) 0x7ffacc0593c0 con 0x7ffae40cc730 2024-05-09T08:07:41.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:41.707+0000 7ffae9200640 1 -- 172.21.15.1:0/3520620894 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7ffac006be30 msgr2=0x7ffac006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:07:41.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:41.707+0000 7ffae9200640 1 --2- 172.21.15.1:0/3520620894 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7ffac006be30 0x7ffac006e2f0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7ffad4034000 tx=0x7ffad402e000 comp rx=0 tx=0).stop 2024-05-09T08:07:41.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:41.707+0000 7ffae9200640 1 -- 172.21.15.1:0/3520620894 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffae40cc730 msgr2=0x7ffae414f030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:07:41.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:41.707+0000 7ffae9200640 1 --2- 172.21.15.1:0/3520620894 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffae40cc730 0x7ffae414f030 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ffacc00ebb0 tx=0x7ffacc00c520 comp rx=0 tx=0).stop 2024-05-09T08:07:41.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:41.707+0000 7ffae9200640 1 -- 172.21.15.1:0/3520620894 shutdown_connections 2024-05-09T08:07:41.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:41.707+0000 7ffae9200640 1 --2- 172.21.15.1:0/3520620894 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7ffac006be30 0x7ffac006e2f0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:41.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:41.707+0000 7ffae9200640 1 --2- 172.21.15.1:0/3520620894 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ffae4152500 0x7ffae40c7af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:41.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:41.707+0000 7ffae9200640 1 --2- 172.21.15.1:0/3520620894 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffae40cc730 0x7ffae414f030 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:41.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:41.707+0000 7ffae9200640 1 --2- 172.21.15.1:0/3520620894 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffae40c7660 0x7ffae41ee410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:41.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:41.707+0000 7ffae9200640 1 -- 172.21.15.1:0/3520620894 >> 172.21.15.1:0/3520620894 conn(0x7ffae40cbc10 msgr2=0x7ffae4072080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:07:41.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:41.707+0000 7ffae9200640 1 -- 172.21.15.1:0/3520620894 shutdown_connections 2024-05-09T08:07:41.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:41.708+0000 7ffae9200640 1 -- 172.21.15.1:0/3520620894 wait complete. 2024-05-09T08:07:41.739 INFO:tasks.mon_thrash.mon_thrasher:waiting for 10.0 secs before continuing thrashing 2024-05-09T08:07:51.703 DEBUG:teuthology.orchestra.run.smithi001:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T08:07:51.712 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T08:07:51.740 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T08:07:51.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:51.941+0000 7fc357000640 1 Processor -- start 2024-05-09T08:07:51.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:51.942+0000 7fc357000640 1 -- start start 2024-05-09T08:07:51.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:51.943+0000 7fc357000640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fc3500c8180 0x7fc3500cc730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:07:51.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:51.943+0000 7fc357000640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fc3500c7660 0x7fc3500c7ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:07:51.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:51.943+0000 7fc357000640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc35015c790 con 0x7fc350152500 2024-05-09T08:07:51.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:51.943+0000 7fc357000640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7fc3500c5e50 con 0x7fc3500c8180 2024-05-09T08:07:51.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:51.943+0000 7fc357000640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7fc3500c22d0 con 0x7fc3500c7660 2024-05-09T08:07:51.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:51.944+0000 7fc356600640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fc3500c7660 0x7fc3500c7ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:07:51.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:51.944+0000 7fc355c00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fc350152500 0x7fc35015c080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.1:60270/0 (socket says 172.21.15.1:60270) 2024-05-09T08:07:51.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:51.944+0000 7fc355c00640 1 -- 172.21.15.1:0/4006663871 learned_addr learned my addr 172.21.15.1:0/4006663871 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T08:07:51.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:51.944+0000 7fc355200640 1 --2- 172.21.15.1:0/4006663871 >> v2:172.21.15.1:3300/0 conn(0x7fc3500c8180 0x7fc3500cc730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:07:51.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:51.944+0000 7fc356600640 1 -- 172.21.15.1:0/4006663871 >> v2:172.21.15.1:3300/0 conn(0x7fc3500c8180 msgr2=0x7fc3500cc730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:07:51.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:51.944+0000 7fc356600640 1 --2- 172.21.15.1:0/4006663871 >> v2:172.21.15.1:3300/0 conn(0x7fc3500c8180 0x7fc3500cc730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:51.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:51.944+0000 7fc356600640 1 -- 172.21.15.1:0/4006663871 >> v1:172.21.15.7:6789/0 conn(0x7fc350152500 legacy=0x7fc35015c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:07:51.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:51.944+0000 7fc34fe00640 1 -- 172.21.15.1:0/4006663871 <== mon.1 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3189647427 0 0) 0x7fc344001110 con 0x7fc350152500 2024-05-09T08:07:51.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:51.944+0000 7fc356600640 1 -- 172.21.15.1:0/4006663871 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3500c6a30 con 0x7fc3500c7660 2024-05-09T08:07:51.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:51.944+0000 7fc355200640 1 --2- 172.21.15.1:0/4006663871 >> v2:172.21.15.1:3300/0 conn(0x7fc3500c8180 0x7fc3500cc730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:07:51.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:51.945+0000 7fc356600640 1 --2- 172.21.15.1:0/4006663871 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fc3500c7660 0x7fc3500c7ab0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fc348009d90 tx=0x7fc34802a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=18acfad5265c1d2e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:07:51.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:51.946+0000 7fc34fe00640 1 -- 172.21.15.1:0/4006663871 <== mon.2 v2:172.21.15.7:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc348007740 con 0x7fc3500c7660 2024-05-09T08:07:51.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:51.946+0000 7fc34fe00640 1 -- 172.21.15.1:0/4006663871 <== mon.2 v2:172.21.15.7:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3480078e0 con 0x7fc3500c7660 2024-05-09T08:07:51.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:51.946+0000 7fc34fe00640 1 -- 172.21.15.1:0/4006663871 <== mon.2 v2:172.21.15.7:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc348007cb0 con 0x7fc3500c7660 2024-05-09T08:07:51.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:51.946+0000 7fc357000640 1 -- 172.21.15.1:0/4006663871 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fc3500c7660 msgr2=0x7fc3500c7ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:07:51.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:51.946+0000 7fc357000640 1 --2- 172.21.15.1:0/4006663871 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fc3500c7660 0x7fc3500c7ab0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fc348009d90 tx=0x7fc34802a660 comp rx=0 tx=0).stop 2024-05-09T08:07:51.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:51.947+0000 7fc357000640 1 -- 172.21.15.1:0/4006663871 shutdown_connections 2024-05-09T08:07:51.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:51.947+0000 7fc357000640 1 --2- 172.21.15.1:0/4006663871 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fc3500c7660 0x7fc3500c7ab0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:51.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:51.947+0000 7fc357000640 1 --2- 172.21.15.1:0/4006663871 >> v2:172.21.15.1:3300/0 conn(0x7fc3500c8180 0x7fc3500cc730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:51.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:51.947+0000 7fc357000640 1 -- 172.21.15.1:0/4006663871 >> 172.21.15.1:0/4006663871 conn(0x7fc3500cbc10 msgr2=0x7fc3500cc020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:07:51.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:51.947+0000 7fc357000640 1 -- 172.21.15.1:0/4006663871 shutdown_connections 2024-05-09T08:07:51.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:51.947+0000 7fc357000640 1 -- 172.21.15.1:0/4006663871 wait complete. 2024-05-09T08:07:51.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:51.948+0000 7fc357000640 1 Processor -- start 2024-05-09T08:07:51.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:51.949+0000 7fc357000640 1 -- start start 2024-05-09T08:07:51.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:51.950+0000 7fc357000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3500c7660 0x7fc350158390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:07:51.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:51.950+0000 7fc357000640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fc3500c8180 0x7fc3501e71d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:07:51.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:51.950+0000 7fc357000640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc350152500 0x7fc3501e3110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:07:51.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:51.950+0000 7fc357000640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fc3500c72b0 con 0x7fc350152500 2024-05-09T08:07:51.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:51.950+0000 7fc357000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc3500c6fd0 con 0x7fc3500c7660 2024-05-09T08:07:51.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:51.950+0000 7fc357000640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7fc35015b880 con 0x7fc3500c8180 2024-05-09T08:07:51.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:51.950+0000 7fc356600640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc350152500 0x7fc3501e3110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:07:51.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:51.950+0000 7fc355c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3500c7660 0x7fc350158390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:07:51.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:51.950+0000 7fc355c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3500c7660 0x7fc350158390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.1:44776/0 (socket says 172.21.15.1:44776) 2024-05-09T08:07:51.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:51.950+0000 7fc355200640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fc3500c8180 0x7fc3501e71d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:07:51.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:51.950+0000 7fc355c00640 1 -- 172.21.15.1:0/1948493701 learned_addr learned my addr 172.21.15.1:0/1948493701 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:07:51.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:51.951+0000 7fc355c00640 1 -- 172.21.15.1:0/1948493701 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fc3500c8180 msgr2=0x7fc3501e71d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:07:51.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:51.951+0000 7fc355c00640 1 --2- 172.21.15.1:0/1948493701 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fc3500c8180 0x7fc3501e71d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:51.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:51.951+0000 7fc355c00640 1 -- 172.21.15.1:0/1948493701 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc350152500 msgr2=0x7fc3501e3110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:07:51.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:51.951+0000 7fc355c00640 1 --2- 172.21.15.1:0/1948493701 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc350152500 0x7fc3501e3110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:51.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:51.951+0000 7fc355c00640 1 -- 172.21.15.1:0/1948493701 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3501e41c0 con 0x7fc3500c7660 2024-05-09T08:07:51.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:51.951+0000 7fc356600640 1 --2- 172.21.15.1:0/1948493701 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc350152500 0x7fc3501e3110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:07:51.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:51.951+0000 7fc355c00640 1 --2- 172.21.15.1:0/1948493701 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3500c7660 0x7fc350158390 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fc34400c430 tx=0x7fc34400c8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:07:51.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:51.951+0000 7fc34d600640 1 -- 172.21.15.1:0/1948493701 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3440074e0 con 0x7fc3500c7660 2024-05-09T08:07:51.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:51.951+0000 7fc34d600640 1 -- 172.21.15.1:0/1948493701 <== mon.1 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3440047e0 con 0x7fc3500c7660 2024-05-09T08:07:51.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:51.951+0000 7fc34d600640 1 -- 172.21.15.1:0/1948493701 <== mon.1 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc344004bd0 con 0x7fc3500c7660 2024-05-09T08:07:51.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:51.951+0000 7fc357000640 1 -- 172.21.15.1:0/1948493701 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3501e4480 con 0x7fc3500c7660 2024-05-09T08:07:51.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:51.951+0000 7fc357000640 1 -- 172.21.15.1:0/1948493701 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3501e2380 con 0x7fc3500c7660 2024-05-09T08:07:51.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:51.953+0000 7fc357000640 1 -- 172.21.15.1:0/1948493701 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc35004f660 con 0x7fc3500c7660 2024-05-09T08:07:51.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:51.954+0000 7fc34d600640 1 -- 172.21.15.1:0/1948493701 <== mon.1 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc344011070 con 0x7fc3500c7660 2024-05-09T08:07:51.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:51.954+0000 7fc34d600640 1 --2- 172.21.15.1:0/1948493701 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fc33006be30 0x7fc33006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:07:51.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:51.955+0000 7fc34d600640 1 -- 172.21.15.1:0/1948493701 <== mon.1 v2:172.21.15.7:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7fc34408ad40 con 0x7fc3500c7660 2024-05-09T08:07:51.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:51.955+0000 7fc355200640 1 --2- 172.21.15.1:0/1948493701 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fc33006be30 0x7fc33006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:07:51.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:51.955+0000 7fc355200640 1 --2- 172.21.15.1:0/1948493701 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fc33006be30 0x7fc33006e2f0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fc33c009560 tx=0x7fc33c009190 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:07:51.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:51.959+0000 7fc34d600640 1 -- 172.21.15.1:0/1948493701 <== mon.1 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fc35015c790 con 0x7fc3500c7660 2024-05-09T08:07:52.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:52.208+0000 7fc357000640 1 -- 172.21.15.1:0/1948493701 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc3500c7b70 con 0x7fc3500c7660 2024-05-09T08:07:52.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:52.209+0000 7fc34d600640 1 -- 172.21.15.1:0/1948493701 <== mon.1 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+6807 (secure 0 0 0) 0x7fc344058ca0 con 0x7fc3500c7660 2024-05-09T08:07:52.219 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T08:07:52.220 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":10,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4220,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/1157980939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":1157980939},{"type":"v1","addr":"172.21.15.7:6837","nonce":1157980939}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4235,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6833/3269798795","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6832","nonce":3269798795},{"type":"v1","addr":"172.21.15.1:6833","nonce":3269798795}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4241,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/725115499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6835","nonce":725115499},{"type":"v1","addr":"172.21.15.1:6837","nonce":725115499}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4276,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6838/1827115274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1827115274},{"type":"v1","addr":"172.21.15.7:6838","nonce":1827115274}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/1032767352","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":1032767352},{"type":"v1","addr":"172.21.15.7:6839","nonce":1032767352}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":10,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-09T08:02:15.332696+0000","modified":"2024-05-09T08:02:49.734561+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4294},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4294":{"gid":4294,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":6,"addr":"172.21.15.1:6836/1509134635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":1509134635},{"type":"v1","addr":"172.21.15.1:6836","nonce":1509134635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4294,"qdb_cluster":[4294]},"id":1}]} 2024-05-09T08:07:52.220 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 10 2024-05-09T08:07:52.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:52.212+0000 7fc357000640 1 -- 172.21.15.1:0/1948493701 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fc33006be30 msgr2=0x7fc33006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:07:52.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:52.212+0000 7fc357000640 1 --2- 172.21.15.1:0/1948493701 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fc33006be30 0x7fc33006e2f0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fc33c009560 tx=0x7fc33c009190 comp rx=0 tx=0).stop 2024-05-09T08:07:52.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:52.212+0000 7fc357000640 1 -- 172.21.15.1:0/1948493701 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3500c7660 msgr2=0x7fc350158390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:07:52.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:52.212+0000 7fc357000640 1 --2- 172.21.15.1:0/1948493701 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3500c7660 0x7fc350158390 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fc34400c430 tx=0x7fc34400c8f0 comp rx=0 tx=0).stop 2024-05-09T08:07:52.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:52.212+0000 7fc357000640 1 -- 172.21.15.1:0/1948493701 shutdown_connections 2024-05-09T08:07:52.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:52.212+0000 7fc357000640 1 --2- 172.21.15.1:0/1948493701 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fc33006be30 0x7fc33006e2f0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:52.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:52.212+0000 7fc357000640 1 --2- 172.21.15.1:0/1948493701 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc350152500 0x7fc3501e3110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:52.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:52.212+0000 7fc357000640 1 --2- 172.21.15.1:0/1948493701 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fc3500c8180 0x7fc3501e71d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:52.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:52.212+0000 7fc357000640 1 --2- 172.21.15.1:0/1948493701 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3500c7660 0x7fc350158390 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:52.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:52.212+0000 7fc357000640 1 -- 172.21.15.1:0/1948493701 >> 172.21.15.1:0/1948493701 conn(0x7fc3500cbc10 msgr2=0x7fc350159780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:07:52.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:52.212+0000 7fc357000640 1 -- 172.21.15.1:0/1948493701 shutdown_connections 2024-05-09T08:07:52.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:52.212+0000 7fc357000640 1 -- 172.21.15.1:0/1948493701 wait complete. 2024-05-09T08:07:52.243 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-05-09T08:07:52.243 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-05-09T08:07:52.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:52.446+0000 7fb344400640 1 Processor -- start 2024-05-09T08:07:52.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:52.447+0000 7fb344400640 1 -- start start 2024-05-09T08:07:52.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:52.448+0000 7fb344400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb33c152b20 0x7fb33c072900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:07:52.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:52.448+0000 7fb344400640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb33c073200 con 0x7fb33c06dca0 2024-05-09T08:07:52.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:52.448+0000 7fb344400640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb33c0734e0 con 0x7fb33c0c6ea0 2024-05-09T08:07:52.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:52.448+0000 7fb344400640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fb33c072e40 con 0x7fb33c152b20 2024-05-09T08:07:52.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:52.449+0000 7fb341a00640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7fb33c06dca0 0x7fb33c06e080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:41848/0 (socket says 172.21.15.1:41848) 2024-05-09T08:07:52.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:52.449+0000 7fb341a00640 1 -- 172.21.15.1:0/2640997017 learned_addr learned my addr 172.21.15.1:0/2640997017 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T08:07:52.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:52.449+0000 7fb342400640 1 --2- 172.21.15.1:0/2640997017 >> v2:172.21.15.7:3300/0 conn(0x7fb33c152b20 0x7fb33c072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:07:52.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:52.449+0000 7fb33be00640 1 -- 172.21.15.1:0/2640997017 <== mon.1 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3623004468 0 0) 0x7fb33c0734e0 con 0x7fb33c0c6ea0 2024-05-09T08:07:52.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:52.449+0000 7fb33be00640 1 -- 172.21.15.1:0/2640997017 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb324003520 con 0x7fb33c0c6ea0 2024-05-09T08:07:52.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:52.449+0000 7fb342400640 1 -- 172.21.15.1:0/2640997017 >> v1:172.21.15.7:6789/0 conn(0x7fb33c0c6ea0 legacy=0x7fb33c152410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:07:52.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:52.449+0000 7fb342400640 1 -- 172.21.15.1:0/2640997017 >> v1:172.21.15.1:6789/0 conn(0x7fb33c06dca0 legacy=0x7fb33c06e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:07:52.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:52.449+0000 7fb33be00640 1 -- 172.21.15.1:0/2640997017 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1775498148 0 0) 0x7fb33c073200 con 0x7fb33c06dca0 2024-05-09T08:07:52.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:52.449+0000 7fb342400640 1 -- 172.21.15.1:0/2640997017 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb33c071e20 con 0x7fb33c152b20 2024-05-09T08:07:52.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:52.450+0000 7fb342400640 1 --2- 172.21.15.1:0/2640997017 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb33c152b20 0x7fb33c072900 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fb334009d90 tx=0x7fb33402a780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c2640c616345959 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:07:52.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:52.451+0000 7fb33be00640 1 -- 172.21.15.1:0/2640997017 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb334037070 con 0x7fb33c152b20 2024-05-09T08:07:52.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:52.451+0000 7fb33be00640 1 -- 172.21.15.1:0/2640997017 <== mon.1 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb334007740 con 0x7fb33c152b20 2024-05-09T08:07:52.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:52.451+0000 7fb33be00640 1 -- 172.21.15.1:0/2640997017 <== mon.1 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb334007b10 con 0x7fb33c152b20 2024-05-09T08:07:52.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:52.451+0000 7fb344400640 1 -- 172.21.15.1:0/2640997017 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb33c152b20 msgr2=0x7fb33c072900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:07:52.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:52.451+0000 7fb344400640 1 --2- 172.21.15.1:0/2640997017 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb33c152b20 0x7fb33c072900 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fb334009d90 tx=0x7fb33402a780 comp rx=0 tx=0).stop 2024-05-09T08:07:52.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:52.451+0000 7fb344400640 1 -- 172.21.15.1:0/2640997017 shutdown_connections 2024-05-09T08:07:52.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:52.451+0000 7fb344400640 1 --2- 172.21.15.1:0/2640997017 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb33c152b20 0x7fb33c072900 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:52.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:52.451+0000 7fb344400640 1 -- 172.21.15.1:0/2640997017 >> 172.21.15.1:0/2640997017 conn(0x7fb33c0cc070 msgr2=0x7fb33c0cc480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:07:52.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:52.451+0000 7fb344400640 1 -- 172.21.15.1:0/2640997017 shutdown_connections 2024-05-09T08:07:52.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:52.451+0000 7fb344400640 1 -- 172.21.15.1:0/2640997017 wait complete. 2024-05-09T08:07:52.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:52.453+0000 7fb344400640 1 Processor -- start 2024-05-09T08:07:52.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:52.453+0000 7fb344400640 1 -- start start 2024-05-09T08:07:52.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:52.454+0000 7fb344400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb33c06dca0 0x7fb33c1ef010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:07:52.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:52.454+0000 7fb344400640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fb33c0c6ea0 0x7fb33c146c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:07:52.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:52.454+0000 7fb344400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb33c152b20 0x7fb33c148630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:07:52.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:52.454+0000 7fb344400640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fb33c072470 con 0x7fb33c06dca0 2024-05-09T08:07:52.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:52.454+0000 7fb344400640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb33c06ece0 con 0x7fb33c152b20 2024-05-09T08:07:52.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:52.454+0000 7fb344400640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7fb33c06ee60 con 0x7fb33c0c6ea0 2024-05-09T08:07:52.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:52.455+0000 7fb341000640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fb33c0c6ea0 0x7fb33c146c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:07:52.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:52.455+0000 7fb342400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb33c152b20 0x7fb33c148630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:07:52.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:52.455+0000 7fb341a00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb33c06dca0 0x7fb33c1ef010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:07:52.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:52.455+0000 7fb342400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb33c152b20 0x7fb33c148630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.1:44788/0 (socket says 172.21.15.1:44788) 2024-05-09T08:07:52.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:52.455+0000 7fb341000640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fb33c0c6ea0 0x7fb33c146c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3301/0 says I am v2:172.21.15.1:46676/0 (socket says 172.21.15.1:46676) 2024-05-09T08:07:52.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:52.455+0000 7fb342400640 1 -- 172.21.15.1:0/3558744168 learned_addr learned my addr 172.21.15.1:0/3558744168 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:07:52.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:52.455+0000 7fb342400640 1 -- 172.21.15.1:0/3558744168 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fb33c0c6ea0 msgr2=0x7fb33c146c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:07:52.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:52.455+0000 7fb342400640 1 --2- 172.21.15.1:0/3558744168 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fb33c0c6ea0 0x7fb33c146c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:52.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:52.455+0000 7fb342400640 1 -- 172.21.15.1:0/3558744168 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb33c06dca0 msgr2=0x7fb33c1ef010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:07:52.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:52.455+0000 7fb342400640 1 --2- 172.21.15.1:0/3558744168 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb33c06dca0 0x7fb33c1ef010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:52.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:52.455+0000 7fb342400640 1 -- 172.21.15.1:0/3558744168 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb33c0cb300 con 0x7fb33c152b20 2024-05-09T08:07:52.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:52.455+0000 7fb341000640 1 --2- 172.21.15.1:0/3558744168 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fb33c0c6ea0 0x7fb33c146c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:07:52.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:52.455+0000 7fb341a00640 1 --2- 172.21.15.1:0/3558744168 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb33c06dca0 0x7fb33c1ef010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:07:52.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:52.456+0000 7fb342400640 1 --2- 172.21.15.1:0/3558744168 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb33c152b20 0x7fb33c148630 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fb334009d90 tx=0x7fb334004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:07:52.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:52.456+0000 7fb339600640 1 -- 172.21.15.1:0/3558744168 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb334042030 con 0x7fb33c152b20 2024-05-09T08:07:52.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:52.456+0000 7fb339600640 1 -- 172.21.15.1:0/3558744168 <== mon.1 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb334007540 con 0x7fb33c152b20 2024-05-09T08:07:52.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:52.456+0000 7fb339600640 1 -- 172.21.15.1:0/3558744168 <== mon.1 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb334037040 con 0x7fb33c152b20 2024-05-09T08:07:52.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:52.456+0000 7fb344400640 1 -- 172.21.15.1:0/3558744168 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb33c0cb5e0 con 0x7fb33c152b20 2024-05-09T08:07:52.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:52.456+0000 7fb344400640 1 -- 172.21.15.1:0/3558744168 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb33c1ee810 con 0x7fb33c152b20 2024-05-09T08:07:52.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:52.457+0000 7fb344400640 1 -- 172.21.15.1:0/3558744168 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb33c06d760 con 0x7fb33c152b20 2024-05-09T08:07:52.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:52.458+0000 7fb339600640 1 -- 172.21.15.1:0/3558744168 <== mon.1 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb334045070 con 0x7fb33c152b20 2024-05-09T08:07:52.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:52.459+0000 7fb339600640 1 --2- 172.21.15.1:0/3558744168 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fb31c06beb0 0x7fb31c06e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:07:52.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:52.459+0000 7fb339600640 1 -- 172.21.15.1:0/3558744168 <== mon.1 v2:172.21.15.7:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7fb3340abe60 con 0x7fb33c152b20 2024-05-09T08:07:52.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:52.459+0000 7fb341a00640 1 --2- 172.21.15.1:0/3558744168 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fb31c06beb0 0x7fb31c06e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:07:52.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:52.460+0000 7fb341a00640 1 --2- 172.21.15.1:0/3558744168 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fb31c06beb0 0x7fb31c06e370 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fb330000f60 tx=0x7fb330007d50 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:07:52.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:52.464+0000 7fb339600640 1 -- 172.21.15.1:0/3558744168 <== mon.1 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fb33c06d760 con 0x7fb33c152b20 2024-05-09T08:07:52.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:52.742+0000 7fb344400640 1 -- 172.21.15.1:0/3558744168 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fb33c14d050 con 0x7fb33c152b20 2024-05-09T08:07:52.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:52.743+0000 7fb339600640 1 -- 172.21.15.1:0/3558744168 <== mon.1 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1545 (secure 0 0 0) 0x7fb33c14d050 con 0x7fb33c152b20 2024-05-09T08:07:52.754 INFO:teuthology.orchestra.run.smithi001.stdout:{"election_epoch":42,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":15,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"54329ec8-3270-4043-9a82-c6cd7420aa7f","modified":"2024-05-09T08:01:21.819270Z","created":"2024-05-09T08:01:21.819270Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:3300","nonce":0},{"type":"v1","addr":"172.21.15.1:6789","nonce":0}]},"addr":"172.21.15.1:6789/0","public_addr":"172.21.15.1:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:3300","nonce":0},{"type":"v1","addr":"172.21.15.7:6789","nonce":0}]},"addr":"172.21.15.7:6789/0","public_addr":"172.21.15.7:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:3301","nonce":0},{"type":"v1","addr":"172.21.15.7:6790","nonce":0}]},"addr":"172.21.15.7:6790/0","public_addr":"172.21.15.7:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-05-09T08:07:52.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:52.746+0000 7fb344400640 1 -- 172.21.15.1:0/3558744168 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fb31c06beb0 msgr2=0x7fb31c06e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:07:52.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:52.746+0000 7fb344400640 1 --2- 172.21.15.1:0/3558744168 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fb31c06beb0 0x7fb31c06e370 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fb330000f60 tx=0x7fb330007d50 comp rx=0 tx=0).stop 2024-05-09T08:07:52.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:52.746+0000 7fb344400640 1 -- 172.21.15.1:0/3558744168 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb33c152b20 msgr2=0x7fb33c148630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:07:52.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:52.746+0000 7fb344400640 1 --2- 172.21.15.1:0/3558744168 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb33c152b20 0x7fb33c148630 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fb334009d90 tx=0x7fb334004a40 comp rx=0 tx=0).stop 2024-05-09T08:07:52.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:52.746+0000 7fb344400640 1 -- 172.21.15.1:0/3558744168 shutdown_connections 2024-05-09T08:07:52.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:52.746+0000 7fb344400640 1 --2- 172.21.15.1:0/3558744168 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fb31c06beb0 0x7fb31c06e370 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:52.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:52.746+0000 7fb344400640 1 --2- 172.21.15.1:0/3558744168 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb33c152b20 0x7fb33c148630 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:52.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:52.746+0000 7fb344400640 1 --2- 172.21.15.1:0/3558744168 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fb33c0c6ea0 0x7fb33c146c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:52.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:52.746+0000 7fb344400640 1 --2- 172.21.15.1:0/3558744168 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb33c06dca0 0x7fb33c1ef010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:52.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:52.746+0000 7fb344400640 1 -- 172.21.15.1:0/3558744168 >> 172.21.15.1:0/3558744168 conn(0x7fb33c0cc070 msgr2=0x7fb33c14d470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:07:52.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:52.747+0000 7fb344400640 1 -- 172.21.15.1:0/3558744168 shutdown_connections 2024-05-09T08:07:52.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:52.747+0000 7fb344400640 1 -- 172.21.15.1:0/3558744168 wait complete. 2024-05-09T08:07:52.776 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-05-09T08:07:52.777 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-05-09T08:07:52.787 INFO:tasks.kclient:Unmounting kernel clients... 2024-05-09T08:07:52.788 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-05-09T08:07:52.788 DEBUG:teuthology.orchestra.run.smithi001:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T08:07:52.838 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-05-09T08:07:52.838 DEBUG:teuthology.orchestra.run.smithi001:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T08:07:52.937 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T08:07:53.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:53.137+0000 7f02a2a00640 1 Processor -- start 2024-05-09T08:07:53.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:53.137+0000 7f02a2a00640 1 -- start start 2024-05-09T08:07:53.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:53.139+0000 7f02a2a00640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f029c0c7dd0 0x7f029c0c44b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:07:53.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:53.140+0000 7f02a2a00640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f029c0c81e0 con 0x7f029c154790 2024-05-09T08:07:53.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:53.140+0000 7f02a2a00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f029c0c6a50 con 0x7f029c15a440 2024-05-09T08:07:53.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:53.140+0000 7f02a2a00640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f029c154b70 con 0x7f029c0c7dd0 2024-05-09T08:07:53.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:53.140+0000 7f02a1600640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f029c154790 0x7f029c06b250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:41852/0 (socket says 172.21.15.1:41852) 2024-05-09T08:07:53.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:53.140+0000 7f02a1600640 1 -- 172.21.15.1:0/617328622 learned_addr learned my addr 172.21.15.1:0/617328622 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T08:07:53.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:53.140+0000 7f02a2000640 1 --2- 172.21.15.1:0/617328622 >> v2:172.21.15.1:3300/0 conn(0x7f029c0c7dd0 0x7f029c0c44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:07:53.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:53.140+0000 7f029be00640 1 -- 172.21.15.1:0/617328622 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 398268895 0 0) 0x7f029c0c81e0 con 0x7f029c154790 2024-05-09T08:07:53.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:53.140+0000 7f029be00640 1 -- 172.21.15.1:0/617328622 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0284003520 con 0x7f029c154790 2024-05-09T08:07:53.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:53.140+0000 7f029be00640 1 -- 172.21.15.1:0/617328622 <== mon.1 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4203599907 0 0) 0x7f029c0c6a50 con 0x7f029c15a440 2024-05-09T08:07:53.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:53.140+0000 7f029be00640 1 -- 172.21.15.1:0/617328622 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f029c0c81e0 con 0x7f029c15a440 2024-05-09T08:07:53.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:53.141+0000 7f02a2000640 1 -- 172.21.15.1:0/617328622 >> v1:172.21.15.7:6789/0 conn(0x7f029c15a440 legacy=0x7f029c06b960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:07:53.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:53.141+0000 7f02a2000640 1 -- 172.21.15.1:0/617328622 >> v1:172.21.15.1:6789/0 conn(0x7f029c154790 legacy=0x7f029c06b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:07:53.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:53.141+0000 7f02a2000640 1 -- 172.21.15.1:0/617328622 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f029c0c6d30 con 0x7f029c0c7dd0 2024-05-09T08:07:53.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:53.142+0000 7f02a2000640 1 --2- 172.21.15.1:0/617328622 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f029c0c7dd0 0x7f029c0c44b0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f02940097a0 tx=0x7f029402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7804aa3b14b4f6b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:07:53.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:53.142+0000 7f029be00640 1 -- 172.21.15.1:0/617328622 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0294037070 con 0x7f029c0c7dd0 2024-05-09T08:07:53.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:53.142+0000 7f029be00640 1 -- 172.21.15.1:0/617328622 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0294032440 con 0x7f029c0c7dd0 2024-05-09T08:07:53.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:53.142+0000 7f029be00640 1 -- 172.21.15.1:0/617328622 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0294032810 con 0x7f029c0c7dd0 2024-05-09T08:07:53.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:53.143+0000 7f02a2a00640 1 -- 172.21.15.1:0/617328622 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f029c0c7dd0 msgr2=0x7f029c0c44b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:07:53.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:53.143+0000 7f02a2a00640 1 --2- 172.21.15.1:0/617328622 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f029c0c7dd0 0x7f029c0c44b0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f02940097a0 tx=0x7f029402a230 comp rx=0 tx=0).stop 2024-05-09T08:07:53.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:53.143+0000 7f02a2a00640 1 -- 172.21.15.1:0/617328622 shutdown_connections 2024-05-09T08:07:53.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:53.143+0000 7f02a2a00640 1 --2- 172.21.15.1:0/617328622 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f029c0c7dd0 0x7f029c0c44b0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:53.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:53.143+0000 7f02a2a00640 1 -- 172.21.15.1:0/617328622 >> 172.21.15.1:0/617328622 conn(0x7f029c0cbc10 msgr2=0x7f029c0cc020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:07:53.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:53.143+0000 7f02a2a00640 1 -- 172.21.15.1:0/617328622 shutdown_connections 2024-05-09T08:07:53.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:53.143+0000 7f02a2a00640 1 -- 172.21.15.1:0/617328622 wait complete. 2024-05-09T08:07:53.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:53.145+0000 7f02a2a00640 1 Processor -- start 2024-05-09T08:07:53.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:53.145+0000 7f02a2a00640 1 -- start start 2024-05-09T08:07:53.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:53.146+0000 7f02a2a00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f029c0c7dd0 0x7f029c14f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:07:53.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:53.146+0000 7f02a2a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f029c154790 0x7f029c0cb1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:07:53.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:53.146+0000 7f02a2a00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f029c15a440 0x7f029c1f1150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:07:53.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:53.146+0000 7f02a2a00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f029c0c75d0 con 0x7f029c15a440 2024-05-09T08:07:53.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:53.146+0000 7f02a2a00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f029c0c72f0 con 0x7f029c154790 2024-05-09T08:07:53.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:53.146+0000 7f02a2a00640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f029c0c78b0 con 0x7f029c0c7dd0 2024-05-09T08:07:53.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:53.146+0000 7f02a2000640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f029c15a440 0x7f029c1f1150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:07:53.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:53.146+0000 7f02a1600640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f029c0c7dd0 0x7f029c14f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:07:53.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:53.146+0000 7f02a2000640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f029c15a440 0x7f029c1f1150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44618/0 (socket says 172.21.15.1:44618) 2024-05-09T08:07:53.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:53.146+0000 7f02a2000640 1 -- 172.21.15.1:0/1129743861 learned_addr learned my addr 172.21.15.1:0/1129743861 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:07:53.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:53.146+0000 7f02a0c00640 1 --2- 172.21.15.1:0/1129743861 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f029c154790 0x7f029c0cb1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:07:53.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:53.147+0000 7f02a2000640 1 -- 172.21.15.1:0/1129743861 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f029c0c7dd0 msgr2=0x7f029c14f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:07:53.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:53.147+0000 7f02a2000640 1 --2- 172.21.15.1:0/1129743861 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f029c0c7dd0 0x7f029c14f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:53.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:53.147+0000 7f02a2000640 1 -- 172.21.15.1:0/1129743861 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f029c154790 msgr2=0x7f029c0cb1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:07:53.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:53.147+0000 7f02a2000640 1 --2- 172.21.15.1:0/1129743861 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f029c154790 0x7f029c0cb1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:53.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:53.147+0000 7f02a2000640 1 -- 172.21.15.1:0/1129743861 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f029c0c9ff0 con 0x7f029c15a440 2024-05-09T08:07:53.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:53.147+0000 7f02a1600640 1 --2- 172.21.15.1:0/1129743861 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f029c0c7dd0 0x7f029c14f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:07:53.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:53.147+0000 7f02a2000640 1 --2- 172.21.15.1:0/1129743861 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f029c15a440 0x7f029c1f1150 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f02940098c0 tx=0x7f02940046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:07:53.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:53.147+0000 7f0299600640 1 -- 172.21.15.1:0/1129743861 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0294041020 con 0x7f029c15a440 2024-05-09T08:07:53.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:53.147+0000 7f0299600640 1 -- 172.21.15.1:0/1129743861 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f029403d640 con 0x7f029c15a440 2024-05-09T08:07:53.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:53.147+0000 7f02a2a00640 1 -- 172.21.15.1:0/1129743861 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f029c0ca2d0 con 0x7f029c15a440 2024-05-09T08:07:53.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:53.147+0000 7f0299600640 1 -- 172.21.15.1:0/1129743861 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0294037040 con 0x7f029c15a440 2024-05-09T08:07:53.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:53.148+0000 7f02a2a00640 1 -- 172.21.15.1:0/1129743861 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f029c1f0620 con 0x7f029c15a440 2024-05-09T08:07:53.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:53.149+0000 7f02a2a00640 1 -- 172.21.15.1:0/1129743861 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f029c04f660 con 0x7f029c15a440 2024-05-09T08:07:53.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:53.150+0000 7f0299600640 1 -- 172.21.15.1:0/1129743861 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0294044020 con 0x7f029c15a440 2024-05-09T08:07:53.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:53.151+0000 7f0299600640 1 --2- 172.21.15.1:0/1129743861 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f027c06be30 0x7f027c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:07:53.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:53.151+0000 7f0299600640 1 -- 172.21.15.1:0/1129743861 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7f02940ab8a0 con 0x7f029c15a440 2024-05-09T08:07:53.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:53.151+0000 7f02a1600640 1 --2- 172.21.15.1:0/1129743861 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f027c06be30 0x7f027c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:07:53.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:53.152+0000 7f02a1600640 1 --2- 172.21.15.1:0/1129743861 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f027c06be30 0x7f027c06e2f0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f02900012e0 tx=0x7f0290001ad0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:07:53.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:53.155+0000 7f0299600640 1 -- 172.21.15.1:0/1129743861 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f029404b030 con 0x7f029c15a440 2024-05-09T08:07:53.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:53.357+0000 7f02a2a00640 1 -- 172.21.15.1:0/1129743861 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f029c14a630 con 0x7f029c15a440 2024-05-09T08:07:53.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:53.360+0000 7f0299600640 1 -- 172.21.15.1:0/1129743861 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18741 (secure 0 0 0) 0x7f0294079780 con 0x7f029c15a440 2024-05-09T08:07:53.371 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T08:07:53.371 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":27,"fsid":"54329ec8-3270-4043-9a82-c6cd7420aa7f","created":"2024-05-09T08:01:56.666970+0000","modified":"2024-05-09T08:03:02.304156+0000","last_up_change":"2024-05-09T08:02:07.146082+0000","last_in_change":"2024-05-09T08:01:59.008225+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T08:02:09.165511+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T08:02:12.925485+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T08:02:14.367455+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-09T08:02:17.967064+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c064454e-a9b5-4a24-9a77-17f0a4085952","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6800","nonce":1513349547},{"type":"v1","addr":"172.21.15.1:6801","nonce":1513349547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":1513349547},{"type":"v1","addr":"172.21.15.1:6803","nonce":1513349547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":1513349547},{"type":"v1","addr":"172.21.15.1:6807","nonce":1513349547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":1513349547},{"type":"v1","addr":"172.21.15.1:6805","nonce":1513349547}]},"public_addr":"172.21.15.1:6801/1513349547","cluster_addr":"172.21.15.1:6803/1513349547","heartbeat_back_addr":"172.21.15.1:6807/1513349547","heartbeat_front_addr":"172.21.15.1:6805/1513349547","state":["exists","up"]},{"osd":1,"uuid":"1a9ea687-2ff6-407a-949b-0e7e1083af21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":2942255259},{"type":"v1","addr":"172.21.15.1:6817","nonce":2942255259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":2942255259},{"type":"v1","addr":"172.21.15.1:6819","nonce":2942255259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":2942255259},{"type":"v1","addr":"172.21.15.1:6823","nonce":2942255259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6820","nonce":2942255259},{"type":"v1","addr":"172.21.15.1:6821","nonce":2942255259}]},"public_addr":"172.21.15.1:6817/2942255259","cluster_addr":"172.21.15.1:6819/2942255259","heartbeat_back_addr":"172.21.15.1:6823/2942255259","heartbeat_front_addr":"172.21.15.1:6821/2942255259","state":["exists","up"]},{"osd":2,"uuid":"c88af08b-7c9a-4dd8-9640-44138239addb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":2925031811},{"type":"v1","addr":"172.21.15.1:6809","nonce":2925031811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":2925031811},{"type":"v1","addr":"172.21.15.1:6811","nonce":2925031811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":2925031811},{"type":"v1","addr":"172.21.15.1:6815","nonce":2925031811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":2925031811},{"type":"v1","addr":"172.21.15.1:6813","nonce":2925031811}]},"public_addr":"172.21.15.1:6809/2925031811","cluster_addr":"172.21.15.1:6811/2925031811","heartbeat_back_addr":"172.21.15.1:6815/2925031811","heartbeat_front_addr":"172.21.15.1:6813/2925031811","state":["exists","up"]},{"osd":3,"uuid":"c087b7e3-a41d-4333-9365-32e3bd8c8e6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6824","nonce":2497776654},{"type":"v1","addr":"172.21.15.1:6825","nonce":2497776654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":2497776654},{"type":"v1","addr":"172.21.15.1:6827","nonce":2497776654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":2497776654},{"type":"v1","addr":"172.21.15.1:6831","nonce":2497776654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6828","nonce":2497776654},{"type":"v1","addr":"172.21.15.1:6829","nonce":2497776654}]},"public_addr":"172.21.15.1:6825/2497776654","cluster_addr":"172.21.15.1:6827/2497776654","heartbeat_back_addr":"172.21.15.1:6831/2497776654","heartbeat_front_addr":"172.21.15.1:6829/2497776654","state":["exists","up"]},{"osd":4,"uuid":"b4576b9c-871d-47a8-8d36-65af525d0200","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":2957026670},{"type":"v1","addr":"172.21.15.7:6801","nonce":2957026670}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2957026670},{"type":"v1","addr":"172.21.15.7:6803","nonce":2957026670}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":2957026670},{"type":"v1","addr":"172.21.15.7:6807","nonce":2957026670}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2957026670},{"type":"v1","addr":"172.21.15.7:6805","nonce":2957026670}]},"public_addr":"172.21.15.7:6801/2957026670","cluster_addr":"172.21.15.7:6803/2957026670","heartbeat_back_addr":"172.21.15.7:6807/2957026670","heartbeat_front_addr":"172.21.15.7:6805/2957026670","state":["exists","up"]},{"osd":5,"uuid":"902b3d94-72f3-4499-b21a-e32453ef5144","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1670295242},{"type":"v1","addr":"172.21.15.7:6809","nonce":1670295242}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1670295242},{"type":"v1","addr":"172.21.15.7:6811","nonce":1670295242}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1670295242},{"type":"v1","addr":"172.21.15.7:6815","nonce":1670295242}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1670295242},{"type":"v1","addr":"172.21.15.7:6813","nonce":1670295242}]},"public_addr":"172.21.15.7:6809/1670295242","cluster_addr":"172.21.15.7:6811/1670295242","heartbeat_back_addr":"172.21.15.7:6815/1670295242","heartbeat_front_addr":"172.21.15.7:6813/1670295242","state":["exists","up"]},{"osd":6,"uuid":"ead1d771-1728-4f85-adb4-26be21cedff3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":2852086880},{"type":"v1","addr":"172.21.15.7:6817","nonce":2852086880}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":2852086880},{"type":"v1","addr":"172.21.15.7:6819","nonce":2852086880}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":2852086880},{"type":"v1","addr":"172.21.15.7:6823","nonce":2852086880}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":2852086880},{"type":"v1","addr":"172.21.15.7:6821","nonce":2852086880}]},"public_addr":"172.21.15.7:6817/2852086880","cluster_addr":"172.21.15.7:6819/2852086880","heartbeat_back_addr":"172.21.15.7:6823/2852086880","heartbeat_front_addr":"172.21.15.7:6821/2852086880","state":["exists","up"]},{"osd":7,"uuid":"3ab5a201-9394-48a8-8918-e95197893669","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":1442884138},{"type":"v1","addr":"172.21.15.7:6825","nonce":1442884138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1442884138},{"type":"v1","addr":"172.21.15.7:6827","nonce":1442884138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1442884138},{"type":"v1","addr":"172.21.15.7:6831","nonce":1442884138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1442884138},{"type":"v1","addr":"172.21.15.7:6829","nonce":1442884138}]},"public_addr":"172.21.15.7:6825/1442884138","cluster_addr":"172.21.15.7:6827/1442884138","heartbeat_back_addr":"172.21.15.7:6831/1442884138","heartbeat_front_addr":"172.21.15.7:6829/1442884138","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:02:04.615675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:02:04.385974+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:02:04.619217+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:02:04.401681+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:02:04.097081+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:02:04.129981+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:02:04.225327+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:02:04.193351+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.35","mappings":[{"from":5,"to":2}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2},{"from":3,"to":1}]},{"pgid":"3.3f","mappings":[{"from":7,"to":2}]},{"pgid":"4.4","mappings":[{"from":1,"to":5}]},{"pgid":"4.e","mappings":[{"from":1,"to":2}]},{"pgid":"4.f","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T08:07:53.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:53.363+0000 7f02a2a00640 1 -- 172.21.15.1:0/1129743861 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f027c06be30 msgr2=0x7f027c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:07:53.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:53.363+0000 7f02a2a00640 1 --2- 172.21.15.1:0/1129743861 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f027c06be30 0x7f027c06e2f0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f02900012e0 tx=0x7f0290001ad0 comp rx=0 tx=0).stop 2024-05-09T08:07:53.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:53.363+0000 7f02a2a00640 1 -- 172.21.15.1:0/1129743861 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f029c15a440 msgr2=0x7f029c1f1150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:07:53.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:53.363+0000 7f02a2a00640 1 --2- 172.21.15.1:0/1129743861 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f029c15a440 0x7f029c1f1150 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f02940098c0 tx=0x7f02940046c0 comp rx=0 tx=0).stop 2024-05-09T08:07:53.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:53.363+0000 7f02a2a00640 1 -- 172.21.15.1:0/1129743861 shutdown_connections 2024-05-09T08:07:53.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:53.363+0000 7f02a2a00640 1 --2- 172.21.15.1:0/1129743861 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f027c06be30 0x7f027c06e2f0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:53.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:53.363+0000 7f02a2a00640 1 --2- 172.21.15.1:0/1129743861 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f029c154790 0x7f029c0cb1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:53.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:53.363+0000 7f02a2a00640 1 --2- 172.21.15.1:0/1129743861 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f029c15a440 0x7f029c1f1150 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:53.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:53.363+0000 7f02a2a00640 1 --2- 172.21.15.1:0/1129743861 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f029c0c7dd0 0x7f029c14f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:53.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:53.363+0000 7f02a2a00640 1 -- 172.21.15.1:0/1129743861 >> 172.21.15.1:0/1129743861 conn(0x7f029c0cbc10 msgr2=0x7f029c146ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:07:53.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:53.363+0000 7f02a2a00640 1 -- 172.21.15.1:0/1129743861 shutdown_connections 2024-05-09T08:07:53.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:53.363+0000 7f02a2a00640 1 -- 172.21.15.1:0/1129743861 wait complete. 2024-05-09T08:07:53.392 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T08:02:09.165511+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-09T08:02:12.925485+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-09T08:02:14.367455+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-09T08:02:17.967064+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-09T08:07:53.393 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-09T08:07:53.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:53.591+0000 7fa1d6000640 1 Processor -- start 2024-05-09T08:07:53.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:53.592+0000 7fa1d6000640 1 -- start start 2024-05-09T08:07:53.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:53.593+0000 7fa1d6000640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa1d00718e0 0x7fa1d015b050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:07:53.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:53.593+0000 7fa1d6000640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fa1d0072940 0x7fa1d0072dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:07:53.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:53.593+0000 7fa1d6000640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1d0073310 con 0x7fa1d00c5890 2024-05-09T08:07:53.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:53.593+0000 7fa1d6000640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fa1d015b590 con 0x7fa1d00718e0 2024-05-09T08:07:53.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:53.593+0000 7fa1d6000640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7fa1d006f060 con 0x7fa1d0072940 2024-05-09T08:07:53.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:53.594+0000 7fa1d4c00640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7fa1d00c5890 0x7fa1d00c69e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:41856/0 (socket says 172.21.15.1:41856) 2024-05-09T08:07:53.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:53.594+0000 7fa1d5600640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fa1d0072940 0x7fa1d0072dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:07:53.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:53.594+0000 7fa1d4c00640 1 -- 172.21.15.1:0/4128736419 learned_addr learned my addr 172.21.15.1:0/4128736419 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T08:07:53.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:53.594+0000 7fa1cfe00640 1 --2- 172.21.15.1:0/4128736419 >> v2:172.21.15.7:3300/0 conn(0x7fa1d00718e0 0x7fa1d015b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:07:53.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:53.594+0000 7fa1d5600640 1 -- 172.21.15.1:0/4128736419 >> v2:172.21.15.7:3300/0 conn(0x7fa1d00718e0 msgr2=0x7fa1d015b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:07:53.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:53.594+0000 7fa1d5600640 1 --2- 172.21.15.1:0/4128736419 >> v2:172.21.15.7:3300/0 conn(0x7fa1d00718e0 0x7fa1d015b050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:53.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:53.594+0000 7fa1d5600640 1 -- 172.21.15.1:0/4128736419 >> v1:172.21.15.1:6789/0 conn(0x7fa1d00c5890 legacy=0x7fa1d00c69e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:07:53.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:53.594+0000 7fa1d5600640 1 -- 172.21.15.1:0/4128736419 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1d0072150 con 0x7fa1d0072940 2024-05-09T08:07:53.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:53.595+0000 7fa1d5600640 1 --2- 172.21.15.1:0/4128736419 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fa1d0072940 0x7fa1d0072dd0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fa1c8009d90 tx=0x7fa1c802a6f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=43684465d7250194 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:07:53.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:53.595+0000 7fa1cf400640 1 -- 172.21.15.1:0/4128736419 <== mon.2 v2:172.21.15.7:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1c8007740 con 0x7fa1d0072940 2024-05-09T08:07:53.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:53.596+0000 7fa1cf400640 1 -- 172.21.15.1:0/4128736419 <== mon.2 v2:172.21.15.7:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1c80078e0 con 0x7fa1d0072940 2024-05-09T08:07:53.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:53.596+0000 7fa1cf400640 1 -- 172.21.15.1:0/4128736419 <== mon.2 v2:172.21.15.7:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1c8007cb0 con 0x7fa1d0072940 2024-05-09T08:07:53.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:53.596+0000 7fa1d6000640 1 -- 172.21.15.1:0/4128736419 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fa1d0072940 msgr2=0x7fa1d0072dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:07:53.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:53.596+0000 7fa1d6000640 1 --2- 172.21.15.1:0/4128736419 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fa1d0072940 0x7fa1d0072dd0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fa1c8009d90 tx=0x7fa1c802a6f0 comp rx=0 tx=0).stop 2024-05-09T08:07:53.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:53.596+0000 7fa1d6000640 1 -- 172.21.15.1:0/4128736419 shutdown_connections 2024-05-09T08:07:53.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:53.596+0000 7fa1d6000640 1 --2- 172.21.15.1:0/4128736419 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fa1d0072940 0x7fa1d0072dd0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:53.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:53.596+0000 7fa1d6000640 1 --2- 172.21.15.1:0/4128736419 >> v2:172.21.15.7:3300/0 conn(0x7fa1d00718e0 0x7fa1d015b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:53.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:53.596+0000 7fa1d6000640 1 -- 172.21.15.1:0/4128736419 >> 172.21.15.1:0/4128736419 conn(0x7fa1d006b250 msgr2=0x7fa1d006b630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:07:53.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:53.596+0000 7fa1d6000640 1 -- 172.21.15.1:0/4128736419 shutdown_connections 2024-05-09T08:07:53.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:53.596+0000 7fa1d6000640 1 -- 172.21.15.1:0/4128736419 wait complete. 2024-05-09T08:07:53.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:53.598+0000 7fa1d6000640 1 Processor -- start 2024-05-09T08:07:53.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:53.598+0000 7fa1d6000640 1 -- start start 2024-05-09T08:07:53.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:53.599+0000 7fa1d6000640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa1d00718e0 0x7fa1d015dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:07:53.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:53.599+0000 7fa1d6000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1d0072940 0x7fa1d0153950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:07:53.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:53.599+0000 7fa1d6000640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fa1d00c5890 0x7fa1d01e3f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:07:53.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:53.599+0000 7fa1d6000640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fa1d006f5e0 con 0x7fa1d00718e0 2024-05-09T08:07:53.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:53.599+0000 7fa1d6000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa1d006f300 con 0x7fa1d0072940 2024-05-09T08:07:53.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:53.599+0000 7fa1d6000640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7fa1d006f8c0 con 0x7fa1d00c5890 2024-05-09T08:07:53.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:53.599+0000 7fa1cfe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1d0072940 0x7fa1d0153950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:07:53.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:53.600+0000 7fa1cfe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1d0072940 0x7fa1d0153950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.1:44818/0 (socket says 172.21.15.1:44818) 2024-05-09T08:07:53.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:53.600+0000 7fa1cfe00640 1 -- 172.21.15.1:0/1723963107 learned_addr learned my addr 172.21.15.1:0/1723963107 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:07:53.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:53.600+0000 7fa1d5600640 1 --2- 172.21.15.1:0/1723963107 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fa1d00c5890 0x7fa1d01e3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:07:53.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:53.600+0000 7fa1d4c00640 1 --2- 172.21.15.1:0/1723963107 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa1d00718e0 0x7fa1d015dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:07:53.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:53.600+0000 7fa1cfe00640 1 -- 172.21.15.1:0/1723963107 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fa1d00c5890 msgr2=0x7fa1d01e3f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:07:53.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:53.600+0000 7fa1cfe00640 1 --2- 172.21.15.1:0/1723963107 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fa1d00c5890 0x7fa1d01e3f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:53.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:53.600+0000 7fa1cfe00640 1 -- 172.21.15.1:0/1723963107 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa1d00718e0 msgr2=0x7fa1d015dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:07:53.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:53.600+0000 7fa1cfe00640 1 --2- 172.21.15.1:0/1723963107 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa1d00718e0 0x7fa1d015dc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:53.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:53.600+0000 7fa1cfe00640 1 -- 172.21.15.1:0/1723963107 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1d01f00a0 con 0x7fa1d0072940 2024-05-09T08:07:53.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:53.600+0000 7fa1cfe00640 1 --2- 172.21.15.1:0/1723963107 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1d0072940 0x7fa1d0153950 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fa1c000e8a0 tx=0x7fa1c000ed60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:07:53.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:53.601+0000 7fa1ccc00640 1 -- 172.21.15.1:0/1723963107 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1c0017070 con 0x7fa1d0072940 2024-05-09T08:07:53.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:53.601+0000 7fa1ccc00640 1 -- 172.21.15.1:0/1723963107 <== mon.1 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1c000c940 con 0x7fa1d0072940 2024-05-09T08:07:53.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:53.601+0000 7fa1d6000640 1 -- 172.21.15.1:0/1723963107 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1d015cbe0 con 0x7fa1d0072940 2024-05-09T08:07:53.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:53.601+0000 7fa1ccc00640 1 -- 172.21.15.1:0/1723963107 <== mon.1 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1c000cd30 con 0x7fa1d0072940 2024-05-09T08:07:53.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:53.601+0000 7fa1d6000640 1 -- 172.21.15.1:0/1723963107 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1d015d1a0 con 0x7fa1d0072940 2024-05-09T08:07:53.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:53.602+0000 7fa1d6000640 1 -- 172.21.15.1:0/1723963107 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1d004f660 con 0x7fa1d0072940 2024-05-09T08:07:53.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:53.603+0000 7fa1ccc00640 1 -- 172.21.15.1:0/1723963107 <== mon.1 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa1c001a070 con 0x7fa1d0072940 2024-05-09T08:07:53.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:53.604+0000 7fa1ccc00640 1 --2- 172.21.15.1:0/1723963107 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fa1b406be30 0x7fa1b406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:07:53.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:53.604+0000 7fa1ccc00640 1 -- 172.21.15.1:0/1723963107 <== mon.1 v2:172.21.15.7:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7fa1c008b3e0 con 0x7fa1d0072940 2024-05-09T08:07:53.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:53.604+0000 7fa1d4c00640 1 --2- 172.21.15.1:0/1723963107 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fa1b406be30 0x7fa1b406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:07:53.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:53.605+0000 7fa1d4c00640 1 --2- 172.21.15.1:0/1723963107 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fa1b406be30 0x7fa1b406e2f0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fa1c4007fd0 tx=0x7fa1c4007e80 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:07:53.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:53.609+0000 7fa1ccc00640 1 -- 172.21.15.1:0/1723963107 <== mon.1 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fa1c0059340 con 0x7fa1d0072940 2024-05-09T08:07:53.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:53.810+0000 7fa1d6000640 1 -- 172.21.15.1:0/1723963107 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa1d00c7520 con 0x7fa1d0072940 2024-05-09T08:07:53.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:53.811+0000 7fa1ccc00640 1 -- 172.21.15.1:0/1723963107 <== mon.1 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) ==== 93+0+10 (secure 0 0 0) 0x7fa1c005d640 con 0x7fa1d0072940 2024-05-09T08:07:53.822 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 1 2024-05-09T08:07:53.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:53.814+0000 7fa1d6000640 1 -- 172.21.15.1:0/1723963107 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fa1b406be30 msgr2=0x7fa1b406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:07:53.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:53.814+0000 7fa1d6000640 1 --2- 172.21.15.1:0/1723963107 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fa1b406be30 0x7fa1b406e2f0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fa1c4007fd0 tx=0x7fa1c4007e80 comp rx=0 tx=0).stop 2024-05-09T08:07:53.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:53.814+0000 7fa1d6000640 1 -- 172.21.15.1:0/1723963107 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1d0072940 msgr2=0x7fa1d0153950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:07:53.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:53.814+0000 7fa1d6000640 1 --2- 172.21.15.1:0/1723963107 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1d0072940 0x7fa1d0153950 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fa1c000e8a0 tx=0x7fa1c000ed60 comp rx=0 tx=0).stop 2024-05-09T08:07:53.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:53.814+0000 7fa1d6000640 1 -- 172.21.15.1:0/1723963107 shutdown_connections 2024-05-09T08:07:53.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:53.814+0000 7fa1d6000640 1 --2- 172.21.15.1:0/1723963107 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fa1b406be30 0x7fa1b406e2f0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:53.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:53.814+0000 7fa1d6000640 1 --2- 172.21.15.1:0/1723963107 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1d0072940 0x7fa1d0153950 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:53.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:53.814+0000 7fa1d6000640 1 --2- 172.21.15.1:0/1723963107 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fa1d00c5890 0x7fa1d01e3f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:53.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:53.814+0000 7fa1d6000640 1 --2- 172.21.15.1:0/1723963107 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa1d00718e0 0x7fa1d015dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:53.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:53.814+0000 7fa1d6000640 1 -- 172.21.15.1:0/1723963107 >> 172.21.15.1:0/1723963107 conn(0x7fa1d006b250 msgr2=0x7fa1d00c7740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:07:53.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:53.814+0000 7fa1d6000640 1 -- 172.21.15.1:0/1723963107 shutdown_connections 2024-05-09T08:07:53.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:53.814+0000 7fa1d6000640 1 -- 172.21.15.1:0/1723963107 wait complete. 2024-05-09T08:07:53.844 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-09T08:07:54.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.045+0000 7f1ba5000640 1 Processor -- start 2024-05-09T08:07:54.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.046+0000 7f1ba5000640 1 -- start start 2024-05-09T08:07:54.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.047+0000 7f1ba5000640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f1ba0154910 0x7f1ba00c7d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:07:54.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.047+0000 7f1ba5000640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f1ba0072890 0x7f1ba015a880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:07:54.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.047+0000 7f1ba5000640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ba0073750 con 0x7f1ba00c9750 2024-05-09T08:07:54.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.047+0000 7f1ba5000640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f1ba0073190 con 0x7f1ba0072890 2024-05-09T08:07:54.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.047+0000 7f1ba5000640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f1ba0073470 con 0x7f1ba0154910 2024-05-09T08:07:54.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.048+0000 7f1b9de00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f1ba0154910 0x7f1ba00c7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:07:54.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.048+0000 7f1b9d400640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f1ba00c9750 0x7f1ba006f720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.1:38670/0 (socket says 172.21.15.1:38670) 2024-05-09T08:07:54.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.048+0000 7f1b9d400640 1 -- 172.21.15.1:0/3147082182 learned_addr learned my addr 172.21.15.1:0/3147082182 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T08:07:54.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.048+0000 7f1b9e800640 1 --2- 172.21.15.1:0/3147082182 >> v2:172.21.15.7:3300/0 conn(0x7f1ba0072890 0x7f1ba015a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:07:54.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.048+0000 7f1b9de00640 1 -- 172.21.15.1:0/3147082182 >> v2:172.21.15.7:3300/0 conn(0x7f1ba0072890 msgr2=0x7f1ba015a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:07:54.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.048+0000 7f1b9de00640 1 --2- 172.21.15.1:0/3147082182 >> v2:172.21.15.7:3300/0 conn(0x7f1ba0072890 0x7f1ba015a880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:54.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.048+0000 7f1b9ca00640 1 -- 172.21.15.1:0/3147082182 <== mon.1 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2550627954 0 0) 0x7f1ba0073750 con 0x7f1ba00c9750 2024-05-09T08:07:54.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.048+0000 7f1b9de00640 1 -- 172.21.15.1:0/3147082182 >> v1:172.21.15.7:6789/0 conn(0x7f1ba00c9750 legacy=0x7f1ba006f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:07:54.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.048+0000 7f1b9de00640 1 -- 172.21.15.1:0/3147082182 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ba0071d90 con 0x7f1ba0154910 2024-05-09T08:07:54.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.050+0000 7f1b9de00640 1 --2- 172.21.15.1:0/3147082182 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f1ba0154910 0x7f1ba00c7d20 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f1b900097a0 tx=0x7f1b9002a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=49d1c54e2e9b44dd server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:07:54.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.050+0000 7f1b9ca00640 1 -- 172.21.15.1:0/3147082182 <== mon.2 v2:172.21.15.7:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b90037070 con 0x7f1ba0154910 2024-05-09T08:07:54.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.050+0000 7f1b9ca00640 1 -- 172.21.15.1:0/3147082182 <== mon.2 v2:172.21.15.7:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b90032440 con 0x7f1ba0154910 2024-05-09T08:07:54.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.050+0000 7f1b9ca00640 1 -- 172.21.15.1:0/3147082182 <== mon.2 v2:172.21.15.7:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b90032810 con 0x7f1ba0154910 2024-05-09T08:07:54.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.051+0000 7f1ba5000640 1 -- 172.21.15.1:0/3147082182 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f1ba0154910 msgr2=0x7f1ba00c7d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:07:54.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.051+0000 7f1ba5000640 1 --2- 172.21.15.1:0/3147082182 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f1ba0154910 0x7f1ba00c7d20 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f1b900097a0 tx=0x7f1b9002a210 comp rx=0 tx=0).stop 2024-05-09T08:07:54.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.051+0000 7f1ba5000640 1 -- 172.21.15.1:0/3147082182 shutdown_connections 2024-05-09T08:07:54.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.051+0000 7f1ba5000640 1 --2- 172.21.15.1:0/3147082182 >> v2:172.21.15.7:3300/0 conn(0x7f1ba0072890 0x7f1ba015a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:54.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.051+0000 7f1ba5000640 1 --2- 172.21.15.1:0/3147082182 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f1ba0154910 0x7f1ba00c7d20 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:54.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.051+0000 7f1ba5000640 1 -- 172.21.15.1:0/3147082182 >> 172.21.15.1:0/3147082182 conn(0x7f1ba006dad0 msgr2=0x7f1ba006deb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:07:54.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.051+0000 7f1ba5000640 1 -- 172.21.15.1:0/3147082182 shutdown_connections 2024-05-09T08:07:54.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.051+0000 7f1ba5000640 1 -- 172.21.15.1:0/3147082182 wait complete. 2024-05-09T08:07:54.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.053+0000 7f1ba5000640 1 Processor -- start 2024-05-09T08:07:54.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.053+0000 7f1ba5000640 1 -- start start 2024-05-09T08:07:54.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.054+0000 7f1ba5000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ba0072890 0x7f1ba014a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:07:54.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.054+0000 7f1ba5000640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1ba00c9750 0x7f1ba0157090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:07:54.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.054+0000 7f1ba5000640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f1ba0154910 0x7f1ba0147eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:07:54.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.054+0000 7f1ba5000640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f1ba006ffb0 con 0x7f1ba00c9750 2024-05-09T08:07:54.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.054+0000 7f1ba5000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1ba006fe30 con 0x7f1ba0072890 2024-05-09T08:07:54.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.054+0000 7f1ba5000640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f1ba015adc0 con 0x7f1ba0154910 2024-05-09T08:07:54.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.054+0000 7f1b9d400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1ba00c9750 0x7f1ba0157090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:07:54.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.054+0000 7f1b9d400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1ba00c9750 0x7f1ba0157090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60524/0 (socket says 172.21.15.1:60524) 2024-05-09T08:07:54.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.054+0000 7f1b9d400640 1 -- 172.21.15.1:0/1798608249 learned_addr learned my addr 172.21.15.1:0/1798608249 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:07:54.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.054+0000 7f1b9de00640 1 --2- 172.21.15.1:0/1798608249 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ba0072890 0x7f1ba014a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:07:54.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.054+0000 7f1b9e800640 1 --2- 172.21.15.1:0/1798608249 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f1ba0154910 0x7f1ba0147eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:07:54.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.055+0000 7f1b9de00640 1 -- 172.21.15.1:0/1798608249 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f1ba0154910 msgr2=0x7f1ba0147eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:07:54.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.055+0000 7f1b9de00640 1 --2- 172.21.15.1:0/1798608249 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f1ba0154910 0x7f1ba0147eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:54.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.055+0000 7f1b9de00640 1 -- 172.21.15.1:0/1798608249 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1ba00c9750 msgr2=0x7f1ba0157090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:07:54.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.055+0000 7f1b9de00640 1 --2- 172.21.15.1:0/1798608249 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1ba00c9750 0x7f1ba0157090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:54.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.055+0000 7f1b9de00640 1 -- 172.21.15.1:0/1798608249 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ba00cba30 con 0x7f1ba0072890 2024-05-09T08:07:54.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.055+0000 7f1b9d400640 1 --2- 172.21.15.1:0/1798608249 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1ba00c9750 0x7f1ba0157090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:07:54.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.055+0000 7f1b9de00640 1 --2- 172.21.15.1:0/1798608249 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ba0072890 0x7f1ba014a510 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f1ba00735f0 tx=0x7f1b900046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:07:54.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.055+0000 7f1b96000640 1 -- 172.21.15.1:0/1798608249 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b90037050 con 0x7f1ba0072890 2024-05-09T08:07:54.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.055+0000 7f1b96000640 1 -- 172.21.15.1:0/1798608249 <== mon.1 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b90038040 con 0x7f1ba0072890 2024-05-09T08:07:54.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.055+0000 7f1b96000640 1 -- 172.21.15.1:0/1798608249 <== mon.1 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b9003c8f0 con 0x7f1ba0072890 2024-05-09T08:07:54.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.056+0000 7f1ba5000640 1 -- 172.21.15.1:0/1798608249 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ba00cbcd0 con 0x7f1ba0072890 2024-05-09T08:07:54.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.056+0000 7f1ba5000640 1 -- 172.21.15.1:0/1798608249 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ba00cc290 con 0x7f1ba0072890 2024-05-09T08:07:54.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.057+0000 7f1ba5000640 1 -- 172.21.15.1:0/1798608249 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ba00723f0 con 0x7f1ba0072890 2024-05-09T08:07:54.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.058+0000 7f1b96000640 1 -- 172.21.15.1:0/1798608249 <== mon.1 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1b90045020 con 0x7f1ba0072890 2024-05-09T08:07:54.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.059+0000 7f1b96000640 1 --2- 172.21.15.1:0/1798608249 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f1b7c06be60 0x7f1b7c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:07:54.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.059+0000 7f1b96000640 1 -- 172.21.15.1:0/1798608249 <== mon.1 v2:172.21.15.7:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7f1b900ab380 con 0x7f1ba0072890 2024-05-09T08:07:54.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.059+0000 7f1b9d400640 1 --2- 172.21.15.1:0/1798608249 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f1b7c06be60 0x7f1b7c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:07:54.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.060+0000 7f1b9d400640 1 --2- 172.21.15.1:0/1798608249 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f1b7c06be60 0x7f1b7c06e320 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f1ba015b5d0 tx=0x7f1b88007de0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:07:54.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.063+0000 7f1b96000640 1 -- 172.21.15.1:0/1798608249 <== mon.1 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f1ba00723f0 con 0x7f1ba0072890 2024-05-09T08:07:54.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.254+0000 7f1ba5000640 1 -- 172.21.15.1:0/1798608249 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f1ba0074d70 con 0x7f1ba0072890 2024-05-09T08:07:54.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.256+0000 7f1b96000640 1 -- 172.21.15.1:0/1798608249 <== mon.1 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) ==== 104+0+11 (secure 0 0 0) 0x7f1b9004c020 con 0x7f1ba0072890 2024-05-09T08:07:54.266 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 32 2024-05-09T08:07:54.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.258+0000 7f1ba5000640 1 -- 172.21.15.1:0/1798608249 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f1b7c06be60 msgr2=0x7f1b7c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:07:54.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.258+0000 7f1ba5000640 1 --2- 172.21.15.1:0/1798608249 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f1b7c06be60 0x7f1b7c06e320 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f1ba015b5d0 tx=0x7f1b88007de0 comp rx=0 tx=0).stop 2024-05-09T08:07:54.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.258+0000 7f1ba5000640 1 -- 172.21.15.1:0/1798608249 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ba0072890 msgr2=0x7f1ba014a510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:07:54.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.258+0000 7f1ba5000640 1 --2- 172.21.15.1:0/1798608249 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ba0072890 0x7f1ba014a510 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f1ba00735f0 tx=0x7f1b900046c0 comp rx=0 tx=0).stop 2024-05-09T08:07:54.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.258+0000 7f1ba5000640 1 -- 172.21.15.1:0/1798608249 shutdown_connections 2024-05-09T08:07:54.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.258+0000 7f1ba5000640 1 --2- 172.21.15.1:0/1798608249 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f1b7c06be60 0x7f1b7c06e320 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:54.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.258+0000 7f1ba5000640 1 --2- 172.21.15.1:0/1798608249 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f1ba0154910 0x7f1ba0147eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:54.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.258+0000 7f1ba5000640 1 --2- 172.21.15.1:0/1798608249 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1ba00c9750 0x7f1ba0157090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:54.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.258+0000 7f1ba5000640 1 --2- 172.21.15.1:0/1798608249 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ba0072890 0x7f1ba014a510 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:54.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.258+0000 7f1ba5000640 1 -- 172.21.15.1:0/1798608249 >> 172.21.15.1:0/1798608249 conn(0x7f1ba006dad0 msgr2=0x7f1ba0152110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:07:54.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.258+0000 7f1ba5000640 1 -- 172.21.15.1:0/1798608249 shutdown_connections 2024-05-09T08:07:54.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.258+0000 7f1ba5000640 1 -- 172.21.15.1:0/1798608249 wait complete. 2024-05-09T08:07:54.287 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-09T08:07:54.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.483+0000 7fc11f400640 1 Processor -- start 2024-05-09T08:07:54.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.483+0000 7fc11f400640 1 -- start start 2024-05-09T08:07:54.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.485+0000 7fc11f400640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fc11815c360 0x7fc1180c9570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:07:54.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.485+0000 7fc11f400640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fc118072c20 0x7fc118073090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:07:54.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.485+0000 7fc11f400640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc118071c80 con 0x7fc1180c5e40 2024-05-09T08:07:54.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.485+0000 7fc11f400640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7fc11815b980 con 0x7fc11815c360 2024-05-09T08:07:54.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.485+0000 7fc11f400640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7fc118071f20 con 0x7fc118072c20 2024-05-09T08:07:54.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.485+0000 7fc11ca00640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fc11815c360 0x7fc1180c9570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:07:54.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.485+0000 7fc11ca00640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fc11815c360 0x7fc1180c9570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60530/0 (socket says 172.21.15.1:60530) 2024-05-09T08:07:54.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.485+0000 7fc11ca00640 1 -- 172.21.15.1:0/1172328198 learned_addr learned my addr 172.21.15.1:0/1172328198 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:07:54.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.485+0000 7fc11d400640 1 --2- 172.21.15.1:0/1172328198 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fc118072c20 0x7fc118073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:07:54.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.486+0000 7fc11ca00640 1 -- 172.21.15.1:0/1172328198 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fc118072c20 msgr2=0x7fc118073090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:07:54.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.486+0000 7fc11ca00640 1 --2- 172.21.15.1:0/1172328198 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fc118072c20 0x7fc118073090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:54.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.486+0000 7fc11ca00640 1 -- 172.21.15.1:0/1172328198 >> v1:172.21.15.1:6789/0 conn(0x7fc1180c5e40 legacy=0x7fc1180cc5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:07:54.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.486+0000 7fc11ca00640 1 -- 172.21.15.1:0/1172328198 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1180721e0 con 0x7fc11815c360 2024-05-09T08:07:54.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.487+0000 7fc11ca00640 1 --2- 172.21.15.1:0/1172328198 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc11815c360 0x7fc1180c9570 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fc108009870 tx=0x7fc10802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a64fc212e3610da server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:07:54.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.487+0000 7fc117400640 1 -- 172.21.15.1:0/1172328198 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc108037070 con 0x7fc11815c360 2024-05-09T08:07:54.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.487+0000 7fc117400640 1 -- 172.21.15.1:0/1172328198 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc108032440 con 0x7fc11815c360 2024-05-09T08:07:54.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.487+0000 7fc117400640 1 -- 172.21.15.1:0/1172328198 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc108032810 con 0x7fc11815c360 2024-05-09T08:07:54.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.488+0000 7fc11f400640 1 -- 172.21.15.1:0/1172328198 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc11815c360 msgr2=0x7fc1180c9570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:07:54.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.488+0000 7fc11f400640 1 --2- 172.21.15.1:0/1172328198 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc11815c360 0x7fc1180c9570 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fc108009870 tx=0x7fc10802a140 comp rx=0 tx=0).stop 2024-05-09T08:07:54.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.488+0000 7fc11f400640 1 -- 172.21.15.1:0/1172328198 shutdown_connections 2024-05-09T08:07:54.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.488+0000 7fc11f400640 1 --2- 172.21.15.1:0/1172328198 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fc118072c20 0x7fc118073090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:54.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.488+0000 7fc11f400640 1 --2- 172.21.15.1:0/1172328198 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc11815c360 0x7fc1180c9570 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:54.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.488+0000 7fc11f400640 1 -- 172.21.15.1:0/1172328198 >> 172.21.15.1:0/1172328198 conn(0x7fc11806dad0 msgr2=0x7fc11806deb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:07:54.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.488+0000 7fc11f400640 1 -- 172.21.15.1:0/1172328198 shutdown_connections 2024-05-09T08:07:54.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.488+0000 7fc11f400640 1 -- 172.21.15.1:0/1172328198 wait complete. 2024-05-09T08:07:54.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.490+0000 7fc11f400640 1 Processor -- start 2024-05-09T08:07:54.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.490+0000 7fc11f400640 1 -- start start 2024-05-09T08:07:54.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.491+0000 7fc11f400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc118072c20 0x7fc11814ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:07:54.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.491+0000 7fc11f400640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fc1180c5e40 0x7fc1180c29a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:07:54.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.491+0000 7fc11f400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc11815c360 0x7fc1180c1430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:07:54.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.491+0000 7fc11f400640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fc1180cc0d0 con 0x7fc11815c360 2024-05-09T08:07:54.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.491+0000 7fc11f400640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc1180727a0 con 0x7fc118072c20 2024-05-09T08:07:54.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.491+0000 7fc11f400640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7fc1180ccd00 con 0x7fc1180c5e40 2024-05-09T08:07:54.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.491+0000 7fc11d400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc11815c360 0x7fc1180c1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:07:54.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.491+0000 7fc11d400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc11815c360 0x7fc1180c1430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60536/0 (socket says 172.21.15.1:60536) 2024-05-09T08:07:54.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.491+0000 7fc11d400640 1 -- 172.21.15.1:0/455394279 learned_addr learned my addr 172.21.15.1:0/455394279 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:07:54.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.491+0000 7fc11ca00640 1 --2- 172.21.15.1:0/455394279 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc118072c20 0x7fc11814ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:07:54.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.491+0000 7fc117e00640 1 --2- 172.21.15.1:0/455394279 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fc1180c5e40 0x7fc1180c29a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:07:54.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.492+0000 7fc11d400640 1 -- 172.21.15.1:0/455394279 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fc1180c5e40 msgr2=0x7fc1180c29a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:07:54.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.492+0000 7fc11d400640 1 --2- 172.21.15.1:0/455394279 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fc1180c5e40 0x7fc1180c29a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:54.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.492+0000 7fc11d400640 1 -- 172.21.15.1:0/455394279 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc118072c20 msgr2=0x7fc11814ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:07:54.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.492+0000 7fc11d400640 1 --2- 172.21.15.1:0/455394279 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc118072c20 0x7fc11814ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:54.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.492+0000 7fc11d400640 1 -- 172.21.15.1:0/455394279 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc11814b390 con 0x7fc11815c360 2024-05-09T08:07:54.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.492+0000 7fc117e00640 1 --2- 172.21.15.1:0/455394279 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fc1180c5e40 0x7fc1180c29a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T08:07:54.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.492+0000 7fc11d400640 1 --2- 172.21.15.1:0/455394279 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc11815c360 0x7fc1180c1430 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fc110009d30 tx=0x7fc11000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:07:54.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.492+0000 7fc114c00640 1 -- 172.21.15.1:0/455394279 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc110018020 con 0x7fc11815c360 2024-05-09T08:07:54.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.492+0000 7fc114c00640 1 -- 172.21.15.1:0/455394279 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc110014480 con 0x7fc11815c360 2024-05-09T08:07:54.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.492+0000 7fc114c00640 1 -- 172.21.15.1:0/455394279 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc110014870 con 0x7fc11815c360 2024-05-09T08:07:54.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.492+0000 7fc11f400640 1 -- 172.21.15.1:0/455394279 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc11814b600 con 0x7fc11815c360 2024-05-09T08:07:54.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.493+0000 7fc11f400640 1 -- 172.21.15.1:0/455394279 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1181e29c0 con 0x7fc11815c360 2024-05-09T08:07:54.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.494+0000 7fc11f400640 1 -- 172.21.15.1:0/455394279 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc118074d70 con 0x7fc11815c360 2024-05-09T08:07:54.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.495+0000 7fc114c00640 1 -- 172.21.15.1:0/455394279 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc110005d10 con 0x7fc11815c360 2024-05-09T08:07:54.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.496+0000 7fc114c00640 1 --2- 172.21.15.1:0/455394279 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fc0f806be30 0x7fc0f806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:07:54.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.496+0000 7fc114c00640 1 -- 172.21.15.1:0/455394279 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7fc11008bfe0 con 0x7fc11815c360 2024-05-09T08:07:54.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.496+0000 7fc11ca00640 1 --2- 172.21.15.1:0/455394279 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fc0f806be30 0x7fc0f806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:07:54.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.497+0000 7fc11ca00640 1 --2- 172.21.15.1:0/455394279 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fc0f806be30 0x7fc0f806e2f0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fc108034000 tx=0x7fc10802e000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:07:54.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.500+0000 7fc114c00640 1 -- 172.21.15.1:0/455394279 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fc110022030 con 0x7fc11815c360 2024-05-09T08:07:54.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.702+0000 7fc11f400640 1 -- 172.21.15.1:0/455394279 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fc118073210 con 0x7fc11815c360 2024-05-09T08:07:54.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.704+0000 7fc114c00640 1 -- 172.21.15.1:0/455394279 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) ==== 100+0+11 (secure 0 0 0) 0x7fc110010040 con 0x7fc11815c360 2024-05-09T08:07:54.714 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 64 2024-05-09T08:07:54.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.706+0000 7fc11f400640 1 -- 172.21.15.1:0/455394279 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fc0f806be30 msgr2=0x7fc0f806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:07:54.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.707+0000 7fc11f400640 1 --2- 172.21.15.1:0/455394279 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fc0f806be30 0x7fc0f806e2f0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fc108034000 tx=0x7fc10802e000 comp rx=0 tx=0).stop 2024-05-09T08:07:54.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.707+0000 7fc11f400640 1 -- 172.21.15.1:0/455394279 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc11815c360 msgr2=0x7fc1180c1430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:07:54.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.707+0000 7fc11f400640 1 --2- 172.21.15.1:0/455394279 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc11815c360 0x7fc1180c1430 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fc110009d30 tx=0x7fc11000c490 comp rx=0 tx=0).stop 2024-05-09T08:07:54.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.707+0000 7fc11f400640 1 -- 172.21.15.1:0/455394279 shutdown_connections 2024-05-09T08:07:54.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.707+0000 7fc11f400640 1 --2- 172.21.15.1:0/455394279 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fc0f806be30 0x7fc0f806e2f0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:54.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.707+0000 7fc11f400640 1 --2- 172.21.15.1:0/455394279 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc11815c360 0x7fc1180c1430 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:54.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.707+0000 7fc11f400640 1 --2- 172.21.15.1:0/455394279 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fc1180c5e40 0x7fc1180c29a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:54.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.707+0000 7fc11f400640 1 --2- 172.21.15.1:0/455394279 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc118072c20 0x7fc11814ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:54.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.707+0000 7fc11f400640 1 -- 172.21.15.1:0/455394279 >> 172.21.15.1:0/455394279 conn(0x7fc11806dad0 msgr2=0x7fc1180c6d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:07:54.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.707+0000 7fc11f400640 1 -- 172.21.15.1:0/455394279 shutdown_connections 2024-05-09T08:07:54.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.707+0000 7fc11f400640 1 -- 172.21.15.1:0/455394279 wait complete. 2024-05-09T08:07:54.736 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-09T08:07:54.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.943+0000 7f4dc6800640 1 Processor -- start 2024-05-09T08:07:54.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.944+0000 7f4dc6800640 1 -- start start 2024-05-09T08:07:54.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.945+0000 7f4dc6800640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f4dc0154910 0x7f4dc00c7d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:07:54.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.945+0000 7f4dc6800640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f4dc0072890 0x7f4dc015a880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:07:54.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.945+0000 7f4dc6800640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4dc0073750 con 0x7f4dc00c9750 2024-05-09T08:07:54.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.945+0000 7f4dc6800640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f4dc0073190 con 0x7f4dc0154910 2024-05-09T08:07:54.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.945+0000 7f4dc6800640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f4dc0073470 con 0x7f4dc0072890 2024-05-09T08:07:54.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.946+0000 7f4dbf400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f4dc0154910 0x7f4dc00c7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:07:54.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.946+0000 7f4dbf400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f4dc0154910 0x7f4dc00c7d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.1:53996/0 (socket says 172.21.15.1:53996) 2024-05-09T08:07:54.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.946+0000 7f4dbf400640 1 -- 172.21.15.1:0/2818842893 learned_addr learned my addr 172.21.15.1:0/2818842893 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:07:54.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.946+0000 7f4dbfe00640 1 --2- 172.21.15.1:0/2818842893 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f4dc0072890 0x7f4dc015a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:07:54.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.946+0000 7f4dbf400640 1 -- 172.21.15.1:0/2818842893 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f4dc0072890 msgr2=0x7f4dc015a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:07:54.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.946+0000 7f4dbf400640 1 --2- 172.21.15.1:0/2818842893 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f4dc0072890 0x7f4dc015a880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:54.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.946+0000 7f4dbf400640 1 -- 172.21.15.1:0/2818842893 >> v1:172.21.15.1:6789/0 conn(0x7f4dc00c9750 legacy=0x7f4dc006f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:07:54.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.946+0000 7f4dbf400640 1 -- 172.21.15.1:0/2818842893 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4dc0071d90 con 0x7f4dc0154910 2024-05-09T08:07:54.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.947+0000 7f4dbf400640 1 --2- 172.21.15.1:0/2818842893 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4dc0154910 0x7f4dc00c7d20 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f4db0009870 tx=0x7f4db002a2f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dab0a1c919dc518f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:07:54.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.948+0000 7f4dbea00640 1 -- 172.21.15.1:0/2818842893 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4db0037070 con 0x7f4dc0154910 2024-05-09T08:07:54.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.948+0000 7f4dbea00640 1 -- 172.21.15.1:0/2818842893 <== mon.1 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4db0032640 con 0x7f4dc0154910 2024-05-09T08:07:54.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.948+0000 7f4dbea00640 1 -- 172.21.15.1:0/2818842893 <== mon.1 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4db0032a10 con 0x7f4dc0154910 2024-05-09T08:07:54.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.948+0000 7f4dc6800640 1 -- 172.21.15.1:0/2818842893 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4dc0154910 msgr2=0x7f4dc00c7d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:07:54.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.948+0000 7f4dc6800640 1 --2- 172.21.15.1:0/2818842893 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4dc0154910 0x7f4dc00c7d20 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f4db0009870 tx=0x7f4db002a2f0 comp rx=0 tx=0).stop 2024-05-09T08:07:54.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.948+0000 7f4dc6800640 1 -- 172.21.15.1:0/2818842893 shutdown_connections 2024-05-09T08:07:54.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.948+0000 7f4dc6800640 1 --2- 172.21.15.1:0/2818842893 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f4dc0072890 0x7f4dc015a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:54.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.948+0000 7f4dc6800640 1 --2- 172.21.15.1:0/2818842893 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4dc0154910 0x7f4dc00c7d20 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:54.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.948+0000 7f4dc6800640 1 -- 172.21.15.1:0/2818842893 >> 172.21.15.1:0/2818842893 conn(0x7f4dc006dad0 msgr2=0x7f4dc006deb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:07:54.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.949+0000 7f4dc6800640 1 -- 172.21.15.1:0/2818842893 shutdown_connections 2024-05-09T08:07:54.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.949+0000 7f4dc6800640 1 -- 172.21.15.1:0/2818842893 wait complete. 2024-05-09T08:07:54.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.951+0000 7f4dc6800640 1 Processor -- start 2024-05-09T08:07:54.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.951+0000 7f4dc6800640 1 -- start start 2024-05-09T08:07:54.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.952+0000 7f4dc6800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4dc0072890 0x7f4dc0155c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:07:54.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.952+0000 7f4dc6800640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4dc00c9750 0x7f4dc01f36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:07:54.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.952+0000 7f4dc6800640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f4dc0154910 0x7f4dc01efee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:07:54.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.952+0000 7f4dc6800640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f4dc006ffb0 con 0x7f4dc00c9750 2024-05-09T08:07:54.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.952+0000 7f4dc6800640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f4dc006fe30 con 0x7f4dc0072890 2024-05-09T08:07:54.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.952+0000 7f4dc6800640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f4dc015adc0 con 0x7f4dc0154910 2024-05-09T08:07:54.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.952+0000 7f4db7e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4dc00c9750 0x7f4dc01f36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:07:54.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.952+0000 7f4db7e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4dc00c9750 0x7f4dc01f36c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60550/0 (socket says 172.21.15.1:60550) 2024-05-09T08:07:54.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.952+0000 7f4db7e00640 1 -- 172.21.15.1:0/4018444271 learned_addr learned my addr 172.21.15.1:0/4018444271 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:07:54.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.952+0000 7f4dbf400640 1 --2- 172.21.15.1:0/4018444271 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4dc0072890 0x7f4dc0155c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:07:54.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.953+0000 7f4dbfe00640 1 --2- 172.21.15.1:0/4018444271 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f4dc0154910 0x7f4dc01efee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:07:54.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.953+0000 7f4dbf400640 1 -- 172.21.15.1:0/4018444271 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f4dc0154910 msgr2=0x7f4dc01efee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:07:54.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.953+0000 7f4dbf400640 1 --2- 172.21.15.1:0/4018444271 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f4dc0154910 0x7f4dc01efee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:54.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.953+0000 7f4dbf400640 1 -- 172.21.15.1:0/4018444271 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4dc00c9750 msgr2=0x7f4dc01f36c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:07:54.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.953+0000 7f4dbf400640 1 --2- 172.21.15.1:0/4018444271 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4dc00c9750 0x7f4dc01f36c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:54.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.953+0000 7f4dbf400640 1 -- 172.21.15.1:0/4018444271 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4dc01f4080 con 0x7f4dc0072890 2024-05-09T08:07:54.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.953+0000 7f4db7e00640 1 --2- 172.21.15.1:0/4018444271 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4dc00c9750 0x7f4dc01f36c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:07:54.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.953+0000 7f4dbf400640 1 --2- 172.21.15.1:0/4018444271 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4dc0072890 0x7f4dc0155c50 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f4db0031000 tx=0x7f4db00046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:07:54.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.954+0000 7f4db7400640 1 -- 172.21.15.1:0/4018444271 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4db0041020 con 0x7f4dc0072890 2024-05-09T08:07:54.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.954+0000 7f4db7400640 1 -- 172.21.15.1:0/4018444271 <== mon.1 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4db0030770 con 0x7f4dc0072890 2024-05-09T08:07:54.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.954+0000 7f4db7400640 1 -- 172.21.15.1:0/4018444271 <== mon.1 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4db0037040 con 0x7f4dc0072890 2024-05-09T08:07:54.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.954+0000 7f4dc6800640 1 -- 172.21.15.1:0/4018444271 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4dc00caa40 con 0x7f4dc0072890 2024-05-09T08:07:54.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.954+0000 7f4dc6800640 1 -- 172.21.15.1:0/4018444271 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4dc0156190 con 0x7f4dc0072890 2024-05-09T08:07:54.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.955+0000 7f4dc6800640 1 -- 172.21.15.1:0/4018444271 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4dc0074d70 con 0x7f4dc0072890 2024-05-09T08:07:54.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.956+0000 7f4db7400640 1 -- 172.21.15.1:0/4018444271 <== mon.1 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4db0044020 con 0x7f4dc0072890 2024-05-09T08:07:54.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.957+0000 7f4db7400640 1 --2- 172.21.15.1:0/4018444271 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f4d9c06be60 0x7f4d9c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:07:54.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.957+0000 7f4db7400640 1 -- 172.21.15.1:0/4018444271 <== mon.1 v2:172.21.15.7:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7f4db00ab8f0 con 0x7f4dc0072890 2024-05-09T08:07:54.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.957+0000 7f4db7e00640 1 --2- 172.21.15.1:0/4018444271 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f4d9c06be60 0x7f4d9c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:07:54.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.958+0000 7f4db7e00640 1 --2- 172.21.15.1:0/4018444271 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f4d9c06be60 0x7f4d9c06e320 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f4dc00b1bf0 tx=0x7f4da8009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:07:54.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:54.962+0000 7f4db7400640 1 -- 172.21.15.1:0/4018444271 <== mon.1 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f4db004b030 con 0x7f4dc0072890 2024-05-09T08:07:55.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:55.161+0000 7f4dc6800640 1 -- 172.21.15.1:0/4018444271 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f4dc00c31e0 con 0x7f4dc0072890 2024-05-09T08:07:55.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:55.162+0000 7f4db7400640 1 -- 172.21.15.1:0/4018444271 <== mon.1 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) ==== 103+0+11 (secure 0 0 0) 0x7f4db00797c0 con 0x7f4dc0072890 2024-05-09T08:07:55.172 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 32 2024-05-09T08:07:55.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:55.165+0000 7f4dc6800640 1 -- 172.21.15.1:0/4018444271 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f4d9c06be60 msgr2=0x7f4d9c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:07:55.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:55.165+0000 7f4dc6800640 1 --2- 172.21.15.1:0/4018444271 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f4d9c06be60 0x7f4d9c06e320 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f4dc00b1bf0 tx=0x7f4da8009040 comp rx=0 tx=0).stop 2024-05-09T08:07:55.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:55.165+0000 7f4dc6800640 1 -- 172.21.15.1:0/4018444271 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4dc0072890 msgr2=0x7f4dc0155c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:07:55.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:55.165+0000 7f4dc6800640 1 --2- 172.21.15.1:0/4018444271 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4dc0072890 0x7f4dc0155c50 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f4db0031000 tx=0x7f4db00046c0 comp rx=0 tx=0).stop 2024-05-09T08:07:55.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:55.165+0000 7f4dc6800640 1 -- 172.21.15.1:0/4018444271 shutdown_connections 2024-05-09T08:07:55.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:55.165+0000 7f4dc6800640 1 --2- 172.21.15.1:0/4018444271 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f4d9c06be60 0x7f4d9c06e320 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:55.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:55.165+0000 7f4dc6800640 1 --2- 172.21.15.1:0/4018444271 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f4dc0154910 0x7f4dc01efee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:55.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:55.165+0000 7f4dc6800640 1 --2- 172.21.15.1:0/4018444271 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4dc00c9750 0x7f4dc01f36c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:55.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:55.165+0000 7f4dc6800640 1 --2- 172.21.15.1:0/4018444271 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4dc0072890 0x7f4dc0155c50 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:55.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:55.165+0000 7f4dc6800640 1 -- 172.21.15.1:0/4018444271 >> 172.21.15.1:0/4018444271 conn(0x7f4dc006dad0 msgr2=0x7f4dc01539f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:07:55.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:55.165+0000 7f4dc6800640 1 -- 172.21.15.1:0/4018444271 shutdown_connections 2024-05-09T08:07:55.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:55.165+0000 7f4dc6800640 1 -- 172.21.15.1:0/4018444271 wait complete. 2024-05-09T08:07:55.194 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-09T08:07:55.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:55.377+0000 7f86f3e00640 1 Processor -- start 2024-05-09T08:07:55.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:55.377+0000 7f86f3e00640 1 -- start start 2024-05-09T08:07:55.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:55.378+0000 7f86f3e00640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f86f40c5890 0x7f86f415b050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:07:55.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:55.378+0000 7f86f3e00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f86f4072940 0x7f86f4072dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:07:55.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:55.378+0000 7f86f3e00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f86f4073310 con 0x7f86f40718e0 2024-05-09T08:07:55.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:55.378+0000 7f86f3e00640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f86f415b590 con 0x7f86f40c5890 2024-05-09T08:07:55.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:55.378+0000 7f86f3e00640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f86f406f060 con 0x7f86f4072940 2024-05-09T08:07:55.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:55.379+0000 7f86f2a00640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f86f40c5890 0x7f86f415b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:07:55.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:55.379+0000 7f86f2a00640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f86f40c5890 0x7f86f415b050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60552/0 (socket says 172.21.15.1:60552) 2024-05-09T08:07:55.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:55.379+0000 7f86f2a00640 1 -- 172.21.15.1:0/2719828761 learned_addr learned my addr 172.21.15.1:0/2719828761 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:07:55.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:55.379+0000 7f86f3400640 1 --2- 172.21.15.1:0/2719828761 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f86f4072940 0x7f86f4072dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:07:55.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:55.379+0000 7f86f1600640 1 -- 172.21.15.1:0/2719828761 <== mon.1 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 919655674 0 0) 0x7f86f4073310 con 0x7f86f40718e0 2024-05-09T08:07:55.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:55.379+0000 7f86f1600640 1 -- 172.21.15.1:0/2719828761 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f86d8003520 con 0x7f86f40718e0 2024-05-09T08:07:55.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:55.379+0000 7f86f2a00640 1 -- 172.21.15.1:0/2719828761 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f86f4072940 msgr2=0x7f86f4072dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:07:55.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:55.379+0000 7f86f2a00640 1 --2- 172.21.15.1:0/2719828761 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f86f4072940 0x7f86f4072dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:55.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:55.379+0000 7f86f2a00640 1 -- 172.21.15.1:0/2719828761 >> v1:172.21.15.7:6789/0 conn(0x7f86f40718e0 legacy=0x7f86f40c69e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:07:55.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:55.379+0000 7f86f2a00640 1 -- 172.21.15.1:0/2719828761 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86f4072150 con 0x7f86f40c5890 2024-05-09T08:07:55.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:55.380+0000 7f86f2a00640 1 --2- 172.21.15.1:0/2719828761 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f86f40c5890 0x7f86f415b050 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f86e4009e50 tx=0x7f86e402a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dca8dbac3161674 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:07:55.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:55.381+0000 7f86f1600640 1 -- 172.21.15.1:0/2719828761 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86e4036070 con 0x7f86f40c5890 2024-05-09T08:07:55.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:55.381+0000 7f86f1600640 1 -- 172.21.15.1:0/2719828761 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86e4031600 con 0x7f86f40c5890 2024-05-09T08:07:55.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:55.381+0000 7f86f1600640 1 -- 172.21.15.1:0/2719828761 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86e40319d0 con 0x7f86f40c5890 2024-05-09T08:07:55.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:55.381+0000 7f86f3e00640 1 -- 172.21.15.1:0/2719828761 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f86f40c5890 msgr2=0x7f86f415b050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:07:55.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:55.381+0000 7f86f3e00640 1 --2- 172.21.15.1:0/2719828761 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f86f40c5890 0x7f86f415b050 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f86e4009e50 tx=0x7f86e402a7a0 comp rx=0 tx=0).stop 2024-05-09T08:07:55.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:55.381+0000 7f86f3e00640 1 -- 172.21.15.1:0/2719828761 shutdown_connections 2024-05-09T08:07:55.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:55.381+0000 7f86f3e00640 1 --2- 172.21.15.1:0/2719828761 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f86f4072940 0x7f86f4072dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:55.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:55.381+0000 7f86f3e00640 1 --2- 172.21.15.1:0/2719828761 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f86f40c5890 0x7f86f415b050 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:55.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:55.381+0000 7f86f3e00640 1 -- 172.21.15.1:0/2719828761 >> 172.21.15.1:0/2719828761 conn(0x7f86f406b250 msgr2=0x7f86f406b630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:07:55.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:55.381+0000 7f86f3e00640 1 -- 172.21.15.1:0/2719828761 shutdown_connections 2024-05-09T08:07:55.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:55.381+0000 7f86f3e00640 1 -- 172.21.15.1:0/2719828761 wait complete. 2024-05-09T08:07:55.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:55.383+0000 7f86f3e00640 1 Processor -- start 2024-05-09T08:07:55.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:55.383+0000 7f86f3e00640 1 -- start start 2024-05-09T08:07:55.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:55.384+0000 7f86f3e00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f86f40718e0 0x7f86f4159c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:07:55.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:55.384+0000 7f86f3e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f86f4072940 0x7f86f41e52f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:07:55.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:55.384+0000 7f86f3e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86f40c5890 0x7f86f41e5830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:07:55.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:55.384+0000 7f86f3e00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f86f406f5e0 con 0x7f86f4072940 2024-05-09T08:07:55.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:55.384+0000 7f86f3e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f86f406f300 con 0x7f86f40c5890 2024-05-09T08:07:55.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:55.384+0000 7f86f3e00640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f86f406f8c0 con 0x7f86f40718e0 2024-05-09T08:07:55.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:55.385+0000 7f86f2000640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f86f4072940 0x7f86f41e52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:07:55.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:55.385+0000 7f86f2a00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f86f40718e0 0x7f86f4159c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:07:55.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:55.385+0000 7f86f2000640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f86f4072940 0x7f86f41e52f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60554/0 (socket says 172.21.15.1:60554) 2024-05-09T08:07:55.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:55.385+0000 7f86f2000640 1 -- 172.21.15.1:0/4158909596 learned_addr learned my addr 172.21.15.1:0/4158909596 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:07:55.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:55.385+0000 7f86f2a00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f86f40718e0 0x7f86f4159c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3301/0 says I am v2:172.21.15.1:58642/0 (socket says 172.21.15.1:58642) 2024-05-09T08:07:55.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:55.385+0000 7f86f3400640 1 --2- 172.21.15.1:0/4158909596 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86f40c5890 0x7f86f41e5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:07:55.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:55.385+0000 7f86f2000640 1 -- 172.21.15.1:0/4158909596 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f86f40718e0 msgr2=0x7f86f4159c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:07:55.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:55.385+0000 7f86f2000640 1 --2- 172.21.15.1:0/4158909596 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f86f40718e0 0x7f86f4159c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:55.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:55.385+0000 7f86f2000640 1 -- 172.21.15.1:0/4158909596 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86f40c5890 msgr2=0x7f86f41e5830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:07:55.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:55.385+0000 7f86f2000640 1 --2- 172.21.15.1:0/4158909596 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86f40c5890 0x7f86f41e5830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:55.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:55.385+0000 7f86f2000640 1 -- 172.21.15.1:0/4158909596 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86f41eff90 con 0x7f86f4072940 2024-05-09T08:07:55.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:55.385+0000 7f86f2000640 1 --2- 172.21.15.1:0/4158909596 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f86f4072940 0x7f86f41e52f0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f86e000c460 tx=0x7f86e000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:07:55.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:55.385+0000 7f86eaa00640 1 -- 172.21.15.1:0/4158909596 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86e0017070 con 0x7f86f4072940 2024-05-09T08:07:55.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:55.385+0000 7f86eaa00640 1 -- 172.21.15.1:0/4158909596 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86e00052f0 con 0x7f86f4072940 2024-05-09T08:07:55.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:55.385+0000 7f86eaa00640 1 -- 172.21.15.1:0/4158909596 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86e0003830 con 0x7f86f4072940 2024-05-09T08:07:55.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:55.385+0000 7f86f3e00640 1 -- 172.21.15.1:0/4158909596 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86f41f0230 con 0x7f86f4072940 2024-05-09T08:07:55.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:55.386+0000 7f86f3e00640 1 -- 172.21.15.1:0/4158909596 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86f41f07f0 con 0x7f86f4072940 2024-05-09T08:07:55.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:55.387+0000 7f86f3e00640 1 -- 172.21.15.1:0/4158909596 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86f404f660 con 0x7f86f4072940 2024-05-09T08:07:55.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:55.388+0000 7f86eaa00640 1 -- 172.21.15.1:0/4158909596 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f86e001a070 con 0x7f86f4072940 2024-05-09T08:07:55.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:55.388+0000 7f86eaa00640 1 --2- 172.21.15.1:0/4158909596 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f86d406be30 0x7f86d406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:07:55.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:55.389+0000 7f86eaa00640 1 -- 172.21.15.1:0/4158909596 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7f86e008a6d0 con 0x7f86f4072940 2024-05-09T08:07:55.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:55.389+0000 7f86f2a00640 1 --2- 172.21.15.1:0/4158909596 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f86d406be30 0x7f86d406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:07:55.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:55.389+0000 7f86f2a00640 1 --2- 172.21.15.1:0/4158909596 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f86d406be30 0x7f86d406e2f0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f86e4033000 tx=0x7f86e4009000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:07:55.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:55.393+0000 7f86eaa00640 1 -- 172.21.15.1:0/4158909596 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f86e0058510 con 0x7f86f4072940 2024-05-09T08:07:55.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:55.609+0000 7f86f3e00640 1 -- 172.21.15.1:0/4158909596 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f86f40c7520 con 0x7f86f4072940 2024-05-09T08:07:55.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:55.610+0000 7f86eaa00640 1 -- 172.21.15.1:0/4158909596 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) ==== 80+0+0 (secure 0 0 0) 0x7f86e005c810 con 0x7f86f4072940 2024-05-09T08:07:55.620 INFO:teuthology.orchestra.run.smithi001.stderr:listed 0 entries 2024-05-09T08:07:55.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:55.613+0000 7f86f3e00640 1 -- 172.21.15.1:0/4158909596 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f86d406be30 msgr2=0x7f86d406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:07:55.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:55.613+0000 7f86f3e00640 1 --2- 172.21.15.1:0/4158909596 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f86d406be30 0x7f86d406e2f0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f86e4033000 tx=0x7f86e4009000 comp rx=0 tx=0).stop 2024-05-09T08:07:55.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:55.613+0000 7f86f3e00640 1 -- 172.21.15.1:0/4158909596 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f86f4072940 msgr2=0x7f86f41e52f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:07:55.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:55.613+0000 7f86f3e00640 1 --2- 172.21.15.1:0/4158909596 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f86f4072940 0x7f86f41e52f0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f86e000c460 tx=0x7f86e000c920 comp rx=0 tx=0).stop 2024-05-09T08:07:55.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:55.613+0000 7f86f3e00640 1 -- 172.21.15.1:0/4158909596 shutdown_connections 2024-05-09T08:07:55.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:55.613+0000 7f86f3e00640 1 --2- 172.21.15.1:0/4158909596 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f86d406be30 0x7f86d406e2f0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:55.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:55.613+0000 7f86f3e00640 1 --2- 172.21.15.1:0/4158909596 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86f40c5890 0x7f86f41e5830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:55.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:55.613+0000 7f86f3e00640 1 --2- 172.21.15.1:0/4158909596 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f86f4072940 0x7f86f41e52f0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:55.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:55.613+0000 7f86f3e00640 1 --2- 172.21.15.1:0/4158909596 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f86f40718e0 0x7f86f4159c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:55.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:55.613+0000 7f86f3e00640 1 -- 172.21.15.1:0/4158909596 >> 172.21.15.1:0/4158909596 conn(0x7f86f406b250 msgr2=0x7f86f40caca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:07:55.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:55.613+0000 7f86f3e00640 1 -- 172.21.15.1:0/4158909596 shutdown_connections 2024-05-09T08:07:55.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:55.613+0000 7f86f3e00640 1 -- 172.21.15.1:0/4158909596 wait complete. 2024-05-09T08:07:55.643 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-05-09T08:07:55.644 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T08:07:55.644 DEBUG:teuthology.orchestra.run.smithi001:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-05-09T08:07:55.776 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi001.front.sepia.ceph.com 2024-05-09T08:07:55.776 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T08:07:55.776 DEBUG:teuthology.orchestra.run.smithi001:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-09T08:07:55.825 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T08:07:55.826 DEBUG:teuthology.orchestra.run.smithi001:> ip netns list 2024-05-09T08:07:55.909 INFO:teuthology.orchestra.run.smithi001.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-09T08:07:55.909 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T08:07:55.910 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-09T08:07:56.035 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T08:07:56.036 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip link delete ceph-brx 2024-05-09T08:07:56.154 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-05-09T08:07:56.163 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-05-09T08:07:56.163 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-09T08:07:56.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:56.333+0000 7f41c8c00640 1 Processor -- start 2024-05-09T08:07:56.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:56.333+0000 7f41c8c00640 1 -- start start 2024-05-09T08:07:56.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:56.335+0000 7f41c8c00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f41c40c8180 0x7f41c40cc730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:07:56.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:56.335+0000 7f41c8c00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f41c40c7660 0x7f41c40c7ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:07:56.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:56.335+0000 7f41c8c00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41c415c790 con 0x7f41c4152500 2024-05-09T08:07:56.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:56.335+0000 7f41c8c00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f41c40c5e50 con 0x7f41c40c7660 2024-05-09T08:07:56.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:56.335+0000 7f41c8c00640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f41c40c22d0 con 0x7f41c40c8180 2024-05-09T08:07:56.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:56.335+0000 7f41c3e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f41c40c7660 0x7f41c40c7ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:07:56.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:56.335+0000 7f41c3400640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f41c4152500 0x7f41c415c080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.1:38680/0 (socket says 172.21.15.1:38680) 2024-05-09T08:07:56.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:56.336+0000 7f41c3400640 1 -- 172.21.15.1:0/3160176082 learned_addr learned my addr 172.21.15.1:0/3160176082 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T08:07:56.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:56.336+0000 7f41c2a00640 1 --2- 172.21.15.1:0/3160176082 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f41c40c8180 0x7f41c40cc730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:07:56.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:56.336+0000 7f41c3e00640 1 -- 172.21.15.1:0/3160176082 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f41c40c8180 msgr2=0x7f41c40cc730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:07:56.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:56.336+0000 7f41c3e00640 1 --2- 172.21.15.1:0/3160176082 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f41c40c8180 0x7f41c40cc730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:56.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:56.336+0000 7f41c3e00640 1 -- 172.21.15.1:0/3160176082 >> v1:172.21.15.7:6789/0 conn(0x7f41c4152500 legacy=0x7f41c415c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:07:56.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:56.336+0000 7f41c3e00640 1 -- 172.21.15.1:0/3160176082 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41c40c6a30 con 0x7f41c40c7660 2024-05-09T08:07:56.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:56.337+0000 7f41c3e00640 1 --2- 172.21.15.1:0/3160176082 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f41c40c7660 0x7f41c40c7ab0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f41bc009d90 tx=0x7f41bc02a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1aaa8ac07a4cfa26 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:07:56.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:56.338+0000 7f41c2000640 1 -- 172.21.15.1:0/3160176082 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41bc007740 con 0x7f41c40c7660 2024-05-09T08:07:56.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:56.338+0000 7f41c2000640 1 -- 172.21.15.1:0/3160176082 <== mon.1 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41bc0078e0 con 0x7f41c40c7660 2024-05-09T08:07:56.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:56.338+0000 7f41c2000640 1 -- 172.21.15.1:0/3160176082 <== mon.1 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41bc007cb0 con 0x7f41c40c7660 2024-05-09T08:07:56.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:56.338+0000 7f41c8c00640 1 -- 172.21.15.1:0/3160176082 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f41c40c7660 msgr2=0x7f41c40c7ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:07:56.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:56.338+0000 7f41c8c00640 1 --2- 172.21.15.1:0/3160176082 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f41c40c7660 0x7f41c40c7ab0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f41bc009d90 tx=0x7f41bc02a660 comp rx=0 tx=0).stop 2024-05-09T08:07:56.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:56.338+0000 7f41c8c00640 1 -- 172.21.15.1:0/3160176082 shutdown_connections 2024-05-09T08:07:56.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:56.338+0000 7f41c8c00640 1 --2- 172.21.15.1:0/3160176082 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f41c40c7660 0x7f41c40c7ab0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:56.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:56.338+0000 7f41c8c00640 1 --2- 172.21.15.1:0/3160176082 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f41c40c8180 0x7f41c40cc730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:56.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:56.338+0000 7f41c8c00640 1 -- 172.21.15.1:0/3160176082 >> 172.21.15.1:0/3160176082 conn(0x7f41c40cbc10 msgr2=0x7f41c40cc020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:07:56.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:56.339+0000 7f41c8c00640 1 -- 172.21.15.1:0/3160176082 shutdown_connections 2024-05-09T08:07:56.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:56.339+0000 7f41c8c00640 1 -- 172.21.15.1:0/3160176082 wait complete. 2024-05-09T08:07:56.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:56.341+0000 7f41c8c00640 1 Processor -- start 2024-05-09T08:07:56.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:56.341+0000 7f41c8c00640 1 -- start start 2024-05-09T08:07:56.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:56.342+0000 7f41c8c00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f41c40c7660 0x7f41c40cc260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:07:56.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:56.342+0000 7f41c8c00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f41c40c8180 0x7f41c414b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:07:56.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:56.342+0000 7f41c8c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f41c4152500 0x7f41c414be60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:07:56.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:56.342+0000 7f41c8c00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f41c40c72b0 con 0x7f41c40c7660 2024-05-09T08:07:56.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:56.342+0000 7f41c8c00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f41c40c6fd0 con 0x7f41c4152500 2024-05-09T08:07:56.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:56.342+0000 7f41c8c00640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f41c415b880 con 0x7f41c40c8180 2024-05-09T08:07:56.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:56.342+0000 7f41c3400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f41c40c7660 0x7f41c40cc260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:07:56.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:56.343+0000 7f41c3e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f41c4152500 0x7f41c414be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:07:56.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:56.343+0000 7f41c3e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f41c4152500 0x7f41c414be60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.1:54042/0 (socket says 172.21.15.1:54042) 2024-05-09T08:07:56.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:56.343+0000 7f41c2a00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f41c40c8180 0x7f41c414b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:07:56.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:56.343+0000 7f41c3e00640 1 -- 172.21.15.1:0/2788653363 learned_addr learned my addr 172.21.15.1:0/2788653363 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:07:56.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:56.343+0000 7f41c3e00640 1 -- 172.21.15.1:0/2788653363 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f41c40c8180 msgr2=0x7f41c414b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:07:56.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:56.343+0000 7f41c3e00640 1 --2- 172.21.15.1:0/2788653363 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f41c40c8180 0x7f41c414b920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:56.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:56.343+0000 7f41c3e00640 1 -- 172.21.15.1:0/2788653363 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f41c40c7660 msgr2=0x7f41c40cc260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:07:56.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:56.343+0000 7f41c3e00640 1 --2- 172.21.15.1:0/2788653363 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f41c40c7660 0x7f41c40cc260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:56.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:56.343+0000 7f41c3e00640 1 -- 172.21.15.1:0/2788653363 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41c41f0210 con 0x7f41c4152500 2024-05-09T08:07:56.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:56.343+0000 7f41c2a00640 1 --2- 172.21.15.1:0/2788653363 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f41c40c8180 0x7f41c414b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:07:56.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:56.343+0000 7f41c3400640 1 --2- 172.21.15.1:0/2788653363 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f41c40c7660 0x7f41c40cc260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:07:56.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:56.343+0000 7f41c3e00640 1 --2- 172.21.15.1:0/2788653363 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f41c4152500 0x7f41c414be60 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f41bc009eb0 tx=0x7f41bc004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:07:56.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:56.344+0000 7f41bb400640 1 -- 172.21.15.1:0/2788653363 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41bc040070 con 0x7f41c4152500 2024-05-09T08:07:56.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:56.344+0000 7f41bb400640 1 -- 172.21.15.1:0/2788653363 <== mon.1 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41bc031bc0 con 0x7f41c4152500 2024-05-09T08:07:56.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:56.344+0000 7f41bb400640 1 -- 172.21.15.1:0/2788653363 <== mon.1 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41bc009450 con 0x7f41c4152500 2024-05-09T08:07:56.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:56.344+0000 7f41c8c00640 1 -- 172.21.15.1:0/2788653363 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41c41f04d0 con 0x7f41c4152500 2024-05-09T08:07:56.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:56.344+0000 7f41c8c00640 1 -- 172.21.15.1:0/2788653363 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41c41f0a90 con 0x7f41c4152500 2024-05-09T08:07:56.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:56.345+0000 7f41c8c00640 1 -- 172.21.15.1:0/2788653363 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41c4072dd0 con 0x7f41c4152500 2024-05-09T08:07:56.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:56.346+0000 7f41bb400640 1 -- 172.21.15.1:0/2788653363 <== mon.1 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f41bc039040 con 0x7f41c4152500 2024-05-09T08:07:56.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:56.347+0000 7f41bb400640 1 --2- 172.21.15.1:0/2788653363 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f41a406be30 0x7f41a406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:07:56.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:56.347+0000 7f41bb400640 1 -- 172.21.15.1:0/2788653363 <== mon.1 v2:172.21.15.7:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7f41bc0aca10 con 0x7f41c4152500 2024-05-09T08:07:56.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:56.347+0000 7f41c3400640 1 --2- 172.21.15.1:0/2788653363 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f41a406be30 0x7f41a406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:07:56.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:56.348+0000 7f41c3400640 1 --2- 172.21.15.1:0/2788653363 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f41a406be30 0x7f41a406e2f0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f41b4005880 tx=0x7f41b4009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:07:56.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:56.352+0000 7f41bb400640 1 -- 172.21.15.1:0/2788653363 <== mon.1 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f41c4072dd0 con 0x7f41c4152500 2024-05-09T08:07:56.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:56.555+0000 7f41c8c00640 1 -- 172.21.15.1:0/2788653363 --> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f41c4070dc0 con 0x7f41a406be30 2024-05-09T08:07:56.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:56.566+0000 7f41bb400640 1 -- 172.21.15.1:0/2788653363 <== mgr.4114 v2:172.21.15.7:6832/3562612464 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+333085 (secure 0 0 0) 0x7f41c4070dc0 con 0x7f41a406be30 2024-05-09T08:07:56.581 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T08:07:56.583 INFO:teuthology.orchestra.run.smithi001.stderr:dumped all 2024-05-09T08:07:56.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:56.573+0000 7f41c8c00640 1 -- 172.21.15.1:0/2788653363 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f41a406be30 msgr2=0x7f41a406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:07:56.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:56.573+0000 7f41c8c00640 1 --2- 172.21.15.1:0/2788653363 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f41a406be30 0x7f41a406e2f0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f41b4005880 tx=0x7f41b4009040 comp rx=0 tx=0).stop 2024-05-09T08:07:56.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:56.573+0000 7f41c8c00640 1 -- 172.21.15.1:0/2788653363 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f41c4152500 msgr2=0x7f41c414be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:07:56.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:56.573+0000 7f41c8c00640 1 --2- 172.21.15.1:0/2788653363 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f41c4152500 0x7f41c414be60 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f41bc009eb0 tx=0x7f41bc004a40 comp rx=0 tx=0).stop 2024-05-09T08:07:56.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:56.573+0000 7f41c8c00640 1 -- 172.21.15.1:0/2788653363 shutdown_connections 2024-05-09T08:07:56.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:56.573+0000 7f41c8c00640 1 --2- 172.21.15.1:0/2788653363 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f41a406be30 0x7f41a406e2f0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:56.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:56.573+0000 7f41c8c00640 1 --2- 172.21.15.1:0/2788653363 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f41c4152500 0x7f41c414be60 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:56.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:56.573+0000 7f41c8c00640 1 --2- 172.21.15.1:0/2788653363 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f41c40c8180 0x7f41c414b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:56.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:56.573+0000 7f41c8c00640 1 --2- 172.21.15.1:0/2788653363 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f41c40c7660 0x7f41c40cc260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:56.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:56.573+0000 7f41c8c00640 1 -- 172.21.15.1:0/2788653363 >> 172.21.15.1:0/2788653363 conn(0x7f41c40cbc10 msgr2=0x7f41c40c9a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:07:56.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:56.574+0000 7f41c8c00640 1 -- 172.21.15.1:0/2788653363 shutdown_connections 2024-05-09T08:07:56.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:56.574+0000 7f41c8c00640 1 -- 172.21.15.1:0/2788653363 wait complete. 2024-05-09T08:07:56.603 INFO:teuthology.orchestra.run.smithi001.stdout:{"pg_ready":true,"pg_map":{"version":194,"stamp":"2024-05-09T08:07:55.184855+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1039130,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":121,"num_read_kb":105,"num_write":9652,"num_write_kb":25168857,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":11133,"ondisk_log_size":11133,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":481240,"kb_used_data":4376,"kb_used_omap":12,"kb_used_meta":476787,"kb_avail":754493480,"statfs":{"total":773094113280,"available":772601323520,"internally_reserved":0,"allocated":4481024,"data_stored":3030012,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":488230450},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004075"},"pg_stats":[{"pgid":"2.1f","version":"24'4","reported_seq":31,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:22.753051+0000","last_change":"2024-05-09T08:02:14.875108+0000","last_active":"2024-05-09T08:04:22.753051+0000","last_peered":"2024-05-09T08:04:22.753051+0000","last_clean":"2024-05-09T08:04:22.753051+0000","last_became_active":"2024-05-09T08:02:14.874841+0000","last_became_peered":"2024-05-09T08:02:14.874841+0000","last_unstale":"2024-05-09T08:04:22.753051+0000","last_undegraded":"2024-05-09T08:04:22.753051+0000","last_fullsized":"2024-05-09T08:04:22.753051+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:00:40.599475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:17.745934+0000","last_change":"2024-05-09T08:02:15.899818+0000","last_active":"2024-05-09T08:04:17.745934+0000","last_peered":"2024-05-09T08:04:17.745934+0000","last_clean":"2024-05-09T08:04:17.745934+0000","last_became_active":"2024-05-09T08:02:15.899479+0000","last_became_peered":"2024-05-09T08:02:15.899479+0000","last_unstale":"2024-05-09T08:04:17.745934+0000","last_undegraded":"2024-05-09T08:04:17.745934+0000","last_fullsized":"2024-05-09T08:04:17.745934+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:18:36.309627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"27'360","reported_seq":380,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:15.262265+0000","last_change":"2024-05-09T08:02:20.057944+0000","last_active":"2024-05-09T08:07:15.262265+0000","last_peered":"2024-05-09T08:07:15.262265+0000","last_clean":"2024-05-09T08:07:15.262265+0000","last_became_active":"2024-05-09T08:02:20.057016+0000","last_became_peered":"2024-05-09T08:02:20.057016+0000","last_unstale":"2024-05-09T08:07:15.262265+0000","last_undegraded":"2024-05-09T08:07:15.262265+0000","last_fullsized":"2024-05-09T08:07:15.262265+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":360,"log_dups_size":0,"ondisk_log_size":360,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:17:01.514168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":306,"num_write_kb":835584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"24'1","reported_seq":28,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:17.745962+0000","last_change":"2024-05-09T08:02:14.886137+0000","last_active":"2024-05-09T08:04:17.745962+0000","last_peered":"2024-05-09T08:04:17.745962+0000","last_clean":"2024-05-09T08:04:17.745962+0000","last_became_active":"2024-05-09T08:02:14.885830+0000","last_became_peered":"2024-05-09T08:02:14.885830+0000","last_unstale":"2024-05-09T08:04:17.745962+0000","last_undegraded":"2024-05-09T08:04:17.745962+0000","last_fullsized":"2024-05-09T08:04:17.745962+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:46:11.160815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:22.753022+0000","last_change":"2024-05-09T08:02:15.906280+0000","last_active":"2024-05-09T08:04:22.753022+0000","last_peered":"2024-05-09T08:04:22.753022+0000","last_clean":"2024-05-09T08:04:22.753022+0000","last_became_active":"2024-05-09T08:02:15.905590+0000","last_became_peered":"2024-05-09T08:02:15.905590+0000","last_unstale":"2024-05-09T08:04:22.753022+0000","last_undegraded":"2024-05-09T08:04:22.753022+0000","last_fullsized":"2024-05-09T08:04:22.753022+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:20:07.368327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"27'377","reported_seq":397,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:15.276306+0000","last_change":"2024-05-09T08:02:19.954548+0000","last_active":"2024-05-09T08:07:15.276306+0000","last_peered":"2024-05-09T08:07:15.276306+0000","last_clean":"2024-05-09T08:07:15.276306+0000","last_became_active":"2024-05-09T08:02:19.954161+0000","last_became_peered":"2024-05-09T08:02:19.954161+0000","last_unstale":"2024-05-09T08:07:15.276306+0000","last_undegraded":"2024-05-09T08:07:15.276306+0000","last_fullsized":"2024-05-09T08:07:15.276306+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":377,"log_dups_size":0,"ondisk_log_size":377,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:53:27.246334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":339,"num_write_kb":925696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"27'3","reported_seq":30,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:05:05.164895+0000","last_change":"2024-05-09T08:02:14.867128+0000","last_active":"2024-05-09T08:05:05.164895+0000","last_peered":"2024-05-09T08:05:05.164895+0000","last_clean":"2024-05-09T08:05:05.164895+0000","last_became_active":"2024-05-09T08:02:14.866809+0000","last_became_peered":"2024-05-09T08:02:14.866809+0000","last_unstale":"2024-05-09T08:05:05.164895+0000","last_undegraded":"2024-05-09T08:05:05.164895+0000","last_fullsized":"2024-05-09T08:05:05.164895+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:57:06.093718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:47.737794+0000","last_change":"2024-05-09T08:02:15.888431+0000","last_active":"2024-05-09T08:04:47.737794+0000","last_peered":"2024-05-09T08:04:47.737794+0000","last_clean":"2024-05-09T08:04:47.737794+0000","last_became_active":"2024-05-09T08:02:15.887509+0000","last_became_peered":"2024-05-09T08:02:15.887509+0000","last_unstale":"2024-05-09T08:04:47.737794+0000","last_undegraded":"2024-05-09T08:04:47.737794+0000","last_fullsized":"2024-05-09T08:04:47.737794+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:25:01.238816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"27'324","reported_seq":344,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:15.265243+0000","last_change":"2024-05-09T08:02:19.953600+0000","last_active":"2024-05-09T08:07:15.265243+0000","last_peered":"2024-05-09T08:07:15.265243+0000","last_clean":"2024-05-09T08:07:15.265243+0000","last_became_active":"2024-05-09T08:02:19.952275+0000","last_became_peered":"2024-05-09T08:02:19.952275+0000","last_unstale":"2024-05-09T08:07:15.265243+0000","last_undegraded":"2024-05-09T08:07:15.265243+0000","last_fullsized":"2024-05-09T08:07:15.265243+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":324,"log_dups_size":0,"ondisk_log_size":324,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:25:05.306541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":273,"num_write_kb":745472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"24'2","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:47.737301+0000","last_change":"2024-05-09T08:02:14.870819+0000","last_active":"2024-05-09T08:04:47.737301+0000","last_peered":"2024-05-09T08:04:47.737301+0000","last_clean":"2024-05-09T08:04:47.737301+0000","last_became_active":"2024-05-09T08:02:14.870476+0000","last_became_peered":"2024-05-09T08:02:14.870476+0000","last_unstale":"2024-05-09T08:04:47.737301+0000","last_undegraded":"2024-05-09T08:04:47.737301+0000","last_fullsized":"2024-05-09T08:04:47.737301+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:43:01.061801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:47.737815+0000","last_change":"2024-05-09T08:02:15.886471+0000","last_active":"2024-05-09T08:04:47.737815+0000","last_peered":"2024-05-09T08:04:47.737815+0000","last_clean":"2024-05-09T08:04:47.737815+0000","last_became_active":"2024-05-09T08:02:15.886168+0000","last_became_peered":"2024-05-09T08:02:15.886168+0000","last_unstale":"2024-05-09T08:04:47.737815+0000","last_undegraded":"2024-05-09T08:04:47.737815+0000","last_fullsized":"2024-05-09T08:04:47.737815+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:30:33.351158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"27'298","reported_seq":318,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:15.295149+0000","last_change":"2024-05-09T08:02:19.947347+0000","last_active":"2024-05-09T08:07:15.295149+0000","last_peered":"2024-05-09T08:07:15.295149+0000","last_clean":"2024-05-09T08:07:15.295149+0000","last_became_active":"2024-05-09T08:02:19.946906+0000","last_became_peered":"2024-05-09T08:02:19.946906+0000","last_unstale":"2024-05-09T08:07:15.295149+0000","last_undegraded":"2024-05-09T08:07:15.295149+0000","last_fullsized":"2024-05-09T08:07:15.295149+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":298,"log_dups_size":0,"ondisk_log_size":298,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:30:37.418883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":252,"num_write_kb":688128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:27.735428+0000","last_change":"2024-05-09T08:02:15.880765+0000","last_active":"2024-05-09T08:04:27.735428+0000","last_peered":"2024-05-09T08:04:27.735428+0000","last_clean":"2024-05-09T08:04:27.735428+0000","last_became_active":"2024-05-09T08:02:15.880439+0000","last_became_peered":"2024-05-09T08:02:15.880439+0000","last_unstale":"2024-05-09T08:04:27.735428+0000","last_undegraded":"2024-05-09T08:04:27.735428+0000","last_fullsized":"2024-05-09T08:04:27.735428+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:29:59.441314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"27'3","reported_seq":30,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:05:05.164991+0000","last_change":"2024-05-09T08:02:14.867587+0000","last_active":"2024-05-09T08:05:05.164991+0000","last_peered":"2024-05-09T08:05:05.164991+0000","last_clean":"2024-05-09T08:05:05.164991+0000","last_became_active":"2024-05-09T08:02:14.867291+0000","last_became_peered":"2024-05-09T08:02:14.867291+0000","last_unstale":"2024-05-09T08:05:05.164991+0000","last_undegraded":"2024-05-09T08:05:05.164991+0000","last_fullsized":"2024-05-09T08:05:05.164991+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:20:08.921752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"27'366","reported_seq":386,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:15.267986+0000","last_change":"2024-05-09T08:02:19.954902+0000","last_active":"2024-05-09T08:07:15.267986+0000","last_peered":"2024-05-09T08:07:15.267986+0000","last_clean":"2024-05-09T08:07:15.267986+0000","last_became_active":"2024-05-09T08:02:19.954007+0000","last_became_peered":"2024-05-09T08:02:19.954007+0000","last_unstale":"2024-05-09T08:07:15.267986+0000","last_undegraded":"2024-05-09T08:07:15.267986+0000","last_fullsized":"2024-05-09T08:07:15.267986+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":366,"log_dups_size":0,"ondisk_log_size":366,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:41:31.470546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":309,"num_write_kb":843776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:32.747723+0000","last_change":"2024-05-09T08:02:15.915155+0000","last_active":"2024-05-09T08:04:32.747723+0000","last_peered":"2024-05-09T08:04:32.747723+0000","last_clean":"2024-05-09T08:04:32.747723+0000","last_became_active":"2024-05-09T08:02:15.914852+0000","last_became_peered":"2024-05-09T08:02:15.914852+0000","last_unstale":"2024-05-09T08:04:32.747723+0000","last_undegraded":"2024-05-09T08:04:32.747723+0000","last_fullsized":"2024-05-09T08:04:32.747723+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:53:23.178610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:27.759691+0000","last_change":"2024-05-09T08:02:14.881770+0000","last_active":"2024-05-09T08:04:27.759691+0000","last_peered":"2024-05-09T08:04:27.759691+0000","last_clean":"2024-05-09T08:04:27.759691+0000","last_became_active":"2024-05-09T08:02:14.881437+0000","last_became_peered":"2024-05-09T08:02:14.881437+0000","last_unstale":"2024-05-09T08:04:27.759691+0000","last_undegraded":"2024-05-09T08:04:27.759691+0000","last_fullsized":"2024-05-09T08:04:27.759691+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:25:23.306827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"27'330","reported_seq":350,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:15.284795+0000","last_change":"2024-05-09T08:02:19.955928+0000","last_active":"2024-05-09T08:07:15.284795+0000","last_peered":"2024-05-09T08:07:15.284795+0000","last_clean":"2024-05-09T08:07:15.284795+0000","last_became_active":"2024-05-09T08:02:19.955432+0000","last_became_peered":"2024-05-09T08:02:19.955432+0000","last_unstale":"2024-05-09T08:07:15.284795+0000","last_undegraded":"2024-05-09T08:07:15.284795+0000","last_fullsized":"2024-05-09T08:07:15.284795+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":330,"log_dups_size":0,"ondisk_log_size":330,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:53:15.458120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":291,"num_write_kb":794624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:27.746932+0000","last_change":"2024-05-09T08:02:15.913311+0000","last_active":"2024-05-09T08:04:27.746932+0000","last_peered":"2024-05-09T08:04:27.746932+0000","last_clean":"2024-05-09T08:04:27.746932+0000","last_became_active":"2024-05-09T08:02:15.912484+0000","last_became_peered":"2024-05-09T08:02:15.912484+0000","last_unstale":"2024-05-09T08:04:27.746932+0000","last_undegraded":"2024-05-09T08:04:27.746932+0000","last_fullsized":"2024-05-09T08:04:27.746932+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:43:06.938904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:27.759654+0000","last_change":"2024-05-09T08:02:14.868884+0000","last_active":"2024-05-09T08:04:27.759654+0000","last_peered":"2024-05-09T08:04:27.759654+0000","last_clean":"2024-05-09T08:04:27.759654+0000","last_became_active":"2024-05-09T08:02:14.868553+0000","last_became_peered":"2024-05-09T08:02:14.868553+0000","last_unstale":"2024-05-09T08:04:27.759654+0000","last_undegraded":"2024-05-09T08:04:27.759654+0000","last_fullsized":"2024-05-09T08:04:27.759654+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:32:48.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"27'321","reported_seq":335,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:15.287758+0000","last_change":"2024-05-09T08:03:02.609342+0000","last_active":"2024-05-09T08:07:15.287758+0000","last_peered":"2024-05-09T08:07:15.287758+0000","last_clean":"2024-05-09T08:07:15.287758+0000","last_became_active":"2024-05-09T08:03:02.607564+0000","last_became_peered":"2024-05-09T08:03:02.607564+0000","last_unstale":"2024-05-09T08:07:15.287758+0000","last_undegraded":"2024-05-09T08:07:15.287758+0000","last_fullsized":"2024-05-09T08:07:15.287758+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":321,"log_dups_size":0,"ondisk_log_size":321,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:23:44.316545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":267,"num_write_kb":729088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4,7],"acting":[2,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:27.735197+0000","last_change":"2024-05-09T08:02:15.891115+0000","last_active":"2024-05-09T08:04:27.735197+0000","last_peered":"2024-05-09T08:04:27.735197+0000","last_clean":"2024-05-09T08:04:27.735197+0000","last_became_active":"2024-05-09T08:02:15.890806+0000","last_became_peered":"2024-05-09T08:02:15.890806+0000","last_unstale":"2024-05-09T08:04:27.735197+0000","last_undegraded":"2024-05-09T08:04:27.735197+0000","last_fullsized":"2024-05-09T08:04:27.735197+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:57:07.107359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:27.735251+0000","last_change":"2024-05-09T08:02:14.874987+0000","last_active":"2024-05-09T08:04:27.735251+0000","last_peered":"2024-05-09T08:04:27.735251+0000","last_clean":"2024-05-09T08:04:27.735251+0000","last_became_active":"2024-05-09T08:02:14.874549+0000","last_became_peered":"2024-05-09T08:02:14.874549+0000","last_unstale":"2024-05-09T08:04:27.735251+0000","last_undegraded":"2024-05-09T08:04:27.735251+0000","last_fullsized":"2024-05-09T08:04:27.735251+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:57:10.338510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"27'339","reported_seq":366,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:15.296068+0000","last_change":"2024-05-09T08:03:02.516739+0000","last_active":"2024-05-09T08:07:15.296068+0000","last_peered":"2024-05-09T08:07:15.296068+0000","last_clean":"2024-05-09T08:07:15.296068+0000","last_became_active":"2024-05-09T08:03:02.516161+0000","last_became_peered":"2024-05-09T08:03:02.516161+0000","last_unstale":"2024-05-09T08:07:15.296068+0000","last_undegraded":"2024-05-09T08:07:15.296068+0000","last_fullsized":"2024-05-09T08:07:15.296068+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":339,"log_dups_size":0,"ondisk_log_size":339,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:10:24.797526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":288,"num_write_kb":786432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,2],"acting":[4,3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:27.753541+0000","last_change":"2024-05-09T08:02:15.913011+0000","last_active":"2024-05-09T08:04:27.753541+0000","last_peered":"2024-05-09T08:04:27.753541+0000","last_clean":"2024-05-09T08:04:27.753541+0000","last_became_active":"2024-05-09T08:02:15.912590+0000","last_became_peered":"2024-05-09T08:02:15.912590+0000","last_unstale":"2024-05-09T08:04:27.753541+0000","last_undegraded":"2024-05-09T08:04:27.753541+0000","last_fullsized":"2024-05-09T08:04:27.753541+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:43:06.938904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"27'3","reported_seq":31,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:06:37.749118+0000","last_change":"2024-05-09T08:02:14.864763+0000","last_active":"2024-05-09T08:06:37.749118+0000","last_peered":"2024-05-09T08:06:37.749118+0000","last_clean":"2024-05-09T08:06:37.749118+0000","last_became_active":"2024-05-09T08:02:14.864470+0000","last_became_peered":"2024-05-09T08:02:14.864470+0000","last_unstale":"2024-05-09T08:06:37.749118+0000","last_undegraded":"2024-05-09T08:06:37.749118+0000","last_fullsized":"2024-05-09T08:06:37.749118+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:00:40.599475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"27'306","reported_seq":326,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:15.290595+0000","last_change":"2024-05-09T08:02:20.054896+0000","last_active":"2024-05-09T08:07:15.290595+0000","last_peered":"2024-05-09T08:07:15.290595+0000","last_clean":"2024-05-09T08:07:15.290595+0000","last_became_active":"2024-05-09T08:02:20.054424+0000","last_became_peered":"2024-05-09T08:02:20.054424+0000","last_unstale":"2024-05-09T08:07:15.290595+0000","last_undegraded":"2024-05-09T08:07:15.290595+0000","last_fullsized":"2024-05-09T08:07:15.290595+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":306,"log_dups_size":0,"ondisk_log_size":306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:23:44.316545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":249,"num_write_kb":679936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:27.747018+0000","last_change":"2024-05-09T08:02:15.905448+0000","last_active":"2024-05-09T08:04:27.747018+0000","last_peered":"2024-05-09T08:04:27.747018+0000","last_clean":"2024-05-09T08:04:27.747018+0000","last_became_active":"2024-05-09T08:02:15.905131+0000","last_became_peered":"2024-05-09T08:02:15.905131+0000","last_unstale":"2024-05-09T08:04:27.747018+0000","last_undegraded":"2024-05-09T08:04:27.747018+0000","last_fullsized":"2024-05-09T08:04:27.747018+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:25:01.238816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"27'21","reported_seq":48,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:15.020483+0000","last_change":"2024-05-09T08:02:14.868410+0000","last_active":"2024-05-09T08:07:15.020483+0000","last_peered":"2024-05-09T08:07:15.020483+0000","last_clean":"2024-05-09T08:07:15.020483+0000","last_became_active":"2024-05-09T08:02:14.865621+0000","last_became_peered":"2024-05-09T08:02:14.865621+0000","last_unstale":"2024-05-09T08:07:15.020483+0000","last_undegraded":"2024-05-09T08:07:15.020483+0000","last_fullsized":"2024-05-09T08:07:15.020483+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:43:01.061801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":41,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"27'328","reported_seq":348,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:15.272939+0000","last_change":"2024-05-09T08:02:20.057603+0000","last_active":"2024-05-09T08:07:15.272939+0000","last_peered":"2024-05-09T08:07:15.272939+0000","last_clean":"2024-05-09T08:07:15.272939+0000","last_became_active":"2024-05-09T08:02:20.055365+0000","last_became_peered":"2024-05-09T08:02:20.055365+0000","last_unstale":"2024-05-09T08:07:15.272939+0000","last_undegraded":"2024-05-09T08:07:15.272939+0000","last_fullsized":"2024-05-09T08:07:15.272939+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":328,"log_dups_size":0,"ondisk_log_size":328,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:48:38.283562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":264,"num_write_kb":720896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:22.759212+0000","last_change":"2024-05-09T08:02:15.890980+0000","last_active":"2024-05-09T08:04:22.759212+0000","last_peered":"2024-05-09T08:04:22.759212+0000","last_clean":"2024-05-09T08:04:22.759212+0000","last_became_active":"2024-05-09T08:02:15.890346+0000","last_became_peered":"2024-05-09T08:02:15.890346+0000","last_unstale":"2024-05-09T08:04:22.759212+0000","last_undegraded":"2024-05-09T08:04:22.759212+0000","last_fullsized":"2024-05-09T08:04:22.759212+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:29:59.441314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:22.734773+0000","last_change":"2024-05-09T08:02:14.876206+0000","last_active":"2024-05-09T08:04:22.734773+0000","last_peered":"2024-05-09T08:04:22.734773+0000","last_clean":"2024-05-09T08:04:22.734773+0000","last_became_active":"2024-05-09T08:02:14.875894+0000","last_became_peered":"2024-05-09T08:02:14.875894+0000","last_unstale":"2024-05-09T08:04:22.734773+0000","last_undegraded":"2024-05-09T08:04:22.734773+0000","last_fullsized":"2024-05-09T08:04:22.734773+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:49:53.882513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"27'297","reported_seq":317,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:15.288524+0000","last_change":"2024-05-09T08:02:20.055222+0000","last_active":"2024-05-09T08:07:15.288524+0000","last_peered":"2024-05-09T08:07:15.288524+0000","last_clean":"2024-05-09T08:07:15.288524+0000","last_became_active":"2024-05-09T08:02:20.053802+0000","last_became_peered":"2024-05-09T08:02:20.053802+0000","last_unstale":"2024-05-09T08:07:15.288524+0000","last_undegraded":"2024-05-09T08:07:15.288524+0000","last_fullsized":"2024-05-09T08:07:15.288524+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":297,"log_dups_size":0,"ondisk_log_size":297,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:42:00.939587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":249,"num_write_kb":679936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:22.735039+0000","last_change":"2024-05-09T08:02:15.904528+0000","last_active":"2024-05-09T08:04:22.735039+0000","last_peered":"2024-05-09T08:04:22.735039+0000","last_clean":"2024-05-09T08:04:22.735039+0000","last_became_active":"2024-05-09T08:02:15.904049+0000","last_became_peered":"2024-05-09T08:02:15.904049+0000","last_unstale":"2024-05-09T08:04:22.735039+0000","last_undegraded":"2024-05-09T08:04:22.735039+0000","last_fullsized":"2024-05-09T08:04:22.735039+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:53:11.390395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:22.759235+0000","last_change":"2024-05-09T08:02:14.872897+0000","last_active":"2024-05-09T08:04:22.759235+0000","last_peered":"2024-05-09T08:04:22.759235+0000","last_clean":"2024-05-09T08:04:22.759235+0000","last_became_active":"2024-05-09T08:02:14.872096+0000","last_became_peered":"2024-05-09T08:02:14.872096+0000","last_unstale":"2024-05-09T08:04:22.759235+0000","last_undegraded":"2024-05-09T08:04:22.759235+0000","last_fullsized":"2024-05-09T08:04:22.759235+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:00:40.599475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"27'320","reported_seq":340,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:15.220609+0000","last_change":"2024-05-09T08:02:19.963716+0000","last_active":"2024-05-09T08:07:15.220609+0000","last_peered":"2024-05-09T08:07:15.220609+0000","last_clean":"2024-05-09T08:07:15.220609+0000","last_became_active":"2024-05-09T08:02:19.962922+0000","last_became_peered":"2024-05-09T08:02:19.962922+0000","last_unstale":"2024-05-09T08:07:15.220609+0000","last_undegraded":"2024-05-09T08:07:15.220609+0000","last_fullsized":"2024-05-09T08:07:15.220609+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":320,"log_dups_size":0,"ondisk_log_size":320,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:42:00.939587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":273,"num_write_kb":745472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:22.734664+0000","last_change":"2024-05-09T08:02:15.901450+0000","last_active":"2024-05-09T08:04:22.734664+0000","last_peered":"2024-05-09T08:04:22.734664+0000","last_clean":"2024-05-09T08:04:22.734664+0000","last_became_active":"2024-05-09T08:02:15.901180+0000","last_became_peered":"2024-05-09T08:02:15.901180+0000","last_unstale":"2024-05-09T08:04:22.734664+0000","last_undegraded":"2024-05-09T08:04:22.734664+0000","last_fullsized":"2024-05-09T08:04:22.734664+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:43:02.075442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:22.734997+0000","last_change":"2024-05-09T08:02:14.876518+0000","last_active":"2024-05-09T08:04:22.734997+0000","last_peered":"2024-05-09T08:04:22.734997+0000","last_clean":"2024-05-09T08:04:22.734997+0000","last_became_active":"2024-05-09T08:02:14.876175+0000","last_became_peered":"2024-05-09T08:02:14.876175+0000","last_unstale":"2024-05-09T08:04:22.734997+0000","last_undegraded":"2024-05-09T08:04:22.734997+0000","last_fullsized":"2024-05-09T08:04:22.734997+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:00:40.599475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"27'301","reported_seq":315,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:15.286265+0000","last_change":"2024-05-09T08:03:02.516595+0000","last_active":"2024-05-09T08:07:15.286265+0000","last_peered":"2024-05-09T08:07:15.286265+0000","last_clean":"2024-05-09T08:07:15.286265+0000","last_became_active":"2024-05-09T08:03:02.516104+0000","last_became_peered":"2024-05-09T08:03:02.516104+0000","last_unstale":"2024-05-09T08:07:15.286265+0000","last_undegraded":"2024-05-09T08:07:15.286265+0000","last_fullsized":"2024-05-09T08:07:15.286265+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":301,"log_dups_size":0,"ondisk_log_size":301,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:47:30.081323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":264,"num_write_kb":720896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3,6],"acting":[5,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:17.758674+0000","last_change":"2024-05-09T08:02:15.891774+0000","last_active":"2024-05-09T08:04:17.758674+0000","last_peered":"2024-05-09T08:04:17.758674+0000","last_clean":"2024-05-09T08:04:17.758674+0000","last_became_active":"2024-05-09T08:02:15.891407+0000","last_became_peered":"2024-05-09T08:02:15.891407+0000","last_unstale":"2024-05-09T08:04:17.758674+0000","last_undegraded":"2024-05-09T08:04:17.758674+0000","last_fullsized":"2024-05-09T08:04:17.758674+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:05:02.181367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"27'6","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:17.381608+0000","last_change":"2024-05-09T08:02:14.883655+0000","last_active":"2024-05-09T08:07:17.381608+0000","last_peered":"2024-05-09T08:07:17.381608+0000","last_clean":"2024-05-09T08:07:17.381608+0000","last_became_active":"2024-05-09T08:02:14.882359+0000","last_became_peered":"2024-05-09T08:02:14.882359+0000","last_unstale":"2024-05-09T08:07:17.381608+0000","last_undegraded":"2024-05-09T08:07:17.381608+0000","last_fullsized":"2024-05-09T08:07:17.381608+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:46:11.160815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":529,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"27'319","reported_seq":339,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:15.257021+0000","last_change":"2024-05-09T08:02:19.953319+0000","last_active":"2024-05-09T08:07:15.257021+0000","last_peered":"2024-05-09T08:07:15.257021+0000","last_clean":"2024-05-09T08:07:15.257021+0000","last_became_active":"2024-05-09T08:02:19.952740+0000","last_became_peered":"2024-05-09T08:02:19.952740+0000","last_unstale":"2024-05-09T08:07:15.257021+0000","last_undegraded":"2024-05-09T08:07:15.257021+0000","last_fullsized":"2024-05-09T08:07:15.257021+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":319,"log_dups_size":0,"ondisk_log_size":319,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:25:28.804048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":270,"num_write_kb":737280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:22.752933+0000","last_change":"2024-05-09T08:02:15.913828+0000","last_active":"2024-05-09T08:04:22.752933+0000","last_peered":"2024-05-09T08:04:22.752933+0000","last_clean":"2024-05-09T08:04:22.752933+0000","last_became_active":"2024-05-09T08:02:15.913313+0000","last_became_peered":"2024-05-09T08:02:15.913313+0000","last_unstale":"2024-05-09T08:04:22.752933+0000","last_undegraded":"2024-05-09T08:04:22.752933+0000","last_fullsized":"2024-05-09T08:04:22.752933+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:20:09.935394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:17.734196+0000","last_change":"2024-05-09T08:02:14.876438+0000","last_active":"2024-05-09T08:04:17.734196+0000","last_peered":"2024-05-09T08:04:17.734196+0000","last_clean":"2024-05-09T08:04:17.734196+0000","last_became_active":"2024-05-09T08:02:14.876146+0000","last_became_peered":"2024-05-09T08:02:14.876146+0000","last_unstale":"2024-05-09T08:04:17.734196+0000","last_undegraded":"2024-05-09T08:04:17.734196+0000","last_fullsized":"2024-05-09T08:04:17.734196+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:36:49.629991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"27'378","reported_seq":398,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:15.271544+0000","last_change":"2024-05-09T08:02:20.051067+0000","last_active":"2024-05-09T08:07:15.271544+0000","last_peered":"2024-05-09T08:07:15.271544+0000","last_clean":"2024-05-09T08:07:15.271544+0000","last_became_active":"2024-05-09T08:02:20.050593+0000","last_became_peered":"2024-05-09T08:02:20.050593+0000","last_unstale":"2024-05-09T08:07:15.271544+0000","last_undegraded":"2024-05-09T08:07:15.271544+0000","last_fullsized":"2024-05-09T08:07:15.271544+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":378,"log_dups_size":0,"ondisk_log_size":378,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:25:28.804048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":339,"num_write_kb":925696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:22.746418+0000","last_change":"2024-05-09T08:02:15.899137+0000","last_active":"2024-05-09T08:04:22.746418+0000","last_peered":"2024-05-09T08:04:22.746418+0000","last_clean":"2024-05-09T08:04:22.746418+0000","last_became_active":"2024-05-09T08:02:15.898766+0000","last_became_peered":"2024-05-09T08:02:15.898766+0000","last_unstale":"2024-05-09T08:04:22.746418+0000","last_undegraded":"2024-05-09T08:04:22.746418+0000","last_fullsized":"2024-05-09T08:04:22.746418+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:47:32.605288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:22.735011+0000","last_change":"2024-05-09T08:02:14.868850+0000","last_active":"2024-05-09T08:04:22.735011+0000","last_peered":"2024-05-09T08:04:22.735011+0000","last_clean":"2024-05-09T08:04:22.735011+0000","last_became_active":"2024-05-09T08:02:14.868384+0000","last_became_peered":"2024-05-09T08:02:14.868384+0000","last_unstale":"2024-05-09T08:04:22.735011+0000","last_undegraded":"2024-05-09T08:04:22.735011+0000","last_fullsized":"2024-05-09T08:04:22.735011+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:14:52.920972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":183,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:12.733624+0000","last_change":"2024-05-09T08:02:11.559804+0000","last_active":"2024-05-09T08:04:12.733624+0000","last_peered":"2024-05-09T08:04:12.733624+0000","last_clean":"2024-05-09T08:04:12.733624+0000","last_became_active":"2024-05-09T08:02:11.558761+0000","last_became_peered":"2024-05-09T08:02:11.558761+0000","last_unstale":"2024-05-09T08:04:12.733624+0000","last_undegraded":"2024-05-09T08:04:12.733624+0000","last_fullsized":"2024-05-09T08:04:12.733624+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:09.800143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:09.800143+0000","last_clean_scrub_stamp":"2024-05-09T08:02:09.800143+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:18:35.964149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"27'315","reported_seq":335,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:15.264268+0000","last_change":"2024-05-09T08:02:20.054746+0000","last_active":"2024-05-09T08:07:15.264268+0000","last_peered":"2024-05-09T08:07:15.264268+0000","last_clean":"2024-05-09T08:07:15.264268+0000","last_became_active":"2024-05-09T08:02:20.054186+0000","last_became_peered":"2024-05-09T08:02:20.054186+0000","last_unstale":"2024-05-09T08:07:15.264268+0000","last_undegraded":"2024-05-09T08:07:15.264268+0000","last_fullsized":"2024-05-09T08:07:15.264268+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":315,"log_dups_size":0,"ondisk_log_size":315,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:53:15.458120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":276,"num_write_kb":753664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:32.735875+0000","last_change":"2024-05-09T08:02:15.889657+0000","last_active":"2024-05-09T08:04:32.735875+0000","last_peered":"2024-05-09T08:04:32.735875+0000","last_clean":"2024-05-09T08:04:32.735875+0000","last_became_active":"2024-05-09T08:02:15.889381+0000","last_became_peered":"2024-05-09T08:02:15.889381+0000","last_unstale":"2024-05-09T08:04:32.735875+0000","last_undegraded":"2024-05-09T08:04:32.735875+0000","last_fullsized":"2024-05-09T08:04:32.735875+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:59:14.957806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:32.747257+0000","last_change":"2024-05-09T08:02:14.868892+0000","last_active":"2024-05-09T08:04:32.747257+0000","last_peered":"2024-05-09T08:04:32.747257+0000","last_clean":"2024-05-09T08:04:32.747257+0000","last_became_active":"2024-05-09T08:02:14.868513+0000","last_became_peered":"2024-05-09T08:02:14.868513+0000","last_unstale":"2024-05-09T08:04:32.747257+0000","last_undegraded":"2024-05-09T08:04:32.747257+0000","last_fullsized":"2024-05-09T08:04:32.747257+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:19:31.913236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"27'338","reported_seq":358,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:15.275986+0000","last_change":"2024-05-09T08:02:20.053097+0000","last_active":"2024-05-09T08:07:15.275986+0000","last_peered":"2024-05-09T08:07:15.275986+0000","last_clean":"2024-05-09T08:07:15.275986+0000","last_became_active":"2024-05-09T08:02:20.052653+0000","last_became_peered":"2024-05-09T08:02:20.052653+0000","last_unstale":"2024-05-09T08:07:15.275986+0000","last_undegraded":"2024-05-09T08:07:15.275986+0000","last_fullsized":"2024-05-09T08:07:15.275986+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":338,"log_dups_size":0,"ondisk_log_size":338,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:25:05.306541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":279,"num_write_kb":761856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:32.736237+0000","last_change":"2024-05-09T08:02:15.899059+0000","last_active":"2024-05-09T08:04:32.736237+0000","last_peered":"2024-05-09T08:04:32.736237+0000","last_clean":"2024-05-09T08:04:32.736237+0000","last_became_active":"2024-05-09T08:02:15.898692+0000","last_became_peered":"2024-05-09T08:02:15.898692+0000","last_unstale":"2024-05-09T08:04:32.736237+0000","last_undegraded":"2024-05-09T08:04:32.736237+0000","last_fullsized":"2024-05-09T08:04:32.736237+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:52:36.461913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"24'1","reported_seq":28,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:32.760505+0000","last_change":"2024-05-09T08:02:14.873071+0000","last_active":"2024-05-09T08:04:32.760505+0000","last_peered":"2024-05-09T08:04:32.760505+0000","last_clean":"2024-05-09T08:04:32.760505+0000","last_became_active":"2024-05-09T08:02:14.872556+0000","last_became_peered":"2024-05-09T08:02:14.872556+0000","last_unstale":"2024-05-09T08:04:32.760505+0000","last_undegraded":"2024-05-09T08:04:32.760505+0000","last_fullsized":"2024-05-09T08:04:32.760505+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:57:06.093718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"27'299","reported_seq":319,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:15.257565+0000","last_change":"2024-05-09T08:02:19.960414+0000","last_active":"2024-05-09T08:07:15.257565+0000","last_peered":"2024-05-09T08:07:15.257565+0000","last_clean":"2024-05-09T08:07:15.257565+0000","last_became_active":"2024-05-09T08:02:19.957941+0000","last_became_peered":"2024-05-09T08:02:19.957941+0000","last_unstale":"2024-05-09T08:07:15.257565+0000","last_undegraded":"2024-05-09T08:07:15.257565+0000","last_fullsized":"2024-05-09T08:07:15.257565+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":299,"log_dups_size":0,"ondisk_log_size":299,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:41:31.470546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":261,"num_write_kb":712704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:32.735920+0000","last_change":"2024-05-09T08:02:15.884395+0000","last_active":"2024-05-09T08:04:32.735920+0000","last_peered":"2024-05-09T08:04:32.735920+0000","last_clean":"2024-05-09T08:04:32.735920+0000","last_became_active":"2024-05-09T08:02:15.883788+0000","last_became_peered":"2024-05-09T08:02:15.883788+0000","last_unstale":"2024-05-09T08:04:32.735920+0000","last_undegraded":"2024-05-09T08:04:32.735920+0000","last_fullsized":"2024-05-09T08:04:32.735920+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:48:34.215837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"24'2","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:32.747303+0000","last_change":"2024-05-09T08:02:14.885044+0000","last_active":"2024-05-09T08:04:32.747303+0000","last_peered":"2024-05-09T08:04:32.747303+0000","last_clean":"2024-05-09T08:04:32.747303+0000","last_became_active":"2024-05-09T08:02:14.884706+0000","last_became_peered":"2024-05-09T08:02:14.884706+0000","last_unstale":"2024-05-09T08:04:32.747303+0000","last_undegraded":"2024-05-09T08:04:32.747303+0000","last_fullsized":"2024-05-09T08:04:32.747303+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:14:52.920972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"27'365","reported_seq":385,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:15.276867+0000","last_change":"2024-05-09T08:02:20.050865+0000","last_active":"2024-05-09T08:07:15.276867+0000","last_peered":"2024-05-09T08:07:15.276867+0000","last_clean":"2024-05-09T08:07:15.276867+0000","last_became_active":"2024-05-09T08:02:20.050270+0000","last_became_peered":"2024-05-09T08:02:20.050270+0000","last_unstale":"2024-05-09T08:07:15.276867+0000","last_undegraded":"2024-05-09T08:07:15.276867+0000","last_fullsized":"2024-05-09T08:07:15.276867+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":365,"log_dups_size":0,"ondisk_log_size":365,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:48:38.283562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":318,"num_write_kb":868352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:32.735905+0000","last_change":"2024-05-09T08:02:15.884737+0000","last_active":"2024-05-09T08:04:32.735905+0000","last_peered":"2024-05-09T08:04:32.735905+0000","last_clean":"2024-05-09T08:04:32.735905+0000","last_became_active":"2024-05-09T08:02:15.884465+0000","last_became_peered":"2024-05-09T08:02:15.884465+0000","last_unstale":"2024-05-09T08:04:32.735905+0000","last_undegraded":"2024-05-09T08:04:32.735905+0000","last_fullsized":"2024-05-09T08:04:32.735905+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:52:36.461913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"24'1","reported_seq":28,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:32.735864+0000","last_change":"2024-05-09T08:02:14.875003+0000","last_active":"2024-05-09T08:04:32.735864+0000","last_peered":"2024-05-09T08:04:32.735864+0000","last_clean":"2024-05-09T08:04:32.735864+0000","last_became_active":"2024-05-09T08:02:14.874665+0000","last_became_peered":"2024-05-09T08:02:14.874665+0000","last_unstale":"2024-05-09T08:04:32.735864+0000","last_undegraded":"2024-05-09T08:04:32.735864+0000","last_fullsized":"2024-05-09T08:04:32.735864+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:18:39.991319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"27'302","reported_seq":322,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:15.294350+0000","last_change":"2024-05-09T08:02:19.952692+0000","last_active":"2024-05-09T08:07:15.294350+0000","last_peered":"2024-05-09T08:07:15.294350+0000","last_clean":"2024-05-09T08:07:15.294350+0000","last_became_active":"2024-05-09T08:02:19.952224+0000","last_became_peered":"2024-05-09T08:02:19.952224+0000","last_unstale":"2024-05-09T08:07:15.294350+0000","last_undegraded":"2024-05-09T08:07:15.294350+0000","last_fullsized":"2024-05-09T08:07:15.294350+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":302,"log_dups_size":0,"ondisk_log_size":302,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:47:36.673013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":258,"num_write_kb":704512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:37.736256+0000","last_change":"2024-05-09T08:02:15.883408+0000","last_active":"2024-05-09T08:04:37.736256+0000","last_peered":"2024-05-09T08:04:37.736256+0000","last_clean":"2024-05-09T08:04:37.736256+0000","last_became_active":"2024-05-09T08:02:15.883103+0000","last_became_peered":"2024-05-09T08:02:15.883103+0000","last_unstale":"2024-05-09T08:04:37.736256+0000","last_undegraded":"2024-05-09T08:04:37.736256+0000","last_fullsized":"2024-05-09T08:04:37.736256+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:53:11.390395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"27'9","reported_seq":36,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:06:57.786502+0000","last_change":"2024-05-09T08:02:14.873184+0000","last_active":"2024-05-09T08:06:57.786502+0000","last_peered":"2024-05-09T08:06:57.786502+0000","last_clean":"2024-05-09T08:06:57.786502+0000","last_became_active":"2024-05-09T08:02:14.871055+0000","last_became_peered":"2024-05-09T08:02:14.871055+0000","last_unstale":"2024-05-09T08:06:57.786502+0000","last_undegraded":"2024-05-09T08:06:57.786502+0000","last_fullsized":"2024-05-09T08:06:57.786502+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:19:56.315972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"27'416","reported_seq":436,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:15.283703+0000","last_change":"2024-05-09T08:02:19.950823+0000","last_active":"2024-05-09T08:07:15.283703+0000","last_peered":"2024-05-09T08:07:15.283703+0000","last_clean":"2024-05-09T08:07:15.283703+0000","last_became_active":"2024-05-09T08:02:19.950352+0000","last_became_peered":"2024-05-09T08:02:19.950352+0000","last_unstale":"2024-05-09T08:07:15.283703+0000","last_undegraded":"2024-05-09T08:07:15.283703+0000","last_fullsized":"2024-05-09T08:07:15.283703+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":416,"log_dups_size":0,"ondisk_log_size":416,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:59:19.025530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":360,"num_write_kb":983040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:37.736615+0000","last_change":"2024-05-09T08:02:15.899115+0000","last_active":"2024-05-09T08:04:37.736615+0000","last_peered":"2024-05-09T08:04:37.736615+0000","last_clean":"2024-05-09T08:04:37.736615+0000","last_became_active":"2024-05-09T08:02:15.898811+0000","last_became_peered":"2024-05-09T08:02:15.898811+0000","last_unstale":"2024-05-09T08:04:37.736615+0000","last_undegraded":"2024-05-09T08:04:37.736615+0000","last_fullsized":"2024-05-09T08:04:37.736615+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:43:06.938904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":28,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:37.736636+0000","last_change":"2024-05-09T08:02:14.860818+0000","last_active":"2024-05-09T08:04:37.736636+0000","last_peered":"2024-05-09T08:04:37.736636+0000","last_clean":"2024-05-09T08:04:37.736636+0000","last_became_active":"2024-05-09T08:02:14.860518+0000","last_became_peered":"2024-05-09T08:02:14.860518+0000","last_unstale":"2024-05-09T08:04:37.736636+0000","last_undegraded":"2024-05-09T08:04:37.736636+0000","last_fullsized":"2024-05-09T08:04:37.736636+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:25:23.306827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"27'318","reported_seq":338,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:15.275345+0000","last_change":"2024-05-09T08:02:20.053251+0000","last_active":"2024-05-09T08:07:15.275345+0000","last_peered":"2024-05-09T08:07:15.275345+0000","last_clean":"2024-05-09T08:07:15.275345+0000","last_became_active":"2024-05-09T08:02:20.052783+0000","last_became_peered":"2024-05-09T08:02:20.052783+0000","last_unstale":"2024-05-09T08:07:15.275345+0000","last_undegraded":"2024-05-09T08:07:15.275345+0000","last_fullsized":"2024-05-09T08:07:15.275345+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":318,"log_dups_size":0,"ondisk_log_size":318,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:16:41.863341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":273,"num_write_kb":745472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:37.736278+0000","last_change":"2024-05-09T08:02:15.883972+0000","last_active":"2024-05-09T08:04:37.736278+0000","last_peered":"2024-05-09T08:04:37.736278+0000","last_clean":"2024-05-09T08:04:37.736278+0000","last_became_active":"2024-05-09T08:02:15.883613+0000","last_became_peered":"2024-05-09T08:02:15.883613+0000","last_unstale":"2024-05-09T08:04:37.736278+0000","last_undegraded":"2024-05-09T08:04:37.736278+0000","last_fullsized":"2024-05-09T08:04:37.736278+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:41:27.402821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:37.736743+0000","last_change":"2024-05-09T08:02:14.883366+0000","last_active":"2024-05-09T08:04:37.736743+0000","last_peered":"2024-05-09T08:04:37.736743+0000","last_clean":"2024-05-09T08:04:37.736743+0000","last_became_active":"2024-05-09T08:02:14.883135+0000","last_became_peered":"2024-05-09T08:02:14.883135+0000","last_unstale":"2024-05-09T08:04:37.736743+0000","last_undegraded":"2024-05-09T08:04:37.736743+0000","last_fullsized":"2024-05-09T08:04:37.736743+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:32:48.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"27'316","reported_seq":336,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:15.284009+0000","last_change":"2024-05-09T08:02:20.050888+0000","last_active":"2024-05-09T08:07:15.284009+0000","last_peered":"2024-05-09T08:07:15.284009+0000","last_clean":"2024-05-09T08:07:15.284009+0000","last_became_active":"2024-05-09T08:02:20.050390+0000","last_became_peered":"2024-05-09T08:02:20.050390+0000","last_unstale":"2024-05-09T08:07:15.284009+0000","last_undegraded":"2024-05-09T08:07:15.284009+0000","last_fullsized":"2024-05-09T08:07:15.284009+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":316,"log_dups_size":0,"ondisk_log_size":316,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:10:24.797526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":273,"num_write_kb":745472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"27'1","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:05:05.201608+0000","last_change":"2024-05-09T08:02:15.899096+0000","last_active":"2024-05-09T08:05:05.201608+0000","last_peered":"2024-05-09T08:05:05.201608+0000","last_clean":"2024-05-09T08:05:05.201608+0000","last_became_active":"2024-05-09T08:02:15.898811+0000","last_became_peered":"2024-05-09T08:02:15.898811+0000","last_unstale":"2024-05-09T08:05:05.201608+0000","last_undegraded":"2024-05-09T08:05:05.201608+0000","last_fullsized":"2024-05-09T08:05:05.201608+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:28:51.229262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"24'2","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:37.755081+0000","last_change":"2024-05-09T08:02:14.868657+0000","last_active":"2024-05-09T08:04:37.755081+0000","last_peered":"2024-05-09T08:04:37.755081+0000","last_clean":"2024-05-09T08:04:37.755081+0000","last_became_active":"2024-05-09T08:02:14.868385+0000","last_became_peered":"2024-05-09T08:02:14.868385+0000","last_unstale":"2024-05-09T08:04:37.755081+0000","last_undegraded":"2024-05-09T08:04:37.755081+0000","last_fullsized":"2024-05-09T08:04:37.755081+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:46:11.160815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"27'378","reported_seq":398,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:15.279263+0000","last_change":"2024-05-09T08:02:20.053155+0000","last_active":"2024-05-09T08:07:15.279263+0000","last_peered":"2024-05-09T08:07:15.279263+0000","last_clean":"2024-05-09T08:07:15.279263+0000","last_became_active":"2024-05-09T08:02:20.052629+0000","last_became_peered":"2024-05-09T08:02:20.052629+0000","last_unstale":"2024-05-09T08:07:15.279263+0000","last_undegraded":"2024-05-09T08:07:15.279263+0000","last_fullsized":"2024-05-09T08:07:15.279263+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":378,"log_dups_size":0,"ondisk_log_size":378,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:35:54.628874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":327,"num_write_kb":892928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:42.736850+0000","last_change":"2024-05-09T08:02:15.905406+0000","last_active":"2024-05-09T08:04:42.736850+0000","last_peered":"2024-05-09T08:04:42.736850+0000","last_clean":"2024-05-09T08:04:42.736850+0000","last_became_active":"2024-05-09T08:02:15.905076+0000","last_became_peered":"2024-05-09T08:02:15.905076+0000","last_unstale":"2024-05-09T08:04:42.736850+0000","last_undegraded":"2024-05-09T08:04:42.736850+0000","last_fullsized":"2024-05-09T08:04:42.736850+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:25:01.238816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"27'189","reported_seq":217,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:14.967504+0000","last_change":"2024-05-09T08:02:14.880410+0000","last_active":"2024-05-09T08:07:14.967504+0000","last_peered":"2024-05-09T08:07:14.967504+0000","last_clean":"2024-05-09T08:07:14.967504+0000","last_became_active":"2024-05-09T08:02:14.880045+0000","last_became_peered":"2024-05-09T08:02:14.880045+0000","last_unstale":"2024-05-09T08:07:14.967504+0000","last_undegraded":"2024-05-09T08:07:14.967504+0000","last_fullsized":"2024-05-09T08:07:14.967504+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":189,"log_dups_size":0,"ondisk_log_size":189,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:19:56.315972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":447435,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":188,"num_write_kb":582,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"27'332","reported_seq":352,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:15.206123+0000","last_change":"2024-05-09T08:02:19.963378+0000","last_active":"2024-05-09T08:07:15.206123+0000","last_peered":"2024-05-09T08:07:15.206123+0000","last_clean":"2024-05-09T08:07:15.206123+0000","last_became_active":"2024-05-09T08:02:19.958964+0000","last_became_peered":"2024-05-09T08:02:19.958964+0000","last_unstale":"2024-05-09T08:07:15.206123+0000","last_undegraded":"2024-05-09T08:07:15.206123+0000","last_fullsized":"2024-05-09T08:07:15.206123+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":332,"log_dups_size":0,"ondisk_log_size":332,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:41:31.470546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":288,"num_write_kb":786432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:37.736256+0000","last_change":"2024-05-09T08:02:15.888428+0000","last_active":"2024-05-09T08:04:37.736256+0000","last_peered":"2024-05-09T08:04:37.736256+0000","last_clean":"2024-05-09T08:04:37.736256+0000","last_became_active":"2024-05-09T08:02:15.888071+0000","last_became_peered":"2024-05-09T08:02:15.888071+0000","last_unstale":"2024-05-09T08:04:37.736256+0000","last_undegraded":"2024-05-09T08:04:37.736256+0000","last_fullsized":"2024-05-09T08:04:37.736256+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:36:50.643632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"27'8","reported_seq":35,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:15.298845+0000","last_change":"2024-05-09T08:02:14.872697+0000","last_active":"2024-05-09T08:07:15.298845+0000","last_peered":"2024-05-09T08:07:15.298845+0000","last_clean":"2024-05-09T08:07:15.298845+0000","last_became_active":"2024-05-09T08:02:14.871945+0000","last_became_peered":"2024-05-09T08:02:14.871945+0000","last_unstale":"2024-05-09T08:07:15.298845+0000","last_undegraded":"2024-05-09T08:07:15.298845+0000","last_fullsized":"2024-05-09T08:07:15.298845+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:19:56.315972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"27'353","reported_seq":373,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:15.280987+0000","last_change":"2024-05-09T08:02:19.953635+0000","last_active":"2024-05-09T08:07:15.280987+0000","last_peered":"2024-05-09T08:07:15.280987+0000","last_clean":"2024-05-09T08:07:15.280987+0000","last_became_active":"2024-05-09T08:02:19.953166+0000","last_became_peered":"2024-05-09T08:02:19.953166+0000","last_unstale":"2024-05-09T08:07:15.280987+0000","last_undegraded":"2024-05-09T08:07:15.280987+0000","last_fullsized":"2024-05-09T08:07:15.280987+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":353,"log_dups_size":0,"ondisk_log_size":353,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:18:40.377352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":297,"num_write_kb":811008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:42.755912+0000","last_change":"2024-05-09T08:02:15.905285+0000","last_active":"2024-05-09T08:04:42.755912+0000","last_peered":"2024-05-09T08:04:42.755912+0000","last_clean":"2024-05-09T08:04:42.755912+0000","last_became_active":"2024-05-09T08:02:15.904898+0000","last_became_peered":"2024-05-09T08:02:15.904898+0000","last_unstale":"2024-05-09T08:04:42.755912+0000","last_undegraded":"2024-05-09T08:04:42.755912+0000","last_fullsized":"2024-05-09T08:04:42.755912+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:47:32.605288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:42.737369+0000","last_change":"2024-05-09T08:02:14.868682+0000","last_active":"2024-05-09T08:04:42.737369+0000","last_peered":"2024-05-09T08:04:42.737369+0000","last_clean":"2024-05-09T08:04:42.737369+0000","last_became_active":"2024-05-09T08:02:14.868294+0000","last_became_peered":"2024-05-09T08:02:14.868294+0000","last_unstale":"2024-05-09T08:04:42.737369+0000","last_undegraded":"2024-05-09T08:04:42.737369+0000","last_fullsized":"2024-05-09T08:04:42.737369+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:57:06.093718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"27'344","reported_seq":364,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:15.272036+0000","last_change":"2024-05-09T08:02:19.956115+0000","last_active":"2024-05-09T08:07:15.272036+0000","last_peered":"2024-05-09T08:07:15.272036+0000","last_clean":"2024-05-09T08:07:15.272036+0000","last_became_active":"2024-05-09T08:02:19.955662+0000","last_became_peered":"2024-05-09T08:02:19.955662+0000","last_unstale":"2024-05-09T08:07:15.272036+0000","last_undegraded":"2024-05-09T08:07:15.272036+0000","last_fullsized":"2024-05-09T08:07:15.272036+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":344,"log_dups_size":0,"ondisk_log_size":344,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:15:28.286330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":297,"num_write_kb":811008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:42.737356+0000","last_change":"2024-05-09T08:02:15.888406+0000","last_active":"2024-05-09T08:04:42.737356+0000","last_peered":"2024-05-09T08:04:42.737356+0000","last_clean":"2024-05-09T08:04:42.737356+0000","last_became_active":"2024-05-09T08:02:15.888090+0000","last_became_peered":"2024-05-09T08:02:15.888090+0000","last_unstale":"2024-05-09T08:04:42.737356+0000","last_undegraded":"2024-05-09T08:04:42.737356+0000","last_fullsized":"2024-05-09T08:04:42.737356+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:53:23.178610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"27'1","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:05:05.164756+0000","last_change":"2024-05-09T08:02:14.862463+0000","last_active":"2024-05-09T08:05:05.164756+0000","last_peered":"2024-05-09T08:05:05.164756+0000","last_clean":"2024-05-09T08:05:05.164756+0000","last_became_active":"2024-05-09T08:02:14.862104+0000","last_became_peered":"2024-05-09T08:02:14.862104+0000","last_unstale":"2024-05-09T08:05:05.164756+0000","last_undegraded":"2024-05-09T08:05:05.164756+0000","last_fullsized":"2024-05-09T08:05:05.164756+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:32:48.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"27'379","reported_seq":399,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:15.252469+0000","last_change":"2024-05-09T08:02:20.053600+0000","last_active":"2024-05-09T08:07:15.252469+0000","last_peered":"2024-05-09T08:07:15.252469+0000","last_clean":"2024-05-09T08:07:15.252469+0000","last_became_active":"2024-05-09T08:02:20.052485+0000","last_became_peered":"2024-05-09T08:02:20.052485+0000","last_unstale":"2024-05-09T08:07:15.252469+0000","last_undegraded":"2024-05-09T08:07:15.252469+0000","last_fullsized":"2024-05-09T08:07:15.252469+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":379,"log_dups_size":0,"ondisk_log_size":379,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:35:54.628874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":327,"num_write_kb":892928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"27'324","reported_seq":344,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:15.288169+0000","last_change":"2024-05-09T08:02:20.054016+0000","last_active":"2024-05-09T08:07:15.288169+0000","last_peered":"2024-05-09T08:07:15.288169+0000","last_clean":"2024-05-09T08:07:15.288169+0000","last_became_active":"2024-05-09T08:02:20.053250+0000","last_became_peered":"2024-05-09T08:02:20.053250+0000","last_unstale":"2024-05-09T08:07:15.288169+0000","last_undegraded":"2024-05-09T08:07:15.288169+0000","last_fullsized":"2024-05-09T08:07:15.288169+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":324,"log_dups_size":0,"ondisk_log_size":324,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:52:40.529638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":273,"num_write_kb":745472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"27'1","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:07.834698+0000","last_change":"2024-05-09T08:02:15.907080+0000","last_active":"2024-05-09T08:07:07.834698+0000","last_peered":"2024-05-09T08:07:07.834698+0000","last_clean":"2024-05-09T08:07:07.834698+0000","last_became_active":"2024-05-09T08:02:15.906600+0000","last_became_peered":"2024-05-09T08:02:15.906600+0000","last_unstale":"2024-05-09T08:07:07.834698+0000","last_undegraded":"2024-05-09T08:07:07.834698+0000","last_fullsized":"2024-05-09T08:07:07.834698+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:43:06.938904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:42.737302+0000","last_change":"2024-05-09T08:02:14.870990+0000","last_active":"2024-05-09T08:04:42.737302+0000","last_peered":"2024-05-09T08:04:42.737302+0000","last_clean":"2024-05-09T08:04:42.737302+0000","last_became_active":"2024-05-09T08:02:14.869818+0000","last_became_peered":"2024-05-09T08:02:14.869818+0000","last_unstale":"2024-05-09T08:04:42.737302+0000","last_undegraded":"2024-05-09T08:04:42.737302+0000","last_fullsized":"2024-05-09T08:04:42.737302+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:25:23.306827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"27'313","reported_seq":333,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:15.240636+0000","last_change":"2024-05-09T08:02:19.952582+0000","last_active":"2024-05-09T08:07:15.240636+0000","last_peered":"2024-05-09T08:07:15.240636+0000","last_clean":"2024-05-09T08:07:15.240636+0000","last_became_active":"2024-05-09T08:02:19.952117+0000","last_became_peered":"2024-05-09T08:02:19.952117+0000","last_unstale":"2024-05-09T08:07:15.240636+0000","last_undegraded":"2024-05-09T08:07:15.240636+0000","last_fullsized":"2024-05-09T08:07:15.240636+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":313,"log_dups_size":0,"ondisk_log_size":313,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:42:00.939587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":267,"num_write_kb":729088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:42.748943+0000","last_change":"2024-05-09T08:02:15.913109+0000","last_active":"2024-05-09T08:04:42.748943+0000","last_peered":"2024-05-09T08:04:42.748943+0000","last_clean":"2024-05-09T08:04:42.748943+0000","last_became_active":"2024-05-09T08:02:15.912883+0000","last_became_peered":"2024-05-09T08:02:15.912883+0000","last_unstale":"2024-05-09T08:04:42.748943+0000","last_undegraded":"2024-05-09T08:04:42.748943+0000","last_fullsized":"2024-05-09T08:04:42.748943+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:29:59.441314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:42.748992+0000","last_change":"2024-05-09T08:02:14.885256+0000","last_active":"2024-05-09T08:04:42.748992+0000","last_peered":"2024-05-09T08:04:42.748992+0000","last_clean":"2024-05-09T08:04:42.748992+0000","last_became_active":"2024-05-09T08:02:14.884965+0000","last_became_peered":"2024-05-09T08:02:14.884965+0000","last_unstale":"2024-05-09T08:04:42.748992+0000","last_undegraded":"2024-05-09T08:04:42.748992+0000","last_fullsized":"2024-05-09T08:04:42.748992+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:14:52.920972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"24'1","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:47.737722+0000","last_change":"2024-05-09T08:02:14.860339+0000","last_active":"2024-05-09T08:04:47.737722+0000","last_peered":"2024-05-09T08:04:47.737722+0000","last_clean":"2024-05-09T08:04:47.737722+0000","last_became_active":"2024-05-09T08:02:14.859917+0000","last_became_peered":"2024-05-09T08:02:14.859917+0000","last_unstale":"2024-05-09T08:04:47.737722+0000","last_undegraded":"2024-05-09T08:04:47.737722+0000","last_fullsized":"2024-05-09T08:04:47.737722+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:14:52.920972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:47.756573+0000","last_change":"2024-05-09T08:02:15.897001+0000","last_active":"2024-05-09T08:04:47.756573+0000","last_peered":"2024-05-09T08:04:47.756573+0000","last_clean":"2024-05-09T08:04:47.756573+0000","last_became_active":"2024-05-09T08:02:15.896708+0000","last_became_peered":"2024-05-09T08:02:15.896708+0000","last_unstale":"2024-05-09T08:04:47.756573+0000","last_undegraded":"2024-05-09T08:04:47.756573+0000","last_fullsized":"2024-05-09T08:04:47.756573+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:29:59.441314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"27'338","reported_seq":358,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:15.293090+0000","last_change":"2024-05-09T08:02:19.955369+0000","last_active":"2024-05-09T08:07:15.293090+0000","last_peered":"2024-05-09T08:07:15.293090+0000","last_clean":"2024-05-09T08:07:15.293090+0000","last_became_active":"2024-05-09T08:02:19.954829+0000","last_became_peered":"2024-05-09T08:02:19.954829+0000","last_unstale":"2024-05-09T08:07:15.293090+0000","last_undegraded":"2024-05-09T08:07:15.293090+0000","last_fullsized":"2024-05-09T08:07:15.293090+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":338,"log_dups_size":0,"ondisk_log_size":338,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:49:58.963879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":297,"num_write_kb":811008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"24'1","reported_seq":28,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:47.753171+0000","last_change":"2024-05-09T08:02:14.883856+0000","last_active":"2024-05-09T08:04:47.753171+0000","last_peered":"2024-05-09T08:04:47.753171+0000","last_clean":"2024-05-09T08:04:47.753171+0000","last_became_active":"2024-05-09T08:02:14.882610+0000","last_became_peered":"2024-05-09T08:02:14.882610+0000","last_unstale":"2024-05-09T08:04:47.753171+0000","last_undegraded":"2024-05-09T08:04:47.753171+0000","last_fullsized":"2024-05-09T08:04:47.753171+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:19:31.913236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:47.737288+0000","last_change":"2024-05-09T08:02:15.889974+0000","last_active":"2024-05-09T08:04:47.737288+0000","last_peered":"2024-05-09T08:04:47.737288+0000","last_clean":"2024-05-09T08:04:47.737288+0000","last_became_active":"2024-05-09T08:02:15.889650+0000","last_became_peered":"2024-05-09T08:02:15.889650+0000","last_unstale":"2024-05-09T08:04:47.737288+0000","last_undegraded":"2024-05-09T08:04:47.737288+0000","last_fullsized":"2024-05-09T08:04:47.737288+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:00:41.613116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"27'361","reported_seq":381,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:15.284334+0000","last_change":"2024-05-09T08:02:19.955362+0000","last_active":"2024-05-09T08:07:15.284334+0000","last_peered":"2024-05-09T08:07:15.284334+0000","last_clean":"2024-05-09T08:07:15.284334+0000","last_became_active":"2024-05-09T08:02:19.954947+0000","last_became_peered":"2024-05-09T08:02:19.954947+0000","last_unstale":"2024-05-09T08:07:15.284334+0000","last_undegraded":"2024-05-09T08:07:15.284334+0000","last_fullsized":"2024-05-09T08:07:15.284334+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":361,"log_dups_size":0,"ondisk_log_size":361,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:43:30.549966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":312,"num_write_kb":851968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:22.734849+0000","last_change":"2024-05-09T08:02:15.909379+0000","last_active":"2024-05-09T08:04:22.734849+0000","last_peered":"2024-05-09T08:04:22.734849+0000","last_clean":"2024-05-09T08:04:22.734849+0000","last_became_active":"2024-05-09T08:02:15.909086+0000","last_became_peered":"2024-05-09T08:02:15.909086+0000","last_unstale":"2024-05-09T08:04:22.734849+0000","last_undegraded":"2024-05-09T08:04:22.734849+0000","last_fullsized":"2024-05-09T08:04:22.734849+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:05:02.181367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:22.745878+0000","last_change":"2024-05-09T08:02:15.891242+0000","last_active":"2024-05-09T08:04:22.745878+0000","last_peered":"2024-05-09T08:04:22.745878+0000","last_clean":"2024-05-09T08:04:22.745878+0000","last_became_active":"2024-05-09T08:02:15.890640+0000","last_became_peered":"2024-05-09T08:02:15.890640+0000","last_unstale":"2024-05-09T08:04:22.745878+0000","last_undegraded":"2024-05-09T08:04:22.745878+0000","last_fullsized":"2024-05-09T08:04:22.745878+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:57:07.107359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:22.734971+0000","last_change":"2024-05-09T08:02:15.899001+0000","last_active":"2024-05-09T08:04:22.734971+0000","last_peered":"2024-05-09T08:04:22.734971+0000","last_clean":"2024-05-09T08:04:22.734971+0000","last_became_active":"2024-05-09T08:02:15.898695+0000","last_became_peered":"2024-05-09T08:02:15.898695+0000","last_unstale":"2024-05-09T08:04:22.734971+0000","last_undegraded":"2024-05-09T08:04:22.734971+0000","last_fullsized":"2024-05-09T08:04:22.734971+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:41:27.402821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:22.752972+0000","last_change":"2024-05-09T08:02:15.907374+0000","last_active":"2024-05-09T08:04:22.752972+0000","last_peered":"2024-05-09T08:04:22.752972+0000","last_clean":"2024-05-09T08:04:22.752972+0000","last_became_active":"2024-05-09T08:02:15.906767+0000","last_became_peered":"2024-05-09T08:02:15.906767+0000","last_unstale":"2024-05-09T08:04:22.752972+0000","last_undegraded":"2024-05-09T08:04:22.752972+0000","last_fullsized":"2024-05-09T08:04:22.752972+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:05:02.181367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:27.746979+0000","last_change":"2024-05-09T08:02:15.905326+0000","last_active":"2024-05-09T08:04:27.746979+0000","last_peered":"2024-05-09T08:04:27.746979+0000","last_clean":"2024-05-09T08:04:27.746979+0000","last_became_active":"2024-05-09T08:02:15.905083+0000","last_became_peered":"2024-05-09T08:02:15.905083+0000","last_unstale":"2024-05-09T08:04:27.746979+0000","last_undegraded":"2024-05-09T08:04:27.746979+0000","last_fullsized":"2024-05-09T08:04:27.746979+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:23:40.248821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:27.759721+0000","last_change":"2024-05-09T08:02:15.907753+0000","last_active":"2024-05-09T08:04:27.759721+0000","last_peered":"2024-05-09T08:04:27.759721+0000","last_clean":"2024-05-09T08:04:27.759721+0000","last_became_active":"2024-05-09T08:02:15.907480+0000","last_became_peered":"2024-05-09T08:02:15.907480+0000","last_unstale":"2024-05-09T08:04:27.759721+0000","last_undegraded":"2024-05-09T08:04:27.759721+0000","last_fullsized":"2024-05-09T08:04:27.759721+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:20:07.368327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:27.735486+0000","last_change":"2024-05-09T08:02:15.888605+0000","last_active":"2024-05-09T08:04:27.735486+0000","last_peered":"2024-05-09T08:04:27.735486+0000","last_clean":"2024-05-09T08:04:27.735486+0000","last_became_active":"2024-05-09T08:02:15.887981+0000","last_became_peered":"2024-05-09T08:02:15.887981+0000","last_unstale":"2024-05-09T08:04:27.735486+0000","last_undegraded":"2024-05-09T08:04:27.735486+0000","last_fullsized":"2024-05-09T08:04:27.735486+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:47:32.605288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:27.747004+0000","last_change":"2024-05-09T08:02:15.904744+0000","last_active":"2024-05-09T08:04:27.747004+0000","last_peered":"2024-05-09T08:04:27.747004+0000","last_clean":"2024-05-09T08:04:27.747004+0000","last_became_active":"2024-05-09T08:02:15.904334+0000","last_became_peered":"2024-05-09T08:02:15.904334+0000","last_unstale":"2024-05-09T08:04:27.747004+0000","last_undegraded":"2024-05-09T08:04:27.747004+0000","last_fullsized":"2024-05-09T08:04:27.747004+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:30:33.351158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:27.735580+0000","last_change":"2024-05-09T08:02:15.904835+0000","last_active":"2024-05-09T08:04:27.735580+0000","last_peered":"2024-05-09T08:04:27.735580+0000","last_clean":"2024-05-09T08:04:27.735580+0000","last_became_active":"2024-05-09T08:02:15.904416+0000","last_became_peered":"2024-05-09T08:02:15.904416+0000","last_unstale":"2024-05-09T08:04:27.735580+0000","last_undegraded":"2024-05-09T08:04:27.735580+0000","last_fullsized":"2024-05-09T08:04:27.735580+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:47:32.605288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:32.747675+0000","last_change":"2024-05-09T08:02:15.898324+0000","last_active":"2024-05-09T08:04:32.747675+0000","last_peered":"2024-05-09T08:04:32.747675+0000","last_clean":"2024-05-09T08:04:32.747675+0000","last_became_active":"2024-05-09T08:02:15.897981+0000","last_became_peered":"2024-05-09T08:02:15.897981+0000","last_unstale":"2024-05-09T08:04:32.747675+0000","last_undegraded":"2024-05-09T08:04:32.747675+0000","last_fullsized":"2024-05-09T08:04:32.747675+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:01:50.341133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:32.736201+0000","last_change":"2024-05-09T08:02:15.890945+0000","last_active":"2024-05-09T08:04:32.736201+0000","last_peered":"2024-05-09T08:04:32.736201+0000","last_clean":"2024-05-09T08:04:32.736201+0000","last_became_active":"2024-05-09T08:02:15.890647+0000","last_became_peered":"2024-05-09T08:02:15.890647+0000","last_unstale":"2024-05-09T08:04:32.736201+0000","last_undegraded":"2024-05-09T08:04:32.736201+0000","last_fullsized":"2024-05-09T08:04:32.736201+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:41:56.871863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:32.754272+0000","last_change":"2024-05-09T08:02:15.913728+0000","last_active":"2024-05-09T08:04:32.754272+0000","last_peered":"2024-05-09T08:04:32.754272+0000","last_clean":"2024-05-09T08:04:32.754272+0000","last_became_active":"2024-05-09T08:02:15.909818+0000","last_became_peered":"2024-05-09T08:02:15.909818+0000","last_unstale":"2024-05-09T08:04:32.754272+0000","last_undegraded":"2024-05-09T08:04:32.754272+0000","last_fullsized":"2024-05-09T08:04:32.754272+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:41:55.788033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:32.736139+0000","last_change":"2024-05-09T08:02:15.885025+0000","last_active":"2024-05-09T08:04:32.736139+0000","last_peered":"2024-05-09T08:04:32.736139+0000","last_clean":"2024-05-09T08:04:32.736139+0000","last_became_active":"2024-05-09T08:02:15.884517+0000","last_became_peered":"2024-05-09T08:02:15.884517+0000","last_unstale":"2024-05-09T08:04:32.736139+0000","last_undegraded":"2024-05-09T08:04:32.736139+0000","last_fullsized":"2024-05-09T08:04:32.736139+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:01:50.341133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:32.735890+0000","last_change":"2024-05-09T08:02:15.890306+0000","last_active":"2024-05-09T08:04:32.735890+0000","last_peered":"2024-05-09T08:04:32.735890+0000","last_clean":"2024-05-09T08:04:32.735890+0000","last_became_active":"2024-05-09T08:02:15.889998+0000","last_became_peered":"2024-05-09T08:02:15.889998+0000","last_unstale":"2024-05-09T08:04:32.735890+0000","last_undegraded":"2024-05-09T08:04:32.735890+0000","last_fullsized":"2024-05-09T08:04:32.735890+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:16:57.446444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:37.747977+0000","last_change":"2024-05-09T08:02:15.886417+0000","last_active":"2024-05-09T08:04:37.747977+0000","last_peered":"2024-05-09T08:04:37.747977+0000","last_clean":"2024-05-09T08:04:37.747977+0000","last_became_active":"2024-05-09T08:02:15.885525+0000","last_became_peered":"2024-05-09T08:02:15.885525+0000","last_unstale":"2024-05-09T08:04:37.747977+0000","last_undegraded":"2024-05-09T08:04:37.747977+0000","last_fullsized":"2024-05-09T08:04:37.747977+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:43:02.075442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:37.736779+0000","last_change":"2024-05-09T08:02:15.893172+0000","last_active":"2024-05-09T08:04:37.736779+0000","last_peered":"2024-05-09T08:04:37.736779+0000","last_clean":"2024-05-09T08:04:37.736779+0000","last_became_active":"2024-05-09T08:02:15.892886+0000","last_became_peered":"2024-05-09T08:02:15.892886+0000","last_unstale":"2024-05-09T08:04:37.736779+0000","last_undegraded":"2024-05-09T08:04:37.736779+0000","last_fullsized":"2024-05-09T08:04:37.736779+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:59:14.957806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:37.736714+0000","last_change":"2024-05-09T08:02:15.908659+0000","last_active":"2024-05-09T08:04:37.736714+0000","last_peered":"2024-05-09T08:04:37.736714+0000","last_clean":"2024-05-09T08:04:37.736714+0000","last_became_active":"2024-05-09T08:02:15.908307+0000","last_became_peered":"2024-05-09T08:02:15.908307+0000","last_unstale":"2024-05-09T08:04:37.736714+0000","last_undegraded":"2024-05-09T08:04:37.736714+0000","last_fullsized":"2024-05-09T08:04:37.736714+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:20:07.368327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:37.761383+0000","last_change":"2024-05-09T08:02:15.885800+0000","last_active":"2024-05-09T08:04:37.761383+0000","last_peered":"2024-05-09T08:04:37.761383+0000","last_clean":"2024-05-09T08:04:37.761383+0000","last_became_active":"2024-05-09T08:02:15.885185+0000","last_became_peered":"2024-05-09T08:02:15.885185+0000","last_unstale":"2024-05-09T08:04:37.761383+0000","last_undegraded":"2024-05-09T08:04:37.761383+0000","last_fullsized":"2024-05-09T08:04:37.761383+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:01:50.341133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:37.755012+0000","last_change":"2024-05-09T08:02:15.911496+0000","last_active":"2024-05-09T08:04:37.755012+0000","last_peered":"2024-05-09T08:04:37.755012+0000","last_clean":"2024-05-09T08:04:37.755012+0000","last_became_active":"2024-05-09T08:02:15.911142+0000","last_became_peered":"2024-05-09T08:02:15.911142+0000","last_unstale":"2024-05-09T08:04:37.755012+0000","last_undegraded":"2024-05-09T08:04:37.755012+0000","last_fullsized":"2024-05-09T08:04:37.755012+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:27:59.315750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:42.749013+0000","last_change":"2024-05-09T08:02:15.898847+0000","last_active":"2024-05-09T08:04:42.749013+0000","last_peered":"2024-05-09T08:04:42.749013+0000","last_clean":"2024-05-09T08:04:42.749013+0000","last_became_active":"2024-05-09T08:02:15.898480+0000","last_became_peered":"2024-05-09T08:02:15.898480+0000","last_unstale":"2024-05-09T08:04:42.749013+0000","last_undegraded":"2024-05-09T08:04:42.749013+0000","last_fullsized":"2024-05-09T08:04:42.749013+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:20:07.368327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:42.736898+0000","last_change":"2024-05-09T08:02:15.883117+0000","last_active":"2024-05-09T08:04:42.736898+0000","last_peered":"2024-05-09T08:04:42.736898+0000","last_clean":"2024-05-09T08:04:42.736898+0000","last_became_active":"2024-05-09T08:02:15.882141+0000","last_became_peered":"2024-05-09T08:02:15.882141+0000","last_unstale":"2024-05-09T08:04:42.736898+0000","last_undegraded":"2024-05-09T08:04:42.736898+0000","last_fullsized":"2024-05-09T08:04:42.736898+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:57:11.352151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":31,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:42.737262+0000","last_change":"2024-05-09T08:03:02.322462+0000","last_active":"2024-05-09T08:04:42.737262+0000","last_peered":"2024-05-09T08:04:42.737262+0000","last_clean":"2024-05-09T08:04:42.737262+0000","last_became_active":"2024-05-09T08:03:02.322073+0000","last_became_peered":"2024-05-09T08:03:02.322073+0000","last_unstale":"2024-05-09T08:04:42.737262+0000","last_undegraded":"2024-05-09T08:04:42.737262+0000","last_fullsized":"2024-05-09T08:04:42.737262+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:16:37.795617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"27'1","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:05:27.208367+0000","last_change":"2024-05-09T08:02:15.886482+0000","last_active":"2024-05-09T08:05:27.208367+0000","last_peered":"2024-05-09T08:05:27.208367+0000","last_clean":"2024-05-09T08:05:27.208367+0000","last_became_active":"2024-05-09T08:02:15.886183+0000","last_became_peered":"2024-05-09T08:02:15.886183+0000","last_unstale":"2024-05-09T08:05:27.208367+0000","last_undegraded":"2024-05-09T08:05:27.208367+0000","last_fullsized":"2024-05-09T08:05:27.208367+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:23:40.248821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:42.736901+0000","last_change":"2024-05-09T08:02:15.887506+0000","last_active":"2024-05-09T08:04:42.736901+0000","last_peered":"2024-05-09T08:04:42.736901+0000","last_clean":"2024-05-09T08:04:42.736901+0000","last_became_active":"2024-05-09T08:02:15.887225+0000","last_became_peered":"2024-05-09T08:02:15.887225+0000","last_unstale":"2024-05-09T08:04:42.736901+0000","last_undegraded":"2024-05-09T08:04:42.736901+0000","last_fullsized":"2024-05-09T08:04:42.736901+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:25:24.736323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:47.753113+0000","last_change":"2024-05-09T08:02:15.902065+0000","last_active":"2024-05-09T08:04:47.753113+0000","last_peered":"2024-05-09T08:04:47.753113+0000","last_clean":"2024-05-09T08:04:47.753113+0000","last_became_active":"2024-05-09T08:02:15.901766+0000","last_became_peered":"2024-05-09T08:02:15.901766+0000","last_unstale":"2024-05-09T08:04:47.753113+0000","last_undegraded":"2024-05-09T08:04:47.753113+0000","last_fullsized":"2024-05-09T08:04:47.753113+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:05:02.181367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:47.737268+0000","last_change":"2024-05-09T08:02:15.911058+0000","last_active":"2024-05-09T08:04:47.737268+0000","last_peered":"2024-05-09T08:04:47.737268+0000","last_clean":"2024-05-09T08:04:47.737268+0000","last_became_active":"2024-05-09T08:02:15.910716+0000","last_became_peered":"2024-05-09T08:02:15.910716+0000","last_unstale":"2024-05-09T08:04:47.737268+0000","last_undegraded":"2024-05-09T08:04:47.737268+0000","last_fullsized":"2024-05-09T08:04:47.737268+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:41:56.871863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:47.737757+0000","last_change":"2024-05-09T08:02:15.908653+0000","last_active":"2024-05-09T08:04:47.737757+0000","last_peered":"2024-05-09T08:04:47.737757+0000","last_clean":"2024-05-09T08:04:47.737757+0000","last_became_active":"2024-05-09T08:02:15.908290+0000","last_became_peered":"2024-05-09T08:02:15.908290+0000","last_unstale":"2024-05-09T08:04:47.737757+0000","last_undegraded":"2024-05-09T08:04:47.737757+0000","last_fullsized":"2024-05-09T08:04:47.737757+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:18:36.309627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:47.737286+0000","last_change":"2024-05-09T08:02:15.896652+0000","last_active":"2024-05-09T08:04:47.737286+0000","last_peered":"2024-05-09T08:04:47.737286+0000","last_clean":"2024-05-09T08:04:47.737286+0000","last_became_active":"2024-05-09T08:02:15.896384+0000","last_became_peered":"2024-05-09T08:02:15.896384+0000","last_unstale":"2024-05-09T08:04:47.737286+0000","last_undegraded":"2024-05-09T08:04:47.737286+0000","last_fullsized":"2024-05-09T08:04:47.737286+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:18:36.309627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:05:02.751368+0000","last_change":"2024-05-09T08:03:02.615230+0000","last_active":"2024-05-09T08:05:02.751368+0000","last_peered":"2024-05-09T08:05:02.751368+0000","last_clean":"2024-05-09T08:05:02.751368+0000","last_became_active":"2024-05-09T08:03:02.614387+0000","last_became_peered":"2024-05-09T08:03:02.614387+0000","last_unstale":"2024-05-09T08:05:02.751368+0000","last_undegraded":"2024-05-09T08:05:02.751368+0000","last_fullsized":"2024-05-09T08:05:02.751368+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:53:23.178610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:22.752990+0000","last_change":"2024-05-09T08:02:15.905078+0000","last_active":"2024-05-09T08:04:22.752990+0000","last_peered":"2024-05-09T08:04:22.752990+0000","last_clean":"2024-05-09T08:04:22.752990+0000","last_became_active":"2024-05-09T08:02:15.904736+0000","last_became_peered":"2024-05-09T08:02:15.904736+0000","last_unstale":"2024-05-09T08:04:22.752990+0000","last_undegraded":"2024-05-09T08:04:22.752990+0000","last_fullsized":"2024-05-09T08:04:22.752990+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:01:50.341133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:22.759254+0000","last_change":"2024-05-09T08:02:15.897743+0000","last_active":"2024-05-09T08:04:22.759254+0000","last_peered":"2024-05-09T08:04:22.759254+0000","last_clean":"2024-05-09T08:04:22.759254+0000","last_became_active":"2024-05-09T08:02:15.897367+0000","last_became_peered":"2024-05-09T08:02:15.897367+0000","last_unstale":"2024-05-09T08:04:22.759254+0000","last_undegraded":"2024-05-09T08:04:22.759254+0000","last_fullsized":"2024-05-09T08:04:22.759254+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:47:32.605288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:05:07.752186+0000","last_change":"2024-05-09T08:03:02.323519+0000","last_active":"2024-05-09T08:05:07.752186+0000","last_peered":"2024-05-09T08:05:07.752186+0000","last_clean":"2024-05-09T08:05:07.752186+0000","last_became_active":"2024-05-09T08:03:02.322397+0000","last_became_peered":"2024-05-09T08:03:02.322397+0000","last_unstale":"2024-05-09T08:05:07.752186+0000","last_undegraded":"2024-05-09T08:05:07.752186+0000","last_fullsized":"2024-05-09T08:05:07.752186+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:47:32.605288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9216,"num_write_kb":25165824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10755,"ondisk_log_size":10755,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":448762,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":277,"num_write_kb":617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":966656,"data_stored":897524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":258,"ondisk_log_size":258,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574919,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50092,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321748,"statfs":{"total":96636764160,"available":96585469952,"internally_reserved":0,"allocated":888832,"data_stored":709919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574919,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":76260,"kb_used_data":732,"kb_used_omap":1,"kb_used_meta":75518,"kb_avail":94295580,"statfs":{"total":96636764160,"available":96558673920,"internally_reserved":0,"allocated":749568,"data_stored":566404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":77330886},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574919,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45352,"kb_used_data":288,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94326488,"statfs":{"total":96636764160,"available":96590323712,"internally_reserved":0,"allocated":294912,"data_stored":119059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574920,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":53680,"kb_used_data":296,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94318160,"statfs":{"total":96636764160,"available":96581795840,"internally_reserved":0,"allocated":303104,"data_stored":119095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":54655431},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574920,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":76264,"kb_used_data":736,"kb_used_omap":1,"kb_used_meta":75518,"kb_avail":94295576,"statfs":{"total":96636764160,"available":96558669824,"internally_reserved":0,"allocated":753664,"data_stored":566933,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":77330887},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574920,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":57832,"kb_used_data":288,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94314008,"statfs":{"total":96636764160,"available":96577544192,"internally_reserved":0,"allocated":294912,"data_stored":119498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574921,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":75820,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":75518,"kb_avail":94296020,"statfs":{"total":96636764160,"available":96559124480,"internally_reserved":0,"allocated":299008,"data_stored":119129,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77330885},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574920,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45940,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325900,"statfs":{"total":96636764160,"available":96589721600,"internally_reserved":0,"allocated":897024,"data_stored":709975,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":462848,"data_stored":447964,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":458752,"data_stored":447435,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T08:07:56.608 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-09T08:07:56.862 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:56.851+0000 7f3c21000640 1 Processor -- start 2024-05-09T08:07:56.862 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:56.852+0000 7f3c21000640 1 -- start start 2024-05-09T08:07:56.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:56.853+0000 7f3c21000640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f3c1c0ca5c0 0x7f3c1c0ca9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:07:56.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:56.853+0000 7f3c21000640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3c1c15ad80 con 0x7f3c1c0ccf70 2024-05-09T08:07:56.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:56.853+0000 7f3c21000640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3c1c15b060 con 0x7f3c1c0cf370 2024-05-09T08:07:56.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:56.853+0000 7f3c21000640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f3c1c06c7d0 con 0x7f3c1c0ca5c0 2024-05-09T08:07:56.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:56.854+0000 7f3c1be00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f3c1c0cf370 0x7f3c1c06d220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.1:38686/0 (socket says 172.21.15.1:38686) 2024-05-09T08:07:56.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:56.854+0000 7f3c1be00640 1 -- 172.21.15.1:0/3201796981 learned_addr learned my addr 172.21.15.1:0/3201796981 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T08:07:56.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:56.854+0000 7f3c1b400640 1 --2- 172.21.15.1:0/3201796981 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f3c1c0ca5c0 0x7f3c1c0ca9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:07:56.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:56.854+0000 7f3c1a000640 1 -- 172.21.15.1:0/3201796981 <== mon.1 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1917575797 0 0) 0x7f3c1c15b060 con 0x7f3c1c0cf370 2024-05-09T08:07:56.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:56.854+0000 7f3c1a000640 1 -- 172.21.15.1:0/3201796981 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3c00003520 con 0x7f3c1c0cf370 2024-05-09T08:07:56.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:56.855+0000 7f3c1a000640 1 -- 172.21.15.1:0/3201796981 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3013741310 0 0) 0x7f3c1c15ad80 con 0x7f3c1c0ccf70 2024-05-09T08:07:56.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:56.855+0000 7f3c1a000640 1 -- 172.21.15.1:0/3201796981 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3c1c15b060 con 0x7f3c1c0ccf70 2024-05-09T08:07:56.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:56.855+0000 7f3c1b400640 1 -- 172.21.15.1:0/3201796981 >> v1:172.21.15.7:6789/0 conn(0x7f3c1c0cf370 legacy=0x7f3c1c06d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:07:56.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:56.855+0000 7f3c1b400640 1 -- 172.21.15.1:0/3201796981 >> v1:172.21.15.1:6789/0 conn(0x7f3c1c0ccf70 legacy=0x7f3c1c15a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:07:56.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:56.855+0000 7f3c1b400640 1 -- 172.21.15.1:0/3201796981 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c1c15b340 con 0x7f3c1c0ca5c0 2024-05-09T08:07:56.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:56.856+0000 7f3c1b400640 1 --2- 172.21.15.1:0/3201796981 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f3c1c0ca5c0 0x7f3c1c0ca9a0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f3c0c009dc0 tx=0x7f3c0c02a760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=28e3a6734ffba37b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:07:56.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:56.856+0000 7f3c1a000640 1 -- 172.21.15.1:0/3201796981 <== mon.2 v2:172.21.15.7:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c0c007740 con 0x7f3c1c0ca5c0 2024-05-09T08:07:56.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:56.856+0000 7f3c1a000640 1 -- 172.21.15.1:0/3201796981 <== mon.2 v2:172.21.15.7:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c0c0078e0 con 0x7f3c1c0ca5c0 2024-05-09T08:07:56.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:56.856+0000 7f3c1a000640 1 -- 172.21.15.1:0/3201796981 <== mon.2 v2:172.21.15.7:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c0c007cb0 con 0x7f3c1c0ca5c0 2024-05-09T08:07:56.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:56.857+0000 7f3c21000640 1 -- 172.21.15.1:0/3201796981 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f3c1c0ca5c0 msgr2=0x7f3c1c0ca9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:07:56.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:56.857+0000 7f3c21000640 1 --2- 172.21.15.1:0/3201796981 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f3c1c0ca5c0 0x7f3c1c0ca9a0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f3c0c009dc0 tx=0x7f3c0c02a760 comp rx=0 tx=0).stop 2024-05-09T08:07:56.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:56.857+0000 7f3c21000640 1 -- 172.21.15.1:0/3201796981 shutdown_connections 2024-05-09T08:07:56.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:56.857+0000 7f3c21000640 1 --2- 172.21.15.1:0/3201796981 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f3c1c0ca5c0 0x7f3c1c0ca9a0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:56.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:56.857+0000 7f3c21000640 1 -- 172.21.15.1:0/3201796981 >> 172.21.15.1:0/3201796981 conn(0x7f3c1c0cfc40 msgr2=0x7f3c1c0d0050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:07:56.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:56.857+0000 7f3c21000640 1 -- 172.21.15.1:0/3201796981 shutdown_connections 2024-05-09T08:07:56.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:56.857+0000 7f3c21000640 1 -- 172.21.15.1:0/3201796981 wait complete. 2024-05-09T08:07:56.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:56.859+0000 7f3c21000640 1 Processor -- start 2024-05-09T08:07:56.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:56.859+0000 7f3c21000640 1 -- start start 2024-05-09T08:07:56.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:56.860+0000 7f3c21000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3c1c0ca5c0 0x7f3c1c1f3730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:07:56.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:56.860+0000 7f3c21000640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3c1c0ccf70 0x7f3c1c156d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:07:56.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:56.860+0000 7f3c21000640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f3c1c0cf370 0x7f3c1c14d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:07:56.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:56.860+0000 7f3c21000640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f3c1c0cf960 con 0x7f3c1c0ccf70 2024-05-09T08:07:56.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:56.860+0000 7f3c21000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f3c1c0cf7e0 con 0x7f3c1c0ca5c0 2024-05-09T08:07:56.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:56.860+0000 7f3c21000640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f3c1c06d930 con 0x7f3c1c0cf370 2024-05-09T08:07:56.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:56.861+0000 7f3c1b400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3c1c0ca5c0 0x7f3c1c1f3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:07:56.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:56.861+0000 7f3c1aa00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3c1c0ccf70 0x7f3c1c156d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:07:56.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:56.861+0000 7f3c1b400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3c1c0ca5c0 0x7f3c1c1f3730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.1:54048/0 (socket says 172.21.15.1:54048) 2024-05-09T08:07:56.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:56.861+0000 7f3c1b400640 1 -- 172.21.15.1:0/3860569727 learned_addr learned my addr 172.21.15.1:0/3860569727 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:07:56.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:56.861+0000 7f3c1be00640 1 --2- 172.21.15.1:0/3860569727 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f3c1c0cf370 0x7f3c1c14d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:07:56.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:56.861+0000 7f3c1b400640 1 -- 172.21.15.1:0/3860569727 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f3c1c0cf370 msgr2=0x7f3c1c14d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:07:56.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:56.861+0000 7f3c1b400640 1 --2- 172.21.15.1:0/3860569727 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f3c1c0cf370 0x7f3c1c14d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:56.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:56.861+0000 7f3c1b400640 1 -- 172.21.15.1:0/3860569727 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3c1c0ccf70 msgr2=0x7f3c1c156d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:07:56.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:56.861+0000 7f3c1b400640 1 --2- 172.21.15.1:0/3860569727 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3c1c0ccf70 0x7f3c1c156d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:56.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:56.861+0000 7f3c1b400640 1 -- 172.21.15.1:0/3860569727 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c1c157a90 con 0x7f3c1c0ca5c0 2024-05-09T08:07:56.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:56.861+0000 7f3c1aa00640 1 --2- 172.21.15.1:0/3860569727 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3c1c0ccf70 0x7f3c1c156d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:07:56.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:56.861+0000 7f3c1b400640 1 --2- 172.21.15.1:0/3860569727 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3c1c0ca5c0 0x7f3c1c1f3730 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f3c0c000f80 tx=0x7f3c0c004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:07:56.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:56.862+0000 7f3c13400640 1 -- 172.21.15.1:0/3860569727 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c0c040070 con 0x7f3c1c0ca5c0 2024-05-09T08:07:56.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:56.862+0000 7f3c13400640 1 -- 172.21.15.1:0/3860569727 <== mon.1 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c0c031c40 con 0x7f3c1c0ca5c0 2024-05-09T08:07:56.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:56.862+0000 7f3c13400640 1 -- 172.21.15.1:0/3860569727 <== mon.1 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c0c009450 con 0x7f3c1c0ca5c0 2024-05-09T08:07:56.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:56.862+0000 7f3c21000640 1 -- 172.21.15.1:0/3860569727 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c1c157240 con 0x7f3c1c0ca5c0 2024-05-09T08:07:56.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:56.862+0000 7f3c21000640 1 -- 172.21.15.1:0/3860569727 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c1c1574d0 con 0x7f3c1c0ca5c0 2024-05-09T08:07:56.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:56.863+0000 7f3c21000640 1 -- 172.21.15.1:0/3860569727 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c1c15ad80 con 0x7f3c1c0ca5c0 2024-05-09T08:07:56.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:56.864+0000 7f3c13400640 1 -- 172.21.15.1:0/3860569727 <== mon.1 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3c0c039040 con 0x7f3c1c0ca5c0 2024-05-09T08:07:56.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:56.865+0000 7f3c13400640 1 --2- 172.21.15.1:0/3860569727 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f3bfc06be30 0x7f3bfc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:07:56.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:56.865+0000 7f3c13400640 1 -- 172.21.15.1:0/3860569727 <== mon.1 v2:172.21.15.7:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7f3c0c0ac8d0 con 0x7f3c1c0ca5c0 2024-05-09T08:07:56.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:56.865+0000 7f3c1aa00640 1 --2- 172.21.15.1:0/3860569727 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f3bfc06be30 0x7f3bfc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:07:56.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:56.866+0000 7f3c1aa00640 1 --2- 172.21.15.1:0/3860569727 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f3bfc06be30 0x7f3bfc06e2f0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f3c08007fd0 tx=0x7f3c08009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:07:56.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:56.870+0000 7f3c13400640 1 -- 172.21.15.1:0/3860569727 <== mon.1 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f3c1c15ad80 con 0x7f3c1c0ca5c0 2024-05-09T08:07:57.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:57.068+0000 7f3c21000640 1 -- 172.21.15.1:0/3860569727 --> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3c1c04f660 con 0x7f3bfc06be30 2024-05-09T08:07:57.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:57.079+0000 7f3c13400640 1 -- 172.21.15.1:0/3860569727 <== mgr.4114 v2:172.21.15.7:6832/3562612464 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+333085 (secure 0 0 0) 0x7f3c1c04f660 con 0x7f3bfc06be30 2024-05-09T08:07:57.101 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T08:07:57.104 INFO:teuthology.orchestra.run.smithi001.stderr:dumped all 2024-05-09T08:07:57.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:57.087+0000 7f3c21000640 1 -- 172.21.15.1:0/3860569727 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f3bfc06be30 msgr2=0x7f3bfc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:07:57.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:57.087+0000 7f3c21000640 1 --2- 172.21.15.1:0/3860569727 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f3bfc06be30 0x7f3bfc06e2f0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f3c08007fd0 tx=0x7f3c08009040 comp rx=0 tx=0).stop 2024-05-09T08:07:57.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:57.087+0000 7f3c21000640 1 -- 172.21.15.1:0/3860569727 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3c1c0ca5c0 msgr2=0x7f3c1c1f3730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:07:57.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:57.087+0000 7f3c21000640 1 --2- 172.21.15.1:0/3860569727 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3c1c0ca5c0 0x7f3c1c1f3730 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f3c0c000f80 tx=0x7f3c0c004a40 comp rx=0 tx=0).stop 2024-05-09T08:07:57.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:57.087+0000 7f3c21000640 1 -- 172.21.15.1:0/3860569727 shutdown_connections 2024-05-09T08:07:57.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:57.087+0000 7f3c21000640 1 --2- 172.21.15.1:0/3860569727 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f3bfc06be30 0x7f3bfc06e2f0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:57.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:57.087+0000 7f3c21000640 1 --2- 172.21.15.1:0/3860569727 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f3c1c0cf370 0x7f3c1c14d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:57.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:57.087+0000 7f3c21000640 1 --2- 172.21.15.1:0/3860569727 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3c1c0ccf70 0x7f3c1c156d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:57.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:57.087+0000 7f3c21000640 1 --2- 172.21.15.1:0/3860569727 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3c1c0ca5c0 0x7f3c1c1f3730 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:57.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:57.087+0000 7f3c21000640 1 -- 172.21.15.1:0/3860569727 >> 172.21.15.1:0/3860569727 conn(0x7f3c1c0cfc40 msgr2=0x7f3c1c06d220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:07:57.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:57.087+0000 7f3c21000640 1 -- 172.21.15.1:0/3860569727 shutdown_connections 2024-05-09T08:07:57.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:57.088+0000 7f3c21000640 1 -- 172.21.15.1:0/3860569727 wait complete. 2024-05-09T08:07:57.119 INFO:teuthology.orchestra.run.smithi001.stdout:{"pg_ready":true,"pg_map":{"version":194,"stamp":"2024-05-09T08:07:55.184855+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1039130,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":121,"num_read_kb":105,"num_write":9652,"num_write_kb":25168857,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":11133,"ondisk_log_size":11133,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":481240,"kb_used_data":4376,"kb_used_omap":12,"kb_used_meta":476787,"kb_avail":754493480,"statfs":{"total":773094113280,"available":772601323520,"internally_reserved":0,"allocated":4481024,"data_stored":3030012,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":488230450},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004075"},"pg_stats":[{"pgid":"2.1f","version":"24'4","reported_seq":31,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:22.753051+0000","last_change":"2024-05-09T08:02:14.875108+0000","last_active":"2024-05-09T08:04:22.753051+0000","last_peered":"2024-05-09T08:04:22.753051+0000","last_clean":"2024-05-09T08:04:22.753051+0000","last_became_active":"2024-05-09T08:02:14.874841+0000","last_became_peered":"2024-05-09T08:02:14.874841+0000","last_unstale":"2024-05-09T08:04:22.753051+0000","last_undegraded":"2024-05-09T08:04:22.753051+0000","last_fullsized":"2024-05-09T08:04:22.753051+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:00:40.599475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:17.745934+0000","last_change":"2024-05-09T08:02:15.899818+0000","last_active":"2024-05-09T08:04:17.745934+0000","last_peered":"2024-05-09T08:04:17.745934+0000","last_clean":"2024-05-09T08:04:17.745934+0000","last_became_active":"2024-05-09T08:02:15.899479+0000","last_became_peered":"2024-05-09T08:02:15.899479+0000","last_unstale":"2024-05-09T08:04:17.745934+0000","last_undegraded":"2024-05-09T08:04:17.745934+0000","last_fullsized":"2024-05-09T08:04:17.745934+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:18:36.309627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"27'360","reported_seq":380,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:15.262265+0000","last_change":"2024-05-09T08:02:20.057944+0000","last_active":"2024-05-09T08:07:15.262265+0000","last_peered":"2024-05-09T08:07:15.262265+0000","last_clean":"2024-05-09T08:07:15.262265+0000","last_became_active":"2024-05-09T08:02:20.057016+0000","last_became_peered":"2024-05-09T08:02:20.057016+0000","last_unstale":"2024-05-09T08:07:15.262265+0000","last_undegraded":"2024-05-09T08:07:15.262265+0000","last_fullsized":"2024-05-09T08:07:15.262265+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":360,"log_dups_size":0,"ondisk_log_size":360,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:17:01.514168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":306,"num_write_kb":835584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"24'1","reported_seq":28,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:17.745962+0000","last_change":"2024-05-09T08:02:14.886137+0000","last_active":"2024-05-09T08:04:17.745962+0000","last_peered":"2024-05-09T08:04:17.745962+0000","last_clean":"2024-05-09T08:04:17.745962+0000","last_became_active":"2024-05-09T08:02:14.885830+0000","last_became_peered":"2024-05-09T08:02:14.885830+0000","last_unstale":"2024-05-09T08:04:17.745962+0000","last_undegraded":"2024-05-09T08:04:17.745962+0000","last_fullsized":"2024-05-09T08:04:17.745962+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:46:11.160815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:22.753022+0000","last_change":"2024-05-09T08:02:15.906280+0000","last_active":"2024-05-09T08:04:22.753022+0000","last_peered":"2024-05-09T08:04:22.753022+0000","last_clean":"2024-05-09T08:04:22.753022+0000","last_became_active":"2024-05-09T08:02:15.905590+0000","last_became_peered":"2024-05-09T08:02:15.905590+0000","last_unstale":"2024-05-09T08:04:22.753022+0000","last_undegraded":"2024-05-09T08:04:22.753022+0000","last_fullsized":"2024-05-09T08:04:22.753022+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:20:07.368327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"27'377","reported_seq":397,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:15.276306+0000","last_change":"2024-05-09T08:02:19.954548+0000","last_active":"2024-05-09T08:07:15.276306+0000","last_peered":"2024-05-09T08:07:15.276306+0000","last_clean":"2024-05-09T08:07:15.276306+0000","last_became_active":"2024-05-09T08:02:19.954161+0000","last_became_peered":"2024-05-09T08:02:19.954161+0000","last_unstale":"2024-05-09T08:07:15.276306+0000","last_undegraded":"2024-05-09T08:07:15.276306+0000","last_fullsized":"2024-05-09T08:07:15.276306+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":377,"log_dups_size":0,"ondisk_log_size":377,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:53:27.246334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":339,"num_write_kb":925696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"27'3","reported_seq":30,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:05:05.164895+0000","last_change":"2024-05-09T08:02:14.867128+0000","last_active":"2024-05-09T08:05:05.164895+0000","last_peered":"2024-05-09T08:05:05.164895+0000","last_clean":"2024-05-09T08:05:05.164895+0000","last_became_active":"2024-05-09T08:02:14.866809+0000","last_became_peered":"2024-05-09T08:02:14.866809+0000","last_unstale":"2024-05-09T08:05:05.164895+0000","last_undegraded":"2024-05-09T08:05:05.164895+0000","last_fullsized":"2024-05-09T08:05:05.164895+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:57:06.093718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:47.737794+0000","last_change":"2024-05-09T08:02:15.888431+0000","last_active":"2024-05-09T08:04:47.737794+0000","last_peered":"2024-05-09T08:04:47.737794+0000","last_clean":"2024-05-09T08:04:47.737794+0000","last_became_active":"2024-05-09T08:02:15.887509+0000","last_became_peered":"2024-05-09T08:02:15.887509+0000","last_unstale":"2024-05-09T08:04:47.737794+0000","last_undegraded":"2024-05-09T08:04:47.737794+0000","last_fullsized":"2024-05-09T08:04:47.737794+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:25:01.238816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"27'324","reported_seq":344,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:15.265243+0000","last_change":"2024-05-09T08:02:19.953600+0000","last_active":"2024-05-09T08:07:15.265243+0000","last_peered":"2024-05-09T08:07:15.265243+0000","last_clean":"2024-05-09T08:07:15.265243+0000","last_became_active":"2024-05-09T08:02:19.952275+0000","last_became_peered":"2024-05-09T08:02:19.952275+0000","last_unstale":"2024-05-09T08:07:15.265243+0000","last_undegraded":"2024-05-09T08:07:15.265243+0000","last_fullsized":"2024-05-09T08:07:15.265243+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":324,"log_dups_size":0,"ondisk_log_size":324,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:25:05.306541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":273,"num_write_kb":745472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"24'2","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:47.737301+0000","last_change":"2024-05-09T08:02:14.870819+0000","last_active":"2024-05-09T08:04:47.737301+0000","last_peered":"2024-05-09T08:04:47.737301+0000","last_clean":"2024-05-09T08:04:47.737301+0000","last_became_active":"2024-05-09T08:02:14.870476+0000","last_became_peered":"2024-05-09T08:02:14.870476+0000","last_unstale":"2024-05-09T08:04:47.737301+0000","last_undegraded":"2024-05-09T08:04:47.737301+0000","last_fullsized":"2024-05-09T08:04:47.737301+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:43:01.061801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:47.737815+0000","last_change":"2024-05-09T08:02:15.886471+0000","last_active":"2024-05-09T08:04:47.737815+0000","last_peered":"2024-05-09T08:04:47.737815+0000","last_clean":"2024-05-09T08:04:47.737815+0000","last_became_active":"2024-05-09T08:02:15.886168+0000","last_became_peered":"2024-05-09T08:02:15.886168+0000","last_unstale":"2024-05-09T08:04:47.737815+0000","last_undegraded":"2024-05-09T08:04:47.737815+0000","last_fullsized":"2024-05-09T08:04:47.737815+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:30:33.351158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"27'298","reported_seq":318,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:15.295149+0000","last_change":"2024-05-09T08:02:19.947347+0000","last_active":"2024-05-09T08:07:15.295149+0000","last_peered":"2024-05-09T08:07:15.295149+0000","last_clean":"2024-05-09T08:07:15.295149+0000","last_became_active":"2024-05-09T08:02:19.946906+0000","last_became_peered":"2024-05-09T08:02:19.946906+0000","last_unstale":"2024-05-09T08:07:15.295149+0000","last_undegraded":"2024-05-09T08:07:15.295149+0000","last_fullsized":"2024-05-09T08:07:15.295149+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":298,"log_dups_size":0,"ondisk_log_size":298,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:30:37.418883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":252,"num_write_kb":688128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:27.735428+0000","last_change":"2024-05-09T08:02:15.880765+0000","last_active":"2024-05-09T08:04:27.735428+0000","last_peered":"2024-05-09T08:04:27.735428+0000","last_clean":"2024-05-09T08:04:27.735428+0000","last_became_active":"2024-05-09T08:02:15.880439+0000","last_became_peered":"2024-05-09T08:02:15.880439+0000","last_unstale":"2024-05-09T08:04:27.735428+0000","last_undegraded":"2024-05-09T08:04:27.735428+0000","last_fullsized":"2024-05-09T08:04:27.735428+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:29:59.441314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"27'3","reported_seq":30,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:05:05.164991+0000","last_change":"2024-05-09T08:02:14.867587+0000","last_active":"2024-05-09T08:05:05.164991+0000","last_peered":"2024-05-09T08:05:05.164991+0000","last_clean":"2024-05-09T08:05:05.164991+0000","last_became_active":"2024-05-09T08:02:14.867291+0000","last_became_peered":"2024-05-09T08:02:14.867291+0000","last_unstale":"2024-05-09T08:05:05.164991+0000","last_undegraded":"2024-05-09T08:05:05.164991+0000","last_fullsized":"2024-05-09T08:05:05.164991+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:20:08.921752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"27'366","reported_seq":386,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:15.267986+0000","last_change":"2024-05-09T08:02:19.954902+0000","last_active":"2024-05-09T08:07:15.267986+0000","last_peered":"2024-05-09T08:07:15.267986+0000","last_clean":"2024-05-09T08:07:15.267986+0000","last_became_active":"2024-05-09T08:02:19.954007+0000","last_became_peered":"2024-05-09T08:02:19.954007+0000","last_unstale":"2024-05-09T08:07:15.267986+0000","last_undegraded":"2024-05-09T08:07:15.267986+0000","last_fullsized":"2024-05-09T08:07:15.267986+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":366,"log_dups_size":0,"ondisk_log_size":366,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:41:31.470546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":309,"num_write_kb":843776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:32.747723+0000","last_change":"2024-05-09T08:02:15.915155+0000","last_active":"2024-05-09T08:04:32.747723+0000","last_peered":"2024-05-09T08:04:32.747723+0000","last_clean":"2024-05-09T08:04:32.747723+0000","last_became_active":"2024-05-09T08:02:15.914852+0000","last_became_peered":"2024-05-09T08:02:15.914852+0000","last_unstale":"2024-05-09T08:04:32.747723+0000","last_undegraded":"2024-05-09T08:04:32.747723+0000","last_fullsized":"2024-05-09T08:04:32.747723+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:53:23.178610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:27.759691+0000","last_change":"2024-05-09T08:02:14.881770+0000","last_active":"2024-05-09T08:04:27.759691+0000","last_peered":"2024-05-09T08:04:27.759691+0000","last_clean":"2024-05-09T08:04:27.759691+0000","last_became_active":"2024-05-09T08:02:14.881437+0000","last_became_peered":"2024-05-09T08:02:14.881437+0000","last_unstale":"2024-05-09T08:04:27.759691+0000","last_undegraded":"2024-05-09T08:04:27.759691+0000","last_fullsized":"2024-05-09T08:04:27.759691+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:25:23.306827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"27'330","reported_seq":350,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:15.284795+0000","last_change":"2024-05-09T08:02:19.955928+0000","last_active":"2024-05-09T08:07:15.284795+0000","last_peered":"2024-05-09T08:07:15.284795+0000","last_clean":"2024-05-09T08:07:15.284795+0000","last_became_active":"2024-05-09T08:02:19.955432+0000","last_became_peered":"2024-05-09T08:02:19.955432+0000","last_unstale":"2024-05-09T08:07:15.284795+0000","last_undegraded":"2024-05-09T08:07:15.284795+0000","last_fullsized":"2024-05-09T08:07:15.284795+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":330,"log_dups_size":0,"ondisk_log_size":330,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:53:15.458120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":291,"num_write_kb":794624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:27.746932+0000","last_change":"2024-05-09T08:02:15.913311+0000","last_active":"2024-05-09T08:04:27.746932+0000","last_peered":"2024-05-09T08:04:27.746932+0000","last_clean":"2024-05-09T08:04:27.746932+0000","last_became_active":"2024-05-09T08:02:15.912484+0000","last_became_peered":"2024-05-09T08:02:15.912484+0000","last_unstale":"2024-05-09T08:04:27.746932+0000","last_undegraded":"2024-05-09T08:04:27.746932+0000","last_fullsized":"2024-05-09T08:04:27.746932+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:43:06.938904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:27.759654+0000","last_change":"2024-05-09T08:02:14.868884+0000","last_active":"2024-05-09T08:04:27.759654+0000","last_peered":"2024-05-09T08:04:27.759654+0000","last_clean":"2024-05-09T08:04:27.759654+0000","last_became_active":"2024-05-09T08:02:14.868553+0000","last_became_peered":"2024-05-09T08:02:14.868553+0000","last_unstale":"2024-05-09T08:04:27.759654+0000","last_undegraded":"2024-05-09T08:04:27.759654+0000","last_fullsized":"2024-05-09T08:04:27.759654+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:32:48.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"27'321","reported_seq":335,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:15.287758+0000","last_change":"2024-05-09T08:03:02.609342+0000","last_active":"2024-05-09T08:07:15.287758+0000","last_peered":"2024-05-09T08:07:15.287758+0000","last_clean":"2024-05-09T08:07:15.287758+0000","last_became_active":"2024-05-09T08:03:02.607564+0000","last_became_peered":"2024-05-09T08:03:02.607564+0000","last_unstale":"2024-05-09T08:07:15.287758+0000","last_undegraded":"2024-05-09T08:07:15.287758+0000","last_fullsized":"2024-05-09T08:07:15.287758+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":321,"log_dups_size":0,"ondisk_log_size":321,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:23:44.316545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":267,"num_write_kb":729088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4,7],"acting":[2,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:27.735197+0000","last_change":"2024-05-09T08:02:15.891115+0000","last_active":"2024-05-09T08:04:27.735197+0000","last_peered":"2024-05-09T08:04:27.735197+0000","last_clean":"2024-05-09T08:04:27.735197+0000","last_became_active":"2024-05-09T08:02:15.890806+0000","last_became_peered":"2024-05-09T08:02:15.890806+0000","last_unstale":"2024-05-09T08:04:27.735197+0000","last_undegraded":"2024-05-09T08:04:27.735197+0000","last_fullsized":"2024-05-09T08:04:27.735197+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:57:07.107359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:27.735251+0000","last_change":"2024-05-09T08:02:14.874987+0000","last_active":"2024-05-09T08:04:27.735251+0000","last_peered":"2024-05-09T08:04:27.735251+0000","last_clean":"2024-05-09T08:04:27.735251+0000","last_became_active":"2024-05-09T08:02:14.874549+0000","last_became_peered":"2024-05-09T08:02:14.874549+0000","last_unstale":"2024-05-09T08:04:27.735251+0000","last_undegraded":"2024-05-09T08:04:27.735251+0000","last_fullsized":"2024-05-09T08:04:27.735251+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:57:10.338510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"27'339","reported_seq":366,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:15.296068+0000","last_change":"2024-05-09T08:03:02.516739+0000","last_active":"2024-05-09T08:07:15.296068+0000","last_peered":"2024-05-09T08:07:15.296068+0000","last_clean":"2024-05-09T08:07:15.296068+0000","last_became_active":"2024-05-09T08:03:02.516161+0000","last_became_peered":"2024-05-09T08:03:02.516161+0000","last_unstale":"2024-05-09T08:07:15.296068+0000","last_undegraded":"2024-05-09T08:07:15.296068+0000","last_fullsized":"2024-05-09T08:07:15.296068+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":339,"log_dups_size":0,"ondisk_log_size":339,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:10:24.797526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":288,"num_write_kb":786432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,2],"acting":[4,3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:27.753541+0000","last_change":"2024-05-09T08:02:15.913011+0000","last_active":"2024-05-09T08:04:27.753541+0000","last_peered":"2024-05-09T08:04:27.753541+0000","last_clean":"2024-05-09T08:04:27.753541+0000","last_became_active":"2024-05-09T08:02:15.912590+0000","last_became_peered":"2024-05-09T08:02:15.912590+0000","last_unstale":"2024-05-09T08:04:27.753541+0000","last_undegraded":"2024-05-09T08:04:27.753541+0000","last_fullsized":"2024-05-09T08:04:27.753541+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:43:06.938904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"27'3","reported_seq":31,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:06:37.749118+0000","last_change":"2024-05-09T08:02:14.864763+0000","last_active":"2024-05-09T08:06:37.749118+0000","last_peered":"2024-05-09T08:06:37.749118+0000","last_clean":"2024-05-09T08:06:37.749118+0000","last_became_active":"2024-05-09T08:02:14.864470+0000","last_became_peered":"2024-05-09T08:02:14.864470+0000","last_unstale":"2024-05-09T08:06:37.749118+0000","last_undegraded":"2024-05-09T08:06:37.749118+0000","last_fullsized":"2024-05-09T08:06:37.749118+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:00:40.599475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"27'306","reported_seq":326,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:15.290595+0000","last_change":"2024-05-09T08:02:20.054896+0000","last_active":"2024-05-09T08:07:15.290595+0000","last_peered":"2024-05-09T08:07:15.290595+0000","last_clean":"2024-05-09T08:07:15.290595+0000","last_became_active":"2024-05-09T08:02:20.054424+0000","last_became_peered":"2024-05-09T08:02:20.054424+0000","last_unstale":"2024-05-09T08:07:15.290595+0000","last_undegraded":"2024-05-09T08:07:15.290595+0000","last_fullsized":"2024-05-09T08:07:15.290595+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":306,"log_dups_size":0,"ondisk_log_size":306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:23:44.316545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":249,"num_write_kb":679936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:27.747018+0000","last_change":"2024-05-09T08:02:15.905448+0000","last_active":"2024-05-09T08:04:27.747018+0000","last_peered":"2024-05-09T08:04:27.747018+0000","last_clean":"2024-05-09T08:04:27.747018+0000","last_became_active":"2024-05-09T08:02:15.905131+0000","last_became_peered":"2024-05-09T08:02:15.905131+0000","last_unstale":"2024-05-09T08:04:27.747018+0000","last_undegraded":"2024-05-09T08:04:27.747018+0000","last_fullsized":"2024-05-09T08:04:27.747018+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:25:01.238816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"27'21","reported_seq":48,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:15.020483+0000","last_change":"2024-05-09T08:02:14.868410+0000","last_active":"2024-05-09T08:07:15.020483+0000","last_peered":"2024-05-09T08:07:15.020483+0000","last_clean":"2024-05-09T08:07:15.020483+0000","last_became_active":"2024-05-09T08:02:14.865621+0000","last_became_peered":"2024-05-09T08:02:14.865621+0000","last_unstale":"2024-05-09T08:07:15.020483+0000","last_undegraded":"2024-05-09T08:07:15.020483+0000","last_fullsized":"2024-05-09T08:07:15.020483+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:43:01.061801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":41,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"27'328","reported_seq":348,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:15.272939+0000","last_change":"2024-05-09T08:02:20.057603+0000","last_active":"2024-05-09T08:07:15.272939+0000","last_peered":"2024-05-09T08:07:15.272939+0000","last_clean":"2024-05-09T08:07:15.272939+0000","last_became_active":"2024-05-09T08:02:20.055365+0000","last_became_peered":"2024-05-09T08:02:20.055365+0000","last_unstale":"2024-05-09T08:07:15.272939+0000","last_undegraded":"2024-05-09T08:07:15.272939+0000","last_fullsized":"2024-05-09T08:07:15.272939+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":328,"log_dups_size":0,"ondisk_log_size":328,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:48:38.283562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":264,"num_write_kb":720896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:22.759212+0000","last_change":"2024-05-09T08:02:15.890980+0000","last_active":"2024-05-09T08:04:22.759212+0000","last_peered":"2024-05-09T08:04:22.759212+0000","last_clean":"2024-05-09T08:04:22.759212+0000","last_became_active":"2024-05-09T08:02:15.890346+0000","last_became_peered":"2024-05-09T08:02:15.890346+0000","last_unstale":"2024-05-09T08:04:22.759212+0000","last_undegraded":"2024-05-09T08:04:22.759212+0000","last_fullsized":"2024-05-09T08:04:22.759212+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:29:59.441314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:22.734773+0000","last_change":"2024-05-09T08:02:14.876206+0000","last_active":"2024-05-09T08:04:22.734773+0000","last_peered":"2024-05-09T08:04:22.734773+0000","last_clean":"2024-05-09T08:04:22.734773+0000","last_became_active":"2024-05-09T08:02:14.875894+0000","last_became_peered":"2024-05-09T08:02:14.875894+0000","last_unstale":"2024-05-09T08:04:22.734773+0000","last_undegraded":"2024-05-09T08:04:22.734773+0000","last_fullsized":"2024-05-09T08:04:22.734773+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:49:53.882513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"27'297","reported_seq":317,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:15.288524+0000","last_change":"2024-05-09T08:02:20.055222+0000","last_active":"2024-05-09T08:07:15.288524+0000","last_peered":"2024-05-09T08:07:15.288524+0000","last_clean":"2024-05-09T08:07:15.288524+0000","last_became_active":"2024-05-09T08:02:20.053802+0000","last_became_peered":"2024-05-09T08:02:20.053802+0000","last_unstale":"2024-05-09T08:07:15.288524+0000","last_undegraded":"2024-05-09T08:07:15.288524+0000","last_fullsized":"2024-05-09T08:07:15.288524+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":297,"log_dups_size":0,"ondisk_log_size":297,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:42:00.939587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":249,"num_write_kb":679936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:22.735039+0000","last_change":"2024-05-09T08:02:15.904528+0000","last_active":"2024-05-09T08:04:22.735039+0000","last_peered":"2024-05-09T08:04:22.735039+0000","last_clean":"2024-05-09T08:04:22.735039+0000","last_became_active":"2024-05-09T08:02:15.904049+0000","last_became_peered":"2024-05-09T08:02:15.904049+0000","last_unstale":"2024-05-09T08:04:22.735039+0000","last_undegraded":"2024-05-09T08:04:22.735039+0000","last_fullsized":"2024-05-09T08:04:22.735039+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:53:11.390395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:22.759235+0000","last_change":"2024-05-09T08:02:14.872897+0000","last_active":"2024-05-09T08:04:22.759235+0000","last_peered":"2024-05-09T08:04:22.759235+0000","last_clean":"2024-05-09T08:04:22.759235+0000","last_became_active":"2024-05-09T08:02:14.872096+0000","last_became_peered":"2024-05-09T08:02:14.872096+0000","last_unstale":"2024-05-09T08:04:22.759235+0000","last_undegraded":"2024-05-09T08:04:22.759235+0000","last_fullsized":"2024-05-09T08:04:22.759235+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:00:40.599475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"27'320","reported_seq":340,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:15.220609+0000","last_change":"2024-05-09T08:02:19.963716+0000","last_active":"2024-05-09T08:07:15.220609+0000","last_peered":"2024-05-09T08:07:15.220609+0000","last_clean":"2024-05-09T08:07:15.220609+0000","last_became_active":"2024-05-09T08:02:19.962922+0000","last_became_peered":"2024-05-09T08:02:19.962922+0000","last_unstale":"2024-05-09T08:07:15.220609+0000","last_undegraded":"2024-05-09T08:07:15.220609+0000","last_fullsized":"2024-05-09T08:07:15.220609+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":320,"log_dups_size":0,"ondisk_log_size":320,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:42:00.939587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":273,"num_write_kb":745472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:22.734664+0000","last_change":"2024-05-09T08:02:15.901450+0000","last_active":"2024-05-09T08:04:22.734664+0000","last_peered":"2024-05-09T08:04:22.734664+0000","last_clean":"2024-05-09T08:04:22.734664+0000","last_became_active":"2024-05-09T08:02:15.901180+0000","last_became_peered":"2024-05-09T08:02:15.901180+0000","last_unstale":"2024-05-09T08:04:22.734664+0000","last_undegraded":"2024-05-09T08:04:22.734664+0000","last_fullsized":"2024-05-09T08:04:22.734664+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:43:02.075442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:22.734997+0000","last_change":"2024-05-09T08:02:14.876518+0000","last_active":"2024-05-09T08:04:22.734997+0000","last_peered":"2024-05-09T08:04:22.734997+0000","last_clean":"2024-05-09T08:04:22.734997+0000","last_became_active":"2024-05-09T08:02:14.876175+0000","last_became_peered":"2024-05-09T08:02:14.876175+0000","last_unstale":"2024-05-09T08:04:22.734997+0000","last_undegraded":"2024-05-09T08:04:22.734997+0000","last_fullsized":"2024-05-09T08:04:22.734997+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:00:40.599475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"27'301","reported_seq":315,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:15.286265+0000","last_change":"2024-05-09T08:03:02.516595+0000","last_active":"2024-05-09T08:07:15.286265+0000","last_peered":"2024-05-09T08:07:15.286265+0000","last_clean":"2024-05-09T08:07:15.286265+0000","last_became_active":"2024-05-09T08:03:02.516104+0000","last_became_peered":"2024-05-09T08:03:02.516104+0000","last_unstale":"2024-05-09T08:07:15.286265+0000","last_undegraded":"2024-05-09T08:07:15.286265+0000","last_fullsized":"2024-05-09T08:07:15.286265+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":301,"log_dups_size":0,"ondisk_log_size":301,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:47:30.081323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":264,"num_write_kb":720896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3,6],"acting":[5,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:17.758674+0000","last_change":"2024-05-09T08:02:15.891774+0000","last_active":"2024-05-09T08:04:17.758674+0000","last_peered":"2024-05-09T08:04:17.758674+0000","last_clean":"2024-05-09T08:04:17.758674+0000","last_became_active":"2024-05-09T08:02:15.891407+0000","last_became_peered":"2024-05-09T08:02:15.891407+0000","last_unstale":"2024-05-09T08:04:17.758674+0000","last_undegraded":"2024-05-09T08:04:17.758674+0000","last_fullsized":"2024-05-09T08:04:17.758674+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:05:02.181367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"27'6","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:17.381608+0000","last_change":"2024-05-09T08:02:14.883655+0000","last_active":"2024-05-09T08:07:17.381608+0000","last_peered":"2024-05-09T08:07:17.381608+0000","last_clean":"2024-05-09T08:07:17.381608+0000","last_became_active":"2024-05-09T08:02:14.882359+0000","last_became_peered":"2024-05-09T08:02:14.882359+0000","last_unstale":"2024-05-09T08:07:17.381608+0000","last_undegraded":"2024-05-09T08:07:17.381608+0000","last_fullsized":"2024-05-09T08:07:17.381608+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:46:11.160815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":529,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"27'319","reported_seq":339,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:15.257021+0000","last_change":"2024-05-09T08:02:19.953319+0000","last_active":"2024-05-09T08:07:15.257021+0000","last_peered":"2024-05-09T08:07:15.257021+0000","last_clean":"2024-05-09T08:07:15.257021+0000","last_became_active":"2024-05-09T08:02:19.952740+0000","last_became_peered":"2024-05-09T08:02:19.952740+0000","last_unstale":"2024-05-09T08:07:15.257021+0000","last_undegraded":"2024-05-09T08:07:15.257021+0000","last_fullsized":"2024-05-09T08:07:15.257021+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":319,"log_dups_size":0,"ondisk_log_size":319,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:25:28.804048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":270,"num_write_kb":737280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:22.752933+0000","last_change":"2024-05-09T08:02:15.913828+0000","last_active":"2024-05-09T08:04:22.752933+0000","last_peered":"2024-05-09T08:04:22.752933+0000","last_clean":"2024-05-09T08:04:22.752933+0000","last_became_active":"2024-05-09T08:02:15.913313+0000","last_became_peered":"2024-05-09T08:02:15.913313+0000","last_unstale":"2024-05-09T08:04:22.752933+0000","last_undegraded":"2024-05-09T08:04:22.752933+0000","last_fullsized":"2024-05-09T08:04:22.752933+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:20:09.935394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:17.734196+0000","last_change":"2024-05-09T08:02:14.876438+0000","last_active":"2024-05-09T08:04:17.734196+0000","last_peered":"2024-05-09T08:04:17.734196+0000","last_clean":"2024-05-09T08:04:17.734196+0000","last_became_active":"2024-05-09T08:02:14.876146+0000","last_became_peered":"2024-05-09T08:02:14.876146+0000","last_unstale":"2024-05-09T08:04:17.734196+0000","last_undegraded":"2024-05-09T08:04:17.734196+0000","last_fullsized":"2024-05-09T08:04:17.734196+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:36:49.629991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"27'378","reported_seq":398,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:15.271544+0000","last_change":"2024-05-09T08:02:20.051067+0000","last_active":"2024-05-09T08:07:15.271544+0000","last_peered":"2024-05-09T08:07:15.271544+0000","last_clean":"2024-05-09T08:07:15.271544+0000","last_became_active":"2024-05-09T08:02:20.050593+0000","last_became_peered":"2024-05-09T08:02:20.050593+0000","last_unstale":"2024-05-09T08:07:15.271544+0000","last_undegraded":"2024-05-09T08:07:15.271544+0000","last_fullsized":"2024-05-09T08:07:15.271544+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":378,"log_dups_size":0,"ondisk_log_size":378,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:25:28.804048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":339,"num_write_kb":925696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:22.746418+0000","last_change":"2024-05-09T08:02:15.899137+0000","last_active":"2024-05-09T08:04:22.746418+0000","last_peered":"2024-05-09T08:04:22.746418+0000","last_clean":"2024-05-09T08:04:22.746418+0000","last_became_active":"2024-05-09T08:02:15.898766+0000","last_became_peered":"2024-05-09T08:02:15.898766+0000","last_unstale":"2024-05-09T08:04:22.746418+0000","last_undegraded":"2024-05-09T08:04:22.746418+0000","last_fullsized":"2024-05-09T08:04:22.746418+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:47:32.605288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:22.735011+0000","last_change":"2024-05-09T08:02:14.868850+0000","last_active":"2024-05-09T08:04:22.735011+0000","last_peered":"2024-05-09T08:04:22.735011+0000","last_clean":"2024-05-09T08:04:22.735011+0000","last_became_active":"2024-05-09T08:02:14.868384+0000","last_became_peered":"2024-05-09T08:02:14.868384+0000","last_unstale":"2024-05-09T08:04:22.735011+0000","last_undegraded":"2024-05-09T08:04:22.735011+0000","last_fullsized":"2024-05-09T08:04:22.735011+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:14:52.920972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":183,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:12.733624+0000","last_change":"2024-05-09T08:02:11.559804+0000","last_active":"2024-05-09T08:04:12.733624+0000","last_peered":"2024-05-09T08:04:12.733624+0000","last_clean":"2024-05-09T08:04:12.733624+0000","last_became_active":"2024-05-09T08:02:11.558761+0000","last_became_peered":"2024-05-09T08:02:11.558761+0000","last_unstale":"2024-05-09T08:04:12.733624+0000","last_undegraded":"2024-05-09T08:04:12.733624+0000","last_fullsized":"2024-05-09T08:04:12.733624+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:09.800143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:09.800143+0000","last_clean_scrub_stamp":"2024-05-09T08:02:09.800143+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:18:35.964149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"27'315","reported_seq":335,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:15.264268+0000","last_change":"2024-05-09T08:02:20.054746+0000","last_active":"2024-05-09T08:07:15.264268+0000","last_peered":"2024-05-09T08:07:15.264268+0000","last_clean":"2024-05-09T08:07:15.264268+0000","last_became_active":"2024-05-09T08:02:20.054186+0000","last_became_peered":"2024-05-09T08:02:20.054186+0000","last_unstale":"2024-05-09T08:07:15.264268+0000","last_undegraded":"2024-05-09T08:07:15.264268+0000","last_fullsized":"2024-05-09T08:07:15.264268+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":315,"log_dups_size":0,"ondisk_log_size":315,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:53:15.458120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":276,"num_write_kb":753664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:32.735875+0000","last_change":"2024-05-09T08:02:15.889657+0000","last_active":"2024-05-09T08:04:32.735875+0000","last_peered":"2024-05-09T08:04:32.735875+0000","last_clean":"2024-05-09T08:04:32.735875+0000","last_became_active":"2024-05-09T08:02:15.889381+0000","last_became_peered":"2024-05-09T08:02:15.889381+0000","last_unstale":"2024-05-09T08:04:32.735875+0000","last_undegraded":"2024-05-09T08:04:32.735875+0000","last_fullsized":"2024-05-09T08:04:32.735875+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:59:14.957806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:32.747257+0000","last_change":"2024-05-09T08:02:14.868892+0000","last_active":"2024-05-09T08:04:32.747257+0000","last_peered":"2024-05-09T08:04:32.747257+0000","last_clean":"2024-05-09T08:04:32.747257+0000","last_became_active":"2024-05-09T08:02:14.868513+0000","last_became_peered":"2024-05-09T08:02:14.868513+0000","last_unstale":"2024-05-09T08:04:32.747257+0000","last_undegraded":"2024-05-09T08:04:32.747257+0000","last_fullsized":"2024-05-09T08:04:32.747257+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:19:31.913236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"27'338","reported_seq":358,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:15.275986+0000","last_change":"2024-05-09T08:02:20.053097+0000","last_active":"2024-05-09T08:07:15.275986+0000","last_peered":"2024-05-09T08:07:15.275986+0000","last_clean":"2024-05-09T08:07:15.275986+0000","last_became_active":"2024-05-09T08:02:20.052653+0000","last_became_peered":"2024-05-09T08:02:20.052653+0000","last_unstale":"2024-05-09T08:07:15.275986+0000","last_undegraded":"2024-05-09T08:07:15.275986+0000","last_fullsized":"2024-05-09T08:07:15.275986+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":338,"log_dups_size":0,"ondisk_log_size":338,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:25:05.306541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":279,"num_write_kb":761856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:32.736237+0000","last_change":"2024-05-09T08:02:15.899059+0000","last_active":"2024-05-09T08:04:32.736237+0000","last_peered":"2024-05-09T08:04:32.736237+0000","last_clean":"2024-05-09T08:04:32.736237+0000","last_became_active":"2024-05-09T08:02:15.898692+0000","last_became_peered":"2024-05-09T08:02:15.898692+0000","last_unstale":"2024-05-09T08:04:32.736237+0000","last_undegraded":"2024-05-09T08:04:32.736237+0000","last_fullsized":"2024-05-09T08:04:32.736237+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:52:36.461913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"24'1","reported_seq":28,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:32.760505+0000","last_change":"2024-05-09T08:02:14.873071+0000","last_active":"2024-05-09T08:04:32.760505+0000","last_peered":"2024-05-09T08:04:32.760505+0000","last_clean":"2024-05-09T08:04:32.760505+0000","last_became_active":"2024-05-09T08:02:14.872556+0000","last_became_peered":"2024-05-09T08:02:14.872556+0000","last_unstale":"2024-05-09T08:04:32.760505+0000","last_undegraded":"2024-05-09T08:04:32.760505+0000","last_fullsized":"2024-05-09T08:04:32.760505+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:57:06.093718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"27'299","reported_seq":319,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:15.257565+0000","last_change":"2024-05-09T08:02:19.960414+0000","last_active":"2024-05-09T08:07:15.257565+0000","last_peered":"2024-05-09T08:07:15.257565+0000","last_clean":"2024-05-09T08:07:15.257565+0000","last_became_active":"2024-05-09T08:02:19.957941+0000","last_became_peered":"2024-05-09T08:02:19.957941+0000","last_unstale":"2024-05-09T08:07:15.257565+0000","last_undegraded":"2024-05-09T08:07:15.257565+0000","last_fullsized":"2024-05-09T08:07:15.257565+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":299,"log_dups_size":0,"ondisk_log_size":299,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:41:31.470546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":261,"num_write_kb":712704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:32.735920+0000","last_change":"2024-05-09T08:02:15.884395+0000","last_active":"2024-05-09T08:04:32.735920+0000","last_peered":"2024-05-09T08:04:32.735920+0000","last_clean":"2024-05-09T08:04:32.735920+0000","last_became_active":"2024-05-09T08:02:15.883788+0000","last_became_peered":"2024-05-09T08:02:15.883788+0000","last_unstale":"2024-05-09T08:04:32.735920+0000","last_undegraded":"2024-05-09T08:04:32.735920+0000","last_fullsized":"2024-05-09T08:04:32.735920+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:48:34.215837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"24'2","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:32.747303+0000","last_change":"2024-05-09T08:02:14.885044+0000","last_active":"2024-05-09T08:04:32.747303+0000","last_peered":"2024-05-09T08:04:32.747303+0000","last_clean":"2024-05-09T08:04:32.747303+0000","last_became_active":"2024-05-09T08:02:14.884706+0000","last_became_peered":"2024-05-09T08:02:14.884706+0000","last_unstale":"2024-05-09T08:04:32.747303+0000","last_undegraded":"2024-05-09T08:04:32.747303+0000","last_fullsized":"2024-05-09T08:04:32.747303+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:14:52.920972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"27'365","reported_seq":385,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:15.276867+0000","last_change":"2024-05-09T08:02:20.050865+0000","last_active":"2024-05-09T08:07:15.276867+0000","last_peered":"2024-05-09T08:07:15.276867+0000","last_clean":"2024-05-09T08:07:15.276867+0000","last_became_active":"2024-05-09T08:02:20.050270+0000","last_became_peered":"2024-05-09T08:02:20.050270+0000","last_unstale":"2024-05-09T08:07:15.276867+0000","last_undegraded":"2024-05-09T08:07:15.276867+0000","last_fullsized":"2024-05-09T08:07:15.276867+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":365,"log_dups_size":0,"ondisk_log_size":365,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:48:38.283562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":318,"num_write_kb":868352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:32.735905+0000","last_change":"2024-05-09T08:02:15.884737+0000","last_active":"2024-05-09T08:04:32.735905+0000","last_peered":"2024-05-09T08:04:32.735905+0000","last_clean":"2024-05-09T08:04:32.735905+0000","last_became_active":"2024-05-09T08:02:15.884465+0000","last_became_peered":"2024-05-09T08:02:15.884465+0000","last_unstale":"2024-05-09T08:04:32.735905+0000","last_undegraded":"2024-05-09T08:04:32.735905+0000","last_fullsized":"2024-05-09T08:04:32.735905+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:52:36.461913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"24'1","reported_seq":28,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:32.735864+0000","last_change":"2024-05-09T08:02:14.875003+0000","last_active":"2024-05-09T08:04:32.735864+0000","last_peered":"2024-05-09T08:04:32.735864+0000","last_clean":"2024-05-09T08:04:32.735864+0000","last_became_active":"2024-05-09T08:02:14.874665+0000","last_became_peered":"2024-05-09T08:02:14.874665+0000","last_unstale":"2024-05-09T08:04:32.735864+0000","last_undegraded":"2024-05-09T08:04:32.735864+0000","last_fullsized":"2024-05-09T08:04:32.735864+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:18:39.991319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"27'302","reported_seq":322,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:15.294350+0000","last_change":"2024-05-09T08:02:19.952692+0000","last_active":"2024-05-09T08:07:15.294350+0000","last_peered":"2024-05-09T08:07:15.294350+0000","last_clean":"2024-05-09T08:07:15.294350+0000","last_became_active":"2024-05-09T08:02:19.952224+0000","last_became_peered":"2024-05-09T08:02:19.952224+0000","last_unstale":"2024-05-09T08:07:15.294350+0000","last_undegraded":"2024-05-09T08:07:15.294350+0000","last_fullsized":"2024-05-09T08:07:15.294350+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":302,"log_dups_size":0,"ondisk_log_size":302,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:47:36.673013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":258,"num_write_kb":704512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:37.736256+0000","last_change":"2024-05-09T08:02:15.883408+0000","last_active":"2024-05-09T08:04:37.736256+0000","last_peered":"2024-05-09T08:04:37.736256+0000","last_clean":"2024-05-09T08:04:37.736256+0000","last_became_active":"2024-05-09T08:02:15.883103+0000","last_became_peered":"2024-05-09T08:02:15.883103+0000","last_unstale":"2024-05-09T08:04:37.736256+0000","last_undegraded":"2024-05-09T08:04:37.736256+0000","last_fullsized":"2024-05-09T08:04:37.736256+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:53:11.390395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"27'9","reported_seq":36,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:06:57.786502+0000","last_change":"2024-05-09T08:02:14.873184+0000","last_active":"2024-05-09T08:06:57.786502+0000","last_peered":"2024-05-09T08:06:57.786502+0000","last_clean":"2024-05-09T08:06:57.786502+0000","last_became_active":"2024-05-09T08:02:14.871055+0000","last_became_peered":"2024-05-09T08:02:14.871055+0000","last_unstale":"2024-05-09T08:06:57.786502+0000","last_undegraded":"2024-05-09T08:06:57.786502+0000","last_fullsized":"2024-05-09T08:06:57.786502+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:19:56.315972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"27'416","reported_seq":436,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:15.283703+0000","last_change":"2024-05-09T08:02:19.950823+0000","last_active":"2024-05-09T08:07:15.283703+0000","last_peered":"2024-05-09T08:07:15.283703+0000","last_clean":"2024-05-09T08:07:15.283703+0000","last_became_active":"2024-05-09T08:02:19.950352+0000","last_became_peered":"2024-05-09T08:02:19.950352+0000","last_unstale":"2024-05-09T08:07:15.283703+0000","last_undegraded":"2024-05-09T08:07:15.283703+0000","last_fullsized":"2024-05-09T08:07:15.283703+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":416,"log_dups_size":0,"ondisk_log_size":416,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:59:19.025530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":360,"num_write_kb":983040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:37.736615+0000","last_change":"2024-05-09T08:02:15.899115+0000","last_active":"2024-05-09T08:04:37.736615+0000","last_peered":"2024-05-09T08:04:37.736615+0000","last_clean":"2024-05-09T08:04:37.736615+0000","last_became_active":"2024-05-09T08:02:15.898811+0000","last_became_peered":"2024-05-09T08:02:15.898811+0000","last_unstale":"2024-05-09T08:04:37.736615+0000","last_undegraded":"2024-05-09T08:04:37.736615+0000","last_fullsized":"2024-05-09T08:04:37.736615+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:43:06.938904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":28,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:37.736636+0000","last_change":"2024-05-09T08:02:14.860818+0000","last_active":"2024-05-09T08:04:37.736636+0000","last_peered":"2024-05-09T08:04:37.736636+0000","last_clean":"2024-05-09T08:04:37.736636+0000","last_became_active":"2024-05-09T08:02:14.860518+0000","last_became_peered":"2024-05-09T08:02:14.860518+0000","last_unstale":"2024-05-09T08:04:37.736636+0000","last_undegraded":"2024-05-09T08:04:37.736636+0000","last_fullsized":"2024-05-09T08:04:37.736636+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:25:23.306827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"27'318","reported_seq":338,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:15.275345+0000","last_change":"2024-05-09T08:02:20.053251+0000","last_active":"2024-05-09T08:07:15.275345+0000","last_peered":"2024-05-09T08:07:15.275345+0000","last_clean":"2024-05-09T08:07:15.275345+0000","last_became_active":"2024-05-09T08:02:20.052783+0000","last_became_peered":"2024-05-09T08:02:20.052783+0000","last_unstale":"2024-05-09T08:07:15.275345+0000","last_undegraded":"2024-05-09T08:07:15.275345+0000","last_fullsized":"2024-05-09T08:07:15.275345+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":318,"log_dups_size":0,"ondisk_log_size":318,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:16:41.863341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":273,"num_write_kb":745472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:37.736278+0000","last_change":"2024-05-09T08:02:15.883972+0000","last_active":"2024-05-09T08:04:37.736278+0000","last_peered":"2024-05-09T08:04:37.736278+0000","last_clean":"2024-05-09T08:04:37.736278+0000","last_became_active":"2024-05-09T08:02:15.883613+0000","last_became_peered":"2024-05-09T08:02:15.883613+0000","last_unstale":"2024-05-09T08:04:37.736278+0000","last_undegraded":"2024-05-09T08:04:37.736278+0000","last_fullsized":"2024-05-09T08:04:37.736278+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:41:27.402821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:37.736743+0000","last_change":"2024-05-09T08:02:14.883366+0000","last_active":"2024-05-09T08:04:37.736743+0000","last_peered":"2024-05-09T08:04:37.736743+0000","last_clean":"2024-05-09T08:04:37.736743+0000","last_became_active":"2024-05-09T08:02:14.883135+0000","last_became_peered":"2024-05-09T08:02:14.883135+0000","last_unstale":"2024-05-09T08:04:37.736743+0000","last_undegraded":"2024-05-09T08:04:37.736743+0000","last_fullsized":"2024-05-09T08:04:37.736743+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:32:48.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"27'316","reported_seq":336,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:15.284009+0000","last_change":"2024-05-09T08:02:20.050888+0000","last_active":"2024-05-09T08:07:15.284009+0000","last_peered":"2024-05-09T08:07:15.284009+0000","last_clean":"2024-05-09T08:07:15.284009+0000","last_became_active":"2024-05-09T08:02:20.050390+0000","last_became_peered":"2024-05-09T08:02:20.050390+0000","last_unstale":"2024-05-09T08:07:15.284009+0000","last_undegraded":"2024-05-09T08:07:15.284009+0000","last_fullsized":"2024-05-09T08:07:15.284009+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":316,"log_dups_size":0,"ondisk_log_size":316,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:10:24.797526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":273,"num_write_kb":745472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"27'1","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:05:05.201608+0000","last_change":"2024-05-09T08:02:15.899096+0000","last_active":"2024-05-09T08:05:05.201608+0000","last_peered":"2024-05-09T08:05:05.201608+0000","last_clean":"2024-05-09T08:05:05.201608+0000","last_became_active":"2024-05-09T08:02:15.898811+0000","last_became_peered":"2024-05-09T08:02:15.898811+0000","last_unstale":"2024-05-09T08:05:05.201608+0000","last_undegraded":"2024-05-09T08:05:05.201608+0000","last_fullsized":"2024-05-09T08:05:05.201608+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:28:51.229262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"24'2","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:37.755081+0000","last_change":"2024-05-09T08:02:14.868657+0000","last_active":"2024-05-09T08:04:37.755081+0000","last_peered":"2024-05-09T08:04:37.755081+0000","last_clean":"2024-05-09T08:04:37.755081+0000","last_became_active":"2024-05-09T08:02:14.868385+0000","last_became_peered":"2024-05-09T08:02:14.868385+0000","last_unstale":"2024-05-09T08:04:37.755081+0000","last_undegraded":"2024-05-09T08:04:37.755081+0000","last_fullsized":"2024-05-09T08:04:37.755081+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:46:11.160815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"27'378","reported_seq":398,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:15.279263+0000","last_change":"2024-05-09T08:02:20.053155+0000","last_active":"2024-05-09T08:07:15.279263+0000","last_peered":"2024-05-09T08:07:15.279263+0000","last_clean":"2024-05-09T08:07:15.279263+0000","last_became_active":"2024-05-09T08:02:20.052629+0000","last_became_peered":"2024-05-09T08:02:20.052629+0000","last_unstale":"2024-05-09T08:07:15.279263+0000","last_undegraded":"2024-05-09T08:07:15.279263+0000","last_fullsized":"2024-05-09T08:07:15.279263+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":378,"log_dups_size":0,"ondisk_log_size":378,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:35:54.628874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":327,"num_write_kb":892928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:42.736850+0000","last_change":"2024-05-09T08:02:15.905406+0000","last_active":"2024-05-09T08:04:42.736850+0000","last_peered":"2024-05-09T08:04:42.736850+0000","last_clean":"2024-05-09T08:04:42.736850+0000","last_became_active":"2024-05-09T08:02:15.905076+0000","last_became_peered":"2024-05-09T08:02:15.905076+0000","last_unstale":"2024-05-09T08:04:42.736850+0000","last_undegraded":"2024-05-09T08:04:42.736850+0000","last_fullsized":"2024-05-09T08:04:42.736850+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:25:01.238816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"27'189","reported_seq":217,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:14.967504+0000","last_change":"2024-05-09T08:02:14.880410+0000","last_active":"2024-05-09T08:07:14.967504+0000","last_peered":"2024-05-09T08:07:14.967504+0000","last_clean":"2024-05-09T08:07:14.967504+0000","last_became_active":"2024-05-09T08:02:14.880045+0000","last_became_peered":"2024-05-09T08:02:14.880045+0000","last_unstale":"2024-05-09T08:07:14.967504+0000","last_undegraded":"2024-05-09T08:07:14.967504+0000","last_fullsized":"2024-05-09T08:07:14.967504+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":189,"log_dups_size":0,"ondisk_log_size":189,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:19:56.315972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":447435,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":188,"num_write_kb":582,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"27'332","reported_seq":352,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:15.206123+0000","last_change":"2024-05-09T08:02:19.963378+0000","last_active":"2024-05-09T08:07:15.206123+0000","last_peered":"2024-05-09T08:07:15.206123+0000","last_clean":"2024-05-09T08:07:15.206123+0000","last_became_active":"2024-05-09T08:02:19.958964+0000","last_became_peered":"2024-05-09T08:02:19.958964+0000","last_unstale":"2024-05-09T08:07:15.206123+0000","last_undegraded":"2024-05-09T08:07:15.206123+0000","last_fullsized":"2024-05-09T08:07:15.206123+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":332,"log_dups_size":0,"ondisk_log_size":332,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:41:31.470546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":288,"num_write_kb":786432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:37.736256+0000","last_change":"2024-05-09T08:02:15.888428+0000","last_active":"2024-05-09T08:04:37.736256+0000","last_peered":"2024-05-09T08:04:37.736256+0000","last_clean":"2024-05-09T08:04:37.736256+0000","last_became_active":"2024-05-09T08:02:15.888071+0000","last_became_peered":"2024-05-09T08:02:15.888071+0000","last_unstale":"2024-05-09T08:04:37.736256+0000","last_undegraded":"2024-05-09T08:04:37.736256+0000","last_fullsized":"2024-05-09T08:04:37.736256+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:36:50.643632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"27'8","reported_seq":35,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:15.298845+0000","last_change":"2024-05-09T08:02:14.872697+0000","last_active":"2024-05-09T08:07:15.298845+0000","last_peered":"2024-05-09T08:07:15.298845+0000","last_clean":"2024-05-09T08:07:15.298845+0000","last_became_active":"2024-05-09T08:02:14.871945+0000","last_became_peered":"2024-05-09T08:02:14.871945+0000","last_unstale":"2024-05-09T08:07:15.298845+0000","last_undegraded":"2024-05-09T08:07:15.298845+0000","last_fullsized":"2024-05-09T08:07:15.298845+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:19:56.315972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"27'353","reported_seq":373,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:15.280987+0000","last_change":"2024-05-09T08:02:19.953635+0000","last_active":"2024-05-09T08:07:15.280987+0000","last_peered":"2024-05-09T08:07:15.280987+0000","last_clean":"2024-05-09T08:07:15.280987+0000","last_became_active":"2024-05-09T08:02:19.953166+0000","last_became_peered":"2024-05-09T08:02:19.953166+0000","last_unstale":"2024-05-09T08:07:15.280987+0000","last_undegraded":"2024-05-09T08:07:15.280987+0000","last_fullsized":"2024-05-09T08:07:15.280987+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":353,"log_dups_size":0,"ondisk_log_size":353,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:18:40.377352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":297,"num_write_kb":811008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:42.755912+0000","last_change":"2024-05-09T08:02:15.905285+0000","last_active":"2024-05-09T08:04:42.755912+0000","last_peered":"2024-05-09T08:04:42.755912+0000","last_clean":"2024-05-09T08:04:42.755912+0000","last_became_active":"2024-05-09T08:02:15.904898+0000","last_became_peered":"2024-05-09T08:02:15.904898+0000","last_unstale":"2024-05-09T08:04:42.755912+0000","last_undegraded":"2024-05-09T08:04:42.755912+0000","last_fullsized":"2024-05-09T08:04:42.755912+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:47:32.605288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:42.737369+0000","last_change":"2024-05-09T08:02:14.868682+0000","last_active":"2024-05-09T08:04:42.737369+0000","last_peered":"2024-05-09T08:04:42.737369+0000","last_clean":"2024-05-09T08:04:42.737369+0000","last_became_active":"2024-05-09T08:02:14.868294+0000","last_became_peered":"2024-05-09T08:02:14.868294+0000","last_unstale":"2024-05-09T08:04:42.737369+0000","last_undegraded":"2024-05-09T08:04:42.737369+0000","last_fullsized":"2024-05-09T08:04:42.737369+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:57:06.093718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"27'344","reported_seq":364,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:15.272036+0000","last_change":"2024-05-09T08:02:19.956115+0000","last_active":"2024-05-09T08:07:15.272036+0000","last_peered":"2024-05-09T08:07:15.272036+0000","last_clean":"2024-05-09T08:07:15.272036+0000","last_became_active":"2024-05-09T08:02:19.955662+0000","last_became_peered":"2024-05-09T08:02:19.955662+0000","last_unstale":"2024-05-09T08:07:15.272036+0000","last_undegraded":"2024-05-09T08:07:15.272036+0000","last_fullsized":"2024-05-09T08:07:15.272036+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":344,"log_dups_size":0,"ondisk_log_size":344,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:15:28.286330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":297,"num_write_kb":811008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:42.737356+0000","last_change":"2024-05-09T08:02:15.888406+0000","last_active":"2024-05-09T08:04:42.737356+0000","last_peered":"2024-05-09T08:04:42.737356+0000","last_clean":"2024-05-09T08:04:42.737356+0000","last_became_active":"2024-05-09T08:02:15.888090+0000","last_became_peered":"2024-05-09T08:02:15.888090+0000","last_unstale":"2024-05-09T08:04:42.737356+0000","last_undegraded":"2024-05-09T08:04:42.737356+0000","last_fullsized":"2024-05-09T08:04:42.737356+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:53:23.178610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"27'1","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:05:05.164756+0000","last_change":"2024-05-09T08:02:14.862463+0000","last_active":"2024-05-09T08:05:05.164756+0000","last_peered":"2024-05-09T08:05:05.164756+0000","last_clean":"2024-05-09T08:05:05.164756+0000","last_became_active":"2024-05-09T08:02:14.862104+0000","last_became_peered":"2024-05-09T08:02:14.862104+0000","last_unstale":"2024-05-09T08:05:05.164756+0000","last_undegraded":"2024-05-09T08:05:05.164756+0000","last_fullsized":"2024-05-09T08:05:05.164756+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:32:48.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"27'379","reported_seq":399,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:15.252469+0000","last_change":"2024-05-09T08:02:20.053600+0000","last_active":"2024-05-09T08:07:15.252469+0000","last_peered":"2024-05-09T08:07:15.252469+0000","last_clean":"2024-05-09T08:07:15.252469+0000","last_became_active":"2024-05-09T08:02:20.052485+0000","last_became_peered":"2024-05-09T08:02:20.052485+0000","last_unstale":"2024-05-09T08:07:15.252469+0000","last_undegraded":"2024-05-09T08:07:15.252469+0000","last_fullsized":"2024-05-09T08:07:15.252469+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":379,"log_dups_size":0,"ondisk_log_size":379,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:35:54.628874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":327,"num_write_kb":892928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"27'324","reported_seq":344,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:15.288169+0000","last_change":"2024-05-09T08:02:20.054016+0000","last_active":"2024-05-09T08:07:15.288169+0000","last_peered":"2024-05-09T08:07:15.288169+0000","last_clean":"2024-05-09T08:07:15.288169+0000","last_became_active":"2024-05-09T08:02:20.053250+0000","last_became_peered":"2024-05-09T08:02:20.053250+0000","last_unstale":"2024-05-09T08:07:15.288169+0000","last_undegraded":"2024-05-09T08:07:15.288169+0000","last_fullsized":"2024-05-09T08:07:15.288169+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":324,"log_dups_size":0,"ondisk_log_size":324,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:52:40.529638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":273,"num_write_kb":745472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"27'1","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:07.834698+0000","last_change":"2024-05-09T08:02:15.907080+0000","last_active":"2024-05-09T08:07:07.834698+0000","last_peered":"2024-05-09T08:07:07.834698+0000","last_clean":"2024-05-09T08:07:07.834698+0000","last_became_active":"2024-05-09T08:02:15.906600+0000","last_became_peered":"2024-05-09T08:02:15.906600+0000","last_unstale":"2024-05-09T08:07:07.834698+0000","last_undegraded":"2024-05-09T08:07:07.834698+0000","last_fullsized":"2024-05-09T08:07:07.834698+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:43:06.938904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:42.737302+0000","last_change":"2024-05-09T08:02:14.870990+0000","last_active":"2024-05-09T08:04:42.737302+0000","last_peered":"2024-05-09T08:04:42.737302+0000","last_clean":"2024-05-09T08:04:42.737302+0000","last_became_active":"2024-05-09T08:02:14.869818+0000","last_became_peered":"2024-05-09T08:02:14.869818+0000","last_unstale":"2024-05-09T08:04:42.737302+0000","last_undegraded":"2024-05-09T08:04:42.737302+0000","last_fullsized":"2024-05-09T08:04:42.737302+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:25:23.306827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"27'313","reported_seq":333,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:15.240636+0000","last_change":"2024-05-09T08:02:19.952582+0000","last_active":"2024-05-09T08:07:15.240636+0000","last_peered":"2024-05-09T08:07:15.240636+0000","last_clean":"2024-05-09T08:07:15.240636+0000","last_became_active":"2024-05-09T08:02:19.952117+0000","last_became_peered":"2024-05-09T08:02:19.952117+0000","last_unstale":"2024-05-09T08:07:15.240636+0000","last_undegraded":"2024-05-09T08:07:15.240636+0000","last_fullsized":"2024-05-09T08:07:15.240636+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":313,"log_dups_size":0,"ondisk_log_size":313,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:42:00.939587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":267,"num_write_kb":729088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:42.748943+0000","last_change":"2024-05-09T08:02:15.913109+0000","last_active":"2024-05-09T08:04:42.748943+0000","last_peered":"2024-05-09T08:04:42.748943+0000","last_clean":"2024-05-09T08:04:42.748943+0000","last_became_active":"2024-05-09T08:02:15.912883+0000","last_became_peered":"2024-05-09T08:02:15.912883+0000","last_unstale":"2024-05-09T08:04:42.748943+0000","last_undegraded":"2024-05-09T08:04:42.748943+0000","last_fullsized":"2024-05-09T08:04:42.748943+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:29:59.441314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:42.748992+0000","last_change":"2024-05-09T08:02:14.885256+0000","last_active":"2024-05-09T08:04:42.748992+0000","last_peered":"2024-05-09T08:04:42.748992+0000","last_clean":"2024-05-09T08:04:42.748992+0000","last_became_active":"2024-05-09T08:02:14.884965+0000","last_became_peered":"2024-05-09T08:02:14.884965+0000","last_unstale":"2024-05-09T08:04:42.748992+0000","last_undegraded":"2024-05-09T08:04:42.748992+0000","last_fullsized":"2024-05-09T08:04:42.748992+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:14:52.920972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"24'1","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:47.737722+0000","last_change":"2024-05-09T08:02:14.860339+0000","last_active":"2024-05-09T08:04:47.737722+0000","last_peered":"2024-05-09T08:04:47.737722+0000","last_clean":"2024-05-09T08:04:47.737722+0000","last_became_active":"2024-05-09T08:02:14.859917+0000","last_became_peered":"2024-05-09T08:02:14.859917+0000","last_unstale":"2024-05-09T08:04:47.737722+0000","last_undegraded":"2024-05-09T08:04:47.737722+0000","last_fullsized":"2024-05-09T08:04:47.737722+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:14:52.920972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:47.756573+0000","last_change":"2024-05-09T08:02:15.897001+0000","last_active":"2024-05-09T08:04:47.756573+0000","last_peered":"2024-05-09T08:04:47.756573+0000","last_clean":"2024-05-09T08:04:47.756573+0000","last_became_active":"2024-05-09T08:02:15.896708+0000","last_became_peered":"2024-05-09T08:02:15.896708+0000","last_unstale":"2024-05-09T08:04:47.756573+0000","last_undegraded":"2024-05-09T08:04:47.756573+0000","last_fullsized":"2024-05-09T08:04:47.756573+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:29:59.441314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"27'338","reported_seq":358,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:15.293090+0000","last_change":"2024-05-09T08:02:19.955369+0000","last_active":"2024-05-09T08:07:15.293090+0000","last_peered":"2024-05-09T08:07:15.293090+0000","last_clean":"2024-05-09T08:07:15.293090+0000","last_became_active":"2024-05-09T08:02:19.954829+0000","last_became_peered":"2024-05-09T08:02:19.954829+0000","last_unstale":"2024-05-09T08:07:15.293090+0000","last_undegraded":"2024-05-09T08:07:15.293090+0000","last_fullsized":"2024-05-09T08:07:15.293090+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":338,"log_dups_size":0,"ondisk_log_size":338,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:49:58.963879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":297,"num_write_kb":811008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"24'1","reported_seq":28,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:47.753171+0000","last_change":"2024-05-09T08:02:14.883856+0000","last_active":"2024-05-09T08:04:47.753171+0000","last_peered":"2024-05-09T08:04:47.753171+0000","last_clean":"2024-05-09T08:04:47.753171+0000","last_became_active":"2024-05-09T08:02:14.882610+0000","last_became_peered":"2024-05-09T08:02:14.882610+0000","last_unstale":"2024-05-09T08:04:47.753171+0000","last_undegraded":"2024-05-09T08:04:47.753171+0000","last_fullsized":"2024-05-09T08:04:47.753171+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:19:31.913236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:47.737288+0000","last_change":"2024-05-09T08:02:15.889974+0000","last_active":"2024-05-09T08:04:47.737288+0000","last_peered":"2024-05-09T08:04:47.737288+0000","last_clean":"2024-05-09T08:04:47.737288+0000","last_became_active":"2024-05-09T08:02:15.889650+0000","last_became_peered":"2024-05-09T08:02:15.889650+0000","last_unstale":"2024-05-09T08:04:47.737288+0000","last_undegraded":"2024-05-09T08:04:47.737288+0000","last_fullsized":"2024-05-09T08:04:47.737288+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:00:41.613116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"27'361","reported_seq":381,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:15.284334+0000","last_change":"2024-05-09T08:02:19.955362+0000","last_active":"2024-05-09T08:07:15.284334+0000","last_peered":"2024-05-09T08:07:15.284334+0000","last_clean":"2024-05-09T08:07:15.284334+0000","last_became_active":"2024-05-09T08:02:19.954947+0000","last_became_peered":"2024-05-09T08:02:19.954947+0000","last_unstale":"2024-05-09T08:07:15.284334+0000","last_undegraded":"2024-05-09T08:07:15.284334+0000","last_fullsized":"2024-05-09T08:07:15.284334+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":361,"log_dups_size":0,"ondisk_log_size":361,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:43:30.549966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":312,"num_write_kb":851968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:22.734849+0000","last_change":"2024-05-09T08:02:15.909379+0000","last_active":"2024-05-09T08:04:22.734849+0000","last_peered":"2024-05-09T08:04:22.734849+0000","last_clean":"2024-05-09T08:04:22.734849+0000","last_became_active":"2024-05-09T08:02:15.909086+0000","last_became_peered":"2024-05-09T08:02:15.909086+0000","last_unstale":"2024-05-09T08:04:22.734849+0000","last_undegraded":"2024-05-09T08:04:22.734849+0000","last_fullsized":"2024-05-09T08:04:22.734849+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:05:02.181367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:22.745878+0000","last_change":"2024-05-09T08:02:15.891242+0000","last_active":"2024-05-09T08:04:22.745878+0000","last_peered":"2024-05-09T08:04:22.745878+0000","last_clean":"2024-05-09T08:04:22.745878+0000","last_became_active":"2024-05-09T08:02:15.890640+0000","last_became_peered":"2024-05-09T08:02:15.890640+0000","last_unstale":"2024-05-09T08:04:22.745878+0000","last_undegraded":"2024-05-09T08:04:22.745878+0000","last_fullsized":"2024-05-09T08:04:22.745878+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:57:07.107359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:22.734971+0000","last_change":"2024-05-09T08:02:15.899001+0000","last_active":"2024-05-09T08:04:22.734971+0000","last_peered":"2024-05-09T08:04:22.734971+0000","last_clean":"2024-05-09T08:04:22.734971+0000","last_became_active":"2024-05-09T08:02:15.898695+0000","last_became_peered":"2024-05-09T08:02:15.898695+0000","last_unstale":"2024-05-09T08:04:22.734971+0000","last_undegraded":"2024-05-09T08:04:22.734971+0000","last_fullsized":"2024-05-09T08:04:22.734971+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:41:27.402821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:22.752972+0000","last_change":"2024-05-09T08:02:15.907374+0000","last_active":"2024-05-09T08:04:22.752972+0000","last_peered":"2024-05-09T08:04:22.752972+0000","last_clean":"2024-05-09T08:04:22.752972+0000","last_became_active":"2024-05-09T08:02:15.906767+0000","last_became_peered":"2024-05-09T08:02:15.906767+0000","last_unstale":"2024-05-09T08:04:22.752972+0000","last_undegraded":"2024-05-09T08:04:22.752972+0000","last_fullsized":"2024-05-09T08:04:22.752972+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:05:02.181367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:27.746979+0000","last_change":"2024-05-09T08:02:15.905326+0000","last_active":"2024-05-09T08:04:27.746979+0000","last_peered":"2024-05-09T08:04:27.746979+0000","last_clean":"2024-05-09T08:04:27.746979+0000","last_became_active":"2024-05-09T08:02:15.905083+0000","last_became_peered":"2024-05-09T08:02:15.905083+0000","last_unstale":"2024-05-09T08:04:27.746979+0000","last_undegraded":"2024-05-09T08:04:27.746979+0000","last_fullsized":"2024-05-09T08:04:27.746979+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:23:40.248821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:27.759721+0000","last_change":"2024-05-09T08:02:15.907753+0000","last_active":"2024-05-09T08:04:27.759721+0000","last_peered":"2024-05-09T08:04:27.759721+0000","last_clean":"2024-05-09T08:04:27.759721+0000","last_became_active":"2024-05-09T08:02:15.907480+0000","last_became_peered":"2024-05-09T08:02:15.907480+0000","last_unstale":"2024-05-09T08:04:27.759721+0000","last_undegraded":"2024-05-09T08:04:27.759721+0000","last_fullsized":"2024-05-09T08:04:27.759721+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:20:07.368327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:27.735486+0000","last_change":"2024-05-09T08:02:15.888605+0000","last_active":"2024-05-09T08:04:27.735486+0000","last_peered":"2024-05-09T08:04:27.735486+0000","last_clean":"2024-05-09T08:04:27.735486+0000","last_became_active":"2024-05-09T08:02:15.887981+0000","last_became_peered":"2024-05-09T08:02:15.887981+0000","last_unstale":"2024-05-09T08:04:27.735486+0000","last_undegraded":"2024-05-09T08:04:27.735486+0000","last_fullsized":"2024-05-09T08:04:27.735486+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:47:32.605288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:27.747004+0000","last_change":"2024-05-09T08:02:15.904744+0000","last_active":"2024-05-09T08:04:27.747004+0000","last_peered":"2024-05-09T08:04:27.747004+0000","last_clean":"2024-05-09T08:04:27.747004+0000","last_became_active":"2024-05-09T08:02:15.904334+0000","last_became_peered":"2024-05-09T08:02:15.904334+0000","last_unstale":"2024-05-09T08:04:27.747004+0000","last_undegraded":"2024-05-09T08:04:27.747004+0000","last_fullsized":"2024-05-09T08:04:27.747004+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:30:33.351158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:27.735580+0000","last_change":"2024-05-09T08:02:15.904835+0000","last_active":"2024-05-09T08:04:27.735580+0000","last_peered":"2024-05-09T08:04:27.735580+0000","last_clean":"2024-05-09T08:04:27.735580+0000","last_became_active":"2024-05-09T08:02:15.904416+0000","last_became_peered":"2024-05-09T08:02:15.904416+0000","last_unstale":"2024-05-09T08:04:27.735580+0000","last_undegraded":"2024-05-09T08:04:27.735580+0000","last_fullsized":"2024-05-09T08:04:27.735580+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:47:32.605288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:32.747675+0000","last_change":"2024-05-09T08:02:15.898324+0000","last_active":"2024-05-09T08:04:32.747675+0000","last_peered":"2024-05-09T08:04:32.747675+0000","last_clean":"2024-05-09T08:04:32.747675+0000","last_became_active":"2024-05-09T08:02:15.897981+0000","last_became_peered":"2024-05-09T08:02:15.897981+0000","last_unstale":"2024-05-09T08:04:32.747675+0000","last_undegraded":"2024-05-09T08:04:32.747675+0000","last_fullsized":"2024-05-09T08:04:32.747675+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:01:50.341133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:32.736201+0000","last_change":"2024-05-09T08:02:15.890945+0000","last_active":"2024-05-09T08:04:32.736201+0000","last_peered":"2024-05-09T08:04:32.736201+0000","last_clean":"2024-05-09T08:04:32.736201+0000","last_became_active":"2024-05-09T08:02:15.890647+0000","last_became_peered":"2024-05-09T08:02:15.890647+0000","last_unstale":"2024-05-09T08:04:32.736201+0000","last_undegraded":"2024-05-09T08:04:32.736201+0000","last_fullsized":"2024-05-09T08:04:32.736201+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:41:56.871863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:32.754272+0000","last_change":"2024-05-09T08:02:15.913728+0000","last_active":"2024-05-09T08:04:32.754272+0000","last_peered":"2024-05-09T08:04:32.754272+0000","last_clean":"2024-05-09T08:04:32.754272+0000","last_became_active":"2024-05-09T08:02:15.909818+0000","last_became_peered":"2024-05-09T08:02:15.909818+0000","last_unstale":"2024-05-09T08:04:32.754272+0000","last_undegraded":"2024-05-09T08:04:32.754272+0000","last_fullsized":"2024-05-09T08:04:32.754272+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:41:55.788033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:32.736139+0000","last_change":"2024-05-09T08:02:15.885025+0000","last_active":"2024-05-09T08:04:32.736139+0000","last_peered":"2024-05-09T08:04:32.736139+0000","last_clean":"2024-05-09T08:04:32.736139+0000","last_became_active":"2024-05-09T08:02:15.884517+0000","last_became_peered":"2024-05-09T08:02:15.884517+0000","last_unstale":"2024-05-09T08:04:32.736139+0000","last_undegraded":"2024-05-09T08:04:32.736139+0000","last_fullsized":"2024-05-09T08:04:32.736139+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:01:50.341133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:32.735890+0000","last_change":"2024-05-09T08:02:15.890306+0000","last_active":"2024-05-09T08:04:32.735890+0000","last_peered":"2024-05-09T08:04:32.735890+0000","last_clean":"2024-05-09T08:04:32.735890+0000","last_became_active":"2024-05-09T08:02:15.889998+0000","last_became_peered":"2024-05-09T08:02:15.889998+0000","last_unstale":"2024-05-09T08:04:32.735890+0000","last_undegraded":"2024-05-09T08:04:32.735890+0000","last_fullsized":"2024-05-09T08:04:32.735890+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:16:57.446444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:37.747977+0000","last_change":"2024-05-09T08:02:15.886417+0000","last_active":"2024-05-09T08:04:37.747977+0000","last_peered":"2024-05-09T08:04:37.747977+0000","last_clean":"2024-05-09T08:04:37.747977+0000","last_became_active":"2024-05-09T08:02:15.885525+0000","last_became_peered":"2024-05-09T08:02:15.885525+0000","last_unstale":"2024-05-09T08:04:37.747977+0000","last_undegraded":"2024-05-09T08:04:37.747977+0000","last_fullsized":"2024-05-09T08:04:37.747977+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:43:02.075442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:37.736779+0000","last_change":"2024-05-09T08:02:15.893172+0000","last_active":"2024-05-09T08:04:37.736779+0000","last_peered":"2024-05-09T08:04:37.736779+0000","last_clean":"2024-05-09T08:04:37.736779+0000","last_became_active":"2024-05-09T08:02:15.892886+0000","last_became_peered":"2024-05-09T08:02:15.892886+0000","last_unstale":"2024-05-09T08:04:37.736779+0000","last_undegraded":"2024-05-09T08:04:37.736779+0000","last_fullsized":"2024-05-09T08:04:37.736779+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:59:14.957806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:37.736714+0000","last_change":"2024-05-09T08:02:15.908659+0000","last_active":"2024-05-09T08:04:37.736714+0000","last_peered":"2024-05-09T08:04:37.736714+0000","last_clean":"2024-05-09T08:04:37.736714+0000","last_became_active":"2024-05-09T08:02:15.908307+0000","last_became_peered":"2024-05-09T08:02:15.908307+0000","last_unstale":"2024-05-09T08:04:37.736714+0000","last_undegraded":"2024-05-09T08:04:37.736714+0000","last_fullsized":"2024-05-09T08:04:37.736714+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:20:07.368327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:37.761383+0000","last_change":"2024-05-09T08:02:15.885800+0000","last_active":"2024-05-09T08:04:37.761383+0000","last_peered":"2024-05-09T08:04:37.761383+0000","last_clean":"2024-05-09T08:04:37.761383+0000","last_became_active":"2024-05-09T08:02:15.885185+0000","last_became_peered":"2024-05-09T08:02:15.885185+0000","last_unstale":"2024-05-09T08:04:37.761383+0000","last_undegraded":"2024-05-09T08:04:37.761383+0000","last_fullsized":"2024-05-09T08:04:37.761383+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:01:50.341133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:37.755012+0000","last_change":"2024-05-09T08:02:15.911496+0000","last_active":"2024-05-09T08:04:37.755012+0000","last_peered":"2024-05-09T08:04:37.755012+0000","last_clean":"2024-05-09T08:04:37.755012+0000","last_became_active":"2024-05-09T08:02:15.911142+0000","last_became_peered":"2024-05-09T08:02:15.911142+0000","last_unstale":"2024-05-09T08:04:37.755012+0000","last_undegraded":"2024-05-09T08:04:37.755012+0000","last_fullsized":"2024-05-09T08:04:37.755012+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:27:59.315750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:42.749013+0000","last_change":"2024-05-09T08:02:15.898847+0000","last_active":"2024-05-09T08:04:42.749013+0000","last_peered":"2024-05-09T08:04:42.749013+0000","last_clean":"2024-05-09T08:04:42.749013+0000","last_became_active":"2024-05-09T08:02:15.898480+0000","last_became_peered":"2024-05-09T08:02:15.898480+0000","last_unstale":"2024-05-09T08:04:42.749013+0000","last_undegraded":"2024-05-09T08:04:42.749013+0000","last_fullsized":"2024-05-09T08:04:42.749013+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:20:07.368327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:42.736898+0000","last_change":"2024-05-09T08:02:15.883117+0000","last_active":"2024-05-09T08:04:42.736898+0000","last_peered":"2024-05-09T08:04:42.736898+0000","last_clean":"2024-05-09T08:04:42.736898+0000","last_became_active":"2024-05-09T08:02:15.882141+0000","last_became_peered":"2024-05-09T08:02:15.882141+0000","last_unstale":"2024-05-09T08:04:42.736898+0000","last_undegraded":"2024-05-09T08:04:42.736898+0000","last_fullsized":"2024-05-09T08:04:42.736898+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:57:11.352151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":31,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:42.737262+0000","last_change":"2024-05-09T08:03:02.322462+0000","last_active":"2024-05-09T08:04:42.737262+0000","last_peered":"2024-05-09T08:04:42.737262+0000","last_clean":"2024-05-09T08:04:42.737262+0000","last_became_active":"2024-05-09T08:03:02.322073+0000","last_became_peered":"2024-05-09T08:03:02.322073+0000","last_unstale":"2024-05-09T08:04:42.737262+0000","last_undegraded":"2024-05-09T08:04:42.737262+0000","last_fullsized":"2024-05-09T08:04:42.737262+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:16:37.795617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"27'1","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:05:27.208367+0000","last_change":"2024-05-09T08:02:15.886482+0000","last_active":"2024-05-09T08:05:27.208367+0000","last_peered":"2024-05-09T08:05:27.208367+0000","last_clean":"2024-05-09T08:05:27.208367+0000","last_became_active":"2024-05-09T08:02:15.886183+0000","last_became_peered":"2024-05-09T08:02:15.886183+0000","last_unstale":"2024-05-09T08:05:27.208367+0000","last_undegraded":"2024-05-09T08:05:27.208367+0000","last_fullsized":"2024-05-09T08:05:27.208367+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:23:40.248821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:42.736901+0000","last_change":"2024-05-09T08:02:15.887506+0000","last_active":"2024-05-09T08:04:42.736901+0000","last_peered":"2024-05-09T08:04:42.736901+0000","last_clean":"2024-05-09T08:04:42.736901+0000","last_became_active":"2024-05-09T08:02:15.887225+0000","last_became_peered":"2024-05-09T08:02:15.887225+0000","last_unstale":"2024-05-09T08:04:42.736901+0000","last_undegraded":"2024-05-09T08:04:42.736901+0000","last_fullsized":"2024-05-09T08:04:42.736901+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:25:24.736323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:47.753113+0000","last_change":"2024-05-09T08:02:15.902065+0000","last_active":"2024-05-09T08:04:47.753113+0000","last_peered":"2024-05-09T08:04:47.753113+0000","last_clean":"2024-05-09T08:04:47.753113+0000","last_became_active":"2024-05-09T08:02:15.901766+0000","last_became_peered":"2024-05-09T08:02:15.901766+0000","last_unstale":"2024-05-09T08:04:47.753113+0000","last_undegraded":"2024-05-09T08:04:47.753113+0000","last_fullsized":"2024-05-09T08:04:47.753113+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:05:02.181367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:47.737268+0000","last_change":"2024-05-09T08:02:15.911058+0000","last_active":"2024-05-09T08:04:47.737268+0000","last_peered":"2024-05-09T08:04:47.737268+0000","last_clean":"2024-05-09T08:04:47.737268+0000","last_became_active":"2024-05-09T08:02:15.910716+0000","last_became_peered":"2024-05-09T08:02:15.910716+0000","last_unstale":"2024-05-09T08:04:47.737268+0000","last_undegraded":"2024-05-09T08:04:47.737268+0000","last_fullsized":"2024-05-09T08:04:47.737268+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:41:56.871863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:47.737757+0000","last_change":"2024-05-09T08:02:15.908653+0000","last_active":"2024-05-09T08:04:47.737757+0000","last_peered":"2024-05-09T08:04:47.737757+0000","last_clean":"2024-05-09T08:04:47.737757+0000","last_became_active":"2024-05-09T08:02:15.908290+0000","last_became_peered":"2024-05-09T08:02:15.908290+0000","last_unstale":"2024-05-09T08:04:47.737757+0000","last_undegraded":"2024-05-09T08:04:47.737757+0000","last_fullsized":"2024-05-09T08:04:47.737757+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:18:36.309627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:47.737286+0000","last_change":"2024-05-09T08:02:15.896652+0000","last_active":"2024-05-09T08:04:47.737286+0000","last_peered":"2024-05-09T08:04:47.737286+0000","last_clean":"2024-05-09T08:04:47.737286+0000","last_became_active":"2024-05-09T08:02:15.896384+0000","last_became_peered":"2024-05-09T08:02:15.896384+0000","last_unstale":"2024-05-09T08:04:47.737286+0000","last_undegraded":"2024-05-09T08:04:47.737286+0000","last_fullsized":"2024-05-09T08:04:47.737286+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:18:36.309627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:05:02.751368+0000","last_change":"2024-05-09T08:03:02.615230+0000","last_active":"2024-05-09T08:05:02.751368+0000","last_peered":"2024-05-09T08:05:02.751368+0000","last_clean":"2024-05-09T08:05:02.751368+0000","last_became_active":"2024-05-09T08:03:02.614387+0000","last_became_peered":"2024-05-09T08:03:02.614387+0000","last_unstale":"2024-05-09T08:05:02.751368+0000","last_undegraded":"2024-05-09T08:05:02.751368+0000","last_fullsized":"2024-05-09T08:05:02.751368+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:53:23.178610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:22.752990+0000","last_change":"2024-05-09T08:02:15.905078+0000","last_active":"2024-05-09T08:04:22.752990+0000","last_peered":"2024-05-09T08:04:22.752990+0000","last_clean":"2024-05-09T08:04:22.752990+0000","last_became_active":"2024-05-09T08:02:15.904736+0000","last_became_peered":"2024-05-09T08:02:15.904736+0000","last_unstale":"2024-05-09T08:04:22.752990+0000","last_undegraded":"2024-05-09T08:04:22.752990+0000","last_fullsized":"2024-05-09T08:04:22.752990+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:01:50.341133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:22.759254+0000","last_change":"2024-05-09T08:02:15.897743+0000","last_active":"2024-05-09T08:04:22.759254+0000","last_peered":"2024-05-09T08:04:22.759254+0000","last_clean":"2024-05-09T08:04:22.759254+0000","last_became_active":"2024-05-09T08:02:15.897367+0000","last_became_peered":"2024-05-09T08:02:15.897367+0000","last_unstale":"2024-05-09T08:04:22.759254+0000","last_undegraded":"2024-05-09T08:04:22.759254+0000","last_fullsized":"2024-05-09T08:04:22.759254+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:47:32.605288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:05:07.752186+0000","last_change":"2024-05-09T08:03:02.323519+0000","last_active":"2024-05-09T08:05:07.752186+0000","last_peered":"2024-05-09T08:05:07.752186+0000","last_clean":"2024-05-09T08:05:07.752186+0000","last_became_active":"2024-05-09T08:03:02.322397+0000","last_became_peered":"2024-05-09T08:03:02.322397+0000","last_unstale":"2024-05-09T08:05:07.752186+0000","last_undegraded":"2024-05-09T08:05:07.752186+0000","last_fullsized":"2024-05-09T08:05:07.752186+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:47:32.605288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9216,"num_write_kb":25165824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10755,"ondisk_log_size":10755,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":448762,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":277,"num_write_kb":617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":966656,"data_stored":897524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":258,"ondisk_log_size":258,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574919,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50092,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321748,"statfs":{"total":96636764160,"available":96585469952,"internally_reserved":0,"allocated":888832,"data_stored":709919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574919,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":76260,"kb_used_data":732,"kb_used_omap":1,"kb_used_meta":75518,"kb_avail":94295580,"statfs":{"total":96636764160,"available":96558673920,"internally_reserved":0,"allocated":749568,"data_stored":566404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":77330886},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574919,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45352,"kb_used_data":288,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94326488,"statfs":{"total":96636764160,"available":96590323712,"internally_reserved":0,"allocated":294912,"data_stored":119059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574920,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":53680,"kb_used_data":296,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94318160,"statfs":{"total":96636764160,"available":96581795840,"internally_reserved":0,"allocated":303104,"data_stored":119095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":54655431},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574920,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":76264,"kb_used_data":736,"kb_used_omap":1,"kb_used_meta":75518,"kb_avail":94295576,"statfs":{"total":96636764160,"available":96558669824,"internally_reserved":0,"allocated":753664,"data_stored":566933,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":77330887},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574920,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":57832,"kb_used_data":288,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94314008,"statfs":{"total":96636764160,"available":96577544192,"internally_reserved":0,"allocated":294912,"data_stored":119498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574921,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":75820,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":75518,"kb_avail":94296020,"statfs":{"total":96636764160,"available":96559124480,"internally_reserved":0,"allocated":299008,"data_stored":119129,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77330885},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574920,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45940,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325900,"statfs":{"total":96636764160,"available":96589721600,"internally_reserved":0,"allocated":897024,"data_stored":709975,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":462848,"data_stored":447964,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":458752,"data_stored":447435,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T08:07:57.125 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-05-09T08:07:57.125 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-09T08:07:57.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:57.369+0000 7fe2fec00640 1 Processor -- start 2024-05-09T08:07:57.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:57.369+0000 7fe2fec00640 1 -- start start 2024-05-09T08:07:57.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:57.371+0000 7fe2fec00640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fe2f815f1f0 0x7fe2f806b250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:07:57.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:57.371+0000 7fe2fec00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fe2f815f610 0x7fe2f806b790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:07:57.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:57.371+0000 7fe2fec00640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe2f806bfd0 con 0x7fe2f8158cf0 2024-05-09T08:07:57.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:57.371+0000 7fe2fec00640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7fe2f815ed60 con 0x7fe2f815f1f0 2024-05-09T08:07:57.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:57.371+0000 7fe2fec00640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7fe2f80c8210 con 0x7fe2f815f610 2024-05-09T08:07:57.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:57.371+0000 7fe2fce00640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fe2f815f1f0 0x7fe2f806b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:07:57.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:57.371+0000 7fe2fce00640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fe2f815f1f0 0x7fe2f806b250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60580/0 (socket says 172.21.15.1:60580) 2024-05-09T08:07:57.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:57.371+0000 7fe2fce00640 1 -- 172.21.15.1:0/2347309418 learned_addr learned my addr 172.21.15.1:0/2347309418 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:07:57.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:57.372+0000 7fe2fe200640 1 --2- 172.21.15.1:0/2347309418 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fe2f815f610 0x7fe2f806b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:07:57.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:57.372+0000 7fe2fce00640 1 -- 172.21.15.1:0/2347309418 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fe2f815f610 msgr2=0x7fe2f806b790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:07:57.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:57.372+0000 7fe2fce00640 1 --2- 172.21.15.1:0/2347309418 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fe2f815f610 0x7fe2f806b790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:57.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:57.372+0000 7fe2fce00640 1 -- 172.21.15.1:0/2347309418 >> v1:172.21.15.1:6789/0 conn(0x7fe2f8158cf0 legacy=0x7fe2f8162850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:07:57.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:57.372+0000 7fe2fce00640 1 -- 172.21.15.1:0/2347309418 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2f80c76b0 con 0x7fe2f815f1f0 2024-05-09T08:07:57.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:57.373+0000 7fe2fce00640 1 --2- 172.21.15.1:0/2347309418 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe2f815f1f0 0x7fe2f806b250 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fe2e4009870 tx=0x7fe2e402a2e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8bda28350fdc3bc server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:07:57.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:57.373+0000 7fe2f7e00640 1 -- 172.21.15.1:0/2347309418 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2e4037070 con 0x7fe2f815f1f0 2024-05-09T08:07:57.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:57.374+0000 7fe2f7e00640 1 -- 172.21.15.1:0/2347309418 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2e4032640 con 0x7fe2f815f1f0 2024-05-09T08:07:57.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:57.374+0000 7fe2f7e00640 1 -- 172.21.15.1:0/2347309418 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2e4032a10 con 0x7fe2f815f1f0 2024-05-09T08:07:57.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:57.374+0000 7fe2fec00640 1 -- 172.21.15.1:0/2347309418 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe2f815f1f0 msgr2=0x7fe2f806b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:07:57.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:57.374+0000 7fe2fec00640 1 --2- 172.21.15.1:0/2347309418 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe2f815f1f0 0x7fe2f806b250 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fe2e4009870 tx=0x7fe2e402a2e0 comp rx=0 tx=0).stop 2024-05-09T08:07:57.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:57.374+0000 7fe2fec00640 1 -- 172.21.15.1:0/2347309418 shutdown_connections 2024-05-09T08:07:57.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:57.374+0000 7fe2fec00640 1 --2- 172.21.15.1:0/2347309418 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fe2f815f610 0x7fe2f806b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:57.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:57.374+0000 7fe2fec00640 1 --2- 172.21.15.1:0/2347309418 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe2f815f1f0 0x7fe2f806b250 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:57.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:57.374+0000 7fe2fec00640 1 -- 172.21.15.1:0/2347309418 >> 172.21.15.1:0/2347309418 conn(0x7fe2f80cbc10 msgr2=0x7fe2f80cc020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:07:57.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:57.374+0000 7fe2fec00640 1 -- 172.21.15.1:0/2347309418 shutdown_connections 2024-05-09T08:07:57.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:57.374+0000 7fe2fec00640 1 -- 172.21.15.1:0/2347309418 wait complete. 2024-05-09T08:07:57.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:57.376+0000 7fe2fec00640 1 Processor -- start 2024-05-09T08:07:57.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:57.376+0000 7fe2fec00640 1 -- start start 2024-05-09T08:07:57.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:57.377+0000 7fe2fec00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fe2f8158cf0 0x7fe2f8157a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:07:57.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:57.377+0000 7fe2fec00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe2f815f1f0 0x7fe2f806b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:07:57.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:57.377+0000 7fe2fec00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe2f815f610 0x7fe2f8151e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:07:57.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:57.377+0000 7fe2fec00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fe2f80c7f30 con 0x7fe2f815f1f0 2024-05-09T08:07:57.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:57.377+0000 7fe2fec00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe2f80c7c50 con 0x7fe2f815f610 2024-05-09T08:07:57.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:57.377+0000 7fe2fec00640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7fe2f815e9f0 con 0x7fe2f8158cf0 2024-05-09T08:07:57.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:57.378+0000 7fe2fe200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe2f815f610 0x7fe2f8151e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:07:57.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:57.378+0000 7fe2fe200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe2f815f610 0x7fe2f8151e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.1:54058/0 (socket says 172.21.15.1:54058) 2024-05-09T08:07:57.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:57.378+0000 7fe2fe200640 1 -- 172.21.15.1:0/1962568742 learned_addr learned my addr 172.21.15.1:0/1962568742 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:07:57.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:57.378+0000 7fe2fd800640 1 --2- 172.21.15.1:0/1962568742 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fe2f8158cf0 0x7fe2f8157a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:07:57.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:57.378+0000 7fe2fce00640 1 --2- 172.21.15.1:0/1962568742 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe2f815f1f0 0x7fe2f806b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:07:57.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:57.378+0000 7fe2fe200640 1 -- 172.21.15.1:0/1962568742 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fe2f8158cf0 msgr2=0x7fe2f8157a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:07:57.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:57.378+0000 7fe2fe200640 1 --2- 172.21.15.1:0/1962568742 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fe2f8158cf0 0x7fe2f8157a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:57.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:57.378+0000 7fe2fe200640 1 -- 172.21.15.1:0/1962568742 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe2f815f1f0 msgr2=0x7fe2f806b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:07:57.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:57.378+0000 7fe2fe200640 1 --2- 172.21.15.1:0/1962568742 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe2f815f1f0 0x7fe2f806b4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:57.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:57.378+0000 7fe2fe200640 1 -- 172.21.15.1:0/1962568742 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2f8155f20 con 0x7fe2f815f610 2024-05-09T08:07:57.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:57.379+0000 7fe2fe200640 1 --2- 172.21.15.1:0/1962568742 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe2f815f610 0x7fe2f8151e50 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fe2f000efc0 tx=0x7fe2f000c490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:07:57.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:57.379+0000 7fe2f5600640 1 -- 172.21.15.1:0/1962568742 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2f0018020 con 0x7fe2f815f610 2024-05-09T08:07:57.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:57.379+0000 7fe2f5600640 1 -- 172.21.15.1:0/1962568742 <== mon.1 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2f0014480 con 0x7fe2f815f610 2024-05-09T08:07:57.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:57.379+0000 7fe2f5600640 1 -- 172.21.15.1:0/1962568742 <== mon.1 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2f0009e60 con 0x7fe2f815f610 2024-05-09T08:07:57.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:57.379+0000 7fe2fec00640 1 -- 172.21.15.1:0/1962568742 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2f81553a0 con 0x7fe2f815f610 2024-05-09T08:07:57.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:57.379+0000 7fe2fec00640 1 -- 172.21.15.1:0/1962568742 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2f8155960 con 0x7fe2f815f610 2024-05-09T08:07:57.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:57.380+0000 7fe2fec00640 1 -- 172.21.15.1:0/1962568742 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2f804f660 con 0x7fe2f815f610 2024-05-09T08:07:57.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:57.381+0000 7fe2f5600640 1 -- 172.21.15.1:0/1962568742 <== mon.1 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe2f0005d10 con 0x7fe2f815f610 2024-05-09T08:07:57.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:57.382+0000 7fe2f5600640 1 --2- 172.21.15.1:0/1962568742 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fe2d806be60 0x7fe2d806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:07:57.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:57.382+0000 7fe2f5600640 1 -- 172.21.15.1:0/1962568742 <== mon.1 v2:172.21.15.7:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7fe2f008be50 con 0x7fe2f815f610 2024-05-09T08:07:57.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:57.383+0000 7fe2fd800640 1 --2- 172.21.15.1:0/1962568742 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fe2d806be60 0x7fe2d806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:07:57.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:57.383+0000 7fe2fd800640 1 --2- 172.21.15.1:0/1962568742 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fe2d806be60 0x7fe2d806e320 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fe2ec001780 tx=0x7fe2ec009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:07:57.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:57.387+0000 7fe2f5600640 1 -- 172.21.15.1:0/1962568742 <== mon.1 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fe2f0022030 con 0x7fe2f815f610 2024-05-09T08:07:57.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:57.587+0000 7fe2fec00640 1 -- 172.21.15.1:0/1962568742 --> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe2f806c1c0 con 0x7fe2d806be60 2024-05-09T08:07:57.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:57.597+0000 7fe2f5600640 1 -- 172.21.15.1:0/1962568742 <== mgr.4114 v2:172.21.15.7:6832/3562612464 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+333085 (secure 0 0 0) 0x7fe2f806c1c0 con 0x7fe2d806be60 2024-05-09T08:07:57.612 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T08:07:57.613 INFO:teuthology.orchestra.run.smithi001.stderr:dumped all 2024-05-09T08:07:57.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:57.603+0000 7fe2fec00640 1 -- 172.21.15.1:0/1962568742 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fe2d806be60 msgr2=0x7fe2d806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:07:57.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:57.603+0000 7fe2fec00640 1 --2- 172.21.15.1:0/1962568742 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fe2d806be60 0x7fe2d806e320 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fe2ec001780 tx=0x7fe2ec009040 comp rx=0 tx=0).stop 2024-05-09T08:07:57.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:57.603+0000 7fe2fec00640 1 -- 172.21.15.1:0/1962568742 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe2f815f610 msgr2=0x7fe2f8151e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:07:57.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:57.603+0000 7fe2fec00640 1 --2- 172.21.15.1:0/1962568742 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe2f815f610 0x7fe2f8151e50 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fe2f000efc0 tx=0x7fe2f000c490 comp rx=0 tx=0).stop 2024-05-09T08:07:57.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:57.603+0000 7fe2fec00640 1 -- 172.21.15.1:0/1962568742 shutdown_connections 2024-05-09T08:07:57.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:57.603+0000 7fe2fec00640 1 --2- 172.21.15.1:0/1962568742 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fe2d806be60 0x7fe2d806e320 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:57.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:57.603+0000 7fe2fec00640 1 --2- 172.21.15.1:0/1962568742 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe2f815f610 0x7fe2f8151e50 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:57.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:57.603+0000 7fe2fec00640 1 --2- 172.21.15.1:0/1962568742 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe2f815f1f0 0x7fe2f806b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:57.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:57.603+0000 7fe2fec00640 1 --2- 172.21.15.1:0/1962568742 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fe2f8158cf0 0x7fe2f8157a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:57.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:57.603+0000 7fe2fec00640 1 -- 172.21.15.1:0/1962568742 >> 172.21.15.1:0/1962568742 conn(0x7fe2f80cbc10 msgr2=0x7fe2f80c48e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:07:57.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:57.603+0000 7fe2fec00640 1 -- 172.21.15.1:0/1962568742 shutdown_connections 2024-05-09T08:07:57.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:57.603+0000 7fe2fec00640 1 -- 172.21.15.1:0/1962568742 wait complete. 2024-05-09T08:07:57.632 INFO:teuthology.orchestra.run.smithi001.stdout:{"pg_ready":true,"pg_map":{"version":195,"stamp":"2024-05-09T08:07:57.185331+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1039130,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":121,"num_read_kb":105,"num_write":9652,"num_write_kb":25168857,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":11133,"ondisk_log_size":11133,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":481240,"kb_used_data":4376,"kb_used_omap":12,"kb_used_meta":476787,"kb_avail":754493480,"statfs":{"total":773094113280,"available":772601323520,"internally_reserved":0,"allocated":4481024,"data_stored":3030012,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":488230450},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003859"},"pg_stats":[{"pgid":"2.1f","version":"24'4","reported_seq":31,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:22.753051+0000","last_change":"2024-05-09T08:02:14.875108+0000","last_active":"2024-05-09T08:04:22.753051+0000","last_peered":"2024-05-09T08:04:22.753051+0000","last_clean":"2024-05-09T08:04:22.753051+0000","last_became_active":"2024-05-09T08:02:14.874841+0000","last_became_peered":"2024-05-09T08:02:14.874841+0000","last_unstale":"2024-05-09T08:04:22.753051+0000","last_undegraded":"2024-05-09T08:04:22.753051+0000","last_fullsized":"2024-05-09T08:04:22.753051+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:00:40.599475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:17.745934+0000","last_change":"2024-05-09T08:02:15.899818+0000","last_active":"2024-05-09T08:04:17.745934+0000","last_peered":"2024-05-09T08:04:17.745934+0000","last_clean":"2024-05-09T08:04:17.745934+0000","last_became_active":"2024-05-09T08:02:15.899479+0000","last_became_peered":"2024-05-09T08:02:15.899479+0000","last_unstale":"2024-05-09T08:04:17.745934+0000","last_undegraded":"2024-05-09T08:04:17.745934+0000","last_fullsized":"2024-05-09T08:04:17.745934+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:18:36.309627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"27'360","reported_seq":380,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:15.262265+0000","last_change":"2024-05-09T08:02:20.057944+0000","last_active":"2024-05-09T08:07:15.262265+0000","last_peered":"2024-05-09T08:07:15.262265+0000","last_clean":"2024-05-09T08:07:15.262265+0000","last_became_active":"2024-05-09T08:02:20.057016+0000","last_became_peered":"2024-05-09T08:02:20.057016+0000","last_unstale":"2024-05-09T08:07:15.262265+0000","last_undegraded":"2024-05-09T08:07:15.262265+0000","last_fullsized":"2024-05-09T08:07:15.262265+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":360,"log_dups_size":0,"ondisk_log_size":360,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:17:01.514168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":306,"num_write_kb":835584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"24'1","reported_seq":28,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:17.745962+0000","last_change":"2024-05-09T08:02:14.886137+0000","last_active":"2024-05-09T08:04:17.745962+0000","last_peered":"2024-05-09T08:04:17.745962+0000","last_clean":"2024-05-09T08:04:17.745962+0000","last_became_active":"2024-05-09T08:02:14.885830+0000","last_became_peered":"2024-05-09T08:02:14.885830+0000","last_unstale":"2024-05-09T08:04:17.745962+0000","last_undegraded":"2024-05-09T08:04:17.745962+0000","last_fullsized":"2024-05-09T08:04:17.745962+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:46:11.160815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:22.753022+0000","last_change":"2024-05-09T08:02:15.906280+0000","last_active":"2024-05-09T08:04:22.753022+0000","last_peered":"2024-05-09T08:04:22.753022+0000","last_clean":"2024-05-09T08:04:22.753022+0000","last_became_active":"2024-05-09T08:02:15.905590+0000","last_became_peered":"2024-05-09T08:02:15.905590+0000","last_unstale":"2024-05-09T08:04:22.753022+0000","last_undegraded":"2024-05-09T08:04:22.753022+0000","last_fullsized":"2024-05-09T08:04:22.753022+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:20:07.368327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"27'377","reported_seq":397,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:15.276306+0000","last_change":"2024-05-09T08:02:19.954548+0000","last_active":"2024-05-09T08:07:15.276306+0000","last_peered":"2024-05-09T08:07:15.276306+0000","last_clean":"2024-05-09T08:07:15.276306+0000","last_became_active":"2024-05-09T08:02:19.954161+0000","last_became_peered":"2024-05-09T08:02:19.954161+0000","last_unstale":"2024-05-09T08:07:15.276306+0000","last_undegraded":"2024-05-09T08:07:15.276306+0000","last_fullsized":"2024-05-09T08:07:15.276306+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":377,"log_dups_size":0,"ondisk_log_size":377,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:53:27.246334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":339,"num_write_kb":925696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"27'3","reported_seq":30,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:05:05.164895+0000","last_change":"2024-05-09T08:02:14.867128+0000","last_active":"2024-05-09T08:05:05.164895+0000","last_peered":"2024-05-09T08:05:05.164895+0000","last_clean":"2024-05-09T08:05:05.164895+0000","last_became_active":"2024-05-09T08:02:14.866809+0000","last_became_peered":"2024-05-09T08:02:14.866809+0000","last_unstale":"2024-05-09T08:05:05.164895+0000","last_undegraded":"2024-05-09T08:05:05.164895+0000","last_fullsized":"2024-05-09T08:05:05.164895+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:57:06.093718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:47.737794+0000","last_change":"2024-05-09T08:02:15.888431+0000","last_active":"2024-05-09T08:04:47.737794+0000","last_peered":"2024-05-09T08:04:47.737794+0000","last_clean":"2024-05-09T08:04:47.737794+0000","last_became_active":"2024-05-09T08:02:15.887509+0000","last_became_peered":"2024-05-09T08:02:15.887509+0000","last_unstale":"2024-05-09T08:04:47.737794+0000","last_undegraded":"2024-05-09T08:04:47.737794+0000","last_fullsized":"2024-05-09T08:04:47.737794+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:25:01.238816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"27'324","reported_seq":344,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:15.265243+0000","last_change":"2024-05-09T08:02:19.953600+0000","last_active":"2024-05-09T08:07:15.265243+0000","last_peered":"2024-05-09T08:07:15.265243+0000","last_clean":"2024-05-09T08:07:15.265243+0000","last_became_active":"2024-05-09T08:02:19.952275+0000","last_became_peered":"2024-05-09T08:02:19.952275+0000","last_unstale":"2024-05-09T08:07:15.265243+0000","last_undegraded":"2024-05-09T08:07:15.265243+0000","last_fullsized":"2024-05-09T08:07:15.265243+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":324,"log_dups_size":0,"ondisk_log_size":324,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:25:05.306541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":273,"num_write_kb":745472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"24'2","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:47.737301+0000","last_change":"2024-05-09T08:02:14.870819+0000","last_active":"2024-05-09T08:04:47.737301+0000","last_peered":"2024-05-09T08:04:47.737301+0000","last_clean":"2024-05-09T08:04:47.737301+0000","last_became_active":"2024-05-09T08:02:14.870476+0000","last_became_peered":"2024-05-09T08:02:14.870476+0000","last_unstale":"2024-05-09T08:04:47.737301+0000","last_undegraded":"2024-05-09T08:04:47.737301+0000","last_fullsized":"2024-05-09T08:04:47.737301+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:43:01.061801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:47.737815+0000","last_change":"2024-05-09T08:02:15.886471+0000","last_active":"2024-05-09T08:04:47.737815+0000","last_peered":"2024-05-09T08:04:47.737815+0000","last_clean":"2024-05-09T08:04:47.737815+0000","last_became_active":"2024-05-09T08:02:15.886168+0000","last_became_peered":"2024-05-09T08:02:15.886168+0000","last_unstale":"2024-05-09T08:04:47.737815+0000","last_undegraded":"2024-05-09T08:04:47.737815+0000","last_fullsized":"2024-05-09T08:04:47.737815+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:30:33.351158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"27'298","reported_seq":318,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:15.295149+0000","last_change":"2024-05-09T08:02:19.947347+0000","last_active":"2024-05-09T08:07:15.295149+0000","last_peered":"2024-05-09T08:07:15.295149+0000","last_clean":"2024-05-09T08:07:15.295149+0000","last_became_active":"2024-05-09T08:02:19.946906+0000","last_became_peered":"2024-05-09T08:02:19.946906+0000","last_unstale":"2024-05-09T08:07:15.295149+0000","last_undegraded":"2024-05-09T08:07:15.295149+0000","last_fullsized":"2024-05-09T08:07:15.295149+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":298,"log_dups_size":0,"ondisk_log_size":298,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:30:37.418883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":252,"num_write_kb":688128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:27.735428+0000","last_change":"2024-05-09T08:02:15.880765+0000","last_active":"2024-05-09T08:04:27.735428+0000","last_peered":"2024-05-09T08:04:27.735428+0000","last_clean":"2024-05-09T08:04:27.735428+0000","last_became_active":"2024-05-09T08:02:15.880439+0000","last_became_peered":"2024-05-09T08:02:15.880439+0000","last_unstale":"2024-05-09T08:04:27.735428+0000","last_undegraded":"2024-05-09T08:04:27.735428+0000","last_fullsized":"2024-05-09T08:04:27.735428+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:29:59.441314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"27'3","reported_seq":30,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:05:05.164991+0000","last_change":"2024-05-09T08:02:14.867587+0000","last_active":"2024-05-09T08:05:05.164991+0000","last_peered":"2024-05-09T08:05:05.164991+0000","last_clean":"2024-05-09T08:05:05.164991+0000","last_became_active":"2024-05-09T08:02:14.867291+0000","last_became_peered":"2024-05-09T08:02:14.867291+0000","last_unstale":"2024-05-09T08:05:05.164991+0000","last_undegraded":"2024-05-09T08:05:05.164991+0000","last_fullsized":"2024-05-09T08:05:05.164991+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:20:08.921752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"27'366","reported_seq":386,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:15.267986+0000","last_change":"2024-05-09T08:02:19.954902+0000","last_active":"2024-05-09T08:07:15.267986+0000","last_peered":"2024-05-09T08:07:15.267986+0000","last_clean":"2024-05-09T08:07:15.267986+0000","last_became_active":"2024-05-09T08:02:19.954007+0000","last_became_peered":"2024-05-09T08:02:19.954007+0000","last_unstale":"2024-05-09T08:07:15.267986+0000","last_undegraded":"2024-05-09T08:07:15.267986+0000","last_fullsized":"2024-05-09T08:07:15.267986+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":366,"log_dups_size":0,"ondisk_log_size":366,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:41:31.470546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":309,"num_write_kb":843776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:32.747723+0000","last_change":"2024-05-09T08:02:15.915155+0000","last_active":"2024-05-09T08:04:32.747723+0000","last_peered":"2024-05-09T08:04:32.747723+0000","last_clean":"2024-05-09T08:04:32.747723+0000","last_became_active":"2024-05-09T08:02:15.914852+0000","last_became_peered":"2024-05-09T08:02:15.914852+0000","last_unstale":"2024-05-09T08:04:32.747723+0000","last_undegraded":"2024-05-09T08:04:32.747723+0000","last_fullsized":"2024-05-09T08:04:32.747723+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:53:23.178610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:27.759691+0000","last_change":"2024-05-09T08:02:14.881770+0000","last_active":"2024-05-09T08:04:27.759691+0000","last_peered":"2024-05-09T08:04:27.759691+0000","last_clean":"2024-05-09T08:04:27.759691+0000","last_became_active":"2024-05-09T08:02:14.881437+0000","last_became_peered":"2024-05-09T08:02:14.881437+0000","last_unstale":"2024-05-09T08:04:27.759691+0000","last_undegraded":"2024-05-09T08:04:27.759691+0000","last_fullsized":"2024-05-09T08:04:27.759691+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:25:23.306827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"27'330","reported_seq":350,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:15.284795+0000","last_change":"2024-05-09T08:02:19.955928+0000","last_active":"2024-05-09T08:07:15.284795+0000","last_peered":"2024-05-09T08:07:15.284795+0000","last_clean":"2024-05-09T08:07:15.284795+0000","last_became_active":"2024-05-09T08:02:19.955432+0000","last_became_peered":"2024-05-09T08:02:19.955432+0000","last_unstale":"2024-05-09T08:07:15.284795+0000","last_undegraded":"2024-05-09T08:07:15.284795+0000","last_fullsized":"2024-05-09T08:07:15.284795+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":330,"log_dups_size":0,"ondisk_log_size":330,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:53:15.458120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":291,"num_write_kb":794624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:27.746932+0000","last_change":"2024-05-09T08:02:15.913311+0000","last_active":"2024-05-09T08:04:27.746932+0000","last_peered":"2024-05-09T08:04:27.746932+0000","last_clean":"2024-05-09T08:04:27.746932+0000","last_became_active":"2024-05-09T08:02:15.912484+0000","last_became_peered":"2024-05-09T08:02:15.912484+0000","last_unstale":"2024-05-09T08:04:27.746932+0000","last_undegraded":"2024-05-09T08:04:27.746932+0000","last_fullsized":"2024-05-09T08:04:27.746932+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:43:06.938904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:27.759654+0000","last_change":"2024-05-09T08:02:14.868884+0000","last_active":"2024-05-09T08:04:27.759654+0000","last_peered":"2024-05-09T08:04:27.759654+0000","last_clean":"2024-05-09T08:04:27.759654+0000","last_became_active":"2024-05-09T08:02:14.868553+0000","last_became_peered":"2024-05-09T08:02:14.868553+0000","last_unstale":"2024-05-09T08:04:27.759654+0000","last_undegraded":"2024-05-09T08:04:27.759654+0000","last_fullsized":"2024-05-09T08:04:27.759654+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:32:48.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"27'321","reported_seq":335,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:15.287758+0000","last_change":"2024-05-09T08:03:02.609342+0000","last_active":"2024-05-09T08:07:15.287758+0000","last_peered":"2024-05-09T08:07:15.287758+0000","last_clean":"2024-05-09T08:07:15.287758+0000","last_became_active":"2024-05-09T08:03:02.607564+0000","last_became_peered":"2024-05-09T08:03:02.607564+0000","last_unstale":"2024-05-09T08:07:15.287758+0000","last_undegraded":"2024-05-09T08:07:15.287758+0000","last_fullsized":"2024-05-09T08:07:15.287758+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":321,"log_dups_size":0,"ondisk_log_size":321,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:23:44.316545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":267,"num_write_kb":729088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4,7],"acting":[2,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:27.735197+0000","last_change":"2024-05-09T08:02:15.891115+0000","last_active":"2024-05-09T08:04:27.735197+0000","last_peered":"2024-05-09T08:04:27.735197+0000","last_clean":"2024-05-09T08:04:27.735197+0000","last_became_active":"2024-05-09T08:02:15.890806+0000","last_became_peered":"2024-05-09T08:02:15.890806+0000","last_unstale":"2024-05-09T08:04:27.735197+0000","last_undegraded":"2024-05-09T08:04:27.735197+0000","last_fullsized":"2024-05-09T08:04:27.735197+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:57:07.107359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:27.735251+0000","last_change":"2024-05-09T08:02:14.874987+0000","last_active":"2024-05-09T08:04:27.735251+0000","last_peered":"2024-05-09T08:04:27.735251+0000","last_clean":"2024-05-09T08:04:27.735251+0000","last_became_active":"2024-05-09T08:02:14.874549+0000","last_became_peered":"2024-05-09T08:02:14.874549+0000","last_unstale":"2024-05-09T08:04:27.735251+0000","last_undegraded":"2024-05-09T08:04:27.735251+0000","last_fullsized":"2024-05-09T08:04:27.735251+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:57:10.338510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"27'339","reported_seq":366,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:15.296068+0000","last_change":"2024-05-09T08:03:02.516739+0000","last_active":"2024-05-09T08:07:15.296068+0000","last_peered":"2024-05-09T08:07:15.296068+0000","last_clean":"2024-05-09T08:07:15.296068+0000","last_became_active":"2024-05-09T08:03:02.516161+0000","last_became_peered":"2024-05-09T08:03:02.516161+0000","last_unstale":"2024-05-09T08:07:15.296068+0000","last_undegraded":"2024-05-09T08:07:15.296068+0000","last_fullsized":"2024-05-09T08:07:15.296068+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":339,"log_dups_size":0,"ondisk_log_size":339,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:10:24.797526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":288,"num_write_kb":786432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,2],"acting":[4,3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:27.753541+0000","last_change":"2024-05-09T08:02:15.913011+0000","last_active":"2024-05-09T08:04:27.753541+0000","last_peered":"2024-05-09T08:04:27.753541+0000","last_clean":"2024-05-09T08:04:27.753541+0000","last_became_active":"2024-05-09T08:02:15.912590+0000","last_became_peered":"2024-05-09T08:02:15.912590+0000","last_unstale":"2024-05-09T08:04:27.753541+0000","last_undegraded":"2024-05-09T08:04:27.753541+0000","last_fullsized":"2024-05-09T08:04:27.753541+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:43:06.938904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"27'3","reported_seq":31,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:06:37.749118+0000","last_change":"2024-05-09T08:02:14.864763+0000","last_active":"2024-05-09T08:06:37.749118+0000","last_peered":"2024-05-09T08:06:37.749118+0000","last_clean":"2024-05-09T08:06:37.749118+0000","last_became_active":"2024-05-09T08:02:14.864470+0000","last_became_peered":"2024-05-09T08:02:14.864470+0000","last_unstale":"2024-05-09T08:06:37.749118+0000","last_undegraded":"2024-05-09T08:06:37.749118+0000","last_fullsized":"2024-05-09T08:06:37.749118+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:00:40.599475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"27'306","reported_seq":326,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:15.290595+0000","last_change":"2024-05-09T08:02:20.054896+0000","last_active":"2024-05-09T08:07:15.290595+0000","last_peered":"2024-05-09T08:07:15.290595+0000","last_clean":"2024-05-09T08:07:15.290595+0000","last_became_active":"2024-05-09T08:02:20.054424+0000","last_became_peered":"2024-05-09T08:02:20.054424+0000","last_unstale":"2024-05-09T08:07:15.290595+0000","last_undegraded":"2024-05-09T08:07:15.290595+0000","last_fullsized":"2024-05-09T08:07:15.290595+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":306,"log_dups_size":0,"ondisk_log_size":306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:23:44.316545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":249,"num_write_kb":679936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:27.747018+0000","last_change":"2024-05-09T08:02:15.905448+0000","last_active":"2024-05-09T08:04:27.747018+0000","last_peered":"2024-05-09T08:04:27.747018+0000","last_clean":"2024-05-09T08:04:27.747018+0000","last_became_active":"2024-05-09T08:02:15.905131+0000","last_became_peered":"2024-05-09T08:02:15.905131+0000","last_unstale":"2024-05-09T08:04:27.747018+0000","last_undegraded":"2024-05-09T08:04:27.747018+0000","last_fullsized":"2024-05-09T08:04:27.747018+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:25:01.238816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"27'21","reported_seq":48,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:15.020483+0000","last_change":"2024-05-09T08:02:14.868410+0000","last_active":"2024-05-09T08:07:15.020483+0000","last_peered":"2024-05-09T08:07:15.020483+0000","last_clean":"2024-05-09T08:07:15.020483+0000","last_became_active":"2024-05-09T08:02:14.865621+0000","last_became_peered":"2024-05-09T08:02:14.865621+0000","last_unstale":"2024-05-09T08:07:15.020483+0000","last_undegraded":"2024-05-09T08:07:15.020483+0000","last_fullsized":"2024-05-09T08:07:15.020483+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:43:01.061801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":41,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"27'328","reported_seq":348,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:15.272939+0000","last_change":"2024-05-09T08:02:20.057603+0000","last_active":"2024-05-09T08:07:15.272939+0000","last_peered":"2024-05-09T08:07:15.272939+0000","last_clean":"2024-05-09T08:07:15.272939+0000","last_became_active":"2024-05-09T08:02:20.055365+0000","last_became_peered":"2024-05-09T08:02:20.055365+0000","last_unstale":"2024-05-09T08:07:15.272939+0000","last_undegraded":"2024-05-09T08:07:15.272939+0000","last_fullsized":"2024-05-09T08:07:15.272939+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":328,"log_dups_size":0,"ondisk_log_size":328,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:48:38.283562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":264,"num_write_kb":720896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:22.759212+0000","last_change":"2024-05-09T08:02:15.890980+0000","last_active":"2024-05-09T08:04:22.759212+0000","last_peered":"2024-05-09T08:04:22.759212+0000","last_clean":"2024-05-09T08:04:22.759212+0000","last_became_active":"2024-05-09T08:02:15.890346+0000","last_became_peered":"2024-05-09T08:02:15.890346+0000","last_unstale":"2024-05-09T08:04:22.759212+0000","last_undegraded":"2024-05-09T08:04:22.759212+0000","last_fullsized":"2024-05-09T08:04:22.759212+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:29:59.441314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:22.734773+0000","last_change":"2024-05-09T08:02:14.876206+0000","last_active":"2024-05-09T08:04:22.734773+0000","last_peered":"2024-05-09T08:04:22.734773+0000","last_clean":"2024-05-09T08:04:22.734773+0000","last_became_active":"2024-05-09T08:02:14.875894+0000","last_became_peered":"2024-05-09T08:02:14.875894+0000","last_unstale":"2024-05-09T08:04:22.734773+0000","last_undegraded":"2024-05-09T08:04:22.734773+0000","last_fullsized":"2024-05-09T08:04:22.734773+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:49:53.882513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"27'297","reported_seq":317,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:15.288524+0000","last_change":"2024-05-09T08:02:20.055222+0000","last_active":"2024-05-09T08:07:15.288524+0000","last_peered":"2024-05-09T08:07:15.288524+0000","last_clean":"2024-05-09T08:07:15.288524+0000","last_became_active":"2024-05-09T08:02:20.053802+0000","last_became_peered":"2024-05-09T08:02:20.053802+0000","last_unstale":"2024-05-09T08:07:15.288524+0000","last_undegraded":"2024-05-09T08:07:15.288524+0000","last_fullsized":"2024-05-09T08:07:15.288524+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":297,"log_dups_size":0,"ondisk_log_size":297,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:42:00.939587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":249,"num_write_kb":679936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:22.735039+0000","last_change":"2024-05-09T08:02:15.904528+0000","last_active":"2024-05-09T08:04:22.735039+0000","last_peered":"2024-05-09T08:04:22.735039+0000","last_clean":"2024-05-09T08:04:22.735039+0000","last_became_active":"2024-05-09T08:02:15.904049+0000","last_became_peered":"2024-05-09T08:02:15.904049+0000","last_unstale":"2024-05-09T08:04:22.735039+0000","last_undegraded":"2024-05-09T08:04:22.735039+0000","last_fullsized":"2024-05-09T08:04:22.735039+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:53:11.390395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:22.759235+0000","last_change":"2024-05-09T08:02:14.872897+0000","last_active":"2024-05-09T08:04:22.759235+0000","last_peered":"2024-05-09T08:04:22.759235+0000","last_clean":"2024-05-09T08:04:22.759235+0000","last_became_active":"2024-05-09T08:02:14.872096+0000","last_became_peered":"2024-05-09T08:02:14.872096+0000","last_unstale":"2024-05-09T08:04:22.759235+0000","last_undegraded":"2024-05-09T08:04:22.759235+0000","last_fullsized":"2024-05-09T08:04:22.759235+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:00:40.599475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"27'320","reported_seq":340,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:15.220609+0000","last_change":"2024-05-09T08:02:19.963716+0000","last_active":"2024-05-09T08:07:15.220609+0000","last_peered":"2024-05-09T08:07:15.220609+0000","last_clean":"2024-05-09T08:07:15.220609+0000","last_became_active":"2024-05-09T08:02:19.962922+0000","last_became_peered":"2024-05-09T08:02:19.962922+0000","last_unstale":"2024-05-09T08:07:15.220609+0000","last_undegraded":"2024-05-09T08:07:15.220609+0000","last_fullsized":"2024-05-09T08:07:15.220609+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":320,"log_dups_size":0,"ondisk_log_size":320,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:42:00.939587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":273,"num_write_kb":745472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:22.734664+0000","last_change":"2024-05-09T08:02:15.901450+0000","last_active":"2024-05-09T08:04:22.734664+0000","last_peered":"2024-05-09T08:04:22.734664+0000","last_clean":"2024-05-09T08:04:22.734664+0000","last_became_active":"2024-05-09T08:02:15.901180+0000","last_became_peered":"2024-05-09T08:02:15.901180+0000","last_unstale":"2024-05-09T08:04:22.734664+0000","last_undegraded":"2024-05-09T08:04:22.734664+0000","last_fullsized":"2024-05-09T08:04:22.734664+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:43:02.075442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:22.734997+0000","last_change":"2024-05-09T08:02:14.876518+0000","last_active":"2024-05-09T08:04:22.734997+0000","last_peered":"2024-05-09T08:04:22.734997+0000","last_clean":"2024-05-09T08:04:22.734997+0000","last_became_active":"2024-05-09T08:02:14.876175+0000","last_became_peered":"2024-05-09T08:02:14.876175+0000","last_unstale":"2024-05-09T08:04:22.734997+0000","last_undegraded":"2024-05-09T08:04:22.734997+0000","last_fullsized":"2024-05-09T08:04:22.734997+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:00:40.599475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"27'301","reported_seq":315,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:15.286265+0000","last_change":"2024-05-09T08:03:02.516595+0000","last_active":"2024-05-09T08:07:15.286265+0000","last_peered":"2024-05-09T08:07:15.286265+0000","last_clean":"2024-05-09T08:07:15.286265+0000","last_became_active":"2024-05-09T08:03:02.516104+0000","last_became_peered":"2024-05-09T08:03:02.516104+0000","last_unstale":"2024-05-09T08:07:15.286265+0000","last_undegraded":"2024-05-09T08:07:15.286265+0000","last_fullsized":"2024-05-09T08:07:15.286265+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":301,"log_dups_size":0,"ondisk_log_size":301,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:47:30.081323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":264,"num_write_kb":720896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3,6],"acting":[5,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:17.758674+0000","last_change":"2024-05-09T08:02:15.891774+0000","last_active":"2024-05-09T08:04:17.758674+0000","last_peered":"2024-05-09T08:04:17.758674+0000","last_clean":"2024-05-09T08:04:17.758674+0000","last_became_active":"2024-05-09T08:02:15.891407+0000","last_became_peered":"2024-05-09T08:02:15.891407+0000","last_unstale":"2024-05-09T08:04:17.758674+0000","last_undegraded":"2024-05-09T08:04:17.758674+0000","last_fullsized":"2024-05-09T08:04:17.758674+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:05:02.181367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"27'6","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:17.381608+0000","last_change":"2024-05-09T08:02:14.883655+0000","last_active":"2024-05-09T08:07:17.381608+0000","last_peered":"2024-05-09T08:07:17.381608+0000","last_clean":"2024-05-09T08:07:17.381608+0000","last_became_active":"2024-05-09T08:02:14.882359+0000","last_became_peered":"2024-05-09T08:02:14.882359+0000","last_unstale":"2024-05-09T08:07:17.381608+0000","last_undegraded":"2024-05-09T08:07:17.381608+0000","last_fullsized":"2024-05-09T08:07:17.381608+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:46:11.160815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":529,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"27'319","reported_seq":339,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:15.257021+0000","last_change":"2024-05-09T08:02:19.953319+0000","last_active":"2024-05-09T08:07:15.257021+0000","last_peered":"2024-05-09T08:07:15.257021+0000","last_clean":"2024-05-09T08:07:15.257021+0000","last_became_active":"2024-05-09T08:02:19.952740+0000","last_became_peered":"2024-05-09T08:02:19.952740+0000","last_unstale":"2024-05-09T08:07:15.257021+0000","last_undegraded":"2024-05-09T08:07:15.257021+0000","last_fullsized":"2024-05-09T08:07:15.257021+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":319,"log_dups_size":0,"ondisk_log_size":319,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:25:28.804048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":270,"num_write_kb":737280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:22.752933+0000","last_change":"2024-05-09T08:02:15.913828+0000","last_active":"2024-05-09T08:04:22.752933+0000","last_peered":"2024-05-09T08:04:22.752933+0000","last_clean":"2024-05-09T08:04:22.752933+0000","last_became_active":"2024-05-09T08:02:15.913313+0000","last_became_peered":"2024-05-09T08:02:15.913313+0000","last_unstale":"2024-05-09T08:04:22.752933+0000","last_undegraded":"2024-05-09T08:04:22.752933+0000","last_fullsized":"2024-05-09T08:04:22.752933+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:20:09.935394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:17.734196+0000","last_change":"2024-05-09T08:02:14.876438+0000","last_active":"2024-05-09T08:04:17.734196+0000","last_peered":"2024-05-09T08:04:17.734196+0000","last_clean":"2024-05-09T08:04:17.734196+0000","last_became_active":"2024-05-09T08:02:14.876146+0000","last_became_peered":"2024-05-09T08:02:14.876146+0000","last_unstale":"2024-05-09T08:04:17.734196+0000","last_undegraded":"2024-05-09T08:04:17.734196+0000","last_fullsized":"2024-05-09T08:04:17.734196+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:36:49.629991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"27'378","reported_seq":398,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:15.271544+0000","last_change":"2024-05-09T08:02:20.051067+0000","last_active":"2024-05-09T08:07:15.271544+0000","last_peered":"2024-05-09T08:07:15.271544+0000","last_clean":"2024-05-09T08:07:15.271544+0000","last_became_active":"2024-05-09T08:02:20.050593+0000","last_became_peered":"2024-05-09T08:02:20.050593+0000","last_unstale":"2024-05-09T08:07:15.271544+0000","last_undegraded":"2024-05-09T08:07:15.271544+0000","last_fullsized":"2024-05-09T08:07:15.271544+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":378,"log_dups_size":0,"ondisk_log_size":378,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:25:28.804048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":339,"num_write_kb":925696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:22.746418+0000","last_change":"2024-05-09T08:02:15.899137+0000","last_active":"2024-05-09T08:04:22.746418+0000","last_peered":"2024-05-09T08:04:22.746418+0000","last_clean":"2024-05-09T08:04:22.746418+0000","last_became_active":"2024-05-09T08:02:15.898766+0000","last_became_peered":"2024-05-09T08:02:15.898766+0000","last_unstale":"2024-05-09T08:04:22.746418+0000","last_undegraded":"2024-05-09T08:04:22.746418+0000","last_fullsized":"2024-05-09T08:04:22.746418+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:47:32.605288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:22.735011+0000","last_change":"2024-05-09T08:02:14.868850+0000","last_active":"2024-05-09T08:04:22.735011+0000","last_peered":"2024-05-09T08:04:22.735011+0000","last_clean":"2024-05-09T08:04:22.735011+0000","last_became_active":"2024-05-09T08:02:14.868384+0000","last_became_peered":"2024-05-09T08:02:14.868384+0000","last_unstale":"2024-05-09T08:04:22.735011+0000","last_undegraded":"2024-05-09T08:04:22.735011+0000","last_fullsized":"2024-05-09T08:04:22.735011+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:14:52.920972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":183,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:12.733624+0000","last_change":"2024-05-09T08:02:11.559804+0000","last_active":"2024-05-09T08:04:12.733624+0000","last_peered":"2024-05-09T08:04:12.733624+0000","last_clean":"2024-05-09T08:04:12.733624+0000","last_became_active":"2024-05-09T08:02:11.558761+0000","last_became_peered":"2024-05-09T08:02:11.558761+0000","last_unstale":"2024-05-09T08:04:12.733624+0000","last_undegraded":"2024-05-09T08:04:12.733624+0000","last_fullsized":"2024-05-09T08:04:12.733624+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:09.800143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:09.800143+0000","last_clean_scrub_stamp":"2024-05-09T08:02:09.800143+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:18:35.964149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"27'315","reported_seq":335,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:15.264268+0000","last_change":"2024-05-09T08:02:20.054746+0000","last_active":"2024-05-09T08:07:15.264268+0000","last_peered":"2024-05-09T08:07:15.264268+0000","last_clean":"2024-05-09T08:07:15.264268+0000","last_became_active":"2024-05-09T08:02:20.054186+0000","last_became_peered":"2024-05-09T08:02:20.054186+0000","last_unstale":"2024-05-09T08:07:15.264268+0000","last_undegraded":"2024-05-09T08:07:15.264268+0000","last_fullsized":"2024-05-09T08:07:15.264268+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":315,"log_dups_size":0,"ondisk_log_size":315,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:53:15.458120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":276,"num_write_kb":753664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:32.735875+0000","last_change":"2024-05-09T08:02:15.889657+0000","last_active":"2024-05-09T08:04:32.735875+0000","last_peered":"2024-05-09T08:04:32.735875+0000","last_clean":"2024-05-09T08:04:32.735875+0000","last_became_active":"2024-05-09T08:02:15.889381+0000","last_became_peered":"2024-05-09T08:02:15.889381+0000","last_unstale":"2024-05-09T08:04:32.735875+0000","last_undegraded":"2024-05-09T08:04:32.735875+0000","last_fullsized":"2024-05-09T08:04:32.735875+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:59:14.957806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:32.747257+0000","last_change":"2024-05-09T08:02:14.868892+0000","last_active":"2024-05-09T08:04:32.747257+0000","last_peered":"2024-05-09T08:04:32.747257+0000","last_clean":"2024-05-09T08:04:32.747257+0000","last_became_active":"2024-05-09T08:02:14.868513+0000","last_became_peered":"2024-05-09T08:02:14.868513+0000","last_unstale":"2024-05-09T08:04:32.747257+0000","last_undegraded":"2024-05-09T08:04:32.747257+0000","last_fullsized":"2024-05-09T08:04:32.747257+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:19:31.913236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"27'338","reported_seq":358,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:15.275986+0000","last_change":"2024-05-09T08:02:20.053097+0000","last_active":"2024-05-09T08:07:15.275986+0000","last_peered":"2024-05-09T08:07:15.275986+0000","last_clean":"2024-05-09T08:07:15.275986+0000","last_became_active":"2024-05-09T08:02:20.052653+0000","last_became_peered":"2024-05-09T08:02:20.052653+0000","last_unstale":"2024-05-09T08:07:15.275986+0000","last_undegraded":"2024-05-09T08:07:15.275986+0000","last_fullsized":"2024-05-09T08:07:15.275986+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":338,"log_dups_size":0,"ondisk_log_size":338,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:25:05.306541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":279,"num_write_kb":761856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:32.736237+0000","last_change":"2024-05-09T08:02:15.899059+0000","last_active":"2024-05-09T08:04:32.736237+0000","last_peered":"2024-05-09T08:04:32.736237+0000","last_clean":"2024-05-09T08:04:32.736237+0000","last_became_active":"2024-05-09T08:02:15.898692+0000","last_became_peered":"2024-05-09T08:02:15.898692+0000","last_unstale":"2024-05-09T08:04:32.736237+0000","last_undegraded":"2024-05-09T08:04:32.736237+0000","last_fullsized":"2024-05-09T08:04:32.736237+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:52:36.461913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"24'1","reported_seq":28,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:32.760505+0000","last_change":"2024-05-09T08:02:14.873071+0000","last_active":"2024-05-09T08:04:32.760505+0000","last_peered":"2024-05-09T08:04:32.760505+0000","last_clean":"2024-05-09T08:04:32.760505+0000","last_became_active":"2024-05-09T08:02:14.872556+0000","last_became_peered":"2024-05-09T08:02:14.872556+0000","last_unstale":"2024-05-09T08:04:32.760505+0000","last_undegraded":"2024-05-09T08:04:32.760505+0000","last_fullsized":"2024-05-09T08:04:32.760505+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:57:06.093718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"27'299","reported_seq":319,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:15.257565+0000","last_change":"2024-05-09T08:02:19.960414+0000","last_active":"2024-05-09T08:07:15.257565+0000","last_peered":"2024-05-09T08:07:15.257565+0000","last_clean":"2024-05-09T08:07:15.257565+0000","last_became_active":"2024-05-09T08:02:19.957941+0000","last_became_peered":"2024-05-09T08:02:19.957941+0000","last_unstale":"2024-05-09T08:07:15.257565+0000","last_undegraded":"2024-05-09T08:07:15.257565+0000","last_fullsized":"2024-05-09T08:07:15.257565+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":299,"log_dups_size":0,"ondisk_log_size":299,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:41:31.470546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":261,"num_write_kb":712704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:32.735920+0000","last_change":"2024-05-09T08:02:15.884395+0000","last_active":"2024-05-09T08:04:32.735920+0000","last_peered":"2024-05-09T08:04:32.735920+0000","last_clean":"2024-05-09T08:04:32.735920+0000","last_became_active":"2024-05-09T08:02:15.883788+0000","last_became_peered":"2024-05-09T08:02:15.883788+0000","last_unstale":"2024-05-09T08:04:32.735920+0000","last_undegraded":"2024-05-09T08:04:32.735920+0000","last_fullsized":"2024-05-09T08:04:32.735920+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:48:34.215837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"24'2","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:32.747303+0000","last_change":"2024-05-09T08:02:14.885044+0000","last_active":"2024-05-09T08:04:32.747303+0000","last_peered":"2024-05-09T08:04:32.747303+0000","last_clean":"2024-05-09T08:04:32.747303+0000","last_became_active":"2024-05-09T08:02:14.884706+0000","last_became_peered":"2024-05-09T08:02:14.884706+0000","last_unstale":"2024-05-09T08:04:32.747303+0000","last_undegraded":"2024-05-09T08:04:32.747303+0000","last_fullsized":"2024-05-09T08:04:32.747303+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:14:52.920972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"27'365","reported_seq":385,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:15.276867+0000","last_change":"2024-05-09T08:02:20.050865+0000","last_active":"2024-05-09T08:07:15.276867+0000","last_peered":"2024-05-09T08:07:15.276867+0000","last_clean":"2024-05-09T08:07:15.276867+0000","last_became_active":"2024-05-09T08:02:20.050270+0000","last_became_peered":"2024-05-09T08:02:20.050270+0000","last_unstale":"2024-05-09T08:07:15.276867+0000","last_undegraded":"2024-05-09T08:07:15.276867+0000","last_fullsized":"2024-05-09T08:07:15.276867+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":365,"log_dups_size":0,"ondisk_log_size":365,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:48:38.283562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":318,"num_write_kb":868352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:32.735905+0000","last_change":"2024-05-09T08:02:15.884737+0000","last_active":"2024-05-09T08:04:32.735905+0000","last_peered":"2024-05-09T08:04:32.735905+0000","last_clean":"2024-05-09T08:04:32.735905+0000","last_became_active":"2024-05-09T08:02:15.884465+0000","last_became_peered":"2024-05-09T08:02:15.884465+0000","last_unstale":"2024-05-09T08:04:32.735905+0000","last_undegraded":"2024-05-09T08:04:32.735905+0000","last_fullsized":"2024-05-09T08:04:32.735905+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:52:36.461913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"24'1","reported_seq":28,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:32.735864+0000","last_change":"2024-05-09T08:02:14.875003+0000","last_active":"2024-05-09T08:04:32.735864+0000","last_peered":"2024-05-09T08:04:32.735864+0000","last_clean":"2024-05-09T08:04:32.735864+0000","last_became_active":"2024-05-09T08:02:14.874665+0000","last_became_peered":"2024-05-09T08:02:14.874665+0000","last_unstale":"2024-05-09T08:04:32.735864+0000","last_undegraded":"2024-05-09T08:04:32.735864+0000","last_fullsized":"2024-05-09T08:04:32.735864+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:18:39.991319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"27'302","reported_seq":322,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:15.294350+0000","last_change":"2024-05-09T08:02:19.952692+0000","last_active":"2024-05-09T08:07:15.294350+0000","last_peered":"2024-05-09T08:07:15.294350+0000","last_clean":"2024-05-09T08:07:15.294350+0000","last_became_active":"2024-05-09T08:02:19.952224+0000","last_became_peered":"2024-05-09T08:02:19.952224+0000","last_unstale":"2024-05-09T08:07:15.294350+0000","last_undegraded":"2024-05-09T08:07:15.294350+0000","last_fullsized":"2024-05-09T08:07:15.294350+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":302,"log_dups_size":0,"ondisk_log_size":302,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:47:36.673013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":258,"num_write_kb":704512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:37.736256+0000","last_change":"2024-05-09T08:02:15.883408+0000","last_active":"2024-05-09T08:04:37.736256+0000","last_peered":"2024-05-09T08:04:37.736256+0000","last_clean":"2024-05-09T08:04:37.736256+0000","last_became_active":"2024-05-09T08:02:15.883103+0000","last_became_peered":"2024-05-09T08:02:15.883103+0000","last_unstale":"2024-05-09T08:04:37.736256+0000","last_undegraded":"2024-05-09T08:04:37.736256+0000","last_fullsized":"2024-05-09T08:04:37.736256+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:53:11.390395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"27'9","reported_seq":36,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:06:57.786502+0000","last_change":"2024-05-09T08:02:14.873184+0000","last_active":"2024-05-09T08:06:57.786502+0000","last_peered":"2024-05-09T08:06:57.786502+0000","last_clean":"2024-05-09T08:06:57.786502+0000","last_became_active":"2024-05-09T08:02:14.871055+0000","last_became_peered":"2024-05-09T08:02:14.871055+0000","last_unstale":"2024-05-09T08:06:57.786502+0000","last_undegraded":"2024-05-09T08:06:57.786502+0000","last_fullsized":"2024-05-09T08:06:57.786502+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:19:56.315972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"27'416","reported_seq":436,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:15.283703+0000","last_change":"2024-05-09T08:02:19.950823+0000","last_active":"2024-05-09T08:07:15.283703+0000","last_peered":"2024-05-09T08:07:15.283703+0000","last_clean":"2024-05-09T08:07:15.283703+0000","last_became_active":"2024-05-09T08:02:19.950352+0000","last_became_peered":"2024-05-09T08:02:19.950352+0000","last_unstale":"2024-05-09T08:07:15.283703+0000","last_undegraded":"2024-05-09T08:07:15.283703+0000","last_fullsized":"2024-05-09T08:07:15.283703+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":416,"log_dups_size":0,"ondisk_log_size":416,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:59:19.025530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":360,"num_write_kb":983040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:37.736615+0000","last_change":"2024-05-09T08:02:15.899115+0000","last_active":"2024-05-09T08:04:37.736615+0000","last_peered":"2024-05-09T08:04:37.736615+0000","last_clean":"2024-05-09T08:04:37.736615+0000","last_became_active":"2024-05-09T08:02:15.898811+0000","last_became_peered":"2024-05-09T08:02:15.898811+0000","last_unstale":"2024-05-09T08:04:37.736615+0000","last_undegraded":"2024-05-09T08:04:37.736615+0000","last_fullsized":"2024-05-09T08:04:37.736615+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:43:06.938904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":28,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:37.736636+0000","last_change":"2024-05-09T08:02:14.860818+0000","last_active":"2024-05-09T08:04:37.736636+0000","last_peered":"2024-05-09T08:04:37.736636+0000","last_clean":"2024-05-09T08:04:37.736636+0000","last_became_active":"2024-05-09T08:02:14.860518+0000","last_became_peered":"2024-05-09T08:02:14.860518+0000","last_unstale":"2024-05-09T08:04:37.736636+0000","last_undegraded":"2024-05-09T08:04:37.736636+0000","last_fullsized":"2024-05-09T08:04:37.736636+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:25:23.306827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"27'318","reported_seq":338,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:15.275345+0000","last_change":"2024-05-09T08:02:20.053251+0000","last_active":"2024-05-09T08:07:15.275345+0000","last_peered":"2024-05-09T08:07:15.275345+0000","last_clean":"2024-05-09T08:07:15.275345+0000","last_became_active":"2024-05-09T08:02:20.052783+0000","last_became_peered":"2024-05-09T08:02:20.052783+0000","last_unstale":"2024-05-09T08:07:15.275345+0000","last_undegraded":"2024-05-09T08:07:15.275345+0000","last_fullsized":"2024-05-09T08:07:15.275345+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":318,"log_dups_size":0,"ondisk_log_size":318,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:16:41.863341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":273,"num_write_kb":745472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:37.736278+0000","last_change":"2024-05-09T08:02:15.883972+0000","last_active":"2024-05-09T08:04:37.736278+0000","last_peered":"2024-05-09T08:04:37.736278+0000","last_clean":"2024-05-09T08:04:37.736278+0000","last_became_active":"2024-05-09T08:02:15.883613+0000","last_became_peered":"2024-05-09T08:02:15.883613+0000","last_unstale":"2024-05-09T08:04:37.736278+0000","last_undegraded":"2024-05-09T08:04:37.736278+0000","last_fullsized":"2024-05-09T08:04:37.736278+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:41:27.402821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:37.736743+0000","last_change":"2024-05-09T08:02:14.883366+0000","last_active":"2024-05-09T08:04:37.736743+0000","last_peered":"2024-05-09T08:04:37.736743+0000","last_clean":"2024-05-09T08:04:37.736743+0000","last_became_active":"2024-05-09T08:02:14.883135+0000","last_became_peered":"2024-05-09T08:02:14.883135+0000","last_unstale":"2024-05-09T08:04:37.736743+0000","last_undegraded":"2024-05-09T08:04:37.736743+0000","last_fullsized":"2024-05-09T08:04:37.736743+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:32:48.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"27'316","reported_seq":336,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:15.284009+0000","last_change":"2024-05-09T08:02:20.050888+0000","last_active":"2024-05-09T08:07:15.284009+0000","last_peered":"2024-05-09T08:07:15.284009+0000","last_clean":"2024-05-09T08:07:15.284009+0000","last_became_active":"2024-05-09T08:02:20.050390+0000","last_became_peered":"2024-05-09T08:02:20.050390+0000","last_unstale":"2024-05-09T08:07:15.284009+0000","last_undegraded":"2024-05-09T08:07:15.284009+0000","last_fullsized":"2024-05-09T08:07:15.284009+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":316,"log_dups_size":0,"ondisk_log_size":316,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:10:24.797526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":273,"num_write_kb":745472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"27'1","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:05:05.201608+0000","last_change":"2024-05-09T08:02:15.899096+0000","last_active":"2024-05-09T08:05:05.201608+0000","last_peered":"2024-05-09T08:05:05.201608+0000","last_clean":"2024-05-09T08:05:05.201608+0000","last_became_active":"2024-05-09T08:02:15.898811+0000","last_became_peered":"2024-05-09T08:02:15.898811+0000","last_unstale":"2024-05-09T08:05:05.201608+0000","last_undegraded":"2024-05-09T08:05:05.201608+0000","last_fullsized":"2024-05-09T08:05:05.201608+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:28:51.229262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"24'2","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:37.755081+0000","last_change":"2024-05-09T08:02:14.868657+0000","last_active":"2024-05-09T08:04:37.755081+0000","last_peered":"2024-05-09T08:04:37.755081+0000","last_clean":"2024-05-09T08:04:37.755081+0000","last_became_active":"2024-05-09T08:02:14.868385+0000","last_became_peered":"2024-05-09T08:02:14.868385+0000","last_unstale":"2024-05-09T08:04:37.755081+0000","last_undegraded":"2024-05-09T08:04:37.755081+0000","last_fullsized":"2024-05-09T08:04:37.755081+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:46:11.160815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"27'378","reported_seq":398,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:15.279263+0000","last_change":"2024-05-09T08:02:20.053155+0000","last_active":"2024-05-09T08:07:15.279263+0000","last_peered":"2024-05-09T08:07:15.279263+0000","last_clean":"2024-05-09T08:07:15.279263+0000","last_became_active":"2024-05-09T08:02:20.052629+0000","last_became_peered":"2024-05-09T08:02:20.052629+0000","last_unstale":"2024-05-09T08:07:15.279263+0000","last_undegraded":"2024-05-09T08:07:15.279263+0000","last_fullsized":"2024-05-09T08:07:15.279263+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":378,"log_dups_size":0,"ondisk_log_size":378,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:35:54.628874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":327,"num_write_kb":892928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:42.736850+0000","last_change":"2024-05-09T08:02:15.905406+0000","last_active":"2024-05-09T08:04:42.736850+0000","last_peered":"2024-05-09T08:04:42.736850+0000","last_clean":"2024-05-09T08:04:42.736850+0000","last_became_active":"2024-05-09T08:02:15.905076+0000","last_became_peered":"2024-05-09T08:02:15.905076+0000","last_unstale":"2024-05-09T08:04:42.736850+0000","last_undegraded":"2024-05-09T08:04:42.736850+0000","last_fullsized":"2024-05-09T08:04:42.736850+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:25:01.238816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"27'189","reported_seq":217,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:14.967504+0000","last_change":"2024-05-09T08:02:14.880410+0000","last_active":"2024-05-09T08:07:14.967504+0000","last_peered":"2024-05-09T08:07:14.967504+0000","last_clean":"2024-05-09T08:07:14.967504+0000","last_became_active":"2024-05-09T08:02:14.880045+0000","last_became_peered":"2024-05-09T08:02:14.880045+0000","last_unstale":"2024-05-09T08:07:14.967504+0000","last_undegraded":"2024-05-09T08:07:14.967504+0000","last_fullsized":"2024-05-09T08:07:14.967504+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":189,"log_dups_size":0,"ondisk_log_size":189,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:19:56.315972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":447435,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":188,"num_write_kb":582,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"27'332","reported_seq":352,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:15.206123+0000","last_change":"2024-05-09T08:02:19.963378+0000","last_active":"2024-05-09T08:07:15.206123+0000","last_peered":"2024-05-09T08:07:15.206123+0000","last_clean":"2024-05-09T08:07:15.206123+0000","last_became_active":"2024-05-09T08:02:19.958964+0000","last_became_peered":"2024-05-09T08:02:19.958964+0000","last_unstale":"2024-05-09T08:07:15.206123+0000","last_undegraded":"2024-05-09T08:07:15.206123+0000","last_fullsized":"2024-05-09T08:07:15.206123+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":332,"log_dups_size":0,"ondisk_log_size":332,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:41:31.470546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":288,"num_write_kb":786432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:37.736256+0000","last_change":"2024-05-09T08:02:15.888428+0000","last_active":"2024-05-09T08:04:37.736256+0000","last_peered":"2024-05-09T08:04:37.736256+0000","last_clean":"2024-05-09T08:04:37.736256+0000","last_became_active":"2024-05-09T08:02:15.888071+0000","last_became_peered":"2024-05-09T08:02:15.888071+0000","last_unstale":"2024-05-09T08:04:37.736256+0000","last_undegraded":"2024-05-09T08:04:37.736256+0000","last_fullsized":"2024-05-09T08:04:37.736256+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:36:50.643632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"27'8","reported_seq":35,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:15.298845+0000","last_change":"2024-05-09T08:02:14.872697+0000","last_active":"2024-05-09T08:07:15.298845+0000","last_peered":"2024-05-09T08:07:15.298845+0000","last_clean":"2024-05-09T08:07:15.298845+0000","last_became_active":"2024-05-09T08:02:14.871945+0000","last_became_peered":"2024-05-09T08:02:14.871945+0000","last_unstale":"2024-05-09T08:07:15.298845+0000","last_undegraded":"2024-05-09T08:07:15.298845+0000","last_fullsized":"2024-05-09T08:07:15.298845+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:19:56.315972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"27'353","reported_seq":373,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:15.280987+0000","last_change":"2024-05-09T08:02:19.953635+0000","last_active":"2024-05-09T08:07:15.280987+0000","last_peered":"2024-05-09T08:07:15.280987+0000","last_clean":"2024-05-09T08:07:15.280987+0000","last_became_active":"2024-05-09T08:02:19.953166+0000","last_became_peered":"2024-05-09T08:02:19.953166+0000","last_unstale":"2024-05-09T08:07:15.280987+0000","last_undegraded":"2024-05-09T08:07:15.280987+0000","last_fullsized":"2024-05-09T08:07:15.280987+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":353,"log_dups_size":0,"ondisk_log_size":353,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:18:40.377352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":297,"num_write_kb":811008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:42.755912+0000","last_change":"2024-05-09T08:02:15.905285+0000","last_active":"2024-05-09T08:04:42.755912+0000","last_peered":"2024-05-09T08:04:42.755912+0000","last_clean":"2024-05-09T08:04:42.755912+0000","last_became_active":"2024-05-09T08:02:15.904898+0000","last_became_peered":"2024-05-09T08:02:15.904898+0000","last_unstale":"2024-05-09T08:04:42.755912+0000","last_undegraded":"2024-05-09T08:04:42.755912+0000","last_fullsized":"2024-05-09T08:04:42.755912+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:47:32.605288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:42.737369+0000","last_change":"2024-05-09T08:02:14.868682+0000","last_active":"2024-05-09T08:04:42.737369+0000","last_peered":"2024-05-09T08:04:42.737369+0000","last_clean":"2024-05-09T08:04:42.737369+0000","last_became_active":"2024-05-09T08:02:14.868294+0000","last_became_peered":"2024-05-09T08:02:14.868294+0000","last_unstale":"2024-05-09T08:04:42.737369+0000","last_undegraded":"2024-05-09T08:04:42.737369+0000","last_fullsized":"2024-05-09T08:04:42.737369+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:57:06.093718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"27'344","reported_seq":364,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:15.272036+0000","last_change":"2024-05-09T08:02:19.956115+0000","last_active":"2024-05-09T08:07:15.272036+0000","last_peered":"2024-05-09T08:07:15.272036+0000","last_clean":"2024-05-09T08:07:15.272036+0000","last_became_active":"2024-05-09T08:02:19.955662+0000","last_became_peered":"2024-05-09T08:02:19.955662+0000","last_unstale":"2024-05-09T08:07:15.272036+0000","last_undegraded":"2024-05-09T08:07:15.272036+0000","last_fullsized":"2024-05-09T08:07:15.272036+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":344,"log_dups_size":0,"ondisk_log_size":344,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:15:28.286330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":297,"num_write_kb":811008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:42.737356+0000","last_change":"2024-05-09T08:02:15.888406+0000","last_active":"2024-05-09T08:04:42.737356+0000","last_peered":"2024-05-09T08:04:42.737356+0000","last_clean":"2024-05-09T08:04:42.737356+0000","last_became_active":"2024-05-09T08:02:15.888090+0000","last_became_peered":"2024-05-09T08:02:15.888090+0000","last_unstale":"2024-05-09T08:04:42.737356+0000","last_undegraded":"2024-05-09T08:04:42.737356+0000","last_fullsized":"2024-05-09T08:04:42.737356+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:53:23.178610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"27'1","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:05:05.164756+0000","last_change":"2024-05-09T08:02:14.862463+0000","last_active":"2024-05-09T08:05:05.164756+0000","last_peered":"2024-05-09T08:05:05.164756+0000","last_clean":"2024-05-09T08:05:05.164756+0000","last_became_active":"2024-05-09T08:02:14.862104+0000","last_became_peered":"2024-05-09T08:02:14.862104+0000","last_unstale":"2024-05-09T08:05:05.164756+0000","last_undegraded":"2024-05-09T08:05:05.164756+0000","last_fullsized":"2024-05-09T08:05:05.164756+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:32:48.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"27'379","reported_seq":399,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:15.252469+0000","last_change":"2024-05-09T08:02:20.053600+0000","last_active":"2024-05-09T08:07:15.252469+0000","last_peered":"2024-05-09T08:07:15.252469+0000","last_clean":"2024-05-09T08:07:15.252469+0000","last_became_active":"2024-05-09T08:02:20.052485+0000","last_became_peered":"2024-05-09T08:02:20.052485+0000","last_unstale":"2024-05-09T08:07:15.252469+0000","last_undegraded":"2024-05-09T08:07:15.252469+0000","last_fullsized":"2024-05-09T08:07:15.252469+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":379,"log_dups_size":0,"ondisk_log_size":379,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:35:54.628874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":327,"num_write_kb":892928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"27'324","reported_seq":344,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:15.288169+0000","last_change":"2024-05-09T08:02:20.054016+0000","last_active":"2024-05-09T08:07:15.288169+0000","last_peered":"2024-05-09T08:07:15.288169+0000","last_clean":"2024-05-09T08:07:15.288169+0000","last_became_active":"2024-05-09T08:02:20.053250+0000","last_became_peered":"2024-05-09T08:02:20.053250+0000","last_unstale":"2024-05-09T08:07:15.288169+0000","last_undegraded":"2024-05-09T08:07:15.288169+0000","last_fullsized":"2024-05-09T08:07:15.288169+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":324,"log_dups_size":0,"ondisk_log_size":324,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:52:40.529638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":273,"num_write_kb":745472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"27'1","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:07.834698+0000","last_change":"2024-05-09T08:02:15.907080+0000","last_active":"2024-05-09T08:07:07.834698+0000","last_peered":"2024-05-09T08:07:07.834698+0000","last_clean":"2024-05-09T08:07:07.834698+0000","last_became_active":"2024-05-09T08:02:15.906600+0000","last_became_peered":"2024-05-09T08:02:15.906600+0000","last_unstale":"2024-05-09T08:07:07.834698+0000","last_undegraded":"2024-05-09T08:07:07.834698+0000","last_fullsized":"2024-05-09T08:07:07.834698+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:43:06.938904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:42.737302+0000","last_change":"2024-05-09T08:02:14.870990+0000","last_active":"2024-05-09T08:04:42.737302+0000","last_peered":"2024-05-09T08:04:42.737302+0000","last_clean":"2024-05-09T08:04:42.737302+0000","last_became_active":"2024-05-09T08:02:14.869818+0000","last_became_peered":"2024-05-09T08:02:14.869818+0000","last_unstale":"2024-05-09T08:04:42.737302+0000","last_undegraded":"2024-05-09T08:04:42.737302+0000","last_fullsized":"2024-05-09T08:04:42.737302+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:25:23.306827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"27'313","reported_seq":333,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:15.240636+0000","last_change":"2024-05-09T08:02:19.952582+0000","last_active":"2024-05-09T08:07:15.240636+0000","last_peered":"2024-05-09T08:07:15.240636+0000","last_clean":"2024-05-09T08:07:15.240636+0000","last_became_active":"2024-05-09T08:02:19.952117+0000","last_became_peered":"2024-05-09T08:02:19.952117+0000","last_unstale":"2024-05-09T08:07:15.240636+0000","last_undegraded":"2024-05-09T08:07:15.240636+0000","last_fullsized":"2024-05-09T08:07:15.240636+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":313,"log_dups_size":0,"ondisk_log_size":313,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:42:00.939587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":267,"num_write_kb":729088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:42.748943+0000","last_change":"2024-05-09T08:02:15.913109+0000","last_active":"2024-05-09T08:04:42.748943+0000","last_peered":"2024-05-09T08:04:42.748943+0000","last_clean":"2024-05-09T08:04:42.748943+0000","last_became_active":"2024-05-09T08:02:15.912883+0000","last_became_peered":"2024-05-09T08:02:15.912883+0000","last_unstale":"2024-05-09T08:04:42.748943+0000","last_undegraded":"2024-05-09T08:04:42.748943+0000","last_fullsized":"2024-05-09T08:04:42.748943+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:29:59.441314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:42.748992+0000","last_change":"2024-05-09T08:02:14.885256+0000","last_active":"2024-05-09T08:04:42.748992+0000","last_peered":"2024-05-09T08:04:42.748992+0000","last_clean":"2024-05-09T08:04:42.748992+0000","last_became_active":"2024-05-09T08:02:14.884965+0000","last_became_peered":"2024-05-09T08:02:14.884965+0000","last_unstale":"2024-05-09T08:04:42.748992+0000","last_undegraded":"2024-05-09T08:04:42.748992+0000","last_fullsized":"2024-05-09T08:04:42.748992+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:14:52.920972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"24'1","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:47.737722+0000","last_change":"2024-05-09T08:02:14.860339+0000","last_active":"2024-05-09T08:04:47.737722+0000","last_peered":"2024-05-09T08:04:47.737722+0000","last_clean":"2024-05-09T08:04:47.737722+0000","last_became_active":"2024-05-09T08:02:14.859917+0000","last_became_peered":"2024-05-09T08:02:14.859917+0000","last_unstale":"2024-05-09T08:04:47.737722+0000","last_undegraded":"2024-05-09T08:04:47.737722+0000","last_fullsized":"2024-05-09T08:04:47.737722+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:14:52.920972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:47.756573+0000","last_change":"2024-05-09T08:02:15.897001+0000","last_active":"2024-05-09T08:04:47.756573+0000","last_peered":"2024-05-09T08:04:47.756573+0000","last_clean":"2024-05-09T08:04:47.756573+0000","last_became_active":"2024-05-09T08:02:15.896708+0000","last_became_peered":"2024-05-09T08:02:15.896708+0000","last_unstale":"2024-05-09T08:04:47.756573+0000","last_undegraded":"2024-05-09T08:04:47.756573+0000","last_fullsized":"2024-05-09T08:04:47.756573+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:29:59.441314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"27'338","reported_seq":358,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:15.293090+0000","last_change":"2024-05-09T08:02:19.955369+0000","last_active":"2024-05-09T08:07:15.293090+0000","last_peered":"2024-05-09T08:07:15.293090+0000","last_clean":"2024-05-09T08:07:15.293090+0000","last_became_active":"2024-05-09T08:02:19.954829+0000","last_became_peered":"2024-05-09T08:02:19.954829+0000","last_unstale":"2024-05-09T08:07:15.293090+0000","last_undegraded":"2024-05-09T08:07:15.293090+0000","last_fullsized":"2024-05-09T08:07:15.293090+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":338,"log_dups_size":0,"ondisk_log_size":338,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:49:58.963879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":297,"num_write_kb":811008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"24'1","reported_seq":28,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:47.753171+0000","last_change":"2024-05-09T08:02:14.883856+0000","last_active":"2024-05-09T08:04:47.753171+0000","last_peered":"2024-05-09T08:04:47.753171+0000","last_clean":"2024-05-09T08:04:47.753171+0000","last_became_active":"2024-05-09T08:02:14.882610+0000","last_became_peered":"2024-05-09T08:02:14.882610+0000","last_unstale":"2024-05-09T08:04:47.753171+0000","last_undegraded":"2024-05-09T08:04:47.753171+0000","last_fullsized":"2024-05-09T08:04:47.753171+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:19:31.913236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:47.737288+0000","last_change":"2024-05-09T08:02:15.889974+0000","last_active":"2024-05-09T08:04:47.737288+0000","last_peered":"2024-05-09T08:04:47.737288+0000","last_clean":"2024-05-09T08:04:47.737288+0000","last_became_active":"2024-05-09T08:02:15.889650+0000","last_became_peered":"2024-05-09T08:02:15.889650+0000","last_unstale":"2024-05-09T08:04:47.737288+0000","last_undegraded":"2024-05-09T08:04:47.737288+0000","last_fullsized":"2024-05-09T08:04:47.737288+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:00:41.613116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"27'361","reported_seq":381,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:15.284334+0000","last_change":"2024-05-09T08:02:19.955362+0000","last_active":"2024-05-09T08:07:15.284334+0000","last_peered":"2024-05-09T08:07:15.284334+0000","last_clean":"2024-05-09T08:07:15.284334+0000","last_became_active":"2024-05-09T08:02:19.954947+0000","last_became_peered":"2024-05-09T08:02:19.954947+0000","last_unstale":"2024-05-09T08:07:15.284334+0000","last_undegraded":"2024-05-09T08:07:15.284334+0000","last_fullsized":"2024-05-09T08:07:15.284334+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":361,"log_dups_size":0,"ondisk_log_size":361,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:43:30.549966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":312,"num_write_kb":851968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:22.734849+0000","last_change":"2024-05-09T08:02:15.909379+0000","last_active":"2024-05-09T08:04:22.734849+0000","last_peered":"2024-05-09T08:04:22.734849+0000","last_clean":"2024-05-09T08:04:22.734849+0000","last_became_active":"2024-05-09T08:02:15.909086+0000","last_became_peered":"2024-05-09T08:02:15.909086+0000","last_unstale":"2024-05-09T08:04:22.734849+0000","last_undegraded":"2024-05-09T08:04:22.734849+0000","last_fullsized":"2024-05-09T08:04:22.734849+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:05:02.181367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:22.745878+0000","last_change":"2024-05-09T08:02:15.891242+0000","last_active":"2024-05-09T08:04:22.745878+0000","last_peered":"2024-05-09T08:04:22.745878+0000","last_clean":"2024-05-09T08:04:22.745878+0000","last_became_active":"2024-05-09T08:02:15.890640+0000","last_became_peered":"2024-05-09T08:02:15.890640+0000","last_unstale":"2024-05-09T08:04:22.745878+0000","last_undegraded":"2024-05-09T08:04:22.745878+0000","last_fullsized":"2024-05-09T08:04:22.745878+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:57:07.107359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:22.734971+0000","last_change":"2024-05-09T08:02:15.899001+0000","last_active":"2024-05-09T08:04:22.734971+0000","last_peered":"2024-05-09T08:04:22.734971+0000","last_clean":"2024-05-09T08:04:22.734971+0000","last_became_active":"2024-05-09T08:02:15.898695+0000","last_became_peered":"2024-05-09T08:02:15.898695+0000","last_unstale":"2024-05-09T08:04:22.734971+0000","last_undegraded":"2024-05-09T08:04:22.734971+0000","last_fullsized":"2024-05-09T08:04:22.734971+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:41:27.402821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:22.752972+0000","last_change":"2024-05-09T08:02:15.907374+0000","last_active":"2024-05-09T08:04:22.752972+0000","last_peered":"2024-05-09T08:04:22.752972+0000","last_clean":"2024-05-09T08:04:22.752972+0000","last_became_active":"2024-05-09T08:02:15.906767+0000","last_became_peered":"2024-05-09T08:02:15.906767+0000","last_unstale":"2024-05-09T08:04:22.752972+0000","last_undegraded":"2024-05-09T08:04:22.752972+0000","last_fullsized":"2024-05-09T08:04:22.752972+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:05:02.181367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:27.746979+0000","last_change":"2024-05-09T08:02:15.905326+0000","last_active":"2024-05-09T08:04:27.746979+0000","last_peered":"2024-05-09T08:04:27.746979+0000","last_clean":"2024-05-09T08:04:27.746979+0000","last_became_active":"2024-05-09T08:02:15.905083+0000","last_became_peered":"2024-05-09T08:02:15.905083+0000","last_unstale":"2024-05-09T08:04:27.746979+0000","last_undegraded":"2024-05-09T08:04:27.746979+0000","last_fullsized":"2024-05-09T08:04:27.746979+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:23:40.248821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:27.759721+0000","last_change":"2024-05-09T08:02:15.907753+0000","last_active":"2024-05-09T08:04:27.759721+0000","last_peered":"2024-05-09T08:04:27.759721+0000","last_clean":"2024-05-09T08:04:27.759721+0000","last_became_active":"2024-05-09T08:02:15.907480+0000","last_became_peered":"2024-05-09T08:02:15.907480+0000","last_unstale":"2024-05-09T08:04:27.759721+0000","last_undegraded":"2024-05-09T08:04:27.759721+0000","last_fullsized":"2024-05-09T08:04:27.759721+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:20:07.368327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:27.735486+0000","last_change":"2024-05-09T08:02:15.888605+0000","last_active":"2024-05-09T08:04:27.735486+0000","last_peered":"2024-05-09T08:04:27.735486+0000","last_clean":"2024-05-09T08:04:27.735486+0000","last_became_active":"2024-05-09T08:02:15.887981+0000","last_became_peered":"2024-05-09T08:02:15.887981+0000","last_unstale":"2024-05-09T08:04:27.735486+0000","last_undegraded":"2024-05-09T08:04:27.735486+0000","last_fullsized":"2024-05-09T08:04:27.735486+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:47:32.605288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:27.747004+0000","last_change":"2024-05-09T08:02:15.904744+0000","last_active":"2024-05-09T08:04:27.747004+0000","last_peered":"2024-05-09T08:04:27.747004+0000","last_clean":"2024-05-09T08:04:27.747004+0000","last_became_active":"2024-05-09T08:02:15.904334+0000","last_became_peered":"2024-05-09T08:02:15.904334+0000","last_unstale":"2024-05-09T08:04:27.747004+0000","last_undegraded":"2024-05-09T08:04:27.747004+0000","last_fullsized":"2024-05-09T08:04:27.747004+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:30:33.351158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:27.735580+0000","last_change":"2024-05-09T08:02:15.904835+0000","last_active":"2024-05-09T08:04:27.735580+0000","last_peered":"2024-05-09T08:04:27.735580+0000","last_clean":"2024-05-09T08:04:27.735580+0000","last_became_active":"2024-05-09T08:02:15.904416+0000","last_became_peered":"2024-05-09T08:02:15.904416+0000","last_unstale":"2024-05-09T08:04:27.735580+0000","last_undegraded":"2024-05-09T08:04:27.735580+0000","last_fullsized":"2024-05-09T08:04:27.735580+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:47:32.605288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:32.747675+0000","last_change":"2024-05-09T08:02:15.898324+0000","last_active":"2024-05-09T08:04:32.747675+0000","last_peered":"2024-05-09T08:04:32.747675+0000","last_clean":"2024-05-09T08:04:32.747675+0000","last_became_active":"2024-05-09T08:02:15.897981+0000","last_became_peered":"2024-05-09T08:02:15.897981+0000","last_unstale":"2024-05-09T08:04:32.747675+0000","last_undegraded":"2024-05-09T08:04:32.747675+0000","last_fullsized":"2024-05-09T08:04:32.747675+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:01:50.341133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:32.736201+0000","last_change":"2024-05-09T08:02:15.890945+0000","last_active":"2024-05-09T08:04:32.736201+0000","last_peered":"2024-05-09T08:04:32.736201+0000","last_clean":"2024-05-09T08:04:32.736201+0000","last_became_active":"2024-05-09T08:02:15.890647+0000","last_became_peered":"2024-05-09T08:02:15.890647+0000","last_unstale":"2024-05-09T08:04:32.736201+0000","last_undegraded":"2024-05-09T08:04:32.736201+0000","last_fullsized":"2024-05-09T08:04:32.736201+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:41:56.871863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:32.754272+0000","last_change":"2024-05-09T08:02:15.913728+0000","last_active":"2024-05-09T08:04:32.754272+0000","last_peered":"2024-05-09T08:04:32.754272+0000","last_clean":"2024-05-09T08:04:32.754272+0000","last_became_active":"2024-05-09T08:02:15.909818+0000","last_became_peered":"2024-05-09T08:02:15.909818+0000","last_unstale":"2024-05-09T08:04:32.754272+0000","last_undegraded":"2024-05-09T08:04:32.754272+0000","last_fullsized":"2024-05-09T08:04:32.754272+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:41:55.788033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:32.736139+0000","last_change":"2024-05-09T08:02:15.885025+0000","last_active":"2024-05-09T08:04:32.736139+0000","last_peered":"2024-05-09T08:04:32.736139+0000","last_clean":"2024-05-09T08:04:32.736139+0000","last_became_active":"2024-05-09T08:02:15.884517+0000","last_became_peered":"2024-05-09T08:02:15.884517+0000","last_unstale":"2024-05-09T08:04:32.736139+0000","last_undegraded":"2024-05-09T08:04:32.736139+0000","last_fullsized":"2024-05-09T08:04:32.736139+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:01:50.341133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:32.735890+0000","last_change":"2024-05-09T08:02:15.890306+0000","last_active":"2024-05-09T08:04:32.735890+0000","last_peered":"2024-05-09T08:04:32.735890+0000","last_clean":"2024-05-09T08:04:32.735890+0000","last_became_active":"2024-05-09T08:02:15.889998+0000","last_became_peered":"2024-05-09T08:02:15.889998+0000","last_unstale":"2024-05-09T08:04:32.735890+0000","last_undegraded":"2024-05-09T08:04:32.735890+0000","last_fullsized":"2024-05-09T08:04:32.735890+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:16:57.446444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:37.747977+0000","last_change":"2024-05-09T08:02:15.886417+0000","last_active":"2024-05-09T08:04:37.747977+0000","last_peered":"2024-05-09T08:04:37.747977+0000","last_clean":"2024-05-09T08:04:37.747977+0000","last_became_active":"2024-05-09T08:02:15.885525+0000","last_became_peered":"2024-05-09T08:02:15.885525+0000","last_unstale":"2024-05-09T08:04:37.747977+0000","last_undegraded":"2024-05-09T08:04:37.747977+0000","last_fullsized":"2024-05-09T08:04:37.747977+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:43:02.075442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:37.736779+0000","last_change":"2024-05-09T08:02:15.893172+0000","last_active":"2024-05-09T08:04:37.736779+0000","last_peered":"2024-05-09T08:04:37.736779+0000","last_clean":"2024-05-09T08:04:37.736779+0000","last_became_active":"2024-05-09T08:02:15.892886+0000","last_became_peered":"2024-05-09T08:02:15.892886+0000","last_unstale":"2024-05-09T08:04:37.736779+0000","last_undegraded":"2024-05-09T08:04:37.736779+0000","last_fullsized":"2024-05-09T08:04:37.736779+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:59:14.957806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:37.736714+0000","last_change":"2024-05-09T08:02:15.908659+0000","last_active":"2024-05-09T08:04:37.736714+0000","last_peered":"2024-05-09T08:04:37.736714+0000","last_clean":"2024-05-09T08:04:37.736714+0000","last_became_active":"2024-05-09T08:02:15.908307+0000","last_became_peered":"2024-05-09T08:02:15.908307+0000","last_unstale":"2024-05-09T08:04:37.736714+0000","last_undegraded":"2024-05-09T08:04:37.736714+0000","last_fullsized":"2024-05-09T08:04:37.736714+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:20:07.368327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:37.761383+0000","last_change":"2024-05-09T08:02:15.885800+0000","last_active":"2024-05-09T08:04:37.761383+0000","last_peered":"2024-05-09T08:04:37.761383+0000","last_clean":"2024-05-09T08:04:37.761383+0000","last_became_active":"2024-05-09T08:02:15.885185+0000","last_became_peered":"2024-05-09T08:02:15.885185+0000","last_unstale":"2024-05-09T08:04:37.761383+0000","last_undegraded":"2024-05-09T08:04:37.761383+0000","last_fullsized":"2024-05-09T08:04:37.761383+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:01:50.341133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:37.755012+0000","last_change":"2024-05-09T08:02:15.911496+0000","last_active":"2024-05-09T08:04:37.755012+0000","last_peered":"2024-05-09T08:04:37.755012+0000","last_clean":"2024-05-09T08:04:37.755012+0000","last_became_active":"2024-05-09T08:02:15.911142+0000","last_became_peered":"2024-05-09T08:02:15.911142+0000","last_unstale":"2024-05-09T08:04:37.755012+0000","last_undegraded":"2024-05-09T08:04:37.755012+0000","last_fullsized":"2024-05-09T08:04:37.755012+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:27:59.315750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:42.749013+0000","last_change":"2024-05-09T08:02:15.898847+0000","last_active":"2024-05-09T08:04:42.749013+0000","last_peered":"2024-05-09T08:04:42.749013+0000","last_clean":"2024-05-09T08:04:42.749013+0000","last_became_active":"2024-05-09T08:02:15.898480+0000","last_became_peered":"2024-05-09T08:02:15.898480+0000","last_unstale":"2024-05-09T08:04:42.749013+0000","last_undegraded":"2024-05-09T08:04:42.749013+0000","last_fullsized":"2024-05-09T08:04:42.749013+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:20:07.368327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:42.736898+0000","last_change":"2024-05-09T08:02:15.883117+0000","last_active":"2024-05-09T08:04:42.736898+0000","last_peered":"2024-05-09T08:04:42.736898+0000","last_clean":"2024-05-09T08:04:42.736898+0000","last_became_active":"2024-05-09T08:02:15.882141+0000","last_became_peered":"2024-05-09T08:02:15.882141+0000","last_unstale":"2024-05-09T08:04:42.736898+0000","last_undegraded":"2024-05-09T08:04:42.736898+0000","last_fullsized":"2024-05-09T08:04:42.736898+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:57:11.352151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":31,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:42.737262+0000","last_change":"2024-05-09T08:03:02.322462+0000","last_active":"2024-05-09T08:04:42.737262+0000","last_peered":"2024-05-09T08:04:42.737262+0000","last_clean":"2024-05-09T08:04:42.737262+0000","last_became_active":"2024-05-09T08:03:02.322073+0000","last_became_peered":"2024-05-09T08:03:02.322073+0000","last_unstale":"2024-05-09T08:04:42.737262+0000","last_undegraded":"2024-05-09T08:04:42.737262+0000","last_fullsized":"2024-05-09T08:04:42.737262+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:16:37.795617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"27'1","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:05:27.208367+0000","last_change":"2024-05-09T08:02:15.886482+0000","last_active":"2024-05-09T08:05:27.208367+0000","last_peered":"2024-05-09T08:05:27.208367+0000","last_clean":"2024-05-09T08:05:27.208367+0000","last_became_active":"2024-05-09T08:02:15.886183+0000","last_became_peered":"2024-05-09T08:02:15.886183+0000","last_unstale":"2024-05-09T08:05:27.208367+0000","last_undegraded":"2024-05-09T08:05:27.208367+0000","last_fullsized":"2024-05-09T08:05:27.208367+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:23:40.248821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:42.736901+0000","last_change":"2024-05-09T08:02:15.887506+0000","last_active":"2024-05-09T08:04:42.736901+0000","last_peered":"2024-05-09T08:04:42.736901+0000","last_clean":"2024-05-09T08:04:42.736901+0000","last_became_active":"2024-05-09T08:02:15.887225+0000","last_became_peered":"2024-05-09T08:02:15.887225+0000","last_unstale":"2024-05-09T08:04:42.736901+0000","last_undegraded":"2024-05-09T08:04:42.736901+0000","last_fullsized":"2024-05-09T08:04:42.736901+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:25:24.736323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:47.753113+0000","last_change":"2024-05-09T08:02:15.902065+0000","last_active":"2024-05-09T08:04:47.753113+0000","last_peered":"2024-05-09T08:04:47.753113+0000","last_clean":"2024-05-09T08:04:47.753113+0000","last_became_active":"2024-05-09T08:02:15.901766+0000","last_became_peered":"2024-05-09T08:02:15.901766+0000","last_unstale":"2024-05-09T08:04:47.753113+0000","last_undegraded":"2024-05-09T08:04:47.753113+0000","last_fullsized":"2024-05-09T08:04:47.753113+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:05:02.181367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:47.737268+0000","last_change":"2024-05-09T08:02:15.911058+0000","last_active":"2024-05-09T08:04:47.737268+0000","last_peered":"2024-05-09T08:04:47.737268+0000","last_clean":"2024-05-09T08:04:47.737268+0000","last_became_active":"2024-05-09T08:02:15.910716+0000","last_became_peered":"2024-05-09T08:02:15.910716+0000","last_unstale":"2024-05-09T08:04:47.737268+0000","last_undegraded":"2024-05-09T08:04:47.737268+0000","last_fullsized":"2024-05-09T08:04:47.737268+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:41:56.871863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:47.737757+0000","last_change":"2024-05-09T08:02:15.908653+0000","last_active":"2024-05-09T08:04:47.737757+0000","last_peered":"2024-05-09T08:04:47.737757+0000","last_clean":"2024-05-09T08:04:47.737757+0000","last_became_active":"2024-05-09T08:02:15.908290+0000","last_became_peered":"2024-05-09T08:02:15.908290+0000","last_unstale":"2024-05-09T08:04:47.737757+0000","last_undegraded":"2024-05-09T08:04:47.737757+0000","last_fullsized":"2024-05-09T08:04:47.737757+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:18:36.309627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:47.737286+0000","last_change":"2024-05-09T08:02:15.896652+0000","last_active":"2024-05-09T08:04:47.737286+0000","last_peered":"2024-05-09T08:04:47.737286+0000","last_clean":"2024-05-09T08:04:47.737286+0000","last_became_active":"2024-05-09T08:02:15.896384+0000","last_became_peered":"2024-05-09T08:02:15.896384+0000","last_unstale":"2024-05-09T08:04:47.737286+0000","last_undegraded":"2024-05-09T08:04:47.737286+0000","last_fullsized":"2024-05-09T08:04:47.737286+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:18:36.309627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:05:02.751368+0000","last_change":"2024-05-09T08:03:02.615230+0000","last_active":"2024-05-09T08:05:02.751368+0000","last_peered":"2024-05-09T08:05:02.751368+0000","last_clean":"2024-05-09T08:05:02.751368+0000","last_became_active":"2024-05-09T08:03:02.614387+0000","last_became_peered":"2024-05-09T08:03:02.614387+0000","last_unstale":"2024-05-09T08:05:02.751368+0000","last_undegraded":"2024-05-09T08:05:02.751368+0000","last_fullsized":"2024-05-09T08:05:02.751368+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:53:23.178610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:22.752990+0000","last_change":"2024-05-09T08:02:15.905078+0000","last_active":"2024-05-09T08:04:22.752990+0000","last_peered":"2024-05-09T08:04:22.752990+0000","last_clean":"2024-05-09T08:04:22.752990+0000","last_became_active":"2024-05-09T08:02:15.904736+0000","last_became_peered":"2024-05-09T08:02:15.904736+0000","last_unstale":"2024-05-09T08:04:22.752990+0000","last_undegraded":"2024-05-09T08:04:22.752990+0000","last_fullsized":"2024-05-09T08:04:22.752990+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:01:50.341133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:22.759254+0000","last_change":"2024-05-09T08:02:15.897743+0000","last_active":"2024-05-09T08:04:22.759254+0000","last_peered":"2024-05-09T08:04:22.759254+0000","last_clean":"2024-05-09T08:04:22.759254+0000","last_became_active":"2024-05-09T08:02:15.897367+0000","last_became_peered":"2024-05-09T08:02:15.897367+0000","last_unstale":"2024-05-09T08:04:22.759254+0000","last_undegraded":"2024-05-09T08:04:22.759254+0000","last_fullsized":"2024-05-09T08:04:22.759254+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:47:32.605288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:05:07.752186+0000","last_change":"2024-05-09T08:03:02.323519+0000","last_active":"2024-05-09T08:05:07.752186+0000","last_peered":"2024-05-09T08:05:07.752186+0000","last_clean":"2024-05-09T08:05:07.752186+0000","last_became_active":"2024-05-09T08:03:02.322397+0000","last_became_peered":"2024-05-09T08:03:02.322397+0000","last_unstale":"2024-05-09T08:05:07.752186+0000","last_undegraded":"2024-05-09T08:05:07.752186+0000","last_fullsized":"2024-05-09T08:05:07.752186+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:47:32.605288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9216,"num_write_kb":25165824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10755,"ondisk_log_size":10755,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":448762,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":277,"num_write_kb":617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":966656,"data_stored":897524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":258,"ondisk_log_size":258,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574919,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50092,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321748,"statfs":{"total":96636764160,"available":96585469952,"internally_reserved":0,"allocated":888832,"data_stored":709919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574919,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":76260,"kb_used_data":732,"kb_used_omap":1,"kb_used_meta":75518,"kb_avail":94295580,"statfs":{"total":96636764160,"available":96558673920,"internally_reserved":0,"allocated":749568,"data_stored":566404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":77330886},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574919,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45352,"kb_used_data":288,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94326488,"statfs":{"total":96636764160,"available":96590323712,"internally_reserved":0,"allocated":294912,"data_stored":119059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574920,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":53680,"kb_used_data":296,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94318160,"statfs":{"total":96636764160,"available":96581795840,"internally_reserved":0,"allocated":303104,"data_stored":119095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":54655431},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574920,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":76264,"kb_used_data":736,"kb_used_omap":1,"kb_used_meta":75518,"kb_avail":94295576,"statfs":{"total":96636764160,"available":96558669824,"internally_reserved":0,"allocated":753664,"data_stored":566933,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":77330887},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574920,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":57832,"kb_used_data":288,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94314008,"statfs":{"total":96636764160,"available":96577544192,"internally_reserved":0,"allocated":294912,"data_stored":119498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574921,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":75820,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":75518,"kb_avail":94296020,"statfs":{"total":96636764160,"available":96559124480,"internally_reserved":0,"allocated":299008,"data_stored":119129,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77330885},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574920,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45940,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325900,"statfs":{"total":96636764160,"available":96589721600,"internally_reserved":0,"allocated":897024,"data_stored":709975,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":462848,"data_stored":447964,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":458752,"data_stored":447435,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T08:07:57.636 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T08:07:57.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:57.881+0000 7f5b91800640 1 Processor -- start 2024-05-09T08:07:57.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:57.881+0000 7f5b91800640 1 -- start start 2024-05-09T08:07:57.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:57.883+0000 7f5b91800640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f5b8c152500 0x7f5b8c0c8180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:07:57.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:57.883+0000 7f5b91800640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f5b8c0cc730 0x7f5b8c15c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:07:57.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:57.883+0000 7f5b91800640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b8c0c6a30 con 0x7f5b8c0c7660 2024-05-09T08:07:57.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:57.883+0000 7f5b91800640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f5b8c0c5e50 con 0x7f5b8c0cc730 2024-05-09T08:07:57.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:57.883+0000 7f5b91800640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f5b8c0c22d0 con 0x7f5b8c152500 2024-05-09T08:07:57.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:57.884+0000 7f5b8b400640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f5b8c0cc730 0x7f5b8c15c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:07:57.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:57.884+0000 7f5b8b400640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f5b8c0cc730 0x7f5b8c15c080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60586/0 (socket says 172.21.15.1:60586) 2024-05-09T08:07:57.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:57.884+0000 7f5b8b400640 1 -- 172.21.15.1:0/3323123557 learned_addr learned my addr 172.21.15.1:0/3323123557 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:07:57.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:57.884+0000 7f5b8be00640 1 --2- 172.21.15.1:0/3323123557 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f5b8c152500 0x7f5b8c0c8180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:07:57.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:57.884+0000 7f5b8be00640 1 -- 172.21.15.1:0/3323123557 >> v2:172.21.15.1:3300/0 conn(0x7f5b8c0cc730 msgr2=0x7f5b8c15c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:07:57.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:57.884+0000 7f5b8be00640 1 --2- 172.21.15.1:0/3323123557 >> v2:172.21.15.1:3300/0 conn(0x7f5b8c0cc730 0x7f5b8c15c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:57.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:57.884+0000 7f5b8be00640 1 -- 172.21.15.1:0/3323123557 >> v1:172.21.15.1:6789/0 conn(0x7f5b8c0c7660 legacy=0x7f5b8c06b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:07:57.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:57.884+0000 7f5b8be00640 1 -- 172.21.15.1:0/3323123557 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b8c0c6cf0 con 0x7f5b8c152500 2024-05-09T08:07:57.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:57.884+0000 7f5b8b400640 1 --2- 172.21.15.1:0/3323123557 >> v2:172.21.15.1:3300/0 conn(0x7f5b8c0cc730 0x7f5b8c15c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:07:57.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:57.886+0000 7f5b8be00640 1 --2- 172.21.15.1:0/3323123557 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f5b8c152500 0x7f5b8c0c8180 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f5b7c0097a0 tx=0x7f5b7c02a230 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5190224055324268 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:07:57.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:57.886+0000 7f5b8aa00640 1 -- 172.21.15.1:0/3323123557 <== mon.2 v2:172.21.15.7:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b7c037070 con 0x7f5b8c152500 2024-05-09T08:07:57.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:57.886+0000 7f5b8aa00640 1 -- 172.21.15.1:0/3323123557 <== mon.2 v2:172.21.15.7:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b7c032440 con 0x7f5b8c152500 2024-05-09T08:07:57.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:57.886+0000 7f5b8aa00640 1 -- 172.21.15.1:0/3323123557 <== mon.2 v2:172.21.15.7:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b7c032810 con 0x7f5b8c152500 2024-05-09T08:07:57.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:57.886+0000 7f5b91800640 1 -- 172.21.15.1:0/3323123557 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f5b8c152500 msgr2=0x7f5b8c0c8180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:07:57.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:57.886+0000 7f5b91800640 1 --2- 172.21.15.1:0/3323123557 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f5b8c152500 0x7f5b8c0c8180 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f5b7c0097a0 tx=0x7f5b7c02a230 comp rx=0 tx=0).stop 2024-05-09T08:07:57.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:57.887+0000 7f5b91800640 1 -- 172.21.15.1:0/3323123557 shutdown_connections 2024-05-09T08:07:57.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:57.887+0000 7f5b91800640 1 --2- 172.21.15.1:0/3323123557 >> v2:172.21.15.1:3300/0 conn(0x7f5b8c0cc730 0x7f5b8c15c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:57.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:57.887+0000 7f5b91800640 1 --2- 172.21.15.1:0/3323123557 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f5b8c152500 0x7f5b8c0c8180 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:57.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:57.887+0000 7f5b91800640 1 -- 172.21.15.1:0/3323123557 >> 172.21.15.1:0/3323123557 conn(0x7f5b8c0cbc10 msgr2=0x7f5b8c0cc020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:07:57.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:57.887+0000 7f5b91800640 1 -- 172.21.15.1:0/3323123557 shutdown_connections 2024-05-09T08:07:57.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:57.887+0000 7f5b91800640 1 -- 172.21.15.1:0/3323123557 wait complete. 2024-05-09T08:07:57.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:57.888+0000 7f5b91800640 1 Processor -- start 2024-05-09T08:07:57.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:57.889+0000 7f5b91800640 1 -- start start 2024-05-09T08:07:57.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:57.889+0000 7f5b91800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b8c0c7660 0x7f5b8c148e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:07:57.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:57.889+0000 7f5b91800640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f5b8c0cc730 0x7f5b8c1e0d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:07:57.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:57.890+0000 7f5b91800640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5b8c152500 0x7f5b8c1473b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:07:57.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:57.890+0000 7f5b91800640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f5b8c0c72b0 con 0x7f5b8c152500 2024-05-09T08:07:57.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:57.890+0000 7f5b91800640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5b8c0ccb50 con 0x7f5b8c0c7660 2024-05-09T08:07:57.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:57.890+0000 7f5b91800640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f5b8c15b880 con 0x7f5b8c0cc730 2024-05-09T08:07:57.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:57.890+0000 7f5b8b400640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f5b8c0cc730 0x7f5b8c1e0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:07:57.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:57.890+0000 7f5b8be00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b8c0c7660 0x7f5b8c148e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:07:57.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:57.890+0000 7f5b90e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5b8c152500 0x7f5b8c1473b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:07:57.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:57.890+0000 7f5b8b400640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f5b8c0cc730 0x7f5b8c1e0d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3301/0 says I am v2:172.21.15.1:58702/0 (socket says 172.21.15.1:58702) 2024-05-09T08:07:57.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:57.890+0000 7f5b8b400640 1 -- 172.21.15.1:0/2058295606 learned_addr learned my addr 172.21.15.1:0/2058295606 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:07:57.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:57.890+0000 7f5b8be00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b8c0c7660 0x7f5b8c148e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.1:54070/0 (socket says 172.21.15.1:54070) 2024-05-09T08:07:57.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:57.890+0000 7f5b8b400640 1 -- 172.21.15.1:0/2058295606 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b8c0c7660 msgr2=0x7f5b8c148e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:07:57.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:57.890+0000 7f5b8b400640 1 --2- 172.21.15.1:0/2058295606 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b8c0c7660 0x7f5b8c148e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:57.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:57.890+0000 7f5b8b400640 1 -- 172.21.15.1:0/2058295606 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5b8c152500 msgr2=0x7f5b8c1473b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:07:57.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:57.890+0000 7f5b8b400640 1 --2- 172.21.15.1:0/2058295606 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5b8c152500 0x7f5b8c1473b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:57.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:57.890+0000 7f5b8b400640 1 -- 172.21.15.1:0/2058295606 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b8c15a870 con 0x7f5b8c0cc730 2024-05-09T08:07:57.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:57.891+0000 7f5b8b400640 1 --2- 172.21.15.1:0/2058295606 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f5b8c0cc730 0x7f5b8c1e0d00 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f5b7800ea00 tx=0x7f5b7800eec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:07:57.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:57.891+0000 7f5b83e00640 1 -- 172.21.15.1:0/2058295606 <== mon.2 v2:172.21.15.7:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b78017070 con 0x7f5b8c0cc730 2024-05-09T08:07:57.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:57.891+0000 7f5b83e00640 1 -- 172.21.15.1:0/2058295606 <== mon.2 v2:172.21.15.7:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b7800ca60 con 0x7f5b8c0cc730 2024-05-09T08:07:57.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:57.891+0000 7f5b83e00640 1 -- 172.21.15.1:0/2058295606 <== mon.2 v2:172.21.15.7:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b7800ce10 con 0x7f5b8c0cc730 2024-05-09T08:07:57.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:57.891+0000 7f5b91800640 1 -- 172.21.15.1:0/2058295606 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b8c147b10 con 0x7f5b8c0cc730 2024-05-09T08:07:57.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:57.891+0000 7f5b91800640 1 -- 172.21.15.1:0/2058295606 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b8c1480d0 con 0x7f5b8c0cc730 2024-05-09T08:07:57.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:57.892+0000 7f5b91800640 1 -- 172.21.15.1:0/2058295606 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b8c04f660 con 0x7f5b8c0cc730 2024-05-09T08:07:57.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:57.893+0000 7f5b83e00640 1 -- 172.21.15.1:0/2058295606 <== mon.2 v2:172.21.15.7:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5b7801a070 con 0x7f5b8c0cc730 2024-05-09T08:07:57.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:57.894+0000 7f5b83e00640 1 --2- 172.21.15.1:0/2058295606 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f5b6c06be30 0x7f5b6c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:07:57.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:57.894+0000 7f5b83e00640 1 -- 172.21.15.1:0/2058295606 <== mon.2 v2:172.21.15.7:3301/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7f5b7808b190 con 0x7f5b8c0cc730 2024-05-09T08:07:57.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:57.894+0000 7f5b8be00640 1 --2- 172.21.15.1:0/2058295606 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f5b6c06be30 0x7f5b6c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:07:57.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:57.895+0000 7f5b8be00640 1 --2- 172.21.15.1:0/2058295606 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f5b6c06be30 0x7f5b6c06e2f0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f5b7c034000 tx=0x7f5b7c02e000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:07:57.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:57.899+0000 7f5b83e00640 1 -- 172.21.15.1:0/2058295606 <== mon.2 v2:172.21.15.7:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f5b78059060 con 0x7f5b8c0cc730 2024-05-09T08:07:58.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:58.103+0000 7f5b91800640 1 -- 172.21.15.1:0/2058295606 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5b8c0cc370 con 0x7f5b8c0cc730 2024-05-09T08:07:58.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:58.105+0000 7f5b83e00640 1 -- 172.21.15.1:0/2058295606 <== mon.2 v2:172.21.15.7:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18741 (secure 0 0 0) 0x7f5b7805d360 con 0x7f5b8c0cc730 2024-05-09T08:07:58.116 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T08:07:58.116 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":27,"fsid":"54329ec8-3270-4043-9a82-c6cd7420aa7f","created":"2024-05-09T08:01:56.666970+0000","modified":"2024-05-09T08:03:02.304156+0000","last_up_change":"2024-05-09T08:02:07.146082+0000","last_in_change":"2024-05-09T08:01:59.008225+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T08:02:09.165511+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T08:02:12.925485+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T08:02:14.367455+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-09T08:02:17.967064+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c064454e-a9b5-4a24-9a77-17f0a4085952","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6800","nonce":1513349547},{"type":"v1","addr":"172.21.15.1:6801","nonce":1513349547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":1513349547},{"type":"v1","addr":"172.21.15.1:6803","nonce":1513349547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":1513349547},{"type":"v1","addr":"172.21.15.1:6807","nonce":1513349547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":1513349547},{"type":"v1","addr":"172.21.15.1:6805","nonce":1513349547}]},"public_addr":"172.21.15.1:6801/1513349547","cluster_addr":"172.21.15.1:6803/1513349547","heartbeat_back_addr":"172.21.15.1:6807/1513349547","heartbeat_front_addr":"172.21.15.1:6805/1513349547","state":["exists","up"]},{"osd":1,"uuid":"1a9ea687-2ff6-407a-949b-0e7e1083af21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":2942255259},{"type":"v1","addr":"172.21.15.1:6817","nonce":2942255259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":2942255259},{"type":"v1","addr":"172.21.15.1:6819","nonce":2942255259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":2942255259},{"type":"v1","addr":"172.21.15.1:6823","nonce":2942255259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6820","nonce":2942255259},{"type":"v1","addr":"172.21.15.1:6821","nonce":2942255259}]},"public_addr":"172.21.15.1:6817/2942255259","cluster_addr":"172.21.15.1:6819/2942255259","heartbeat_back_addr":"172.21.15.1:6823/2942255259","heartbeat_front_addr":"172.21.15.1:6821/2942255259","state":["exists","up"]},{"osd":2,"uuid":"c88af08b-7c9a-4dd8-9640-44138239addb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":2925031811},{"type":"v1","addr":"172.21.15.1:6809","nonce":2925031811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":2925031811},{"type":"v1","addr":"172.21.15.1:6811","nonce":2925031811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":2925031811},{"type":"v1","addr":"172.21.15.1:6815","nonce":2925031811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":2925031811},{"type":"v1","addr":"172.21.15.1:6813","nonce":2925031811}]},"public_addr":"172.21.15.1:6809/2925031811","cluster_addr":"172.21.15.1:6811/2925031811","heartbeat_back_addr":"172.21.15.1:6815/2925031811","heartbeat_front_addr":"172.21.15.1:6813/2925031811","state":["exists","up"]},{"osd":3,"uuid":"c087b7e3-a41d-4333-9365-32e3bd8c8e6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6824","nonce":2497776654},{"type":"v1","addr":"172.21.15.1:6825","nonce":2497776654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":2497776654},{"type":"v1","addr":"172.21.15.1:6827","nonce":2497776654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":2497776654},{"type":"v1","addr":"172.21.15.1:6831","nonce":2497776654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6828","nonce":2497776654},{"type":"v1","addr":"172.21.15.1:6829","nonce":2497776654}]},"public_addr":"172.21.15.1:6825/2497776654","cluster_addr":"172.21.15.1:6827/2497776654","heartbeat_back_addr":"172.21.15.1:6831/2497776654","heartbeat_front_addr":"172.21.15.1:6829/2497776654","state":["exists","up"]},{"osd":4,"uuid":"b4576b9c-871d-47a8-8d36-65af525d0200","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":2957026670},{"type":"v1","addr":"172.21.15.7:6801","nonce":2957026670}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2957026670},{"type":"v1","addr":"172.21.15.7:6803","nonce":2957026670}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":2957026670},{"type":"v1","addr":"172.21.15.7:6807","nonce":2957026670}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2957026670},{"type":"v1","addr":"172.21.15.7:6805","nonce":2957026670}]},"public_addr":"172.21.15.7:6801/2957026670","cluster_addr":"172.21.15.7:6803/2957026670","heartbeat_back_addr":"172.21.15.7:6807/2957026670","heartbeat_front_addr":"172.21.15.7:6805/2957026670","state":["exists","up"]},{"osd":5,"uuid":"902b3d94-72f3-4499-b21a-e32453ef5144","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1670295242},{"type":"v1","addr":"172.21.15.7:6809","nonce":1670295242}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1670295242},{"type":"v1","addr":"172.21.15.7:6811","nonce":1670295242}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1670295242},{"type":"v1","addr":"172.21.15.7:6815","nonce":1670295242}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1670295242},{"type":"v1","addr":"172.21.15.7:6813","nonce":1670295242}]},"public_addr":"172.21.15.7:6809/1670295242","cluster_addr":"172.21.15.7:6811/1670295242","heartbeat_back_addr":"172.21.15.7:6815/1670295242","heartbeat_front_addr":"172.21.15.7:6813/1670295242","state":["exists","up"]},{"osd":6,"uuid":"ead1d771-1728-4f85-adb4-26be21cedff3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":2852086880},{"type":"v1","addr":"172.21.15.7:6817","nonce":2852086880}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":2852086880},{"type":"v1","addr":"172.21.15.7:6819","nonce":2852086880}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":2852086880},{"type":"v1","addr":"172.21.15.7:6823","nonce":2852086880}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":2852086880},{"type":"v1","addr":"172.21.15.7:6821","nonce":2852086880}]},"public_addr":"172.21.15.7:6817/2852086880","cluster_addr":"172.21.15.7:6819/2852086880","heartbeat_back_addr":"172.21.15.7:6823/2852086880","heartbeat_front_addr":"172.21.15.7:6821/2852086880","state":["exists","up"]},{"osd":7,"uuid":"3ab5a201-9394-48a8-8918-e95197893669","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":1442884138},{"type":"v1","addr":"172.21.15.7:6825","nonce":1442884138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1442884138},{"type":"v1","addr":"172.21.15.7:6827","nonce":1442884138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1442884138},{"type":"v1","addr":"172.21.15.7:6831","nonce":1442884138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1442884138},{"type":"v1","addr":"172.21.15.7:6829","nonce":1442884138}]},"public_addr":"172.21.15.7:6825/1442884138","cluster_addr":"172.21.15.7:6827/1442884138","heartbeat_back_addr":"172.21.15.7:6831/1442884138","heartbeat_front_addr":"172.21.15.7:6829/1442884138","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:02:04.615675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:02:04.385974+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:02:04.619217+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:02:04.401681+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:02:04.097081+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:02:04.129981+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:02:04.225327+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:02:04.193351+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.35","mappings":[{"from":5,"to":2}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2},{"from":3,"to":1}]},{"pgid":"3.3f","mappings":[{"from":7,"to":2}]},{"pgid":"4.4","mappings":[{"from":1,"to":5}]},{"pgid":"4.e","mappings":[{"from":1,"to":2}]},{"pgid":"4.f","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T08:07:58.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:58.108+0000 7f5b91800640 1 -- 172.21.15.1:0/2058295606 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f5b6c06be30 msgr2=0x7f5b6c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:07:58.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:58.108+0000 7f5b91800640 1 --2- 172.21.15.1:0/2058295606 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f5b6c06be30 0x7f5b6c06e2f0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f5b7c034000 tx=0x7f5b7c02e000 comp rx=0 tx=0).stop 2024-05-09T08:07:58.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:58.108+0000 7f5b91800640 1 -- 172.21.15.1:0/2058295606 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f5b8c0cc730 msgr2=0x7f5b8c1e0d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:07:58.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:58.108+0000 7f5b91800640 1 --2- 172.21.15.1:0/2058295606 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f5b8c0cc730 0x7f5b8c1e0d00 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f5b7800ea00 tx=0x7f5b7800eec0 comp rx=0 tx=0).stop 2024-05-09T08:07:58.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:58.108+0000 7f5b91800640 1 -- 172.21.15.1:0/2058295606 shutdown_connections 2024-05-09T08:07:58.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:58.108+0000 7f5b91800640 1 --2- 172.21.15.1:0/2058295606 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f5b6c06be30 0x7f5b6c06e2f0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:58.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:58.108+0000 7f5b91800640 1 --2- 172.21.15.1:0/2058295606 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5b8c152500 0x7f5b8c1473b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:58.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:58.108+0000 7f5b91800640 1 --2- 172.21.15.1:0/2058295606 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f5b8c0cc730 0x7f5b8c1e0d00 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:58.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:58.108+0000 7f5b91800640 1 --2- 172.21.15.1:0/2058295606 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b8c0c7660 0x7f5b8c148e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:58.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:58.108+0000 7f5b91800640 1 -- 172.21.15.1:0/2058295606 >> 172.21.15.1:0/2058295606 conn(0x7f5b8c0cbc10 msgr2=0x7f5b8c14ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:07:58.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:58.109+0000 7f5b91800640 1 -- 172.21.15.1:0/2058295606 shutdown_connections 2024-05-09T08:07:58.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:58.109+0000 7f5b91800640 1 -- 172.21.15.1:0/2058295606 wait complete. 2024-05-09T08:07:59.139 INFO:tasks.ceph:Scrubbing osd.0 2024-05-09T08:07:59.140 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-05-09T08:07:59.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:59.353+0000 7f35f6400640 1 Processor -- start 2024-05-09T08:07:59.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:59.354+0000 7f35f6400640 1 -- start start 2024-05-09T08:07:59.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:59.355+0000 7f35f6400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f35f00c54f0 0x7f35f006bc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:07:59.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:59.355+0000 7f35f6400640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f35f01528e0 0x7f35f0072280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:07:59.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:59.355+0000 7f35f6400640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f35f01587e0 con 0x7f35f00c58d0 2024-05-09T08:07:59.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:59.355+0000 7f35f6400640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f35f0159360 con 0x7f35f01528e0 2024-05-09T08:07:59.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:59.355+0000 7f35f6400640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f35f0071df0 con 0x7f35f00c54f0 2024-05-09T08:07:59.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:59.356+0000 7f35f5000640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f35f00c54f0 0x7f35f006bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:07:59.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:59.356+0000 7f35f5a00640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f35f01528e0 0x7f35f0072280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:07:59.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:59.356+0000 7f35f5000640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f35f00c54f0 0x7f35f006bc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.1:54076/0 (socket says 172.21.15.1:54076) 2024-05-09T08:07:59.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:59.356+0000 7f35f5000640 1 -- 172.21.15.1:0/3439775686 learned_addr learned my addr 172.21.15.1:0/3439775686 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:07:59.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:59.356+0000 7f35f5000640 1 -- 172.21.15.1:0/3439775686 >> v2:172.21.15.1:3300/0 conn(0x7f35f01528e0 msgr2=0x7f35f0072280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:07:59.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:59.356+0000 7f35f5000640 1 --2- 172.21.15.1:0/3439775686 >> v2:172.21.15.1:3300/0 conn(0x7f35f01528e0 0x7f35f0072280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:59.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:59.356+0000 7f35f5000640 1 -- 172.21.15.1:0/3439775686 >> v1:172.21.15.1:6789/0 conn(0x7f35f00c58d0 legacy=0x7f35f01521d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:07:59.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:59.356+0000 7f35f5000640 1 -- 172.21.15.1:0/3439775686 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35f0158ac0 con 0x7f35f00c54f0 2024-05-09T08:07:59.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:59.356+0000 7f35f5a00640 1 --2- 172.21.15.1:0/3439775686 >> v2:172.21.15.1:3300/0 conn(0x7f35f01528e0 0x7f35f0072280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:07:59.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:59.357+0000 7f35f5000640 1 --2- 172.21.15.1:0/3439775686 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35f00c54f0 0x7f35f006bc10 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f35f0070180 tx=0x7f35e002a310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=401928e06731b918 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:07:59.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:59.358+0000 7f35ef400640 1 -- 172.21.15.1:0/3439775686 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35e0037070 con 0x7f35f00c54f0 2024-05-09T08:07:59.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:59.358+0000 7f35ef400640 1 -- 172.21.15.1:0/3439775686 <== mon.1 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35e0032640 con 0x7f35f00c54f0 2024-05-09T08:07:59.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:59.358+0000 7f35ef400640 1 -- 172.21.15.1:0/3439775686 <== mon.1 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35e002ae60 con 0x7f35f00c54f0 2024-05-09T08:07:59.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:59.358+0000 7f35f6400640 1 -- 172.21.15.1:0/3439775686 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35f00c54f0 msgr2=0x7f35f006bc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:07:59.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:59.358+0000 7f35f6400640 1 --2- 172.21.15.1:0/3439775686 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35f00c54f0 0x7f35f006bc10 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f35f0070180 tx=0x7f35e002a310 comp rx=0 tx=0).stop 2024-05-09T08:07:59.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:59.358+0000 7f35f6400640 1 -- 172.21.15.1:0/3439775686 shutdown_connections 2024-05-09T08:07:59.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:59.358+0000 7f35f6400640 1 --2- 172.21.15.1:0/3439775686 >> v2:172.21.15.1:3300/0 conn(0x7f35f01528e0 0x7f35f0072280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:59.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:59.358+0000 7f35f6400640 1 --2- 172.21.15.1:0/3439775686 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35f00c54f0 0x7f35f006bc10 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:59.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:59.358+0000 7f35f6400640 1 -- 172.21.15.1:0/3439775686 >> 172.21.15.1:0/3439775686 conn(0x7f35f006e080 msgr2=0x7f35f006f510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:07:59.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:59.359+0000 7f35f6400640 1 -- 172.21.15.1:0/3439775686 shutdown_connections 2024-05-09T08:07:59.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:59.359+0000 7f35f6400640 1 -- 172.21.15.1:0/3439775686 wait complete. 2024-05-09T08:07:59.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:59.360+0000 7f35f6400640 1 Processor -- start 2024-05-09T08:07:59.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:59.360+0000 7f35f6400640 1 -- start start 2024-05-09T08:07:59.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:59.361+0000 7f35f6400640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f35f00c54f0 0x7f35f00c8f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:07:59.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:59.361+0000 7f35f6400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f35f00c58d0 0x7f35f01f0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:07:59.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:59.361+0000 7f35f6400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35f01528e0 0x7f35f00c1140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:07:59.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:59.361+0000 7f35f6400640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f35f01583a0 con 0x7f35f00c58d0 2024-05-09T08:07:59.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:59.361+0000 7f35f6400640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f35f0159080 con 0x7f35f01528e0 2024-05-09T08:07:59.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:59.361+0000 7f35f6400640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f35f0158520 con 0x7f35f00c54f0 2024-05-09T08:07:59.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:59.362+0000 7f35f5a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35f01528e0 0x7f35f00c1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:07:59.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:59.362+0000 7f35efe00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f35f00c58d0 0x7f35f01f0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:07:59.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:59.362+0000 7f35f5a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35f01528e0 0x7f35f00c1140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.1:54084/0 (socket says 172.21.15.1:54084) 2024-05-09T08:07:59.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:59.362+0000 7f35f5000640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f35f00c54f0 0x7f35f00c8f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:07:59.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:59.362+0000 7f35f5a00640 1 -- 172.21.15.1:0/878291379 learned_addr learned my addr 172.21.15.1:0/878291379 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:07:59.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:59.362+0000 7f35f5a00640 1 -- 172.21.15.1:0/878291379 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f35f00c54f0 msgr2=0x7f35f00c8f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:07:59.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:59.362+0000 7f35f5a00640 1 --2- 172.21.15.1:0/878291379 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f35f00c54f0 0x7f35f00c8f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:59.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:59.362+0000 7f35f5a00640 1 -- 172.21.15.1:0/878291379 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f35f00c58d0 msgr2=0x7f35f01f0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:07:59.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:59.362+0000 7f35f5a00640 1 --2- 172.21.15.1:0/878291379 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f35f00c58d0 0x7f35f01f0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:59.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:59.362+0000 7f35f5a00640 1 -- 172.21.15.1:0/878291379 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35f0155650 con 0x7f35f01528e0 2024-05-09T08:07:59.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:59.362+0000 7f35efe00640 1 --2- 172.21.15.1:0/878291379 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f35f00c58d0 0x7f35f01f0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:07:59.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:59.362+0000 7f35f5000640 1 --2- 172.21.15.1:0/878291379 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f35f00c54f0 0x7f35f00c8f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:07:59.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:59.363+0000 7f35f5a00640 1 --2- 172.21.15.1:0/878291379 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35f01528e0 0x7f35f00c1140 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f35e800e860 tx=0x7f35e800ed20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:07:59.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:59.363+0000 7f35ecc00640 1 -- 172.21.15.1:0/878291379 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35e8017070 con 0x7f35f01528e0 2024-05-09T08:07:59.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:59.363+0000 7f35ecc00640 1 -- 172.21.15.1:0/878291379 <== mon.1 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35e800c940 con 0x7f35f01528e0 2024-05-09T08:07:59.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:59.363+0000 7f35ecc00640 1 -- 172.21.15.1:0/878291379 <== mon.1 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35e800cd10 con 0x7f35f01528e0 2024-05-09T08:07:59.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:59.363+0000 7f35f6400640 1 -- 172.21.15.1:0/878291379 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35f00c6300 con 0x7f35f01528e0 2024-05-09T08:07:59.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:59.363+0000 7f35f6400640 1 -- 172.21.15.1:0/878291379 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35f01efe20 con 0x7f35f01528e0 2024-05-09T08:07:59.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:59.364+0000 7f35f6400640 1 -- 172.21.15.1:0/878291379 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f35f00cb120 con 0x7f35f01528e0 2024-05-09T08:07:59.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:59.366+0000 7f35ecc00640 1 -- 172.21.15.1:0/878291379 <== mon.1 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f35e801a070 con 0x7f35f01528e0 2024-05-09T08:07:59.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:59.366+0000 7f35ecc00640 1 --2- 172.21.15.1:0/878291379 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f35d006be60 0x7f35d006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:07:59.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:59.367+0000 7f35ecc00640 1 -- 172.21.15.1:0/878291379 <== mon.1 v2:172.21.15.7:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7f35e808b500 con 0x7f35f01528e0 2024-05-09T08:07:59.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:59.367+0000 7f35f5000640 1 --2- 172.21.15.1:0/878291379 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f35d006be60 0x7f35d006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:07:59.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:59.367+0000 7f35ecc00640 1 --2- 172.21.15.1:0/878291379 >> [v2:172.21.15.1:6800/1513349547,v1:172.21.15.1:6801/1513349547] conn(0x7f35d0075800 0x7f35d0077ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:07:59.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:59.367+0000 7f35ecc00640 1 -- 172.21.15.1:0/878291379 --> [v2:172.21.15.1:6800/1513349547,v1:172.21.15.1:6801/1513349547] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f35e8013e50 con 0x7f35d0075800 2024-05-09T08:07:59.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:59.367+0000 7f35ecc00640 1 -- 172.21.15.1:0/878291379 <== mon.1 v2:172.21.15.7:3300/0 6 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f35e808b8b0 con 0x7f35f01528e0 2024-05-09T08:07:59.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:59.367+0000 7f35efe00640 1 --2- 172.21.15.1:0/878291379 >> [v2:172.21.15.1:6800/1513349547,v1:172.21.15.1:6801/1513349547] conn(0x7f35d0075800 0x7f35d0077ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:07:59.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:59.368+0000 7f35f5000640 1 --2- 172.21.15.1:0/878291379 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f35d006be60 0x7f35d006e320 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f35e0034000 tx=0x7f35e002e000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:07:59.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:59.368+0000 7f35efe00640 1 --2- 172.21.15.1:0/878291379 >> [v2:172.21.15.1:6800/1513349547,v1:172.21.15.1:6801/1513349547] conn(0x7f35d0075800 0x7f35d0077ca0 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:07:59.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:59.369+0000 7f35ecc00640 1 -- 172.21.15.1:0/878291379 <== osd.0 v2:172.21.15.1:6800/1513349547 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f35e8013e50 con 0x7f35d0075800 2024-05-09T08:07:59.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:59.384+0000 7f35f6400640 1 -- 172.21.15.1:0/878291379 --> [v2:172.21.15.1:6800/1513349547,v1:172.21.15.1:6801/1513349547] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f35f0074d70 con 0x7f35d0075800 2024-05-09T08:07:59.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:59.384+0000 7f35ecc00640 1 -- 172.21.15.1:0/878291379 <== osd.0 v2:172.21.15.1:6800/1513349547 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f35f0074d70 con 0x7f35d0075800 2024-05-09T08:07:59.394 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-05-09T08:07:59.394 INFO:teuthology.orchestra.run.smithi001.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-09T08:07:59.395 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-05-09T08:07:59.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:59.384+0000 7f35f6400640 1 -- 172.21.15.1:0/878291379 >> [v2:172.21.15.1:6800/1513349547,v1:172.21.15.1:6801/1513349547] conn(0x7f35d0075800 msgr2=0x7f35d0077ca0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:07:59.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:59.384+0000 7f35f6400640 1 --2- 172.21.15.1:0/878291379 >> [v2:172.21.15.1:6800/1513349547,v1:172.21.15.1:6801/1513349547] conn(0x7f35d0075800 0x7f35d0077ca0 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:59.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:59.385+0000 7f35f6400640 1 -- 172.21.15.1:0/878291379 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f35d006be60 msgr2=0x7f35d006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:07:59.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:59.385+0000 7f35f6400640 1 --2- 172.21.15.1:0/878291379 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f35d006be60 0x7f35d006e320 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f35e0034000 tx=0x7f35e002e000 comp rx=0 tx=0).stop 2024-05-09T08:07:59.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:59.385+0000 7f35f6400640 1 -- 172.21.15.1:0/878291379 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35f01528e0 msgr2=0x7f35f00c1140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:07:59.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:59.385+0000 7f35f6400640 1 --2- 172.21.15.1:0/878291379 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35f01528e0 0x7f35f00c1140 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f35e800e860 tx=0x7f35e800ed20 comp rx=0 tx=0).stop 2024-05-09T08:07:59.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:59.385+0000 7f35f5a00640 1 -- 172.21.15.1:0/878291379 reap_dead start 2024-05-09T08:07:59.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:59.385+0000 7f35f6400640 1 -- 172.21.15.1:0/878291379 shutdown_connections 2024-05-09T08:07:59.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:59.385+0000 7f35f6400640 1 -- 172.21.15.1:0/878291379 >> 172.21.15.1:0/878291379 conn(0x7f35f006e080 msgr2=0x7f35f006bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:07:59.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:59.385+0000 7f35f6400640 1 -- 172.21.15.1:0/878291379 shutdown_connections 2024-05-09T08:07:59.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:59.385+0000 7f35f6400640 1 -- 172.21.15.1:0/878291379 wait complete. 2024-05-09T08:07:59.413 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-05-09T08:07:59.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:59.610+0000 7fa0b5a00640 1 Processor -- start 2024-05-09T08:07:59.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:59.611+0000 7fa0b5a00640 1 -- start start 2024-05-09T08:07:59.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:59.612+0000 7fa0b5a00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fa0b00c54f0 0x7fa0b006f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:07:59.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:59.612+0000 7fa0b5a00640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa0b00718e0 con 0x7fa0b00c58d0 2024-05-09T08:07:59.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:59.612+0000 7fa0b5a00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa0b0071b60 con 0x7fa0b00c6470 2024-05-09T08:07:59.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:59.612+0000 7fa0b5a00640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7fa0b00726e0 con 0x7fa0b00c54f0 2024-05-09T08:07:59.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:59.613+0000 7fa0af400640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7fa0b00c58d0 0x7fa0b01589e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:33632/0 (socket says 172.21.15.1:33632) 2024-05-09T08:07:59.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:59.613+0000 7fa0af400640 1 -- 172.21.15.1:0/982645092 learned_addr learned my addr 172.21.15.1:0/982645092 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T08:07:59.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:59.613+0000 7fa0afe00640 1 --2- 172.21.15.1:0/982645092 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fa0b00c54f0 0x7fa0b006f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:07:59.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:59.613+0000 7fa0aea00640 1 -- 172.21.15.1:0/982645092 <== mon.1 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 694085873 0 0) 0x7fa0b0071b60 con 0x7fa0b00c6470 2024-05-09T08:07:59.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:59.613+0000 7fa0aea00640 1 -- 172.21.15.1:0/982645092 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa094003520 con 0x7fa0b00c6470 2024-05-09T08:07:59.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:59.613+0000 7fa0aea00640 1 -- 172.21.15.1:0/982645092 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 413096586 0 0) 0x7fa098001110 con 0x7fa0b00c58d0 2024-05-09T08:07:59.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:59.613+0000 7fa0aea00640 1 -- 172.21.15.1:0/982645092 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa0b0071b60 con 0x7fa0b00c58d0 2024-05-09T08:07:59.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:59.613+0000 7fa0afe00640 1 -- 172.21.15.1:0/982645092 >> v1:172.21.15.7:6789/0 conn(0x7fa0b00c6470 legacy=0x7fa0b0151d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:07:59.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:59.613+0000 7fa0afe00640 1 -- 172.21.15.1:0/982645092 >> v1:172.21.15.1:6789/0 conn(0x7fa0b00c58d0 legacy=0x7fa0b01589e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:07:59.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:59.614+0000 7fa0afe00640 1 -- 172.21.15.1:0/982645092 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0b0071e40 con 0x7fa0b00c54f0 2024-05-09T08:07:59.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:59.615+0000 7fa0afe00640 1 --2- 172.21.15.1:0/982645092 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fa0b00c54f0 0x7fa0b006f930 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fa0a00097d0 tx=0x7fa0a002a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ec4f82267e991729 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:07:59.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:59.615+0000 7fa0aea00640 1 -- 172.21.15.1:0/982645092 <== mon.2 v2:172.21.15.7:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0a0037070 con 0x7fa0b00c54f0 2024-05-09T08:07:59.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:59.615+0000 7fa0aea00640 1 -- 172.21.15.1:0/982645092 <== mon.2 v2:172.21.15.7:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0a0032440 con 0x7fa0b00c54f0 2024-05-09T08:07:59.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:59.615+0000 7fa0aea00640 1 -- 172.21.15.1:0/982645092 <== mon.2 v2:172.21.15.7:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0a0032810 con 0x7fa0b00c54f0 2024-05-09T08:07:59.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:59.616+0000 7fa0b5a00640 1 -- 172.21.15.1:0/982645092 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fa0b00c54f0 msgr2=0x7fa0b006f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:07:59.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:59.616+0000 7fa0b5a00640 1 --2- 172.21.15.1:0/982645092 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fa0b00c54f0 0x7fa0b006f930 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fa0a00097d0 tx=0x7fa0a002a180 comp rx=0 tx=0).stop 2024-05-09T08:07:59.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:59.616+0000 7fa0b5a00640 1 -- 172.21.15.1:0/982645092 shutdown_connections 2024-05-09T08:07:59.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:59.616+0000 7fa0b5a00640 1 --2- 172.21.15.1:0/982645092 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fa0b00c54f0 0x7fa0b006f930 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:59.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:59.616+0000 7fa0b5a00640 1 -- 172.21.15.1:0/982645092 >> 172.21.15.1:0/982645092 conn(0x7fa0b006b250 msgr2=0x7fa0b006b630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:07:59.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:59.616+0000 7fa0b5a00640 1 -- 172.21.15.1:0/982645092 shutdown_connections 2024-05-09T08:07:59.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:59.616+0000 7fa0b5a00640 1 -- 172.21.15.1:0/982645092 wait complete. 2024-05-09T08:07:59.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:59.618+0000 7fa0b5a00640 1 Processor -- start 2024-05-09T08:07:59.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:59.618+0000 7fa0b5a00640 1 -- start start 2024-05-09T08:07:59.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:59.619+0000 7fa0b5a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa0b00c54f0 0x7fa0b01e28d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:07:59.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:59.619+0000 7fa0b5a00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa0b00c58d0 0x7fa0b01578c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:07:59.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:59.619+0000 7fa0b5a00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fa0b00c6470 0x7fa0b01e39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:07:59.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:59.619+0000 7fa0b5a00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fa0b01584c0 con 0x7fa0b00c58d0 2024-05-09T08:07:59.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:59.619+0000 7fa0b5a00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa0b0072400 con 0x7fa0b00c54f0 2024-05-09T08:07:59.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:59.619+0000 7fa0b5a00640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7fa0b01590f0 con 0x7fa0b00c6470 2024-05-09T08:07:59.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:59.619+0000 7fa0af400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa0b00c58d0 0x7fa0b01578c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:07:59.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:59.619+0000 7fa0af400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa0b00c58d0 0x7fa0b01578c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60612/0 (socket says 172.21.15.1:60612) 2024-05-09T08:07:59.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:59.619+0000 7fa0af400640 1 -- 172.21.15.1:0/3314813170 learned_addr learned my addr 172.21.15.1:0/3314813170 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:07:59.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:59.619+0000 7fa0afe00640 1 --2- 172.21.15.1:0/3314813170 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa0b00c54f0 0x7fa0b01e28d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:07:59.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:59.619+0000 7fa0b5000640 1 --2- 172.21.15.1:0/3314813170 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fa0b00c6470 0x7fa0b01e39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:07:59.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:59.620+0000 7fa0af400640 1 -- 172.21.15.1:0/3314813170 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fa0b00c6470 msgr2=0x7fa0b01e39c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:07:59.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:59.620+0000 7fa0af400640 1 --2- 172.21.15.1:0/3314813170 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fa0b00c6470 0x7fa0b01e39c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:59.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:59.620+0000 7fa0af400640 1 -- 172.21.15.1:0/3314813170 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa0b00c54f0 msgr2=0x7fa0b01e28d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:07:59.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:59.620+0000 7fa0af400640 1 --2- 172.21.15.1:0/3314813170 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa0b00c54f0 0x7fa0b01e28d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:59.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:59.620+0000 7fa0af400640 1 -- 172.21.15.1:0/3314813170 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0b01e3230 con 0x7fa0b00c58d0 2024-05-09T08:07:59.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:59.620+0000 7fa0af400640 1 --2- 172.21.15.1:0/3314813170 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa0b00c58d0 0x7fa0b01578c0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fa09800c730 tx=0x7fa09800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:07:59.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:59.620+0000 7fa0a7e00640 1 -- 172.21.15.1:0/3314813170 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa098005ab0 con 0x7fa0b00c58d0 2024-05-09T08:07:59.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:59.620+0000 7fa0a7e00640 1 -- 172.21.15.1:0/3314813170 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa098005c70 con 0x7fa0b00c58d0 2024-05-09T08:07:59.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:59.620+0000 7fa0b5a00640 1 -- 172.21.15.1:0/3314813170 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0b01569f0 con 0x7fa0b00c58d0 2024-05-09T08:07:59.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:59.620+0000 7fa0a7e00640 1 -- 172.21.15.1:0/3314813170 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0980072b0 con 0x7fa0b00c58d0 2024-05-09T08:07:59.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:59.620+0000 7fa0b5a00640 1 -- 172.21.15.1:0/3314813170 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0b0156fb0 con 0x7fa0b00c58d0 2024-05-09T08:07:59.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:59.622+0000 7fa0b5a00640 1 -- 172.21.15.1:0/3314813170 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0b004f660 con 0x7fa0b00c58d0 2024-05-09T08:07:59.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:59.623+0000 7fa0a7e00640 1 -- 172.21.15.1:0/3314813170 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa098007450 con 0x7fa0b00c58d0 2024-05-09T08:07:59.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:59.624+0000 7fa0a7e00640 1 --2- 172.21.15.1:0/3314813170 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fa08c06be30 0x7fa08c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:07:59.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:59.624+0000 7fa0a7e00640 1 -- 172.21.15.1:0/3314813170 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7fa09808ae30 con 0x7fa0b00c58d0 2024-05-09T08:07:59.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:59.624+0000 7fa0afe00640 1 --2- 172.21.15.1:0/3314813170 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fa08c06be30 0x7fa08c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:07:59.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:59.625+0000 7fa0afe00640 1 --2- 172.21.15.1:0/3314813170 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fa08c06be30 0x7fa08c06e2f0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fa0a0034000 tx=0x7fa0a002e000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:07:59.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:59.628+0000 7fa0a7e00640 1 -- 172.21.15.1:0/3314813170 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fa0b00718e0 con 0x7fa0b00c58d0 2024-05-09T08:07:59.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:59.835+0000 7fa0b5a00640 1 -- 172.21.15.1:0/3314813170 --> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) -- 0x7fa0b00bf850 con 0x7fa08c06be30 2024-05-09T08:07:59.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:59.837+0000 7fa0a7e00640 1 -- 172.21.15.1:0/3314813170 <== mgr.4114 v2:172.21.15.7:6832/3562612464 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fa0b00bf850 con 0x7fa08c06be30 2024-05-09T08:07:59.847 INFO:teuthology.orchestra.run.smithi001.stderr:instructed osd(s) 0 to deep-scrub 2024-05-09T08:07:59.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:59.840+0000 7fa0b5a00640 1 -- 172.21.15.1:0/3314813170 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fa08c06be30 msgr2=0x7fa08c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:07:59.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:59.840+0000 7fa0b5a00640 1 --2- 172.21.15.1:0/3314813170 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fa08c06be30 0x7fa08c06e2f0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fa0a0034000 tx=0x7fa0a002e000 comp rx=0 tx=0).stop 2024-05-09T08:07:59.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:59.840+0000 7fa0b5a00640 1 -- 172.21.15.1:0/3314813170 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa0b00c58d0 msgr2=0x7fa0b01578c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:07:59.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:59.840+0000 7fa0b5a00640 1 --2- 172.21.15.1:0/3314813170 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa0b00c58d0 0x7fa0b01578c0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fa09800c730 tx=0x7fa09800cbf0 comp rx=0 tx=0).stop 2024-05-09T08:07:59.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:59.840+0000 7fa0b5a00640 1 -- 172.21.15.1:0/3314813170 shutdown_connections 2024-05-09T08:07:59.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:59.840+0000 7fa0b5a00640 1 --2- 172.21.15.1:0/3314813170 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fa08c06be30 0x7fa08c06e2f0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:59.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:59.840+0000 7fa0b5a00640 1 --2- 172.21.15.1:0/3314813170 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fa0b00c6470 0x7fa0b01e39c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:59.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:59.840+0000 7fa0b5a00640 1 --2- 172.21.15.1:0/3314813170 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa0b00c58d0 0x7fa0b01578c0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:59.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:59.840+0000 7fa0b5a00640 1 --2- 172.21.15.1:0/3314813170 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa0b00c54f0 0x7fa0b01e28d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:07:59.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:59.840+0000 7fa0b5a00640 1 -- 172.21.15.1:0/3314813170 >> 172.21.15.1:0/3314813170 conn(0x7fa0b006b250 msgr2=0x7fa0b0150c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:07:59.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:59.840+0000 7fa0b5a00640 1 -- 172.21.15.1:0/3314813170 shutdown_connections 2024-05-09T08:07:59.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:07:59.840+0000 7fa0b5a00640 1 -- 172.21.15.1:0/3314813170 wait complete. 2024-05-09T08:07:59.870 INFO:tasks.ceph:Scrubbing osd.1 2024-05-09T08:07:59.870 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-05-09T08:08:00.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.070+0000 7f5423800640 1 Processor -- start 2024-05-09T08:08:00.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.070+0000 7f5423800640 1 -- start start 2024-05-09T08:08:00.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.072+0000 7f5423800640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f541c158f90 0x7f541c0c7350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:08:00.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.072+0000 7f5423800640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f541c06e0a0 0x7f541c1692c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:08:00.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.072+0000 7f5423800640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f541c15f5b0 con 0x7f541c0c6c70 2024-05-09T08:08:00.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.072+0000 7f5423800640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f541c0c7180 con 0x7f541c158f90 2024-05-09T08:08:00.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.072+0000 7f5423800640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f541c0b2070 con 0x7f541c06e0a0 2024-05-09T08:08:00.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.072+0000 7f5420e00640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f541c158f90 0x7f541c0c7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:08:00.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.072+0000 7f5420e00640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f541c158f90 0x7f541c0c7350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60616/0 (socket says 172.21.15.1:60616) 2024-05-09T08:08:00.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.072+0000 7f5420e00640 1 -- 172.21.15.1:0/2994756297 learned_addr learned my addr 172.21.15.1:0/2994756297 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:08:00.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.072+0000 7f5421800640 1 --2- 172.21.15.1:0/2994756297 >> v2:172.21.15.7:3300/0 conn(0x7f541c06e0a0 0x7f541c1692c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:08:00.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.073+0000 7f5420e00640 1 -- 172.21.15.1:0/2994756297 >> v2:172.21.15.7:3300/0 conn(0x7f541c06e0a0 msgr2=0x7f541c1692c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:08:00.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.073+0000 7f5420e00640 1 --2- 172.21.15.1:0/2994756297 >> v2:172.21.15.7:3300/0 conn(0x7f541c06e0a0 0x7f541c1692c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:00.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.073+0000 7f5420e00640 1 -- 172.21.15.1:0/2994756297 >> v1:172.21.15.1:6789/0 conn(0x7f541c0c6c70 legacy=0x7f541c06d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:08:00.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.073+0000 7f5420e00640 1 -- 172.21.15.1:0/2994756297 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f541c15f890 con 0x7f541c158f90 2024-05-09T08:08:00.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.074+0000 7f5420e00640 1 --2- 172.21.15.1:0/2994756297 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f541c158f90 0x7f541c0c7350 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f541c06f090 tx=0x7f541002a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c87cde07d9c25c8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:08:00.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.074+0000 7f541b400640 1 -- 172.21.15.1:0/2994756297 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5410037070 con 0x7f541c158f90 2024-05-09T08:08:00.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.074+0000 7f541b400640 1 -- 172.21.15.1:0/2994756297 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f541002ae50 con 0x7f541c158f90 2024-05-09T08:08:00.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.074+0000 7f541b400640 1 -- 172.21.15.1:0/2994756297 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5410032860 con 0x7f541c158f90 2024-05-09T08:08:00.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.075+0000 7f5423800640 1 -- 172.21.15.1:0/2994756297 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f541c158f90 msgr2=0x7f541c0c7350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:08:00.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.075+0000 7f5423800640 1 --2- 172.21.15.1:0/2994756297 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f541c158f90 0x7f541c0c7350 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f541c06f090 tx=0x7f541002a300 comp rx=0 tx=0).stop 2024-05-09T08:08:00.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.075+0000 7f5423800640 1 -- 172.21.15.1:0/2994756297 shutdown_connections 2024-05-09T08:08:00.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.075+0000 7f5423800640 1 --2- 172.21.15.1:0/2994756297 >> v2:172.21.15.7:3300/0 conn(0x7f541c06e0a0 0x7f541c1692c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:00.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.075+0000 7f5423800640 1 --2- 172.21.15.1:0/2994756297 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f541c158f90 0x7f541c0c7350 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:00.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.075+0000 7f5423800640 1 -- 172.21.15.1:0/2994756297 >> 172.21.15.1:0/2994756297 conn(0x7f541c0c89b0 msgr2=0x7f541c0c8dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:08:00.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.075+0000 7f5423800640 1 -- 172.21.15.1:0/2994756297 shutdown_connections 2024-05-09T08:08:00.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.075+0000 7f5423800640 1 -- 172.21.15.1:0/2994756297 wait complete. 2024-05-09T08:08:00.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.077+0000 7f5423800640 1 Processor -- start 2024-05-09T08:08:00.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.077+0000 7f5423800640 1 -- start start 2024-05-09T08:08:00.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.078+0000 7f5423800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f541c06e0a0 0x7f541c1f4f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:08:00.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.078+0000 7f5423800640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f541c0c6c70 0x7f541c147c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:08:00.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.078+0000 7f5423800640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f541c158f90 0x7f541c148150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:08:00.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.078+0000 7f5423800640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f541c0c29b0 con 0x7f541c158f90 2024-05-09T08:08:00.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.078+0000 7f5423800640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f541c0c26d0 con 0x7f541c06e0a0 2024-05-09T08:08:00.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.078+0000 7f5423800640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f541c0c20f0 con 0x7f541c0c6c70 2024-05-09T08:08:00.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.078+0000 7f5421800640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f541c158f90 0x7f541c148150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:08:00.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.078+0000 7f5421800640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f541c158f90 0x7f541c148150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60626/0 (socket says 172.21.15.1:60626) 2024-05-09T08:08:00.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.078+0000 7f5421800640 1 -- 172.21.15.1:0/2960587701 learned_addr learned my addr 172.21.15.1:0/2960587701 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:08:00.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.078+0000 7f541be00640 1 --2- 172.21.15.1:0/2960587701 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f541c0c6c70 0x7f541c147c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:08:00.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.078+0000 7f5420e00640 1 --2- 172.21.15.1:0/2960587701 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f541c06e0a0 0x7f541c1f4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:08:00.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.079+0000 7f5421800640 1 -- 172.21.15.1:0/2960587701 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f541c0c6c70 msgr2=0x7f541c147c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:08:00.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.079+0000 7f5421800640 1 --2- 172.21.15.1:0/2960587701 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f541c0c6c70 0x7f541c147c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:00.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.079+0000 7f5421800640 1 -- 172.21.15.1:0/2960587701 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f541c06e0a0 msgr2=0x7f541c1f4f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:08:00.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.079+0000 7f5421800640 1 --2- 172.21.15.1:0/2960587701 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f541c06e0a0 0x7f541c1f4f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:00.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.079+0000 7f5421800640 1 -- 172.21.15.1:0/2960587701 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f541c1f8730 con 0x7f541c158f90 2024-05-09T08:08:00.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.079+0000 7f5421800640 1 --2- 172.21.15.1:0/2960587701 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f541c158f90 0x7f541c148150 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f541400efc0 tx=0x7f541400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:08:00.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.079+0000 7f5418c00640 1 -- 172.21.15.1:0/2960587701 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5414018020 con 0x7f541c158f90 2024-05-09T08:08:00.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.080+0000 7f5418c00640 1 -- 172.21.15.1:0/2960587701 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5414014480 con 0x7f541c158f90 2024-05-09T08:08:00.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.080+0000 7f5423800640 1 -- 172.21.15.1:0/2960587701 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f541c15d6c0 con 0x7f541c158f90 2024-05-09T08:08:00.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.080+0000 7f5418c00640 1 -- 172.21.15.1:0/2960587701 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5414014870 con 0x7f541c158f90 2024-05-09T08:08:00.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.080+0000 7f5423800640 1 -- 172.21.15.1:0/2960587701 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f541c1f89d0 con 0x7f541c158f90 2024-05-09T08:08:00.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.081+0000 7f5423800640 1 -- 172.21.15.1:0/2960587701 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f541c068520 con 0x7f541c158f90 2024-05-09T08:08:00.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.082+0000 7f5418c00640 1 -- 172.21.15.1:0/2960587701 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5414005d10 con 0x7f541c158f90 2024-05-09T08:08:00.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.083+0000 7f5418c00640 1 --2- 172.21.15.1:0/2960587701 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f53fc06be30 0x7f53fc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:08:00.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.083+0000 7f5418c00640 1 -- 172.21.15.1:0/2960587701 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7f541408bc10 con 0x7f541c158f90 2024-05-09T08:08:00.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.083+0000 7f5418c00640 1 --2- 172.21.15.1:0/2960587701 >> [v2:172.21.15.1:6816/2942255259,v1:172.21.15.1:6817/2942255259] conn(0x7f53fc0757c0 0x7f53fc077c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:08:00.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.083+0000 7f5418c00640 1 -- 172.21.15.1:0/2960587701 --> [v2:172.21.15.1:6816/2942255259,v1:172.21.15.1:6817/2942255259] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f541400ae50 con 0x7f53fc0757c0 2024-05-09T08:08:00.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.083+0000 7f5418c00640 1 -- 172.21.15.1:0/2960587701 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f54140902a0 con 0x7f541c158f90 2024-05-09T08:08:00.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.083+0000 7f5420e00640 1 --2- 172.21.15.1:0/2960587701 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f53fc06be30 0x7f53fc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:08:00.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.083+0000 7f541be00640 1 --2- 172.21.15.1:0/2960587701 >> [v2:172.21.15.1:6816/2942255259,v1:172.21.15.1:6817/2942255259] conn(0x7f53fc0757c0 0x7f53fc077c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:08:00.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.084+0000 7f5420e00640 1 --2- 172.21.15.1:0/2960587701 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f53fc06be30 0x7f53fc06e2f0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f5410034000 tx=0x7f541002e000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:08:00.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.084+0000 7f541be00640 1 --2- 172.21.15.1:0/2960587701 >> [v2:172.21.15.1:6816/2942255259,v1:172.21.15.1:6817/2942255259] conn(0x7f53fc0757c0 0x7f53fc077c60 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:08:00.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.085+0000 7f5418c00640 1 -- 172.21.15.1:0/2960587701 <== osd.1 v2:172.21.15.1:6816/2942255259 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f541400ae50 con 0x7f53fc0757c0 2024-05-09T08:08:00.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.100+0000 7f5423800640 1 -- 172.21.15.1:0/2960587701 --> [v2:172.21.15.1:6816/2942255259,v1:172.21.15.1:6817/2942255259] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f541c072b80 con 0x7f53fc0757c0 2024-05-09T08:08:00.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.100+0000 7f5418c00640 1 -- 172.21.15.1:0/2960587701 <== osd.1 v2:172.21.15.1:6816/2942255259 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f541c072b80 con 0x7f53fc0757c0 2024-05-09T08:08:00.110 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-05-09T08:08:00.111 INFO:teuthology.orchestra.run.smithi001.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-09T08:08:00.111 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-05-09T08:08:00.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.101+0000 7f5423800640 1 -- 172.21.15.1:0/2960587701 >> [v2:172.21.15.1:6816/2942255259,v1:172.21.15.1:6817/2942255259] conn(0x7f53fc0757c0 msgr2=0x7f53fc077c60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:08:00.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.101+0000 7f5423800640 1 --2- 172.21.15.1:0/2960587701 >> [v2:172.21.15.1:6816/2942255259,v1:172.21.15.1:6817/2942255259] conn(0x7f53fc0757c0 0x7f53fc077c60 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:00.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.101+0000 7f5423800640 1 -- 172.21.15.1:0/2960587701 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f53fc06be30 msgr2=0x7f53fc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:08:00.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.101+0000 7f5423800640 1 --2- 172.21.15.1:0/2960587701 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f53fc06be30 0x7f53fc06e2f0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f5410034000 tx=0x7f541002e000 comp rx=0 tx=0).stop 2024-05-09T08:08:00.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.101+0000 7f5423800640 1 -- 172.21.15.1:0/2960587701 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f541c158f90 msgr2=0x7f541c148150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:08:00.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.101+0000 7f5423800640 1 --2- 172.21.15.1:0/2960587701 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f541c158f90 0x7f541c148150 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f541400efc0 tx=0x7f541400c490 comp rx=0 tx=0).stop 2024-05-09T08:08:00.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.101+0000 7f5421800640 1 -- 172.21.15.1:0/2960587701 reap_dead start 2024-05-09T08:08:00.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.101+0000 7f5423800640 1 -- 172.21.15.1:0/2960587701 shutdown_connections 2024-05-09T08:08:00.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.101+0000 7f5423800640 1 -- 172.21.15.1:0/2960587701 >> 172.21.15.1:0/2960587701 conn(0x7f541c0c89b0 msgr2=0x7f541c06d990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:08:00.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.101+0000 7f5423800640 1 -- 172.21.15.1:0/2960587701 shutdown_connections 2024-05-09T08:08:00.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.101+0000 7f5423800640 1 -- 172.21.15.1:0/2960587701 wait complete. 2024-05-09T08:08:00.129 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-05-09T08:08:00.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.328+0000 7f7297200640 1 Processor -- start 2024-05-09T08:08:00.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.328+0000 7f7297200640 1 -- start start 2024-05-09T08:08:00.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.329+0000 7f7297200640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f72980c54f0 0x7f729806f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:08:00.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.329+0000 7f7297200640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f72980c58d0 0x7f72981589e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:08:00.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.330+0000 7f7297200640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f72980c6470 0x7f7298158f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:08:00.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.330+0000 7f7297200640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f72980726e0 con 0x7f72980c58d0 2024-05-09T08:08:00.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.330+0000 7f7297200640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f72980718e0 con 0x7f72980c6470 2024-05-09T08:08:00.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.330+0000 7f7297200640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f7298071b60 con 0x7f72980c54f0 2024-05-09T08:08:00.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.330+0000 7f7295400640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f72980c58d0 0x7f72981589e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:08:00.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.330+0000 7f7295e00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f72980c54f0 0x7f729806f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:08:00.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.330+0000 7f7296800640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f72980c6470 0x7f7298158f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:08:00.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.330+0000 7f7295400640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f72980c58d0 0x7f72981589e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60630/0 (socket says 172.21.15.1:60630) 2024-05-09T08:08:00.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.330+0000 7f7295400640 1 -- 172.21.15.1:0/4161551955 learned_addr learned my addr 172.21.15.1:0/4161551955 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:08:00.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.330+0000 7f7296800640 1 -- 172.21.15.1:0/4161551955 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f72980c54f0 msgr2=0x7f729806f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:08:00.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.330+0000 7f7296800640 1 --2- 172.21.15.1:0/4161551955 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f72980c54f0 0x7f729806f930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:00.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.330+0000 7f7296800640 1 -- 172.21.15.1:0/4161551955 >> v2:172.21.15.1:3300/0 conn(0x7f72980c58d0 msgr2=0x7f72981589e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:08:00.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.330+0000 7f7296800640 1 --2- 172.21.15.1:0/4161551955 >> v2:172.21.15.1:3300/0 conn(0x7f72980c58d0 0x7f72981589e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:00.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.330+0000 7f7296800640 1 -- 172.21.15.1:0/4161551955 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7298071e40 con 0x7f72980c6470 2024-05-09T08:08:00.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.330+0000 7f7295400640 1 --2- 172.21.15.1:0/4161551955 >> v2:172.21.15.1:3300/0 conn(0x7f72980c58d0 0x7f72981589e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:08:00.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.332+0000 7f7296800640 1 --2- 172.21.15.1:0/4161551955 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f72980c6470 0x7f7298158f20 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f729806ec20 tx=0x7f729002a540 comp rx=0 tx=0).ready entity=mon.1 client_cookie=49c127afa925997b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:08:00.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.332+0000 7f7294a00640 1 -- 172.21.15.1:0/4161551955 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7290007740 con 0x7f72980c6470 2024-05-09T08:08:00.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.332+0000 7f7294a00640 1 -- 172.21.15.1:0/4161551955 <== mon.1 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72900078e0 con 0x7f72980c6470 2024-05-09T08:08:00.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.332+0000 7f7294a00640 1 -- 172.21.15.1:0/4161551955 <== mon.1 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7290007cb0 con 0x7f72980c6470 2024-05-09T08:08:00.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.332+0000 7f7297200640 1 -- 172.21.15.1:0/4161551955 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f72980c6470 msgr2=0x7f7298158f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:08:00.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.332+0000 7f7297200640 1 --2- 172.21.15.1:0/4161551955 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f72980c6470 0x7f7298158f20 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f729806ec20 tx=0x7f729002a540 comp rx=0 tx=0).stop 2024-05-09T08:08:00.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.333+0000 7f7297200640 1 -- 172.21.15.1:0/4161551955 shutdown_connections 2024-05-09T08:08:00.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.333+0000 7f7297200640 1 --2- 172.21.15.1:0/4161551955 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f72980c6470 0x7f7298158f20 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:00.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.333+0000 7f7297200640 1 --2- 172.21.15.1:0/4161551955 >> v2:172.21.15.1:3300/0 conn(0x7f72980c58d0 0x7f72981589e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:00.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.333+0000 7f7297200640 1 --2- 172.21.15.1:0/4161551955 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f72980c54f0 0x7f729806f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:00.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.333+0000 7f7297200640 1 -- 172.21.15.1:0/4161551955 >> 172.21.15.1:0/4161551955 conn(0x7f729806b250 msgr2=0x7f729806b630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:08:00.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.333+0000 7f7297200640 1 -- 172.21.15.1:0/4161551955 shutdown_connections 2024-05-09T08:08:00.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.333+0000 7f7297200640 1 -- 172.21.15.1:0/4161551955 wait complete. 2024-05-09T08:08:00.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.334+0000 7f7297200640 1 Processor -- start 2024-05-09T08:08:00.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.334+0000 7f7297200640 1 -- start start 2024-05-09T08:08:00.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.335+0000 7f7297200640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f72980c54f0 0x7f72981f0000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:08:00.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.335+0000 7f7297200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f72980c58d0 0x7f7298152e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:08:00.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.335+0000 7f7297200640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f72980c6470 0x7f729814eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:08:00.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.335+0000 7f7297200640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f72981584c0 con 0x7f72980c6470 2024-05-09T08:08:00.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.335+0000 7f7297200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7298072400 con 0x7f72980c58d0 2024-05-09T08:08:00.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.335+0000 7f7297200640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f729806fe70 con 0x7f72980c54f0 2024-05-09T08:08:00.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.336+0000 7f7295e00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f72980c54f0 0x7f72981f0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:08:00.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.336+0000 7f7295e00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f72980c54f0 0x7f72981f0000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3301/0 says I am v2:172.21.15.1:58746/0 (socket says 172.21.15.1:58746) 2024-05-09T08:08:00.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.336+0000 7f7296800640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f72980c6470 0x7f729814eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:08:00.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.336+0000 7f7295e00640 1 -- 172.21.15.1:0/977245915 learned_addr learned my addr 172.21.15.1:0/977245915 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:08:00.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.336+0000 7f7295400640 1 --2- 172.21.15.1:0/977245915 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f72980c58d0 0x7f7298152e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:08:00.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.336+0000 7f7295e00640 1 -- 172.21.15.1:0/977245915 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f72980c58d0 msgr2=0x7f7298152e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:08:00.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.336+0000 7f7295e00640 1 --2- 172.21.15.1:0/977245915 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f72980c58d0 0x7f7298152e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:00.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.336+0000 7f7295e00640 1 -- 172.21.15.1:0/977245915 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f72980c6470 msgr2=0x7f729814eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:08:00.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.336+0000 7f7295e00640 1 --2- 172.21.15.1:0/977245915 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f72980c6470 0x7f729814eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:00.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.336+0000 7f7295e00640 1 -- 172.21.15.1:0/977245915 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7298155ba0 con 0x7f72980c54f0 2024-05-09T08:08:00.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.337+0000 7f7295e00640 1 --2- 172.21.15.1:0/977245915 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f72980c54f0 0x7f72981f0000 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f728800e890 tx=0x7f728800ed50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:08:00.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.337+0000 7f728e000640 1 -- 172.21.15.1:0/977245915 <== mon.2 v2:172.21.15.7:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7288017070 con 0x7f72980c54f0 2024-05-09T08:08:00.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.337+0000 7f7297200640 1 -- 172.21.15.1:0/977245915 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7298153ac0 con 0x7f72980c54f0 2024-05-09T08:08:00.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.337+0000 7f728e000640 1 -- 172.21.15.1:0/977245915 <== mon.2 v2:172.21.15.7:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f728800c940 con 0x7f72980c54f0 2024-05-09T08:08:00.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.337+0000 7f728e000640 1 -- 172.21.15.1:0/977245915 <== mon.2 v2:172.21.15.7:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f728800cd10 con 0x7f72980c54f0 2024-05-09T08:08:00.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.337+0000 7f7297200640 1 -- 172.21.15.1:0/977245915 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f7298154080 con 0x7f72980c54f0 2024-05-09T08:08:00.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.338+0000 7f7297200640 1 -- 172.21.15.1:0/977245915 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7298151340 con 0x7f72980c54f0 2024-05-09T08:08:00.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.339+0000 7f728e000640 1 -- 172.21.15.1:0/977245915 <== mon.2 v2:172.21.15.7:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f728801a070 con 0x7f72980c54f0 2024-05-09T08:08:00.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.340+0000 7f728e000640 1 --2- 172.21.15.1:0/977245915 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f727406be60 0x7f727406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:08:00.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.340+0000 7f728e000640 1 -- 172.21.15.1:0/977245915 <== mon.2 v2:172.21.15.7:3301/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7f728808b180 con 0x7f72980c54f0 2024-05-09T08:08:00.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.340+0000 7f7295400640 1 --2- 172.21.15.1:0/977245915 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f727406be60 0x7f727406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:08:00.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.341+0000 7f7295400640 1 --2- 172.21.15.1:0/977245915 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f727406be60 0x7f727406e320 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f729814e010 tx=0x7f7280009240 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:08:00.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.345+0000 7f728e000640 1 -- 172.21.15.1:0/977245915 <== mon.2 v2:172.21.15.7:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f7298151340 con 0x7f72980c54f0 2024-05-09T08:08:00.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.550+0000 7f7297200640 1 -- 172.21.15.1:0/977245915 --> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) -- 0x7f729804f660 con 0x7f727406be60 2024-05-09T08:08:00.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.551+0000 7f728e000640 1 -- 172.21.15.1:0/977245915 <== mgr.4114 v2:172.21.15.7:6832/3562612464 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f729804f660 con 0x7f727406be60 2024-05-09T08:08:00.562 INFO:teuthology.orchestra.run.smithi001.stderr:instructed osd(s) 1 to deep-scrub 2024-05-09T08:08:00.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.554+0000 7f7297200640 1 -- 172.21.15.1:0/977245915 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f727406be60 msgr2=0x7f727406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:08:00.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.554+0000 7f7297200640 1 --2- 172.21.15.1:0/977245915 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f727406be60 0x7f727406e320 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f729814e010 tx=0x7f7280009240 comp rx=0 tx=0).stop 2024-05-09T08:08:00.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.554+0000 7f7297200640 1 -- 172.21.15.1:0/977245915 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f72980c54f0 msgr2=0x7f72981f0000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:08:00.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.554+0000 7f7297200640 1 --2- 172.21.15.1:0/977245915 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f72980c54f0 0x7f72981f0000 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f728800e890 tx=0x7f728800ed50 comp rx=0 tx=0).stop 2024-05-09T08:08:00.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.554+0000 7f7297200640 1 -- 172.21.15.1:0/977245915 shutdown_connections 2024-05-09T08:08:00.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.554+0000 7f7297200640 1 --2- 172.21.15.1:0/977245915 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f727406be60 0x7f727406e320 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:00.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.554+0000 7f7297200640 1 --2- 172.21.15.1:0/977245915 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f72980c58d0 0x7f7298152e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:00.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.554+0000 7f7297200640 1 --2- 172.21.15.1:0/977245915 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f72980c6470 0x7f729814eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:00.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.554+0000 7f7297200640 1 --2- 172.21.15.1:0/977245915 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f72980c54f0 0x7f72981f0000 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:00.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.554+0000 7f7297200640 1 -- 172.21.15.1:0/977245915 >> 172.21.15.1:0/977245915 conn(0x7f729806b250 msgr2=0x7f729806b630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:08:00.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.554+0000 7f7297200640 1 -- 172.21.15.1:0/977245915 shutdown_connections 2024-05-09T08:08:00.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.555+0000 7f7297200640 1 -- 172.21.15.1:0/977245915 wait complete. 2024-05-09T08:08:00.584 INFO:tasks.ceph:Scrubbing osd.2 2024-05-09T08:08:00.584 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-05-09T08:08:00.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.794+0000 7fc950800640 1 Processor -- start 2024-05-09T08:08:00.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.795+0000 7fc950800640 1 -- start start 2024-05-09T08:08:00.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.796+0000 7fc950800640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fc9481512f0 0x7fc9480c0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:08:00.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.796+0000 7fc950800640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fc948072ce0 0x7fc948073150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:08:00.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.796+0000 7fc950800640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc9480c7d60 con 0x7fc9480c94d0 2024-05-09T08:08:00.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.797+0000 7fc950800640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7fc9480c1530 con 0x7fc9481512f0 2024-05-09T08:08:00.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.797+0000 7fc950800640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7fc9480c17f0 con 0x7fc948072ce0 2024-05-09T08:08:00.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.797+0000 7fc94d400640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fc9480c94d0 0x7fc9480c7650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.1:38696/0 (socket says 172.21.15.1:38696) 2024-05-09T08:08:00.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.797+0000 7fc94d400640 1 -- 172.21.15.1:0/1937784143 learned_addr learned my addr 172.21.15.1:0/1937784143 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T08:08:00.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.797+0000 7fc94e800640 1 --2- 172.21.15.1:0/1937784143 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fc948072ce0 0x7fc948073150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:08:00.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.797+0000 7fc94de00640 1 --2- 172.21.15.1:0/1937784143 >> v2:172.21.15.1:3300/0 conn(0x7fc9481512f0 0x7fc9480c0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:08:00.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.797+0000 7fc94ca00640 1 -- 172.21.15.1:0/1937784143 <== mon.1 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1947793157 0 0) 0x7fc9480c7d60 con 0x7fc9480c94d0 2024-05-09T08:08:00.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.798+0000 7fc94ca00640 1 -- 172.21.15.1:0/1937784143 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc930003520 con 0x7fc9480c94d0 2024-05-09T08:08:00.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.798+0000 7fc94e800640 1 -- 172.21.15.1:0/1937784143 >> v2:172.21.15.1:3300/0 conn(0x7fc9481512f0 msgr2=0x7fc9480c0ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:08:00.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.798+0000 7fc94e800640 1 --2- 172.21.15.1:0/1937784143 >> v2:172.21.15.1:3300/0 conn(0x7fc9481512f0 0x7fc9480c0ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:00.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.798+0000 7fc94e800640 1 -- 172.21.15.1:0/1937784143 >> v1:172.21.15.7:6789/0 conn(0x7fc9480c94d0 legacy=0x7fc9480c7650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:08:00.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.798+0000 7fc94e800640 1 -- 172.21.15.1:0/1937784143 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc948157240 con 0x7fc948072ce0 2024-05-09T08:08:00.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.798+0000 7fc94de00640 1 --2- 172.21.15.1:0/1937784143 >> v2:172.21.15.1:3300/0 conn(0x7fc9481512f0 0x7fc9480c0ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:08:00.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.799+0000 7fc94e800640 1 --2- 172.21.15.1:0/1937784143 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fc948072ce0 0x7fc948073150 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fc944009d90 tx=0x7fc94402a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=494bca84f88beb3a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:08:00.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.800+0000 7fc94ca00640 1 -- 172.21.15.1:0/1937784143 <== mon.2 v2:172.21.15.7:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc944007740 con 0x7fc948072ce0 2024-05-09T08:08:00.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.800+0000 7fc94ca00640 1 -- 172.21.15.1:0/1937784143 <== mon.2 v2:172.21.15.7:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9440078e0 con 0x7fc948072ce0 2024-05-09T08:08:00.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.800+0000 7fc94ca00640 1 -- 172.21.15.1:0/1937784143 <== mon.2 v2:172.21.15.7:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc944007cb0 con 0x7fc948072ce0 2024-05-09T08:08:00.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.800+0000 7fc950800640 1 -- 172.21.15.1:0/1937784143 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fc948072ce0 msgr2=0x7fc948073150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:08:00.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.800+0000 7fc950800640 1 --2- 172.21.15.1:0/1937784143 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fc948072ce0 0x7fc948073150 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fc944009d90 tx=0x7fc94402a660 comp rx=0 tx=0).stop 2024-05-09T08:08:00.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.801+0000 7fc950800640 1 -- 172.21.15.1:0/1937784143 shutdown_connections 2024-05-09T08:08:00.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.801+0000 7fc950800640 1 --2- 172.21.15.1:0/1937784143 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fc948072ce0 0x7fc948073150 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:00.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.801+0000 7fc950800640 1 --2- 172.21.15.1:0/1937784143 >> v2:172.21.15.1:3300/0 conn(0x7fc9481512f0 0x7fc9480c0ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:00.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.801+0000 7fc950800640 1 -- 172.21.15.1:0/1937784143 >> 172.21.15.1:0/1937784143 conn(0x7fc9480c89b0 msgr2=0x7fc9480c8dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:08:00.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.801+0000 7fc950800640 1 -- 172.21.15.1:0/1937784143 shutdown_connections 2024-05-09T08:08:00.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.801+0000 7fc950800640 1 -- 172.21.15.1:0/1937784143 wait complete. 2024-05-09T08:08:00.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.803+0000 7fc950800640 1 Processor -- start 2024-05-09T08:08:00.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.803+0000 7fc950800640 1 -- start start 2024-05-09T08:08:00.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.804+0000 7fc950800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc948072ce0 0x7fc94814f450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:08:00.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.804+0000 7fc950800640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc9480c94d0 0x7fc94814d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:08:00.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.804+0000 7fc950800640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fc9481512f0 0x7fc948148830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:08:00.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.804+0000 7fc950800640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fc948157ae0 con 0x7fc9480c94d0 2024-05-09T08:08:00.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.804+0000 7fc950800640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc948157800 con 0x7fc948072ce0 2024-05-09T08:08:00.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.804+0000 7fc950800640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7fc9480c6210 con 0x7fc9481512f0 2024-05-09T08:08:00.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.805+0000 7fc94d400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc9480c94d0 0x7fc94814d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:08:00.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.805+0000 7fc94d400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc9480c94d0 0x7fc94814d570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60662/0 (socket says 172.21.15.1:60662) 2024-05-09T08:08:00.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.805+0000 7fc94d400640 1 -- 172.21.15.1:0/916340060 learned_addr learned my addr 172.21.15.1:0/916340060 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:08:00.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.805+0000 7fc94e800640 1 --2- 172.21.15.1:0/916340060 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fc9481512f0 0x7fc948148830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:08:00.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.805+0000 7fc94de00640 1 --2- 172.21.15.1:0/916340060 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc948072ce0 0x7fc94814f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:08:00.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.805+0000 7fc94d400640 1 -- 172.21.15.1:0/916340060 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fc9481512f0 msgr2=0x7fc948148830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:08:00.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.805+0000 7fc94d400640 1 --2- 172.21.15.1:0/916340060 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fc9481512f0 0x7fc948148830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:00.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.805+0000 7fc94d400640 1 -- 172.21.15.1:0/916340060 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc948072ce0 msgr2=0x7fc94814f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:08:00.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.805+0000 7fc94d400640 1 --2- 172.21.15.1:0/916340060 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc948072ce0 0x7fc94814f450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:00.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.805+0000 7fc94d400640 1 -- 172.21.15.1:0/916340060 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9481e3f60 con 0x7fc9480c94d0 2024-05-09T08:08:00.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.806+0000 7fc94d400640 1 --2- 172.21.15.1:0/916340060 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc9480c94d0 0x7fc94814d570 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fc93800c390 tx=0x7fc93800c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:08:00.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.806+0000 7fc942000640 1 -- 172.21.15.1:0/916340060 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc938017070 con 0x7fc9480c94d0 2024-05-09T08:08:00.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.806+0000 7fc942000640 1 -- 172.21.15.1:0/916340060 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc938005ce0 con 0x7fc9480c94d0 2024-05-09T08:08:00.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.806+0000 7fc950800640 1 -- 172.21.15.1:0/916340060 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9481e4230 con 0x7fc9480c94d0 2024-05-09T08:08:00.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.806+0000 7fc942000640 1 -- 172.21.15.1:0/916340060 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9380072f0 con 0x7fc9480c94d0 2024-05-09T08:08:00.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.806+0000 7fc950800640 1 -- 172.21.15.1:0/916340060 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc94814ec50 con 0x7fc9480c94d0 2024-05-09T08:08:00.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.807+0000 7fc950800640 1 -- 172.21.15.1:0/916340060 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc94814f030 con 0x7fc9480c94d0 2024-05-09T08:08:00.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.808+0000 7fc942000640 1 -- 172.21.15.1:0/916340060 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc93801a070 con 0x7fc9480c94d0 2024-05-09T08:08:00.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.810+0000 7fc942000640 1 --2- 172.21.15.1:0/916340060 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fc92c06be30 0x7fc92c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:08:00.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.810+0000 7fc942000640 1 -- 172.21.15.1:0/916340060 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7fc93808a660 con 0x7fc9480c94d0 2024-05-09T08:08:00.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.810+0000 7fc94de00640 1 --2- 172.21.15.1:0/916340060 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fc92c06be30 0x7fc92c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:08:00.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.810+0000 7fc942000640 1 --2- 172.21.15.1:0/916340060 >> [v2:172.21.15.1:6808/2925031811,v1:172.21.15.1:6809/2925031811] conn(0x7fc92c0757d0 0x7fc92c077c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:08:00.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.810+0000 7fc942000640 1 -- 172.21.15.1:0/916340060 --> [v2:172.21.15.1:6808/2925031811,v1:172.21.15.1:6809/2925031811] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc938015e70 con 0x7fc92c0757d0 2024-05-09T08:08:00.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.810+0000 7fc942000640 1 -- 172.21.15.1:0/916340060 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc938091050 con 0x7fc9480c94d0 2024-05-09T08:08:00.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.810+0000 7fc94e800640 1 --2- 172.21.15.1:0/916340060 >> [v2:172.21.15.1:6808/2925031811,v1:172.21.15.1:6809/2925031811] conn(0x7fc92c0757d0 0x7fc92c077c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:08:00.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.811+0000 7fc94de00640 1 --2- 172.21.15.1:0/916340060 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fc92c06be30 0x7fc92c06e2f0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fc93c009560 tx=0x7fc93c009190 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:08:00.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.811+0000 7fc94e800640 1 --2- 172.21.15.1:0/916340060 >> [v2:172.21.15.1:6808/2925031811,v1:172.21.15.1:6809/2925031811] conn(0x7fc92c0757d0 0x7fc92c077c70 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:08:00.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.812+0000 7fc942000640 1 -- 172.21.15.1:0/916340060 <== osd.2 v2:172.21.15.1:6808/2925031811 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc938015e70 con 0x7fc92c0757d0 2024-05-09T08:08:00.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.827+0000 7fc950800640 1 -- 172.21.15.1:0/916340060 --> [v2:172.21.15.1:6808/2925031811,v1:172.21.15.1:6809/2925031811] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fc948068520 con 0x7fc92c0757d0 2024-05-09T08:08:00.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.828+0000 7fc942000640 1 -- 172.21.15.1:0/916340060 <== osd.2 v2:172.21.15.1:6808/2925031811 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fc948068520 con 0x7fc92c0757d0 2024-05-09T08:08:00.838 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-05-09T08:08:00.838 INFO:teuthology.orchestra.run.smithi001.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-09T08:08:00.838 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-05-09T08:08:00.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.828+0000 7fc950800640 1 -- 172.21.15.1:0/916340060 >> [v2:172.21.15.1:6808/2925031811,v1:172.21.15.1:6809/2925031811] conn(0x7fc92c0757d0 msgr2=0x7fc92c077c70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:08:00.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.829+0000 7fc950800640 1 --2- 172.21.15.1:0/916340060 >> [v2:172.21.15.1:6808/2925031811,v1:172.21.15.1:6809/2925031811] conn(0x7fc92c0757d0 0x7fc92c077c70 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:00.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.829+0000 7fc950800640 1 -- 172.21.15.1:0/916340060 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fc92c06be30 msgr2=0x7fc92c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:08:00.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.829+0000 7fc950800640 1 --2- 172.21.15.1:0/916340060 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fc92c06be30 0x7fc92c06e2f0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fc93c009560 tx=0x7fc93c009190 comp rx=0 tx=0).stop 2024-05-09T08:08:00.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.829+0000 7fc950800640 1 -- 172.21.15.1:0/916340060 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc9480c94d0 msgr2=0x7fc94814d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:08:00.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.829+0000 7fc950800640 1 --2- 172.21.15.1:0/916340060 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc9480c94d0 0x7fc94814d570 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fc93800c390 tx=0x7fc93800c850 comp rx=0 tx=0).stop 2024-05-09T08:08:00.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.829+0000 7fc94e800640 1 -- 172.21.15.1:0/916340060 reap_dead start 2024-05-09T08:08:00.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.829+0000 7fc950800640 1 -- 172.21.15.1:0/916340060 shutdown_connections 2024-05-09T08:08:00.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.829+0000 7fc950800640 1 -- 172.21.15.1:0/916340060 >> 172.21.15.1:0/916340060 conn(0x7fc9480c89b0 msgr2=0x7fc9480c7f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:08:00.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.829+0000 7fc950800640 1 -- 172.21.15.1:0/916340060 shutdown_connections 2024-05-09T08:08:00.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:00.829+0000 7fc950800640 1 -- 172.21.15.1:0/916340060 wait complete. 2024-05-09T08:08:00.856 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-05-09T08:08:01.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.062+0000 7fc05f200640 1 Processor -- start 2024-05-09T08:08:01.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.062+0000 7fc05f200640 1 -- start start 2024-05-09T08:08:01.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.064+0000 7fc05f200640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fc0600ccd10 0x7fc0600cf1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:08:01.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.064+0000 7fc05f200640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fc0600ce900 0x7fc0600cf730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:08:01.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.064+0000 7fc05f200640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc06015abd0 con 0x7fc06015a690 2024-05-09T08:08:01.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.064+0000 7fc05f200640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7fc060154e50 con 0x7fc0600ccd10 2024-05-09T08:08:01.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.064+0000 7fc05f200640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7fc0600cda80 con 0x7fc0600ce900 2024-05-09T08:08:01.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.064+0000 7fc05de00640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fc0600ccd10 0x7fc0600cf1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:08:01.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.064+0000 7fc05e800640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fc0600ce900 0x7fc0600cf730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:08:01.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.064+0000 7fc05de00640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fc0600ccd10 0x7fc0600cf1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60666/0 (socket says 172.21.15.1:60666) 2024-05-09T08:08:01.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.064+0000 7fc05de00640 1 -- 172.21.15.1:0/216613730 learned_addr learned my addr 172.21.15.1:0/216613730 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:08:01.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.065+0000 7fc05d400640 1 -- 172.21.15.1:0/216613730 <== mon.1 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2203346464 0 0) 0x7fc06015abd0 con 0x7fc06015a690 2024-05-09T08:08:01.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.065+0000 7fc05d400640 1 -- 172.21.15.1:0/216613730 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc044003520 con 0x7fc06015a690 2024-05-09T08:08:01.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.065+0000 7fc05e800640 1 -- 172.21.15.1:0/216613730 >> v2:172.21.15.1:3300/0 conn(0x7fc0600ccd10 msgr2=0x7fc0600cf1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:08:01.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.065+0000 7fc05e800640 1 --2- 172.21.15.1:0/216613730 >> v2:172.21.15.1:3300/0 conn(0x7fc0600ccd10 0x7fc0600cf1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:01.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.065+0000 7fc05e800640 1 -- 172.21.15.1:0/216613730 >> v1:172.21.15.7:6789/0 conn(0x7fc06015a690 legacy=0x7fc0600ce1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:08:01.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.065+0000 7fc05e800640 1 -- 172.21.15.1:0/216613730 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc06015aeb0 con 0x7fc0600ce900 2024-05-09T08:08:01.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.065+0000 7fc05de00640 1 --2- 172.21.15.1:0/216613730 >> v2:172.21.15.1:3300/0 conn(0x7fc0600ccd10 0x7fc0600cf1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:08:01.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.066+0000 7fc05e800640 1 --2- 172.21.15.1:0/216613730 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fc0600ce900 0x7fc0600cf730 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fc058009d90 tx=0x7fc05802a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aea60f70f5ea0a00 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:08:01.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.066+0000 7fc05d400640 1 -- 172.21.15.1:0/216613730 <== mon.2 v2:172.21.15.7:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc058007740 con 0x7fc0600ce900 2024-05-09T08:08:01.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.067+0000 7fc05d400640 1 -- 172.21.15.1:0/216613730 <== mon.2 v2:172.21.15.7:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0580078e0 con 0x7fc0600ce900 2024-05-09T08:08:01.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.067+0000 7fc05d400640 1 -- 172.21.15.1:0/216613730 <== mon.2 v2:172.21.15.7:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc058007c90 con 0x7fc0600ce900 2024-05-09T08:08:01.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.067+0000 7fc05f200640 1 -- 172.21.15.1:0/216613730 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fc0600ce900 msgr2=0x7fc0600cf730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:08:01.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.067+0000 7fc05f200640 1 --2- 172.21.15.1:0/216613730 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fc0600ce900 0x7fc0600cf730 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fc058009d90 tx=0x7fc05802a660 comp rx=0 tx=0).stop 2024-05-09T08:08:01.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.067+0000 7fc05f200640 1 -- 172.21.15.1:0/216613730 shutdown_connections 2024-05-09T08:08:01.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.067+0000 7fc05f200640 1 --2- 172.21.15.1:0/216613730 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fc0600ce900 0x7fc0600cf730 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:01.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.067+0000 7fc05f200640 1 --2- 172.21.15.1:0/216613730 >> v2:172.21.15.1:3300/0 conn(0x7fc0600ccd10 0x7fc0600cf1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:01.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.067+0000 7fc05f200640 1 -- 172.21.15.1:0/216613730 >> 172.21.15.1:0/216613730 conn(0x7fc0600b1790 msgr2=0x7fc0600b1ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:08:01.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.067+0000 7fc05f200640 1 -- 172.21.15.1:0/216613730 shutdown_connections 2024-05-09T08:08:01.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.068+0000 7fc05f200640 1 -- 172.21.15.1:0/216613730 wait complete. 2024-05-09T08:08:01.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.070+0000 7fc05f200640 1 Processor -- start 2024-05-09T08:08:01.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.070+0000 7fc05f200640 1 -- start start 2024-05-09T08:08:01.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.071+0000 7fc05f200640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fc0600ccd10 0x7fc060153530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:08:01.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.071+0000 7fc05f200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc0600ce900 0x7fc0601f11e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:08:01.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.071+0000 7fc05f200640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc06015a690 0x7fc0601f32e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:08:01.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.071+0000 7fc05f200640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fc06015b470 con 0x7fc06015a690 2024-05-09T08:08:01.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.071+0000 7fc05f200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc0600cdd60 con 0x7fc0600ce900 2024-05-09T08:08:01.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.071+0000 7fc05f200640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7fc0600ca600 con 0x7fc0600ccd10 2024-05-09T08:08:01.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.071+0000 7fc05de00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fc0600ccd10 0x7fc060153530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:08:01.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.072+0000 7fc05de00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fc0600ccd10 0x7fc060153530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3301/0 says I am v2:172.21.15.1:58778/0 (socket says 172.21.15.1:58778) 2024-05-09T08:08:01.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.072+0000 7fc05de00640 1 -- 172.21.15.1:0/3634540309 learned_addr learned my addr 172.21.15.1:0/3634540309 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:08:01.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.072+0000 7fc05e800640 1 --2- 172.21.15.1:0/3634540309 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc06015a690 0x7fc0601f32e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:08:01.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.072+0000 7fc057e00640 1 --2- 172.21.15.1:0/3634540309 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc0600ce900 0x7fc0601f11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:08:01.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.072+0000 7fc05de00640 1 -- 172.21.15.1:0/3634540309 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc0600ce900 msgr2=0x7fc0601f11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:08:01.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.072+0000 7fc05de00640 1 --2- 172.21.15.1:0/3634540309 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc0600ce900 0x7fc0601f11e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:01.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.072+0000 7fc05de00640 1 -- 172.21.15.1:0/3634540309 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc06015a690 msgr2=0x7fc0601f32e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:08:01.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.072+0000 7fc05de00640 1 --2- 172.21.15.1:0/3634540309 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc06015a690 0x7fc0601f32e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:01.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.072+0000 7fc05de00640 1 -- 172.21.15.1:0/3634540309 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0600c4a30 con 0x7fc0600ccd10 2024-05-09T08:08:01.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.072+0000 7fc05de00640 1 --2- 172.21.15.1:0/3634540309 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fc0600ccd10 0x7fc060153530 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fc05000ea00 tx=0x7fc05000eec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:08:01.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.073+0000 7fc056000640 1 -- 172.21.15.1:0/3634540309 <== mon.2 v2:172.21.15.7:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc050017070 con 0x7fc0600ccd10 2024-05-09T08:08:01.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.073+0000 7fc056000640 1 -- 172.21.15.1:0/3634540309 <== mon.2 v2:172.21.15.7:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc05000ca60 con 0x7fc0600ccd10 2024-05-09T08:08:01.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.073+0000 7fc056000640 1 -- 172.21.15.1:0/3634540309 <== mon.2 v2:172.21.15.7:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc05000ce10 con 0x7fc0600ccd10 2024-05-09T08:08:01.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.073+0000 7fc05f200640 1 -- 172.21.15.1:0/3634540309 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0600c4270 con 0x7fc0600ccd10 2024-05-09T08:08:01.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.073+0000 7fc05f200640 1 -- 172.21.15.1:0/3634540309 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0600c65b0 con 0x7fc0600ccd10 2024-05-09T08:08:01.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.074+0000 7fc05f200640 1 -- 172.21.15.1:0/3634540309 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0600b2320 con 0x7fc0600ccd10 2024-05-09T08:08:01.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.075+0000 7fc056000640 1 -- 172.21.15.1:0/3634540309 <== mon.2 v2:172.21.15.7:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc05001a070 con 0x7fc0600ccd10 2024-05-09T08:08:01.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.076+0000 7fc056000640 1 --2- 172.21.15.1:0/3634540309 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fc03c06be60 0x7fc03c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:08:01.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.076+0000 7fc056000640 1 -- 172.21.15.1:0/3634540309 <== mon.2 v2:172.21.15.7:3301/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7fc05008b1e0 con 0x7fc0600ccd10 2024-05-09T08:08:01.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.076+0000 7fc057e00640 1 --2- 172.21.15.1:0/3634540309 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fc03c06be60 0x7fc03c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:08:01.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.077+0000 7fc057e00640 1 --2- 172.21.15.1:0/3634540309 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fc03c06be60 0x7fc03c06e320 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fc048007fd0 tx=0x7fc048009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:08:01.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.081+0000 7fc056000640 1 -- 172.21.15.1:0/3634540309 <== mon.2 v2:172.21.15.7:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fc0600b2320 con 0x7fc0600ccd10 2024-05-09T08:08:01.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.284+0000 7fc05f200640 1 -- 172.21.15.1:0/3634540309 --> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) -- 0x7fc06006dfc0 con 0x7fc03c06be60 2024-05-09T08:08:01.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.285+0000 7fc056000640 1 -- 172.21.15.1:0/3634540309 <== mgr.4114 v2:172.21.15.7:6832/3562612464 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fc06006dfc0 con 0x7fc03c06be60 2024-05-09T08:08:01.296 INFO:teuthology.orchestra.run.smithi001.stderr:instructed osd(s) 2 to deep-scrub 2024-05-09T08:08:01.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.288+0000 7fc05f200640 1 -- 172.21.15.1:0/3634540309 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fc03c06be60 msgr2=0x7fc03c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:08:01.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.288+0000 7fc05f200640 1 --2- 172.21.15.1:0/3634540309 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fc03c06be60 0x7fc03c06e320 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fc048007fd0 tx=0x7fc048009040 comp rx=0 tx=0).stop 2024-05-09T08:08:01.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.288+0000 7fc05f200640 1 -- 172.21.15.1:0/3634540309 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fc0600ccd10 msgr2=0x7fc060153530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:08:01.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.288+0000 7fc05f200640 1 --2- 172.21.15.1:0/3634540309 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fc0600ccd10 0x7fc060153530 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fc05000ea00 tx=0x7fc05000eec0 comp rx=0 tx=0).stop 2024-05-09T08:08:01.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.288+0000 7fc05f200640 1 -- 172.21.15.1:0/3634540309 shutdown_connections 2024-05-09T08:08:01.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.288+0000 7fc05f200640 1 --2- 172.21.15.1:0/3634540309 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fc03c06be60 0x7fc03c06e320 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:01.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.288+0000 7fc05f200640 1 --2- 172.21.15.1:0/3634540309 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc0600ce900 0x7fc0601f11e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:01.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.288+0000 7fc05f200640 1 --2- 172.21.15.1:0/3634540309 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc06015a690 0x7fc0601f32e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:01.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.288+0000 7fc05f200640 1 --2- 172.21.15.1:0/3634540309 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fc0600ccd10 0x7fc060153530 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:01.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.288+0000 7fc05f200640 1 -- 172.21.15.1:0/3634540309 >> 172.21.15.1:0/3634540309 conn(0x7fc0600b1790 msgr2=0x7fc0600c6f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:08:01.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.288+0000 7fc05f200640 1 -- 172.21.15.1:0/3634540309 shutdown_connections 2024-05-09T08:08:01.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.288+0000 7fc05f200640 1 -- 172.21.15.1:0/3634540309 wait complete. 2024-05-09T08:08:01.319 INFO:tasks.ceph:Scrubbing osd.3 2024-05-09T08:08:01.319 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-05-09T08:08:01.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.526+0000 7fa5bb200640 1 Processor -- start 2024-05-09T08:08:01.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.526+0000 7fa5bb200640 1 -- start start 2024-05-09T08:08:01.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.528+0000 7fa5bb200640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa5bc1546c0 0x7fa5bc15aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:08:01.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.528+0000 7fa5bb200640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fa5bc0cb330 0x7fa5bc15dae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:08:01.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.528+0000 7fa5bb200640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa5bc15b3f0 con 0x7fa5bc0cb8f0 2024-05-09T08:08:01.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.528+0000 7fa5bb200640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7fa5bc0cacf0 con 0x7fa5bc0cb330 2024-05-09T08:08:01.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.528+0000 7fa5bb200640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fa5bc0ca6c0 con 0x7fa5bc1546c0 2024-05-09T08:08:01.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.528+0000 7fa5ba800640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fa5bc0cb330 0x7fa5bc15dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:08:01.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.528+0000 7fa5b9e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa5bc1546c0 0x7fa5bc15aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:08:01.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.528+0000 7fa5b9e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa5bc1546c0 0x7fa5bc15aeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.1:54134/0 (socket says 172.21.15.1:54134) 2024-05-09T08:08:01.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.528+0000 7fa5ba800640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fa5bc0cb330 0x7fa5bc15dae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60688/0 (socket says 172.21.15.1:60688) 2024-05-09T08:08:01.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.528+0000 7fa5b9e00640 1 -- 172.21.15.1:0/3731538488 learned_addr learned my addr 172.21.15.1:0/3731538488 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:08:01.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.529+0000 7fa5b9e00640 1 -- 172.21.15.1:0/3731538488 >> v2:172.21.15.1:3300/0 conn(0x7fa5bc0cb330 msgr2=0x7fa5bc15dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:08:01.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.529+0000 7fa5b9e00640 1 --2- 172.21.15.1:0/3731538488 >> v2:172.21.15.1:3300/0 conn(0x7fa5bc0cb330 0x7fa5bc15dae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:01.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.529+0000 7fa5b9e00640 1 -- 172.21.15.1:0/3731538488 >> v1:172.21.15.1:6789/0 conn(0x7fa5bc0cb8f0 legacy=0x7fa5bc0cfcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:08:01.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.529+0000 7fa5b9e00640 1 -- 172.21.15.1:0/3731538488 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5bc0cbe60 con 0x7fa5bc1546c0 2024-05-09T08:08:01.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.530+0000 7fa5b9e00640 1 --2- 172.21.15.1:0/3731538488 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5bc1546c0 0x7fa5bc15aeb0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fa5bc08d0f0 tx=0x7fa5a802a300 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a456b3c6df1140a3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:08:01.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.530+0000 7fa5b8a00640 1 -- 172.21.15.1:0/3731538488 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5a8037070 con 0x7fa5bc1546c0 2024-05-09T08:08:01.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.530+0000 7fa5b8a00640 1 -- 172.21.15.1:0/3731538488 <== mon.1 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5a802ae50 con 0x7fa5bc1546c0 2024-05-09T08:08:01.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.530+0000 7fa5b8a00640 1 -- 172.21.15.1:0/3731538488 <== mon.1 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5a8032860 con 0x7fa5bc1546c0 2024-05-09T08:08:01.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.531+0000 7fa5bb200640 1 -- 172.21.15.1:0/3731538488 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5bc1546c0 msgr2=0x7fa5bc15aeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:08:01.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.531+0000 7fa5bb200640 1 --2- 172.21.15.1:0/3731538488 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5bc1546c0 0x7fa5bc15aeb0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fa5bc08d0f0 tx=0x7fa5a802a300 comp rx=0 tx=0).stop 2024-05-09T08:08:01.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.531+0000 7fa5bb200640 1 -- 172.21.15.1:0/3731538488 shutdown_connections 2024-05-09T08:08:01.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.531+0000 7fa5bb200640 1 --2- 172.21.15.1:0/3731538488 >> v2:172.21.15.1:3300/0 conn(0x7fa5bc0cb330 0x7fa5bc15dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:01.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.531+0000 7fa5bb200640 1 --2- 172.21.15.1:0/3731538488 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5bc1546c0 0x7fa5bc15aeb0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:01.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.531+0000 7fa5bb200640 1 -- 172.21.15.1:0/3731538488 >> 172.21.15.1:0/3731538488 conn(0x7fa5bc066ed0 msgr2=0x7fa5bc0672e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:08:01.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.531+0000 7fa5bb200640 1 -- 172.21.15.1:0/3731538488 shutdown_connections 2024-05-09T08:08:01.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.531+0000 7fa5bb200640 1 -- 172.21.15.1:0/3731538488 wait complete. 2024-05-09T08:08:01.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.533+0000 7fa5bb200640 1 Processor -- start 2024-05-09T08:08:01.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.533+0000 7fa5bb200640 1 -- start start 2024-05-09T08:08:01.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.534+0000 7fa5bb200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5bc0cb330 0x7fa5bc14aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:08:01.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.534+0000 7fa5bb200640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa5bc0cb8f0 0x7fa5bc159b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:08:01.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.534+0000 7fa5bb200640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fa5bc1546c0 0x7fa5bc154d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:08:01.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.534+0000 7fa5bb200640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fa5bc0ca980 con 0x7fa5bc0cb8f0 2024-05-09T08:08:01.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.534+0000 7fa5bb200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa5bc0cc420 con 0x7fa5bc0cb330 2024-05-09T08:08:01.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.534+0000 7fa5bb200640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7fa5bc154aa0 con 0x7fa5bc1546c0 2024-05-09T08:08:01.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.535+0000 7fa5b9e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5bc0cb330 0x7fa5bc14aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:08:01.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.535+0000 7fa5b9e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5bc0cb330 0x7fa5bc14aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.1:54136/0 (socket says 172.21.15.1:54136) 2024-05-09T08:08:01.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.535+0000 7fa5b9e00640 1 -- 172.21.15.1:0/3344564923 learned_addr learned my addr 172.21.15.1:0/3344564923 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:08:01.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.535+0000 7fa5b9400640 1 --2- 172.21.15.1:0/3344564923 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa5bc0cb8f0 0x7fa5bc159b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:08:01.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.535+0000 7fa5ba800640 1 --2- 172.21.15.1:0/3344564923 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fa5bc1546c0 0x7fa5bc154d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:08:01.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.535+0000 7fa5b9e00640 1 -- 172.21.15.1:0/3344564923 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fa5bc1546c0 msgr2=0x7fa5bc154d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:08:01.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.535+0000 7fa5b9e00640 1 --2- 172.21.15.1:0/3344564923 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fa5bc1546c0 0x7fa5bc154d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:01.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.535+0000 7fa5b9e00640 1 -- 172.21.15.1:0/3344564923 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa5bc0cb8f0 msgr2=0x7fa5bc159b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:08:01.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.535+0000 7fa5b9e00640 1 --2- 172.21.15.1:0/3344564923 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa5bc0cb8f0 0x7fa5bc159b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:01.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.535+0000 7fa5b9e00640 1 -- 172.21.15.1:0/3344564923 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5bc156df0 con 0x7fa5bc0cb330 2024-05-09T08:08:01.547 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.536+0000 7fa5b9e00640 1 --2- 172.21.15.1:0/3344564923 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5bc0cb330 0x7fa5bc14aa70 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fa5a802a340 tx=0x7fa5a80046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:08:01.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.536+0000 7fa5b2000640 1 -- 172.21.15.1:0/3344564923 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5a8041070 con 0x7fa5bc0cb330 2024-05-09T08:08:01.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.536+0000 7fa5b2000640 1 -- 172.21.15.1:0/3344564923 <== mon.1 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5a803c680 con 0x7fa5bc0cb330 2024-05-09T08:08:01.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.536+0000 7fa5b2000640 1 -- 172.21.15.1:0/3344564923 <== mon.1 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5a8037040 con 0x7fa5bc0cb330 2024-05-09T08:08:01.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.536+0000 7fa5bb200640 1 -- 172.21.15.1:0/3344564923 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5bc14afb0 con 0x7fa5bc0cb330 2024-05-09T08:08:01.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.536+0000 7fa5bb200640 1 -- 172.21.15.1:0/3344564923 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5bc157060 con 0x7fa5bc0cb330 2024-05-09T08:08:01.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.537+0000 7fa5bb200640 1 -- 172.21.15.1:0/3344564923 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa5bc074d70 con 0x7fa5bc0cb330 2024-05-09T08:08:01.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.538+0000 7fa5b2000640 1 -- 172.21.15.1:0/3344564923 <== mon.1 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa5a8044070 con 0x7fa5bc0cb330 2024-05-09T08:08:01.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.539+0000 7fa5b2000640 1 --2- 172.21.15.1:0/3344564923 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fa59c06be30 0x7fa59c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:08:01.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.539+0000 7fa5b2000640 1 -- 172.21.15.1:0/3344564923 <== mon.1 v2:172.21.15.7:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7fa5a80abb00 con 0x7fa5bc0cb330 2024-05-09T08:08:01.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.539+0000 7fa5b2000640 1 --2- 172.21.15.1:0/3344564923 >> [v2:172.21.15.1:6824/2497776654,v1:172.21.15.1:6825/2497776654] conn(0x7fa59c0757d0 0x7fa59c077c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:08:01.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.539+0000 7fa5b2000640 1 -- 172.21.15.1:0/3344564923 --> [v2:172.21.15.1:6824/2497776654,v1:172.21.15.1:6825/2497776654] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa59c078340 con 0x7fa59c0757d0 2024-05-09T08:08:01.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.539+0000 7fa5b2000640 1 -- 172.21.15.1:0/3344564923 <== mon.1 v2:172.21.15.7:3300/0 6 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa5a80b02a0 con 0x7fa5bc0cb330 2024-05-09T08:08:01.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.540+0000 7fa5b9400640 1 --2- 172.21.15.1:0/3344564923 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fa59c06be30 0x7fa59c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:08:01.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.540+0000 7fa5ba800640 1 --2- 172.21.15.1:0/3344564923 >> [v2:172.21.15.1:6824/2497776654,v1:172.21.15.1:6825/2497776654] conn(0x7fa59c0757d0 0x7fa59c077c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:08:01.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.540+0000 7fa5b9400640 1 --2- 172.21.15.1:0/3344564923 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fa59c06be30 0x7fa59c06e2f0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fa5bc0b5520 tx=0x7fa5ac009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:08:01.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.540+0000 7fa5ba800640 1 --2- 172.21.15.1:0/3344564923 >> [v2:172.21.15.1:6824/2497776654,v1:172.21.15.1:6825/2497776654] conn(0x7fa59c0757d0 0x7fa59c077c70 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:08:01.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.541+0000 7fa5b2000640 1 -- 172.21.15.1:0/3344564923 <== osd.3 v2:172.21.15.1:6824/2497776654 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa59c078340 con 0x7fa59c0757d0 2024-05-09T08:08:01.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.556+0000 7fa5bb200640 1 -- 172.21.15.1:0/3344564923 --> [v2:172.21.15.1:6824/2497776654,v1:172.21.15.1:6825/2497776654] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fa5bc0c5ec0 con 0x7fa59c0757d0 2024-05-09T08:08:01.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.557+0000 7fa5b2000640 1 -- 172.21.15.1:0/3344564923 <== osd.3 v2:172.21.15.1:6824/2497776654 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fa5bc0c5ec0 con 0x7fa59c0757d0 2024-05-09T08:08:01.567 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-05-09T08:08:01.567 INFO:teuthology.orchestra.run.smithi001.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-09T08:08:01.567 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-05-09T08:08:01.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.557+0000 7fa5bb200640 1 -- 172.21.15.1:0/3344564923 >> [v2:172.21.15.1:6824/2497776654,v1:172.21.15.1:6825/2497776654] conn(0x7fa59c0757d0 msgr2=0x7fa59c077c70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:08:01.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.557+0000 7fa5bb200640 1 --2- 172.21.15.1:0/3344564923 >> [v2:172.21.15.1:6824/2497776654,v1:172.21.15.1:6825/2497776654] conn(0x7fa59c0757d0 0x7fa59c077c70 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:01.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.557+0000 7fa5bb200640 1 -- 172.21.15.1:0/3344564923 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fa59c06be30 msgr2=0x7fa59c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:08:01.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.557+0000 7fa5bb200640 1 --2- 172.21.15.1:0/3344564923 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fa59c06be30 0x7fa59c06e2f0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fa5bc0b5520 tx=0x7fa5ac009040 comp rx=0 tx=0).stop 2024-05-09T08:08:01.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.557+0000 7fa5bb200640 1 -- 172.21.15.1:0/3344564923 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5bc0cb330 msgr2=0x7fa5bc14aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:08:01.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.557+0000 7fa5bb200640 1 --2- 172.21.15.1:0/3344564923 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5bc0cb330 0x7fa5bc14aa70 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fa5a802a340 tx=0x7fa5a80046c0 comp rx=0 tx=0).stop 2024-05-09T08:08:01.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.557+0000 7fa5ba800640 1 -- 172.21.15.1:0/3344564923 reap_dead start 2024-05-09T08:08:01.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.557+0000 7fa5bb200640 1 -- 172.21.15.1:0/3344564923 shutdown_connections 2024-05-09T08:08:01.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.557+0000 7fa5bb200640 1 -- 172.21.15.1:0/3344564923 >> 172.21.15.1:0/3344564923 conn(0x7fa5bc066ed0 msgr2=0x7fa5bc0c5650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:08:01.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.558+0000 7fa5bb200640 1 -- 172.21.15.1:0/3344564923 shutdown_connections 2024-05-09T08:08:01.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.558+0000 7fa5bb200640 1 -- 172.21.15.1:0/3344564923 wait complete. 2024-05-09T08:08:01.587 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-05-09T08:08:01.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.791+0000 7f7638c00640 1 Processor -- start 2024-05-09T08:08:01.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.791+0000 7f7638c00640 1 -- start start 2024-05-09T08:08:01.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.792+0000 7f7638c00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f76341513a0 0x7f76340c3070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:08:01.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.792+0000 7f7638c00640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f76340c5df0 0x7f7634157240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:08:01.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.792+0000 7f7638c00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f76340c5350 con 0x7f76340c7b10 2024-05-09T08:08:01.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.792+0000 7f7638c00640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f76340c3670 con 0x7f76340c5df0 2024-05-09T08:08:01.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.793+0000 7f7638c00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f76340c3950 con 0x7f76341513a0 2024-05-09T08:08:01.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.793+0000 7f762aa00640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f76340c5df0 0x7f7634157240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:08:01.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.793+0000 7f762aa00640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f76340c5df0 0x7f7634157240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60710/0 (socket says 172.21.15.1:60710) 2024-05-09T08:08:01.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.793+0000 7f762aa00640 1 -- 172.21.15.1:0/2002270560 learned_addr learned my addr 172.21.15.1:0/2002270560 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:08:01.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.793+0000 7f7633400640 1 --2- 172.21.15.1:0/2002270560 >> v2:172.21.15.7:3300/0 conn(0x7f76341513a0 0x7f76340c3070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:08:01.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.793+0000 7f762aa00640 1 -- 172.21.15.1:0/2002270560 >> v2:172.21.15.7:3300/0 conn(0x7f76341513a0 msgr2=0x7f76340c3070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:08:01.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.793+0000 7f762aa00640 1 --2- 172.21.15.1:0/2002270560 >> v2:172.21.15.7:3300/0 conn(0x7f76341513a0 0x7f76340c3070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:01.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.793+0000 7f762aa00640 1 -- 172.21.15.1:0/2002270560 >> v1:172.21.15.7:6789/0 conn(0x7f76340c7b10 legacy=0x7f76340c8600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:08:01.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.794+0000 7f762aa00640 1 -- 172.21.15.1:0/2002270560 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76340c55f0 con 0x7f76340c5df0 2024-05-09T08:08:01.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.794+0000 7f7632a00640 1 -- 172.21.15.1:0/2002270560 <== mon.1 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3493691042 0 0) 0x7f76340c5350 con 0x7f76340c7b10 2024-05-09T08:08:01.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.795+0000 7f762aa00640 1 --2- 172.21.15.1:0/2002270560 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f76340c5df0 0x7f7634157240 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f7634070250 tx=0x7f761c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82edb578fc95b1d2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:08:01.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.795+0000 7f7632a00640 1 -- 172.21.15.1:0/2002270560 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f761c037070 con 0x7f76340c5df0 2024-05-09T08:08:01.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.795+0000 7f7632a00640 1 -- 172.21.15.1:0/2002270560 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f761c032440 con 0x7f76340c5df0 2024-05-09T08:08:01.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.795+0000 7f7632a00640 1 -- 172.21.15.1:0/2002270560 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f761c032810 con 0x7f76340c5df0 2024-05-09T08:08:01.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.795+0000 7f7638c00640 1 -- 172.21.15.1:0/2002270560 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f76340c5df0 msgr2=0x7f7634157240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:08:01.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.795+0000 7f7638c00640 1 --2- 172.21.15.1:0/2002270560 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f76340c5df0 0x7f7634157240 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f7634070250 tx=0x7f761c02a270 comp rx=0 tx=0).stop 2024-05-09T08:08:01.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.796+0000 7f7638c00640 1 -- 172.21.15.1:0/2002270560 shutdown_connections 2024-05-09T08:08:01.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.796+0000 7f7638c00640 1 --2- 172.21.15.1:0/2002270560 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f76340c5df0 0x7f7634157240 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:01.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.796+0000 7f7638c00640 1 --2- 172.21.15.1:0/2002270560 >> v2:172.21.15.7:3300/0 conn(0x7f76341513a0 0x7f76340c3070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:01.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.796+0000 7f7638c00640 1 -- 172.21.15.1:0/2002270560 >> 172.21.15.1:0/2002270560 conn(0x7f763406b250 msgr2=0x7f763406b630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:08:01.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.796+0000 7f7638c00640 1 -- 172.21.15.1:0/2002270560 shutdown_connections 2024-05-09T08:08:01.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.796+0000 7f7638c00640 1 -- 172.21.15.1:0/2002270560 wait complete. 2024-05-09T08:08:01.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.798+0000 7f7638c00640 1 Processor -- start 2024-05-09T08:08:01.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.798+0000 7f7638c00640 1 -- start start 2024-05-09T08:08:01.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.799+0000 7f7638c00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f76340c5df0 0x7f763414d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:08:01.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.799+0000 7f7638c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f76340c7b10 0x7f76341f0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:08:01.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.799+0000 7f7638c00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f76341513a0 0x7f763414af20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:08:01.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.799+0000 7f7638c00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f76340c3db0 con 0x7f76340c5df0 2024-05-09T08:08:01.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.799+0000 7f7638c00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f76340c3c30 con 0x7f76340c7b10 2024-05-09T08:08:01.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.799+0000 7f7638c00640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f76340c8d10 con 0x7f76341513a0 2024-05-09T08:08:01.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.799+0000 7f7633400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f76340c5df0 0x7f763414d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:08:01.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.799+0000 7f7633400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f76340c5df0 0x7f763414d0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60726/0 (socket says 172.21.15.1:60726) 2024-05-09T08:08:01.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.799+0000 7f7633400640 1 -- 172.21.15.1:0/2264256052 learned_addr learned my addr 172.21.15.1:0/2264256052 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:08:01.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.799+0000 7f762aa00640 1 --2- 172.21.15.1:0/2264256052 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f76340c7b10 0x7f76341f0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:08:01.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.799+0000 7f7633e00640 1 --2- 172.21.15.1:0/2264256052 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f76341513a0 0x7f763414af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:08:01.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.799+0000 7f7633400640 1 -- 172.21.15.1:0/2264256052 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f76341513a0 msgr2=0x7f763414af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:08:01.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.799+0000 7f7633400640 1 --2- 172.21.15.1:0/2264256052 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f76341513a0 0x7f763414af20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:01.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.800+0000 7f7633400640 1 -- 172.21.15.1:0/2264256052 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f76340c7b10 msgr2=0x7f76341f0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:08:01.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.800+0000 7f7633400640 1 --2- 172.21.15.1:0/2264256052 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f76340c7b10 0x7f76341f0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:01.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.800+0000 7f7633400640 1 -- 172.21.15.1:0/2264256052 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76340bfc10 con 0x7f76340c5df0 2024-05-09T08:08:01.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.800+0000 7f7633400640 1 --2- 172.21.15.1:0/2264256052 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f76340c5df0 0x7f763414d0e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f7624009db0 tx=0x7f762400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:08:01.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.800+0000 7f762be00640 1 -- 172.21.15.1:0/2264256052 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7624018020 con 0x7f76340c5df0 2024-05-09T08:08:01.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.800+0000 7f762be00640 1 -- 172.21.15.1:0/2264256052 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7624014480 con 0x7f76340c5df0 2024-05-09T08:08:01.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.800+0000 7f762be00640 1 -- 172.21.15.1:0/2264256052 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7624014850 con 0x7f76340c5df0 2024-05-09T08:08:01.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.800+0000 7f7638c00640 1 -- 172.21.15.1:0/2264256052 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76340bfef0 con 0x7f76340c5df0 2024-05-09T08:08:01.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.800+0000 7f7638c00640 1 -- 172.21.15.1:0/2264256052 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76341efe50 con 0x7f76340c5df0 2024-05-09T08:08:01.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.801+0000 7f7638c00640 1 -- 172.21.15.1:0/2264256052 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76340cca50 con 0x7f76340c5df0 2024-05-09T08:08:01.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.802+0000 7f762be00640 1 -- 172.21.15.1:0/2264256052 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7624005d10 con 0x7f76340c5df0 2024-05-09T08:08:01.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.803+0000 7f762be00640 1 --2- 172.21.15.1:0/2264256052 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f761006be30 0x7f761006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:08:01.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.803+0000 7f762be00640 1 -- 172.21.15.1:0/2264256052 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7f762408c050 con 0x7f76340c5df0 2024-05-09T08:08:01.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.804+0000 7f762aa00640 1 --2- 172.21.15.1:0/2264256052 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f761006be30 0x7f761006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:08:01.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.804+0000 7f762aa00640 1 --2- 172.21.15.1:0/2264256052 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f761006be30 0x7f761006e2f0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f761c034000 tx=0x7f761c02e000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:08:01.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:01.808+0000 7f762be00640 1 -- 172.21.15.1:0/2264256052 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f76340cca50 con 0x7f76340c5df0 2024-05-09T08:08:02.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.010+0000 7f7638c00640 1 -- 172.21.15.1:0/2264256052 --> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) -- 0x7f763404f660 con 0x7f761006be30 2024-05-09T08:08:02.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.011+0000 7f762be00640 1 -- 172.21.15.1:0/2264256052 <== mgr.4114 v2:172.21.15.7:6832/3562612464 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f763404f660 con 0x7f761006be30 2024-05-09T08:08:02.021 INFO:teuthology.orchestra.run.smithi001.stderr:instructed osd(s) 3 to deep-scrub 2024-05-09T08:08:02.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.014+0000 7f7638c00640 1 -- 172.21.15.1:0/2264256052 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f761006be30 msgr2=0x7f761006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:08:02.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.014+0000 7f7638c00640 1 --2- 172.21.15.1:0/2264256052 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f761006be30 0x7f761006e2f0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f761c034000 tx=0x7f761c02e000 comp rx=0 tx=0).stop 2024-05-09T08:08:02.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.014+0000 7f7638c00640 1 -- 172.21.15.1:0/2264256052 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f76340c5df0 msgr2=0x7f763414d0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:08:02.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.014+0000 7f7638c00640 1 --2- 172.21.15.1:0/2264256052 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f76340c5df0 0x7f763414d0e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f7624009db0 tx=0x7f762400c490 comp rx=0 tx=0).stop 2024-05-09T08:08:02.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.014+0000 7f7638c00640 1 -- 172.21.15.1:0/2264256052 shutdown_connections 2024-05-09T08:08:02.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.014+0000 7f7638c00640 1 --2- 172.21.15.1:0/2264256052 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f761006be30 0x7f761006e2f0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:02.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.014+0000 7f7638c00640 1 --2- 172.21.15.1:0/2264256052 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f76340c7b10 0x7f76341f0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:02.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.014+0000 7f7638c00640 1 --2- 172.21.15.1:0/2264256052 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f76341513a0 0x7f763414af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:02.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.014+0000 7f7638c00640 1 --2- 172.21.15.1:0/2264256052 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f76340c5df0 0x7f763414d0e0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:02.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.014+0000 7f7638c00640 1 -- 172.21.15.1:0/2264256052 >> 172.21.15.1:0/2264256052 conn(0x7f763406b250 msgr2=0x7f7634071e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:08:02.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.015+0000 7f7638c00640 1 -- 172.21.15.1:0/2264256052 shutdown_connections 2024-05-09T08:08:02.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.015+0000 7f7638c00640 1 -- 172.21.15.1:0/2264256052 wait complete. 2024-05-09T08:08:02.046 INFO:tasks.ceph:Scrubbing osd.4 2024-05-09T08:08:02.046 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-05-09T08:08:02.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.257+0000 7f6983200640 1 Processor -- start 2024-05-09T08:08:02.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.257+0000 7f6983200640 1 -- start start 2024-05-09T08:08:02.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.258+0000 7f6983200640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f69840c9fd0 0x7f69841542c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:08:02.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.258+0000 7f6983200640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f698415e510 con 0x7f698406f0e0 2024-05-09T08:08:02.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.258+0000 7f6983200640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f698415e7f0 con 0x7f69840c3f50 2024-05-09T08:08:02.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.258+0000 7f6983200640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f69840c4b40 con 0x7f69840c9fd0 2024-05-09T08:08:02.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.258+0000 7f6981400640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f698406f0e0 0x7f69840c4330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:33676/0 (socket says 172.21.15.1:33676) 2024-05-09T08:08:02.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.258+0000 7f6981400640 1 -- 172.21.15.1:0/2093500497 learned_addr learned my addr 172.21.15.1:0/2093500497 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T08:08:02.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.259+0000 7f6982800640 1 --2- 172.21.15.1:0/2093500497 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f69840c9fd0 0x7f69841542c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:08:02.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.259+0000 7f6980a00640 1 -- 172.21.15.1:0/2093500497 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 271223691 0 0) 0x7f698415e510 con 0x7f698406f0e0 2024-05-09T08:08:02.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.259+0000 7f6980a00640 1 -- 172.21.15.1:0/2093500497 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6968003520 con 0x7f698406f0e0 2024-05-09T08:08:02.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.259+0000 7f6980a00640 1 -- 172.21.15.1:0/2093500497 <== mon.1 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1526354721 0 0) 0x7f698415e7f0 con 0x7f69840c3f50 2024-05-09T08:08:02.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.259+0000 7f6980a00640 1 -- 172.21.15.1:0/2093500497 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f698415e510 con 0x7f69840c3f50 2024-05-09T08:08:02.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.259+0000 7f6982800640 1 -- 172.21.15.1:0/2093500497 >> v1:172.21.15.7:6789/0 conn(0x7f69840c3f50 legacy=0x7f69840c92b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:08:02.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.259+0000 7f6982800640 1 -- 172.21.15.1:0/2093500497 >> v1:172.21.15.1:6789/0 conn(0x7f698406f0e0 legacy=0x7f69840c4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:08:02.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.259+0000 7f6982800640 1 -- 172.21.15.1:0/2093500497 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69840c87d0 con 0x7f69840c9fd0 2024-05-09T08:08:02.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.260+0000 7f6982800640 1 --2- 172.21.15.1:0/2093500497 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f69840c9fd0 0x7f69841542c0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f697c0097d0 tx=0x7f697c02a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a518f566fecd5b50 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:08:02.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.261+0000 7f6980a00640 1 -- 172.21.15.1:0/2093500497 <== mon.2 v2:172.21.15.7:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f697c037070 con 0x7f69840c9fd0 2024-05-09T08:08:02.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.261+0000 7f6980a00640 1 -- 172.21.15.1:0/2093500497 <== mon.2 v2:172.21.15.7:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f697c032440 con 0x7f69840c9fd0 2024-05-09T08:08:02.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.261+0000 7f6980a00640 1 -- 172.21.15.1:0/2093500497 <== mon.2 v2:172.21.15.7:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f697c032810 con 0x7f69840c9fd0 2024-05-09T08:08:02.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.261+0000 7f6983200640 1 -- 172.21.15.1:0/2093500497 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f69840c9fd0 msgr2=0x7f69841542c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:08:02.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.261+0000 7f6983200640 1 --2- 172.21.15.1:0/2093500497 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f69840c9fd0 0x7f69841542c0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f697c0097d0 tx=0x7f697c02a210 comp rx=0 tx=0).stop 2024-05-09T08:08:02.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.261+0000 7f6983200640 1 -- 172.21.15.1:0/2093500497 shutdown_connections 2024-05-09T08:08:02.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.261+0000 7f6983200640 1 --2- 172.21.15.1:0/2093500497 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f69840c9fd0 0x7f69841542c0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:02.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.261+0000 7f6983200640 1 -- 172.21.15.1:0/2093500497 >> 172.21.15.1:0/2093500497 conn(0x7f69840cddd0 msgr2=0x7f69840ce1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:08:02.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.261+0000 7f6983200640 1 -- 172.21.15.1:0/2093500497 shutdown_connections 2024-05-09T08:08:02.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.262+0000 7f6983200640 1 -- 172.21.15.1:0/2093500497 wait complete. 2024-05-09T08:08:02.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.263+0000 7f6983200640 1 Processor -- start 2024-05-09T08:08:02.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.263+0000 7f6983200640 1 -- start start 2024-05-09T08:08:02.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.264+0000 7f6983200640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f698406f0e0 0x7f69841505b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:08:02.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.264+0000 7f6983200640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f69840c3f50 0x7f69840c5dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:08:02.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.264+0000 7f6983200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f69840c9fd0 0x7f69840c6310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:08:02.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.264+0000 7f6983200640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f69840ca520 con 0x7f69840c3f50 2024-05-09T08:08:02.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.264+0000 7f6983200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f69840c8e20 con 0x7f69840c9fd0 2024-05-09T08:08:02.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.264+0000 7f6983200640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f69840ca6a0 con 0x7f698406f0e0 2024-05-09T08:08:02.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.265+0000 7f6981400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f69840c3f50 0x7f69840c5dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:08:02.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.265+0000 7f6981e00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f698406f0e0 0x7f69841505b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:08:02.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.265+0000 7f6981e00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f698406f0e0 0x7f69841505b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3301/0 says I am v2:172.21.15.1:58796/0 (socket says 172.21.15.1:58796) 2024-05-09T08:08:02.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.265+0000 7f6981400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f69840c3f50 0x7f69840c5dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60742/0 (socket says 172.21.15.1:60742) 2024-05-09T08:08:02.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.265+0000 7f6981e00640 1 -- 172.21.15.1:0/4236282332 learned_addr learned my addr 172.21.15.1:0/4236282332 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:08:02.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.265+0000 7f6982800640 1 --2- 172.21.15.1:0/4236282332 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f69840c9fd0 0x7f69840c6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:08:02.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.265+0000 7f6981e00640 1 -- 172.21.15.1:0/4236282332 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f69840c9fd0 msgr2=0x7f69840c6310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:08:02.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.265+0000 7f6981e00640 1 --2- 172.21.15.1:0/4236282332 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f69840c9fd0 0x7f69840c6310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:02.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.265+0000 7f6981e00640 1 -- 172.21.15.1:0/4236282332 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f69840c3f50 msgr2=0x7f69840c5dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:08:02.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.265+0000 7f6981e00640 1 --2- 172.21.15.1:0/4236282332 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f69840c3f50 0x7f69840c5dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:02.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.265+0000 7f6981e00640 1 -- 172.21.15.1:0/4236282332 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69840c5610 con 0x7f698406f0e0 2024-05-09T08:08:02.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.265+0000 7f6981400640 1 --2- 172.21.15.1:0/4236282332 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f69840c3f50 0x7f69840c5dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:08:02.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.266+0000 7f6981e00640 1 --2- 172.21.15.1:0/4236282332 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f698406f0e0 0x7f69841505b0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f697400c7f0 tx=0x7f697400ccb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:08:02.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.266+0000 7f697a000640 1 -- 172.21.15.1:0/4236282332 <== mon.2 v2:172.21.15.7:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6974003b70 con 0x7f698406f0e0 2024-05-09T08:08:02.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.266+0000 7f697a000640 1 -- 172.21.15.1:0/4236282332 <== mon.2 v2:172.21.15.7:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6974003d10 con 0x7f698406f0e0 2024-05-09T08:08:02.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.266+0000 7f697a000640 1 -- 172.21.15.1:0/4236282332 <== mon.2 v2:172.21.15.7:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f697400aa90 con 0x7f698406f0e0 2024-05-09T08:08:02.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.266+0000 7f6983200640 1 -- 172.21.15.1:0/4236282332 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6984158ce0 con 0x7f698406f0e0 2024-05-09T08:08:02.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.266+0000 7f6983200640 1 -- 172.21.15.1:0/4236282332 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f6984158fa0 con 0x7f698406f0e0 2024-05-09T08:08:02.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.267+0000 7f6983200640 1 -- 172.21.15.1:0/4236282332 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f69840c59b0 con 0x7f698406f0e0 2024-05-09T08:08:02.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.268+0000 7f697a000640 1 -- 172.21.15.1:0/4236282332 <== mon.2 v2:172.21.15.7:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f697400ac30 con 0x7f698406f0e0 2024-05-09T08:08:02.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.269+0000 7f697a000640 1 --2- 172.21.15.1:0/4236282332 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f696406be30 0x7f696406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:08:02.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.269+0000 7f697a000640 1 -- 172.21.15.1:0/4236282332 <== mon.2 v2:172.21.15.7:3301/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7f697408b2f0 con 0x7f698406f0e0 2024-05-09T08:08:02.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.269+0000 7f697a000640 1 --2- 172.21.15.1:0/4236282332 >> [v2:172.21.15.7:6800/2957026670,v1:172.21.15.7:6801/2957026670] conn(0x7f69640757c0 0x7f6964077c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:08:02.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.269+0000 7f6981400640 1 --2- 172.21.15.1:0/4236282332 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f696406be30 0x7f696406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:08:02.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.269+0000 7f697a000640 1 -- 172.21.15.1:0/4236282332 --> [v2:172.21.15.7:6800/2957026670,v1:172.21.15.7:6801/2957026670] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6974004ec0 con 0x7f69640757c0 2024-05-09T08:08:02.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.269+0000 7f697a000640 1 -- 172.21.15.1:0/4236282332 <== mon.2 v2:172.21.15.7:3301/0 6 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6974092050 con 0x7f698406f0e0 2024-05-09T08:08:02.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.270+0000 7f6982800640 1 --2- 172.21.15.1:0/4236282332 >> [v2:172.21.15.7:6800/2957026670,v1:172.21.15.7:6801/2957026670] conn(0x7f69640757c0 0x7f6964077c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:08:02.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.270+0000 7f6981400640 1 --2- 172.21.15.1:0/4236282332 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f696406be30 0x7f696406e2f0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f6984150e60 tx=0x7f6970007d70 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:08:02.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.270+0000 7f6982800640 1 --2- 172.21.15.1:0/4236282332 >> [v2:172.21.15.7:6800/2957026670,v1:172.21.15.7:6801/2957026670] conn(0x7f69640757c0 0x7f6964077c60 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:08:02.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.271+0000 7f697a000640 1 -- 172.21.15.1:0/4236282332 <== osd.4 v2:172.21.15.7:6800/2957026670 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6974004ec0 con 0x7f69640757c0 2024-05-09T08:08:02.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.286+0000 7f6983200640 1 -- 172.21.15.1:0/4236282332 --> [v2:172.21.15.7:6800/2957026670,v1:172.21.15.7:6801/2957026670] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f6984074d70 con 0x7f69640757c0 2024-05-09T08:08:02.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.287+0000 7f697a000640 1 -- 172.21.15.1:0/4236282332 <== osd.4 v2:172.21.15.7:6800/2957026670 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f6984074d70 con 0x7f69640757c0 2024-05-09T08:08:02.297 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-05-09T08:08:02.297 INFO:teuthology.orchestra.run.smithi001.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-09T08:08:02.297 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-05-09T08:08:02.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.287+0000 7f6983200640 1 -- 172.21.15.1:0/4236282332 >> [v2:172.21.15.7:6800/2957026670,v1:172.21.15.7:6801/2957026670] conn(0x7f69640757c0 msgr2=0x7f6964077c60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:08:02.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.287+0000 7f6983200640 1 --2- 172.21.15.1:0/4236282332 >> [v2:172.21.15.7:6800/2957026670,v1:172.21.15.7:6801/2957026670] conn(0x7f69640757c0 0x7f6964077c60 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:02.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.288+0000 7f6983200640 1 -- 172.21.15.1:0/4236282332 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f696406be30 msgr2=0x7f696406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:08:02.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.288+0000 7f6983200640 1 --2- 172.21.15.1:0/4236282332 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f696406be30 0x7f696406e2f0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f6984150e60 tx=0x7f6970007d70 comp rx=0 tx=0).stop 2024-05-09T08:08:02.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.288+0000 7f6983200640 1 -- 172.21.15.1:0/4236282332 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f698406f0e0 msgr2=0x7f69841505b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:08:02.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.288+0000 7f6983200640 1 --2- 172.21.15.1:0/4236282332 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f698406f0e0 0x7f69841505b0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f697400c7f0 tx=0x7f697400ccb0 comp rx=0 tx=0).stop 2024-05-09T08:08:02.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.288+0000 7f6982800640 1 -- 172.21.15.1:0/4236282332 reap_dead start 2024-05-09T08:08:02.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.288+0000 7f6983200640 1 -- 172.21.15.1:0/4236282332 shutdown_connections 2024-05-09T08:08:02.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.288+0000 7f6983200640 1 -- 172.21.15.1:0/4236282332 >> 172.21.15.1:0/4236282332 conn(0x7f69840cddd0 msgr2=0x7f69840ce1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:08:02.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.288+0000 7f6983200640 1 -- 172.21.15.1:0/4236282332 shutdown_connections 2024-05-09T08:08:02.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.288+0000 7f6983200640 1 -- 172.21.15.1:0/4236282332 wait complete. 2024-05-09T08:08:02.317 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-05-09T08:08:02.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.521+0000 7f5837200640 1 Processor -- start 2024-05-09T08:08:02.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.521+0000 7f5837200640 1 -- start start 2024-05-09T08:08:02.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.522+0000 7f5837200640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f58300cac40 0x7f583006da40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:08:02.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.522+0000 7f5837200640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f58300cd2a0 0x7f583015ba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:08:02.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.522+0000 7f5837200640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f583015a610 0x7f583015aac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:08:02.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.522+0000 7f5837200640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f5830154dd0 con 0x7f58300cd2a0 2024-05-09T08:08:02.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.522+0000 7f5837200640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f58300cce10 con 0x7f58300cac40 2024-05-09T08:08:02.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.522+0000 7f5837200640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f58300cb020 con 0x7f583015a610 2024-05-09T08:08:02.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.523+0000 7f5835400640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f58300cd2a0 0x7f583015ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:08:02.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.523+0000 7f5835400640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f58300cd2a0 0x7f583015ba80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60746/0 (socket says 172.21.15.1:60746) 2024-05-09T08:08:02.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.523+0000 7f5835400640 1 -- 172.21.15.1:0/1047356097 learned_addr learned my addr 172.21.15.1:0/1047356097 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:08:02.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.523+0000 7f5835e00640 1 --2- 172.21.15.1:0/1047356097 >> v2:172.21.15.7:3300/0 conn(0x7f58300cac40 0x7f583006da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:08:02.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.523+0000 7f5836800640 1 --2- 172.21.15.1:0/1047356097 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f583015a610 0x7f583015aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:08:02.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.523+0000 7f5835e00640 1 -- 172.21.15.1:0/1047356097 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f583015a610 msgr2=0x7f583015aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:08:02.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.523+0000 7f5835e00640 1 --2- 172.21.15.1:0/1047356097 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f583015a610 0x7f583015aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:02.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.523+0000 7f5835e00640 1 -- 172.21.15.1:0/1047356097 >> v2:172.21.15.1:3300/0 conn(0x7f58300cd2a0 msgr2=0x7f583015ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:08:02.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.523+0000 7f5835e00640 1 --2- 172.21.15.1:0/1047356097 >> v2:172.21.15.1:3300/0 conn(0x7f58300cd2a0 0x7f583015ba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:02.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.523+0000 7f5835e00640 1 -- 172.21.15.1:0/1047356097 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58300cc550 con 0x7f58300cac40 2024-05-09T08:08:02.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.523+0000 7f5835400640 1 --2- 172.21.15.1:0/1047356097 >> v2:172.21.15.1:3300/0 conn(0x7f58300cd2a0 0x7f583015ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:08:02.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.524+0000 7f5835e00640 1 --2- 172.21.15.1:0/1047356097 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58300cac40 0x7f583006da40 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f58300d0c90 tx=0x7f582402a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ac50f788be1524e6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:08:02.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.525+0000 7f5834a00640 1 -- 172.21.15.1:0/1047356097 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5824037070 con 0x7f58300cac40 2024-05-09T08:08:02.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.525+0000 7f5834a00640 1 -- 172.21.15.1:0/1047356097 <== mon.1 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f582402ae50 con 0x7f58300cac40 2024-05-09T08:08:02.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.525+0000 7f5834a00640 1 -- 172.21.15.1:0/1047356097 <== mon.1 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5824032660 con 0x7f58300cac40 2024-05-09T08:08:02.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.525+0000 7f5837200640 1 -- 172.21.15.1:0/1047356097 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58300cac40 msgr2=0x7f583006da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:08:02.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.525+0000 7f5837200640 1 --2- 172.21.15.1:0/1047356097 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58300cac40 0x7f583006da40 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f58300d0c90 tx=0x7f582402a100 comp rx=0 tx=0).stop 2024-05-09T08:08:02.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.525+0000 7f5837200640 1 -- 172.21.15.1:0/1047356097 shutdown_connections 2024-05-09T08:08:02.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.525+0000 7f5837200640 1 --2- 172.21.15.1:0/1047356097 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f583015a610 0x7f583015aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:02.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.525+0000 7f5837200640 1 --2- 172.21.15.1:0/1047356097 >> v2:172.21.15.1:3300/0 conn(0x7f58300cd2a0 0x7f583015ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:02.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.525+0000 7f5837200640 1 --2- 172.21.15.1:0/1047356097 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58300cac40 0x7f583006da40 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:02.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.525+0000 7f5837200640 1 -- 172.21.15.1:0/1047356097 >> 172.21.15.1:0/1047356097 conn(0x7f58300670b0 msgr2=0x7f58300674c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:08:02.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.525+0000 7f5837200640 1 -- 172.21.15.1:0/1047356097 shutdown_connections 2024-05-09T08:08:02.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.525+0000 7f5837200640 1 -- 172.21.15.1:0/1047356097 wait complete. 2024-05-09T08:08:02.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.527+0000 7f5837200640 1 Processor -- start 2024-05-09T08:08:02.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.527+0000 7f5837200640 1 -- start start 2024-05-09T08:08:02.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.528+0000 7f5837200640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f58300cac40 0x7f5830156e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:08:02.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.528+0000 7f5837200640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f583015a610 0x7f58301551c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:08:02.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.528+0000 7f5837200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58301573d0 0x7f5830155700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:08:02.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.528+0000 7f5837200640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f58300cc7c0 con 0x7f58300cac40 2024-05-09T08:08:02.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.528+0000 7f5837200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f583015b410 con 0x7f58301573d0 2024-05-09T08:08:02.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.528+0000 7f5837200640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f58300ccaa0 con 0x7f583015a610 2024-05-09T08:08:02.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.528+0000 7f5835e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f58300cac40 0x7f5830156e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:08:02.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.528+0000 7f5835e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f58300cac40 0x7f5830156e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60752/0 (socket says 172.21.15.1:60752) 2024-05-09T08:08:02.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.528+0000 7f5835e00640 1 -- 172.21.15.1:0/2271628744 learned_addr learned my addr 172.21.15.1:0/2271628744 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:08:02.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.528+0000 7f5836800640 1 --2- 172.21.15.1:0/2271628744 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58301573d0 0x7f5830155700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:08:02.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.528+0000 7f5835400640 1 --2- 172.21.15.1:0/2271628744 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f583015a610 0x7f58301551c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:08:02.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.529+0000 7f5835e00640 1 -- 172.21.15.1:0/2271628744 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f583015a610 msgr2=0x7f58301551c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:08:02.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.529+0000 7f5835e00640 1 --2- 172.21.15.1:0/2271628744 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f583015a610 0x7f58301551c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:02.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.529+0000 7f5835e00640 1 -- 172.21.15.1:0/2271628744 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58301573d0 msgr2=0x7f5830155700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:08:02.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.529+0000 7f5835e00640 1 --2- 172.21.15.1:0/2271628744 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58301573d0 0x7f5830155700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:02.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.529+0000 7f5835e00640 1 -- 172.21.15.1:0/2271628744 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58301e67c0 con 0x7f58300cac40 2024-05-09T08:08:02.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.529+0000 7f5835e00640 1 --2- 172.21.15.1:0/2271628744 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f58300cac40 0x7f5830156e90 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f5824000c00 tx=0x7f58240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:08:02.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.529+0000 7f582a000640 1 -- 172.21.15.1:0/2271628744 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5824041070 con 0x7f58300cac40 2024-05-09T08:08:02.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.530+0000 7f582a000640 1 -- 172.21.15.1:0/2271628744 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f582403c460 con 0x7f58300cac40 2024-05-09T08:08:02.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.530+0000 7f582a000640 1 -- 172.21.15.1:0/2271628744 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5824037040 con 0x7f58300cac40 2024-05-09T08:08:02.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.530+0000 7f5837200640 1 -- 172.21.15.1:0/2271628744 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58301578e0 con 0x7f58300cac40 2024-05-09T08:08:02.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.530+0000 7f5837200640 1 -- 172.21.15.1:0/2271628744 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58301e6d20 con 0x7f58300cac40 2024-05-09T08:08:02.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.531+0000 7f5837200640 1 -- 172.21.15.1:0/2271628744 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f583004f660 con 0x7f58300cac40 2024-05-09T08:08:02.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.532+0000 7f582a000640 1 -- 172.21.15.1:0/2271628744 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5824044070 con 0x7f58300cac40 2024-05-09T08:08:02.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.533+0000 7f582a000640 1 --2- 172.21.15.1:0/2271628744 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f581406be30 0x7f581406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:08:02.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.533+0000 7f582a000640 1 -- 172.21.15.1:0/2271628744 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7f58240ab5d0 con 0x7f58300cac40 2024-05-09T08:08:02.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.533+0000 7f5835400640 1 --2- 172.21.15.1:0/2271628744 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f581406be30 0x7f581406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:08:02.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.534+0000 7f5835400640 1 --2- 172.21.15.1:0/2271628744 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f581406be30 0x7f581406e2f0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f5820009630 tx=0x7f5820009240 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:08:02.547 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.537+0000 7f582a000640 1 -- 172.21.15.1:0/2271628744 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f58301e67c0 con 0x7f58300cac40 2024-05-09T08:08:02.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.743+0000 7f5837200640 1 -- 172.21.15.1:0/2271628744 --> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) -- 0x7f58300ce3b0 con 0x7f581406be30 2024-05-09T08:08:02.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.744+0000 7f582a000640 1 -- 172.21.15.1:0/2271628744 <== mgr.4114 v2:172.21.15.7:6832/3562612464 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f58300ce3b0 con 0x7f581406be30 2024-05-09T08:08:02.754 INFO:teuthology.orchestra.run.smithi001.stderr:instructed osd(s) 4 to deep-scrub 2024-05-09T08:08:02.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.747+0000 7f5837200640 1 -- 172.21.15.1:0/2271628744 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f581406be30 msgr2=0x7f581406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:08:02.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.747+0000 7f5837200640 1 --2- 172.21.15.1:0/2271628744 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f581406be30 0x7f581406e2f0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f5820009630 tx=0x7f5820009240 comp rx=0 tx=0).stop 2024-05-09T08:08:02.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.747+0000 7f5837200640 1 -- 172.21.15.1:0/2271628744 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f58300cac40 msgr2=0x7f5830156e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:08:02.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.747+0000 7f5837200640 1 --2- 172.21.15.1:0/2271628744 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f58300cac40 0x7f5830156e90 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f5824000c00 tx=0x7f58240046c0 comp rx=0 tx=0).stop 2024-05-09T08:08:02.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.747+0000 7f5837200640 1 -- 172.21.15.1:0/2271628744 shutdown_connections 2024-05-09T08:08:02.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.747+0000 7f5837200640 1 --2- 172.21.15.1:0/2271628744 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f581406be30 0x7f581406e2f0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:02.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.747+0000 7f5837200640 1 --2- 172.21.15.1:0/2271628744 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58301573d0 0x7f5830155700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:02.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.747+0000 7f5837200640 1 --2- 172.21.15.1:0/2271628744 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f583015a610 0x7f58301551c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:02.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.747+0000 7f5837200640 1 --2- 172.21.15.1:0/2271628744 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f58300cac40 0x7f5830156e90 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:02.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.747+0000 7f5837200640 1 -- 172.21.15.1:0/2271628744 >> 172.21.15.1:0/2271628744 conn(0x7f58300670b0 msgr2=0x7f5830158480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:08:02.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.747+0000 7f5837200640 1 -- 172.21.15.1:0/2271628744 shutdown_connections 2024-05-09T08:08:02.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.747+0000 7f5837200640 1 -- 172.21.15.1:0/2271628744 wait complete. 2024-05-09T08:08:02.776 INFO:tasks.ceph:Scrubbing osd.5 2024-05-09T08:08:02.776 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-05-09T08:08:02.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.987+0000 7f8439200640 1 Processor -- start 2024-05-09T08:08:02.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.987+0000 7f8439200640 1 -- start start 2024-05-09T08:08:02.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.989+0000 7f8439200640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f84340734e0 0x7f843406d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:08:02.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.989+0000 7f8439200640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f843406dc80 0x7f843406e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:08:02.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.989+0000 7f8439200640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8434158f20 con 0x7f8434152550 2024-05-09T08:08:02.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.989+0000 7f8439200640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f843406ed90 con 0x7f843406dc80 2024-05-09T08:08:02.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.989+0000 7f8439200640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f84340c6b50 con 0x7f84340734e0 2024-05-09T08:08:03.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.989+0000 7f8432800640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f843406dc80 0x7f843406e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:08:03.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.989+0000 7f8429400640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f84340734e0 0x7f843406d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:08:03.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.989+0000 7f8429400640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f84340734e0 0x7f843406d740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3301/0 says I am v2:172.21.15.1:58818/0 (socket says 172.21.15.1:58818) 2024-05-09T08:08:03.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.989+0000 7f8432800640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f843406dc80 0x7f843406e100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60760/0 (socket says 172.21.15.1:60760) 2024-05-09T08:08:03.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.989+0000 7f8429400640 1 -- 172.21.15.1:0/3933778514 learned_addr learned my addr 172.21.15.1:0/3933778514 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:08:03.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.990+0000 7f8429400640 1 -- 172.21.15.1:0/3933778514 >> v2:172.21.15.1:3300/0 conn(0x7f843406dc80 msgr2=0x7f843406e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:08:03.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.990+0000 7f8429400640 1 --2- 172.21.15.1:0/3933778514 >> v2:172.21.15.1:3300/0 conn(0x7f843406dc80 0x7f843406e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:03.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.990+0000 7f8429400640 1 -- 172.21.15.1:0/3933778514 >> v1:172.21.15.1:6789/0 conn(0x7f8434152550 legacy=0x7f84340723d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:08:03.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.990+0000 7f8429400640 1 -- 172.21.15.1:0/3933778514 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f843406eab0 con 0x7f84340734e0 2024-05-09T08:08:03.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.991+0000 7f8429400640 1 --2- 172.21.15.1:0/3933778514 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f84340734e0 0x7f843406d740 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f841c009870 tx=0x7f841c02a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bda7181ed3a7a5b0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:08:03.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.991+0000 7f8431400640 1 -- 172.21.15.1:0/3933778514 <== mon.2 v2:172.21.15.7:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f841c037070 con 0x7f84340734e0 2024-05-09T08:08:03.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.991+0000 7f8431400640 1 -- 172.21.15.1:0/3933778514 <== mon.2 v2:172.21.15.7:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f841c032440 con 0x7f84340734e0 2024-05-09T08:08:03.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.991+0000 7f8431400640 1 -- 172.21.15.1:0/3933778514 <== mon.2 v2:172.21.15.7:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f841c032810 con 0x7f84340734e0 2024-05-09T08:08:03.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.991+0000 7f8439200640 1 -- 172.21.15.1:0/3933778514 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f84340734e0 msgr2=0x7f843406d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:08:03.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.991+0000 7f8439200640 1 --2- 172.21.15.1:0/3933778514 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f84340734e0 0x7f843406d740 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f841c009870 tx=0x7f841c02a1c0 comp rx=0 tx=0).stop 2024-05-09T08:08:03.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.992+0000 7f8439200640 1 -- 172.21.15.1:0/3933778514 shutdown_connections 2024-05-09T08:08:03.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.992+0000 7f8439200640 1 --2- 172.21.15.1:0/3933778514 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f84340734e0 0x7f843406d740 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:03.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.992+0000 7f8439200640 1 --2- 172.21.15.1:0/3933778514 >> v2:172.21.15.1:3300/0 conn(0x7f843406dc80 0x7f843406e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:03.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.992+0000 7f8439200640 1 -- 172.21.15.1:0/3933778514 >> 172.21.15.1:0/3933778514 conn(0x7f84340c89b0 msgr2=0x7f84340c8dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:08:03.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.992+0000 7f8439200640 1 -- 172.21.15.1:0/3933778514 shutdown_connections 2024-05-09T08:08:03.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.992+0000 7f8439200640 1 -- 172.21.15.1:0/3933778514 wait complete. 2024-05-09T08:08:03.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.994+0000 7f8439200640 1 Processor -- start 2024-05-09T08:08:03.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.994+0000 7f8439200640 1 -- start start 2024-05-09T08:08:03.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.995+0000 7f8439200640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f843406dc80 0x7f84340c5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:08:03.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.995+0000 7f8439200640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f84340734e0 0x7f84341f3880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:08:03.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.995+0000 7f8439200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8434152550 0x7f84341f3dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:08:03.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.995+0000 7f8439200640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f8434071bd0 con 0x7f84340734e0 2024-05-09T08:08:03.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.995+0000 7f8439200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f843406e860 con 0x7f8434152550 2024-05-09T08:08:03.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.995+0000 7f8439200640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f8434071eb0 con 0x7f843406dc80 2024-05-09T08:08:03.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.995+0000 7f8429400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f84340734e0 0x7f84341f3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:08:03.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.995+0000 7f8429400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f84340734e0 0x7f84341f3880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60774/0 (socket says 172.21.15.1:60774) 2024-05-09T08:08:03.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.995+0000 7f8431e00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f843406dc80 0x7f84340c5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:08:03.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.995+0000 7f8429400640 1 -- 172.21.15.1:0/2583874850 learned_addr learned my addr 172.21.15.1:0/2583874850 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:08:03.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.995+0000 7f8432800640 1 --2- 172.21.15.1:0/2583874850 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8434152550 0x7f84341f3dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:08:03.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.996+0000 7f8429400640 1 -- 172.21.15.1:0/2583874850 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f843406dc80 msgr2=0x7f84340c5710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:08:03.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.996+0000 7f8429400640 1 --2- 172.21.15.1:0/2583874850 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f843406dc80 0x7f84340c5710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:03.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.996+0000 7f8429400640 1 -- 172.21.15.1:0/2583874850 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8434152550 msgr2=0x7f84341f3dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:08:03.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.996+0000 7f8429400640 1 --2- 172.21.15.1:0/2583874850 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8434152550 0x7f84341f3dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:03.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.996+0000 7f8429400640 1 -- 172.21.15.1:0/2583874850 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8434156350 con 0x7f84340734e0 2024-05-09T08:08:03.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.996+0000 7f8431e00640 1 --2- 172.21.15.1:0/2583874850 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f843406dc80 0x7f84340c5710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:08:03.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.996+0000 7f8429400640 1 --2- 172.21.15.1:0/2583874850 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f84340734e0 0x7f84341f3880 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f841c02a200 tx=0x7f841c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:08:03.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.996+0000 7f842aa00640 1 -- 172.21.15.1:0/2583874850 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f841c037040 con 0x7f84340734e0 2024-05-09T08:08:03.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.996+0000 7f842aa00640 1 -- 172.21.15.1:0/2583874850 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f841c042070 con 0x7f84340734e0 2024-05-09T08:08:03.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.996+0000 7f842aa00640 1 -- 172.21.15.1:0/2583874850 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f841c030880 con 0x7f84340734e0 2024-05-09T08:08:03.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.996+0000 7f8439200640 1 -- 172.21.15.1:0/2583874850 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f843414de10 con 0x7f84340734e0 2024-05-09T08:08:03.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.997+0000 7f8439200640 1 -- 172.21.15.1:0/2583874850 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f843414e3d0 con 0x7f84340734e0 2024-05-09T08:08:03.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.997+0000 7f8439200640 1 -- 172.21.15.1:0/2583874850 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8434068520 con 0x7f84340734e0 2024-05-09T08:08:03.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:02.999+0000 7f842aa00640 1 -- 172.21.15.1:0/2583874850 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f841c02ad10 con 0x7f84340734e0 2024-05-09T08:08:03.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.000+0000 7f842aa00640 1 --2- 172.21.15.1:0/2583874850 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f841006be30 0x7f841006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:08:03.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.000+0000 7f842aa00640 1 -- 172.21.15.1:0/2583874850 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7f841c0ab740 con 0x7f84340734e0 2024-05-09T08:08:03.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.000+0000 7f842aa00640 1 --2- 172.21.15.1:0/2583874850 >> [v2:172.21.15.7:6808/1670295242,v1:172.21.15.7:6809/1670295242] conn(0x7f84100757c0 0x7f8410077c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:08:03.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.000+0000 7f842aa00640 1 -- 172.21.15.1:0/2583874850 --> [v2:172.21.15.7:6808/1670295242,v1:172.21.15.7:6809/1670295242] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f841c032440 con 0x7f84100757c0 2024-05-09T08:08:03.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.000+0000 7f842aa00640 1 -- 172.21.15.1:0/2583874850 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f841c0b2050 con 0x7f84340734e0 2024-05-09T08:08:03.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.000+0000 7f8431e00640 1 --2- 172.21.15.1:0/2583874850 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f841006be30 0x7f841006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:08:03.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.000+0000 7f8432800640 1 --2- 172.21.15.1:0/2583874850 >> [v2:172.21.15.7:6808/1670295242,v1:172.21.15.7:6809/1670295242] conn(0x7f84100757c0 0x7f8410077c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:08:03.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.001+0000 7f8431e00640 1 --2- 172.21.15.1:0/2583874850 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f841006be30 0x7f841006e2f0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f8424001b70 tx=0x7f8424009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:08:03.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.001+0000 7f8432800640 1 --2- 172.21.15.1:0/2583874850 >> [v2:172.21.15.7:6808/1670295242,v1:172.21.15.7:6809/1670295242] conn(0x7f84100757c0 0x7f8410077c60 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:08:03.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.002+0000 7f842aa00640 1 -- 172.21.15.1:0/2583874850 <== osd.5 v2:172.21.15.7:6808/1670295242 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f841c032440 con 0x7f84100757c0 2024-05-09T08:08:03.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.017+0000 7f8439200640 1 -- 172.21.15.1:0/2583874850 --> [v2:172.21.15.7:6808/1670295242,v1:172.21.15.7:6809/1670295242] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f84340c6e10 con 0x7f84100757c0 2024-05-09T08:08:03.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.017+0000 7f842aa00640 1 -- 172.21.15.1:0/2583874850 <== osd.5 v2:172.21.15.7:6808/1670295242 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f84340c6e10 con 0x7f84100757c0 2024-05-09T08:08:03.027 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-05-09T08:08:03.028 INFO:teuthology.orchestra.run.smithi001.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-09T08:08:03.028 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-05-09T08:08:03.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.018+0000 7f8439200640 1 -- 172.21.15.1:0/2583874850 >> [v2:172.21.15.7:6808/1670295242,v1:172.21.15.7:6809/1670295242] conn(0x7f84100757c0 msgr2=0x7f8410077c60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:08:03.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.018+0000 7f8439200640 1 --2- 172.21.15.1:0/2583874850 >> [v2:172.21.15.7:6808/1670295242,v1:172.21.15.7:6809/1670295242] conn(0x7f84100757c0 0x7f8410077c60 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:03.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.018+0000 7f8439200640 1 -- 172.21.15.1:0/2583874850 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f841006be30 msgr2=0x7f841006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:08:03.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.018+0000 7f8439200640 1 --2- 172.21.15.1:0/2583874850 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f841006be30 0x7f841006e2f0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f8424001b70 tx=0x7f8424009040 comp rx=0 tx=0).stop 2024-05-09T08:08:03.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.018+0000 7f8439200640 1 -- 172.21.15.1:0/2583874850 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f84340734e0 msgr2=0x7f84341f3880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:08:03.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.018+0000 7f8439200640 1 --2- 172.21.15.1:0/2583874850 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f84340734e0 0x7f84341f3880 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f841c02a200 tx=0x7f841c0046c0 comp rx=0 tx=0).stop 2024-05-09T08:08:03.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.018+0000 7f8432800640 1 -- 172.21.15.1:0/2583874850 reap_dead start 2024-05-09T08:08:03.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.018+0000 7f8439200640 1 -- 172.21.15.1:0/2583874850 shutdown_connections 2024-05-09T08:08:03.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.018+0000 7f8439200640 1 -- 172.21.15.1:0/2583874850 >> 172.21.15.1:0/2583874850 conn(0x7f84340c89b0 msgr2=0x7f8434150950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:08:03.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.019+0000 7f8439200640 1 -- 172.21.15.1:0/2583874850 shutdown_connections 2024-05-09T08:08:03.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.019+0000 7f8439200640 1 -- 172.21.15.1:0/2583874850 wait complete. 2024-05-09T08:08:03.046 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-05-09T08:08:03.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.259+0000 7f6126800640 1 Processor -- start 2024-05-09T08:08:03.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.260+0000 7f6126800640 1 -- start start 2024-05-09T08:08:03.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.261+0000 7f6126800640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f61200cac60 0x7f61200cda00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:08:03.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.261+0000 7f6126800640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f612015ac80 0x7f61200cdf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:08:03.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.261+0000 7f6126800640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f61200ccb20 con 0x7f61200cc5f0 2024-05-09T08:08:03.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.261+0000 7f6126800640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f61200cce00 con 0x7f612015ac80 2024-05-09T08:08:03.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.261+0000 7f6126800640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f61200cf120 con 0x7f61200cac60 2024-05-09T08:08:03.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.261+0000 7f6125400640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f61200cac60 0x7f61200cda00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:08:03.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.261+0000 7f6125e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f612015ac80 0x7f61200cdf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:08:03.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.261+0000 7f6125400640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f61200cac60 0x7f61200cda00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3301/0 says I am v2:172.21.15.1:58834/0 (socket says 172.21.15.1:58834) 2024-05-09T08:08:03.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.261+0000 7f6125400640 1 -- 172.21.15.1:0/1080669072 learned_addr learned my addr 172.21.15.1:0/1080669072 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:08:03.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.262+0000 7f6125400640 1 -- 172.21.15.1:0/1080669072 >> v2:172.21.15.7:3300/0 conn(0x7f612015ac80 msgr2=0x7f61200cdf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:08:03.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.262+0000 7f6125400640 1 --2- 172.21.15.1:0/1080669072 >> v2:172.21.15.7:3300/0 conn(0x7f612015ac80 0x7f61200cdf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:03.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.262+0000 7f6125400640 1 -- 172.21.15.1:0/1080669072 >> v1:172.21.15.7:6789/0 conn(0x7f61200cc5f0 legacy=0x7f612015a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:08:03.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.262+0000 7f6125400640 1 -- 172.21.15.1:0/1080669072 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61200cf400 con 0x7f61200cac60 2024-05-09T08:08:03.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.263+0000 7f6125400640 1 --2- 172.21.15.1:0/1080669072 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f61200cac60 0x7f61200cda00 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f6114009870 tx=0x7f611402a2f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ab463de67400673 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:08:03.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.263+0000 7f611fe00640 1 -- 172.21.15.1:0/1080669072 <== mon.2 v2:172.21.15.7:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6114037070 con 0x7f61200cac60 2024-05-09T08:08:03.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.263+0000 7f611fe00640 1 -- 172.21.15.1:0/1080669072 <== mon.2 v2:172.21.15.7:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6114032640 con 0x7f61200cac60 2024-05-09T08:08:03.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.263+0000 7f611fe00640 1 -- 172.21.15.1:0/1080669072 <== mon.2 v2:172.21.15.7:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6114032a10 con 0x7f61200cac60 2024-05-09T08:08:03.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.264+0000 7f6126800640 1 -- 172.21.15.1:0/1080669072 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f61200cac60 msgr2=0x7f61200cda00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:08:03.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.264+0000 7f6126800640 1 --2- 172.21.15.1:0/1080669072 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f61200cac60 0x7f61200cda00 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f6114009870 tx=0x7f611402a2f0 comp rx=0 tx=0).stop 2024-05-09T08:08:03.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.264+0000 7f6126800640 1 -- 172.21.15.1:0/1080669072 shutdown_connections 2024-05-09T08:08:03.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.264+0000 7f6126800640 1 --2- 172.21.15.1:0/1080669072 >> v2:172.21.15.7:3300/0 conn(0x7f612015ac80 0x7f61200cdf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:03.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.264+0000 7f6126800640 1 --2- 172.21.15.1:0/1080669072 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f61200cac60 0x7f61200cda00 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:03.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.264+0000 7f6126800640 1 -- 172.21.15.1:0/1080669072 >> 172.21.15.1:0/1080669072 conn(0x7f61200cfc20 msgr2=0x7f61200d0030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:08:03.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.264+0000 7f6126800640 1 -- 172.21.15.1:0/1080669072 shutdown_connections 2024-05-09T08:08:03.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.264+0000 7f6126800640 1 -- 172.21.15.1:0/1080669072 wait complete. 2024-05-09T08:08:03.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.266+0000 7f6126800640 1 Processor -- start 2024-05-09T08:08:03.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.266+0000 7f6126800640 1 -- start start 2024-05-09T08:08:03.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.267+0000 7f6126800640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f61200cac60 0x7f6120152890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:08:03.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.267+0000 7f6126800640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f61200cc5f0 0x7f61200b2a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:08:03.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.267+0000 7f6126800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f612015ac80 0x7f61201f4150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:08:03.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.267+0000 7f6126800640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f61200ce4e0 con 0x7f61200cac60 2024-05-09T08:08:03.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.267+0000 7f6126800640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f61200cd550 con 0x7f612015ac80 2024-05-09T08:08:03.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.267+0000 7f6126800640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f61200ce660 con 0x7f61200cc5f0 2024-05-09T08:08:03.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.267+0000 7f6125400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f61200cac60 0x7f6120152890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:08:03.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.267+0000 7f6125400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f61200cac60 0x7f6120152890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60784/0 (socket says 172.21.15.1:60784) 2024-05-09T08:08:03.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.267+0000 7f6125400640 1 -- 172.21.15.1:0/49626236 learned_addr learned my addr 172.21.15.1:0/49626236 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:08:03.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.267+0000 7f6125e00640 1 --2- 172.21.15.1:0/49626236 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f612015ac80 0x7f61201f4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:08:03.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.267+0000 7f6124a00640 1 --2- 172.21.15.1:0/49626236 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f61200cc5f0 0x7f61200b2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:08:03.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.268+0000 7f6125400640 1 -- 172.21.15.1:0/49626236 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f61200cc5f0 msgr2=0x7f61200b2a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:08:03.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.268+0000 7f6125400640 1 --2- 172.21.15.1:0/49626236 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f61200cc5f0 0x7f61200b2a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:03.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.268+0000 7f6125400640 1 -- 172.21.15.1:0/49626236 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f612015ac80 msgr2=0x7f61201f4150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:08:03.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.268+0000 7f6125400640 1 --2- 172.21.15.1:0/49626236 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f612015ac80 0x7f61201f4150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:03.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.268+0000 7f6125400640 1 -- 172.21.15.1:0/49626236 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61200b5440 con 0x7f61200cac60 2024-05-09T08:08:03.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.268+0000 7f6125400640 1 --2- 172.21.15.1:0/49626236 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f61200cac60 0x7f6120152890 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f6114009870 tx=0x7f61140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:08:03.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.268+0000 7f611d600640 1 -- 172.21.15.1:0/49626236 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6114041070 con 0x7f61200cac60 2024-05-09T08:08:03.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.268+0000 7f611d600640 1 -- 172.21.15.1:0/49626236 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f611403c870 con 0x7f61200cac60 2024-05-09T08:08:03.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.268+0000 7f611d600640 1 -- 172.21.15.1:0/49626236 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6114037040 con 0x7f61200cac60 2024-05-09T08:08:03.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.268+0000 7f6126800640 1 -- 172.21.15.1:0/49626236 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61200b48c0 con 0x7f61200cac60 2024-05-09T08:08:03.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.269+0000 7f6126800640 1 -- 172.21.15.1:0/49626236 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61200b4e80 con 0x7f61200cac60 2024-05-09T08:08:03.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.270+0000 7f6126800640 1 -- 172.21.15.1:0/49626236 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f612004f660 con 0x7f61200cac60 2024-05-09T08:08:03.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.271+0000 7f611d600640 1 -- 172.21.15.1:0/49626236 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6114044070 con 0x7f61200cac60 2024-05-09T08:08:03.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.272+0000 7f611d600640 1 --2- 172.21.15.1:0/49626236 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f610006be30 0x7f610006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:08:03.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.272+0000 7f611d600640 1 -- 172.21.15.1:0/49626236 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7f61140abe60 con 0x7f61200cac60 2024-05-09T08:08:03.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.272+0000 7f6124a00640 1 --2- 172.21.15.1:0/49626236 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f610006be30 0x7f610006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:08:03.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.273+0000 7f6124a00640 1 --2- 172.21.15.1:0/49626236 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f610006be30 0x7f610006e2f0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f61200cc580 tx=0x7f610c009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:08:03.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.277+0000 7f611d600640 1 -- 172.21.15.1:0/49626236 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f6114079ee0 con 0x7f61200cac60 2024-05-09T08:08:03.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.479+0000 7f6126800640 1 -- 172.21.15.1:0/49626236 --> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) -- 0x7f612015e830 con 0x7f610006be30 2024-05-09T08:08:03.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.480+0000 7f611d600640 1 -- 172.21.15.1:0/49626236 <== mgr.4114 v2:172.21.15.7:6832/3562612464 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f612015e830 con 0x7f610006be30 2024-05-09T08:08:03.490 INFO:teuthology.orchestra.run.smithi001.stderr:instructed osd(s) 5 to deep-scrub 2024-05-09T08:08:03.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.483+0000 7f6126800640 1 -- 172.21.15.1:0/49626236 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f610006be30 msgr2=0x7f610006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:08:03.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.483+0000 7f6126800640 1 --2- 172.21.15.1:0/49626236 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f610006be30 0x7f610006e2f0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f61200cc580 tx=0x7f610c009040 comp rx=0 tx=0).stop 2024-05-09T08:08:03.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.483+0000 7f6126800640 1 -- 172.21.15.1:0/49626236 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f61200cac60 msgr2=0x7f6120152890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:08:03.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.483+0000 7f6126800640 1 --2- 172.21.15.1:0/49626236 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f61200cac60 0x7f6120152890 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f6114009870 tx=0x7f61140046c0 comp rx=0 tx=0).stop 2024-05-09T08:08:03.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.483+0000 7f6126800640 1 -- 172.21.15.1:0/49626236 shutdown_connections 2024-05-09T08:08:03.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.483+0000 7f6126800640 1 --2- 172.21.15.1:0/49626236 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f610006be30 0x7f610006e2f0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:03.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.483+0000 7f6126800640 1 --2- 172.21.15.1:0/49626236 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f612015ac80 0x7f61201f4150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:03.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.483+0000 7f6126800640 1 --2- 172.21.15.1:0/49626236 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f61200cc5f0 0x7f61200b2a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:03.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.483+0000 7f6126800640 1 --2- 172.21.15.1:0/49626236 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f61200cac60 0x7f6120152890 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:03.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.483+0000 7f6126800640 1 -- 172.21.15.1:0/49626236 >> 172.21.15.1:0/49626236 conn(0x7f61200cfc20 msgr2=0x7f6120159960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:08:03.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.483+0000 7f6126800640 1 -- 172.21.15.1:0/49626236 shutdown_connections 2024-05-09T08:08:03.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.483+0000 7f6126800640 1 -- 172.21.15.1:0/49626236 wait complete. 2024-05-09T08:08:03.515 INFO:tasks.ceph:Scrubbing osd.6 2024-05-09T08:08:03.515 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-05-09T08:08:03.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.725+0000 7f7719000640 1 Processor -- start 2024-05-09T08:08:03.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.726+0000 7f7719000640 1 -- start start 2024-05-09T08:08:03.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.727+0000 7f7719000640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f77140734e0 0x7f771406d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:08:03.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.727+0000 7f7719000640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f771406dc80 0x7f771406e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:08:03.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.727+0000 7f7719000640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7714158f20 con 0x7f7714152550 2024-05-09T08:08:03.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.727+0000 7f7719000640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f771406ed90 con 0x7f771406dc80 2024-05-09T08:08:03.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.727+0000 7f7719000640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f77140c6b50 con 0x7f77140734e0 2024-05-09T08:08:03.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.727+0000 7f7712800640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f771406dc80 0x7f771406e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:08:03.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.727+0000 7f7711400640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f77140734e0 0x7f771406d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:08:03.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.727+0000 7f7711e00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f7714152550 0x7f77140723d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.1:38728/0 (socket says 172.21.15.1:38728) 2024-05-09T08:08:03.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.727+0000 7f7712800640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f771406dc80 0x7f771406e100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.1:54208/0 (socket says 172.21.15.1:54208) 2024-05-09T08:08:03.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.728+0000 7f7711e00640 1 -- 172.21.15.1:0/838699194 learned_addr learned my addr 172.21.15.1:0/838699194 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T08:08:03.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.728+0000 7f7712800640 1 -- 172.21.15.1:0/838699194 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f77140734e0 msgr2=0x7f771406d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:08:03.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.728+0000 7f7712800640 1 --2- 172.21.15.1:0/838699194 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f77140734e0 0x7f771406d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:03.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.728+0000 7f7712800640 1 -- 172.21.15.1:0/838699194 >> v1:172.21.15.7:6789/0 conn(0x7f7714152550 legacy=0x7f77140723d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:08:03.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.728+0000 7f7712800640 1 -- 172.21.15.1:0/838699194 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f771406eab0 con 0x7f771406dc80 2024-05-09T08:08:03.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.728+0000 7f7711400640 1 --2- 172.21.15.1:0/838699194 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f77140734e0 0x7f771406d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:08:03.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.728+0000 7f7710a00640 1 -- 172.21.15.1:0/838699194 <== mon.1 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3199575161 0 0) 0x7f7714158f20 con 0x7f7714152550 2024-05-09T08:08:03.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.729+0000 7f7712800640 1 --2- 172.21.15.1:0/838699194 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f771406dc80 0x7f771406e100 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f770c0097a0 tx=0x7f770c02a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e80e00513e394a94 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:08:03.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.730+0000 7f7710a00640 1 -- 172.21.15.1:0/838699194 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f770c037070 con 0x7f771406dc80 2024-05-09T08:08:03.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.730+0000 7f7710a00640 1 -- 172.21.15.1:0/838699194 <== mon.1 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f770c02ae50 con 0x7f771406dc80 2024-05-09T08:08:03.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.730+0000 7f7710a00640 1 -- 172.21.15.1:0/838699194 <== mon.1 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f770c032660 con 0x7f771406dc80 2024-05-09T08:08:03.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.730+0000 7f7719000640 1 -- 172.21.15.1:0/838699194 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f771406dc80 msgr2=0x7f771406e100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:08:03.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.730+0000 7f7719000640 1 --2- 172.21.15.1:0/838699194 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f771406dc80 0x7f771406e100 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f770c0097a0 tx=0x7f770c02a100 comp rx=0 tx=0).stop 2024-05-09T08:08:03.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.731+0000 7f7719000640 1 -- 172.21.15.1:0/838699194 shutdown_connections 2024-05-09T08:08:03.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.731+0000 7f7719000640 1 --2- 172.21.15.1:0/838699194 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f771406dc80 0x7f771406e100 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:03.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.731+0000 7f7719000640 1 --2- 172.21.15.1:0/838699194 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f77140734e0 0x7f771406d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:03.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.731+0000 7f7719000640 1 -- 172.21.15.1:0/838699194 >> 172.21.15.1:0/838699194 conn(0x7f77140c89b0 msgr2=0x7f77140c8dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:08:03.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.731+0000 7f7719000640 1 -- 172.21.15.1:0/838699194 shutdown_connections 2024-05-09T08:08:03.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.731+0000 7f7719000640 1 -- 172.21.15.1:0/838699194 wait complete. 2024-05-09T08:08:03.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.733+0000 7f7719000640 1 Processor -- start 2024-05-09T08:08:03.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.733+0000 7f7719000640 1 -- start start 2024-05-09T08:08:03.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.735+0000 7f7719000640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f771406dc80 0x7f77141562e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:08:03.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.735+0000 7f7719000640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f77140734e0 0x7f77141570f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:08:03.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.735+0000 7f7719000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7714152550 0x7f7714153960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:08:03.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.735+0000 7f7719000640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f7714071bd0 con 0x7f771406dc80 2024-05-09T08:08:03.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.735+0000 7f7719000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f771406e860 con 0x7f7714152550 2024-05-09T08:08:03.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.735+0000 7f7719000640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f7714071eb0 con 0x7f77140734e0 2024-05-09T08:08:03.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.735+0000 7f7711e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f771406dc80 0x7f77141562e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:08:03.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.735+0000 7f7711e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f771406dc80 0x7f77141562e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60792/0 (socket says 172.21.15.1:60792) 2024-05-09T08:08:03.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.735+0000 7f7711e00640 1 -- 172.21.15.1:0/3310686413 learned_addr learned my addr 172.21.15.1:0/3310686413 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:08:03.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.735+0000 7f7711400640 1 --2- 172.21.15.1:0/3310686413 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f77140734e0 0x7f77141570f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:08:03.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.735+0000 7f7712800640 1 --2- 172.21.15.1:0/3310686413 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7714152550 0x7f7714153960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:08:03.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.735+0000 7f7711400640 1 -- 172.21.15.1:0/3310686413 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7714152550 msgr2=0x7f7714153960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:08:03.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.735+0000 7f7711400640 1 --2- 172.21.15.1:0/3310686413 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7714152550 0x7f7714153960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:03.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.736+0000 7f7711400640 1 -- 172.21.15.1:0/3310686413 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f771406dc80 msgr2=0x7f77141562e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:08:03.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.736+0000 7f7711400640 1 --2- 172.21.15.1:0/3310686413 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f771406dc80 0x7f77141562e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:03.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.736+0000 7f7711400640 1 -- 172.21.15.1:0/3310686413 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f771415c1e0 con 0x7f77140734e0 2024-05-09T08:08:03.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.736+0000 7f7711e00640 1 --2- 172.21.15.1:0/3310686413 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f771406dc80 0x7f77141562e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:08:03.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.736+0000 7f7711400640 1 --2- 172.21.15.1:0/3310686413 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f77140734e0 0x7f77141570f0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f770400ef90 tx=0x7f770400c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:08:03.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.736+0000 7f770a000640 1 -- 172.21.15.1:0/3310686413 <== mon.2 v2:172.21.15.7:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7704018020 con 0x7f77140734e0 2024-05-09T08:08:03.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.736+0000 7f770a000640 1 -- 172.21.15.1:0/3310686413 <== mon.2 v2:172.21.15.7:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7704014680 con 0x7f77140734e0 2024-05-09T08:08:03.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.736+0000 7f7719000640 1 -- 172.21.15.1:0/3310686413 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f771415c480 con 0x7f77140734e0 2024-05-09T08:08:03.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.736+0000 7f770a000640 1 -- 172.21.15.1:0/3310686413 <== mon.2 v2:172.21.15.7:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7704014a70 con 0x7f77140734e0 2024-05-09T08:08:03.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.736+0000 7f7719000640 1 -- 172.21.15.1:0/3310686413 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f7714153160 con 0x7f77140734e0 2024-05-09T08:08:03.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.737+0000 7f7719000640 1 -- 172.21.15.1:0/3310686413 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f771406d740 con 0x7f77140734e0 2024-05-09T08:08:03.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.739+0000 7f770a000640 1 -- 172.21.15.1:0/3310686413 <== mon.2 v2:172.21.15.7:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f770401b020 con 0x7f77140734e0 2024-05-09T08:08:03.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.740+0000 7f770a000640 1 --2- 172.21.15.1:0/3310686413 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f76f006be30 0x7f76f006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:08:03.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.740+0000 7f770a000640 1 -- 172.21.15.1:0/3310686413 <== mon.2 v2:172.21.15.7:3301/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7f770408c040 con 0x7f77140734e0 2024-05-09T08:08:03.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.740+0000 7f7711e00640 1 --2- 172.21.15.1:0/3310686413 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f76f006be30 0x7f76f006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:08:03.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.740+0000 7f770a000640 1 --2- 172.21.15.1:0/3310686413 >> [v2:172.21.15.7:6816/2852086880,v1:172.21.15.7:6817/2852086880] conn(0x7f76f00757c0 0x7f76f0077c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:08:03.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.740+0000 7f770a000640 1 -- 172.21.15.1:0/3310686413 --> [v2:172.21.15.7:6816/2852086880,v1:172.21.15.7:6817/2852086880] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f76f0078330 con 0x7f76f00757c0 2024-05-09T08:08:03.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.740+0000 7f770a000640 1 -- 172.21.15.1:0/3310686413 <== mon.2 v2:172.21.15.7:3301/0 6 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f770408c3f0 con 0x7f77140734e0 2024-05-09T08:08:03.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.741+0000 7f7712800640 1 --2- 172.21.15.1:0/3310686413 >> [v2:172.21.15.7:6816/2852086880,v1:172.21.15.7:6817/2852086880] conn(0x7f76f00757c0 0x7f76f0077c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:08:03.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.741+0000 7f7711e00640 1 --2- 172.21.15.1:0/3310686413 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f76f006be30 0x7f76f006e2f0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f76fc000f60 tx=0x7f76fc007e10 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:08:03.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.741+0000 7f7712800640 1 --2- 172.21.15.1:0/3310686413 >> [v2:172.21.15.7:6816/2852086880,v1:172.21.15.7:6817/2852086880] conn(0x7f76f00757c0 0x7f76f0077c60 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:08:03.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.742+0000 7f770a000640 1 -- 172.21.15.1:0/3310686413 <== osd.6 v2:172.21.15.7:6816/2852086880 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f76f0078330 con 0x7f76f00757c0 2024-05-09T08:08:03.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.757+0000 7f7719000640 1 -- 172.21.15.1:0/3310686413 --> [v2:172.21.15.7:6816/2852086880,v1:172.21.15.7:6817/2852086880] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f7714068520 con 0x7f76f00757c0 2024-05-09T08:08:03.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.758+0000 7f770a000640 1 -- 172.21.15.1:0/3310686413 <== osd.6 v2:172.21.15.7:6816/2852086880 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f7714068520 con 0x7f76f00757c0 2024-05-09T08:08:03.768 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-05-09T08:08:03.768 INFO:teuthology.orchestra.run.smithi001.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-09T08:08:03.768 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-05-09T08:08:03.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.758+0000 7f7719000640 1 -- 172.21.15.1:0/3310686413 >> [v2:172.21.15.7:6816/2852086880,v1:172.21.15.7:6817/2852086880] conn(0x7f76f00757c0 msgr2=0x7f76f0077c60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:08:03.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.758+0000 7f7719000640 1 --2- 172.21.15.1:0/3310686413 >> [v2:172.21.15.7:6816/2852086880,v1:172.21.15.7:6817/2852086880] conn(0x7f76f00757c0 0x7f76f0077c60 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:03.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.758+0000 7f7719000640 1 -- 172.21.15.1:0/3310686413 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f76f006be30 msgr2=0x7f76f006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:08:03.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.758+0000 7f7719000640 1 --2- 172.21.15.1:0/3310686413 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f76f006be30 0x7f76f006e2f0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f76fc000f60 tx=0x7f76fc007e10 comp rx=0 tx=0).stop 2024-05-09T08:08:03.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.758+0000 7f7719000640 1 -- 172.21.15.1:0/3310686413 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f77140734e0 msgr2=0x7f77141570f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:08:03.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.758+0000 7f7719000640 1 --2- 172.21.15.1:0/3310686413 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f77140734e0 0x7f77141570f0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f770400ef90 tx=0x7f770400c490 comp rx=0 tx=0).stop 2024-05-09T08:08:03.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.758+0000 7f7712800640 1 -- 172.21.15.1:0/3310686413 reap_dead start 2024-05-09T08:08:03.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.759+0000 7f7719000640 1 -- 172.21.15.1:0/3310686413 shutdown_connections 2024-05-09T08:08:03.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.759+0000 7f7719000640 1 -- 172.21.15.1:0/3310686413 >> 172.21.15.1:0/3310686413 conn(0x7f77140c89b0 msgr2=0x7f771414fd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:08:03.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.759+0000 7f7719000640 1 -- 172.21.15.1:0/3310686413 shutdown_connections 2024-05-09T08:08:03.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.759+0000 7f7719000640 1 -- 172.21.15.1:0/3310686413 wait complete. 2024-05-09T08:08:03.786 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-05-09T08:08:03.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.987+0000 7f59b8e00640 1 Processor -- start 2024-05-09T08:08:03.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.987+0000 7f59b8e00640 1 -- start start 2024-05-09T08:08:04.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.989+0000 7f59b8e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f59b415c360 0x7f59b40c9570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:08:04.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.989+0000 7f59b8e00640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f59b40c5e40 0x7f59b40cc5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:08:04.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.989+0000 7f59b8e00640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f59b4071c80 con 0x7f59b40ccb30 2024-05-09T08:08:04.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.989+0000 7f59b8e00640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f59b415b980 con 0x7f59b40c5e40 2024-05-09T08:08:04.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.989+0000 7f59b8e00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f59b4071f20 con 0x7f59b415c360 2024-05-09T08:08:04.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.989+0000 7f59b1400640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f59b40c5e40 0x7f59b40cc5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:08:04.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.989+0000 7f59b1400640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f59b40c5e40 0x7f59b40cc5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:36896/0 (socket says 172.21.15.1:36896) 2024-05-09T08:08:04.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.989+0000 7f59b1400640 1 -- 172.21.15.1:0/796968705 learned_addr learned my addr 172.21.15.1:0/796968705 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:08:04.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.989+0000 7f59b1e00640 1 --2- 172.21.15.1:0/796968705 >> v2:172.21.15.7:3300/0 conn(0x7f59b415c360 0x7f59b40c9570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:08:04.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.990+0000 7f59b1400640 1 -- 172.21.15.1:0/796968705 >> v2:172.21.15.7:3300/0 conn(0x7f59b415c360 msgr2=0x7f59b40c9570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:08:04.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.990+0000 7f59b1400640 1 --2- 172.21.15.1:0/796968705 >> v2:172.21.15.7:3300/0 conn(0x7f59b415c360 0x7f59b40c9570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:04.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.990+0000 7f59b1400640 1 -- 172.21.15.1:0/796968705 >> v1:172.21.15.1:6789/0 conn(0x7f59b40ccb30 legacy=0x7f59b4072c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:08:04.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.990+0000 7f59b1400640 1 -- 172.21.15.1:0/796968705 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59b40721e0 con 0x7f59b40c5e40 2024-05-09T08:08:04.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.990+0000 7f59b1e00640 1 --2- 172.21.15.1:0/796968705 >> v2:172.21.15.7:3300/0 conn(0x7f59b415c360 0x7f59b40c9570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:08:04.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.991+0000 7f59b1400640 1 --2- 172.21.15.1:0/796968705 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f59b40c5e40 0x7f59b40cc5f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f59b40c8c60 tx=0x7f59a002a160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a47e414a27ea734f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:08:04.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.992+0000 7f59b0a00640 1 -- 172.21.15.1:0/796968705 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59a0037070 con 0x7f59b40c5e40 2024-05-09T08:08:04.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.992+0000 7f59b0a00640 1 -- 172.21.15.1:0/796968705 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59a0032440 con 0x7f59b40c5e40 2024-05-09T08:08:04.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.992+0000 7f59b0a00640 1 -- 172.21.15.1:0/796968705 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59a0032810 con 0x7f59b40c5e40 2024-05-09T08:08:04.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.992+0000 7f59b8e00640 1 -- 172.21.15.1:0/796968705 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f59b40c5e40 msgr2=0x7f59b40cc5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:08:04.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.992+0000 7f59b8e00640 1 --2- 172.21.15.1:0/796968705 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f59b40c5e40 0x7f59b40cc5f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f59b40c8c60 tx=0x7f59a002a160 comp rx=0 tx=0).stop 2024-05-09T08:08:04.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.992+0000 7f59b8e00640 1 -- 172.21.15.1:0/796968705 shutdown_connections 2024-05-09T08:08:04.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.992+0000 7f59b8e00640 1 --2- 172.21.15.1:0/796968705 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f59b40c5e40 0x7f59b40cc5f0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:04.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.992+0000 7f59b8e00640 1 --2- 172.21.15.1:0/796968705 >> v2:172.21.15.7:3300/0 conn(0x7f59b415c360 0x7f59b40c9570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:04.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.992+0000 7f59b8e00640 1 -- 172.21.15.1:0/796968705 >> 172.21.15.1:0/796968705 conn(0x7f59b406dad0 msgr2=0x7f59b406deb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:08:04.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.992+0000 7f59b8e00640 1 -- 172.21.15.1:0/796968705 shutdown_connections 2024-05-09T08:08:04.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.992+0000 7f59b8e00640 1 -- 172.21.15.1:0/796968705 wait complete. 2024-05-09T08:08:04.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.994+0000 7f59b8e00640 1 Processor -- start 2024-05-09T08:08:04.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.994+0000 7f59b8e00640 1 -- start start 2024-05-09T08:08:04.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.995+0000 7f59b8e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f59b40c5e40 0x7f59b4149d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:08:04.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.995+0000 7f59b8e00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f59b40ccb30 0x7f59b41537c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:08:04.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.995+0000 7f59b8e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f59b415c360 0x7f59b41f0580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:08:04.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.995+0000 7f59b8e00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f59b40cc0d0 con 0x7f59b415c360 2024-05-09T08:08:04.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.995+0000 7f59b8e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f59b40727a0 con 0x7f59b40c5e40 2024-05-09T08:08:04.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.995+0000 7f59b8e00640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f59b415c740 con 0x7f59b40ccb30 2024-05-09T08:08:04.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.995+0000 7f59b2800640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f59b415c360 0x7f59b41f0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:08:04.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.996+0000 7f59b1e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f59b40c5e40 0x7f59b4149d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:08:04.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.996+0000 7f59b1400640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f59b40ccb30 0x7f59b41537c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:08:04.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.996+0000 7f59b2800640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f59b415c360 0x7f59b41f0580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:36906/0 (socket says 172.21.15.1:36906) 2024-05-09T08:08:04.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.996+0000 7f59b2800640 1 -- 172.21.15.1:0/2931912336 learned_addr learned my addr 172.21.15.1:0/2931912336 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:08:04.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.996+0000 7f59b1400640 1 -- 172.21.15.1:0/2931912336 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f59b40c5e40 msgr2=0x7f59b4149d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:08:04.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.996+0000 7f59b1400640 1 --2- 172.21.15.1:0/2931912336 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f59b40c5e40 0x7f59b4149d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:04.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.996+0000 7f59b1400640 1 -- 172.21.15.1:0/2931912336 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f59b415c360 msgr2=0x7f59b41f0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:08:04.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.996+0000 7f59b1400640 1 --2- 172.21.15.1:0/2931912336 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f59b415c360 0x7f59b41f0580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:04.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.996+0000 7f59b1400640 1 -- 172.21.15.1:0/2931912336 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59b41f0080 con 0x7f59b40ccb30 2024-05-09T08:08:04.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.996+0000 7f59b2800640 1 --2- 172.21.15.1:0/2931912336 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f59b415c360 0x7f59b41f0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:08:04.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.996+0000 7f59b1400640 1 --2- 172.21.15.1:0/2931912336 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f59b40ccb30 0x7f59b41537c0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f59a0031000 tx=0x7f59a00046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:08:04.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.997+0000 7f59aa000640 1 -- 172.21.15.1:0/2931912336 <== mon.2 v2:172.21.15.7:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59a0041030 con 0x7f59b40ccb30 2024-05-09T08:08:04.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.997+0000 7f59aa000640 1 -- 172.21.15.1:0/2931912336 <== mon.2 v2:172.21.15.7:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59a0030440 con 0x7f59b40ccb30 2024-05-09T08:08:04.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.997+0000 7f59aa000640 1 -- 172.21.15.1:0/2931912336 <== mon.2 v2:172.21.15.7:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59a0037040 con 0x7f59b40ccb30 2024-05-09T08:08:04.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.997+0000 7f59b8e00640 1 -- 172.21.15.1:0/2931912336 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59b41f3790 con 0x7f59b40ccb30 2024-05-09T08:08:04.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.997+0000 7f59b8e00640 1 -- 172.21.15.1:0/2931912336 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f59b41f3d50 con 0x7f59b40ccb30 2024-05-09T08:08:04.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.998+0000 7f59b8e00640 1 -- 172.21.15.1:0/2931912336 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59b41f0250 con 0x7f59b40ccb30 2024-05-09T08:08:04.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:03.999+0000 7f59aa000640 1 -- 172.21.15.1:0/2931912336 <== mon.2 v2:172.21.15.7:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f59a0044070 con 0x7f59b40ccb30 2024-05-09T08:08:04.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:04.000+0000 7f59aa000640 1 --2- 172.21.15.1:0/2931912336 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f599406be30 0x7f599406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:08:04.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:04.000+0000 7f59aa000640 1 -- 172.21.15.1:0/2931912336 <== mon.2 v2:172.21.15.7:3301/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7f59a00ab740 con 0x7f59b40ccb30 2024-05-09T08:08:04.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:04.000+0000 7f59b1e00640 1 --2- 172.21.15.1:0/2931912336 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f599406be30 0x7f599406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:08:04.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:04.001+0000 7f59b1e00640 1 --2- 172.21.15.1:0/2931912336 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f599406be30 0x7f599406e2f0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f59a4009560 tx=0x7f59a4009190 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:08:04.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:04.005+0000 7f59aa000640 1 -- 172.21.15.1:0/2931912336 <== mon.2 v2:172.21.15.7:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f59a00797c0 con 0x7f59b40ccb30 2024-05-09T08:08:04.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:04.209+0000 7f59b8e00640 1 -- 172.21.15.1:0/2931912336 --> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) -- 0x7f59b414e930 con 0x7f599406be30 2024-05-09T08:08:04.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:04.210+0000 7f59aa000640 1 -- 172.21.15.1:0/2931912336 <== mgr.4114 v2:172.21.15.7:6832/3562612464 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f59b414e930 con 0x7f599406be30 2024-05-09T08:08:04.220 INFO:teuthology.orchestra.run.smithi001.stderr:instructed osd(s) 6 to deep-scrub 2024-05-09T08:08:04.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:04.212+0000 7f59b8e00640 1 -- 172.21.15.1:0/2931912336 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f599406be30 msgr2=0x7f599406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:08:04.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:04.212+0000 7f59b8e00640 1 --2- 172.21.15.1:0/2931912336 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f599406be30 0x7f599406e2f0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f59a4009560 tx=0x7f59a4009190 comp rx=0 tx=0).stop 2024-05-09T08:08:04.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:04.212+0000 7f59b8e00640 1 -- 172.21.15.1:0/2931912336 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f59b40ccb30 msgr2=0x7f59b41537c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:08:04.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:04.212+0000 7f59b8e00640 1 --2- 172.21.15.1:0/2931912336 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f59b40ccb30 0x7f59b41537c0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f59a0031000 tx=0x7f59a00046c0 comp rx=0 tx=0).stop 2024-05-09T08:08:04.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:04.213+0000 7f59b8e00640 1 -- 172.21.15.1:0/2931912336 shutdown_connections 2024-05-09T08:08:04.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:04.213+0000 7f59b8e00640 1 --2- 172.21.15.1:0/2931912336 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f599406be30 0x7f599406e2f0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:04.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:04.213+0000 7f59b8e00640 1 --2- 172.21.15.1:0/2931912336 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f59b415c360 0x7f59b41f0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:04.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:04.213+0000 7f59b8e00640 1 --2- 172.21.15.1:0/2931912336 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f59b40ccb30 0x7f59b41537c0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:04.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:04.213+0000 7f59b8e00640 1 --2- 172.21.15.1:0/2931912336 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f59b40c5e40 0x7f59b4149d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:04.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:04.213+0000 7f59b8e00640 1 -- 172.21.15.1:0/2931912336 >> 172.21.15.1:0/2931912336 conn(0x7f59b406dad0 msgr2=0x7f59b406deb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:08:04.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:04.213+0000 7f59b8e00640 1 -- 172.21.15.1:0/2931912336 shutdown_connections 2024-05-09T08:08:04.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:04.213+0000 7f59b8e00640 1 -- 172.21.15.1:0/2931912336 wait complete. 2024-05-09T08:08:04.243 INFO:tasks.ceph:Scrubbing osd.7 2024-05-09T08:08:04.243 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-05-09T08:08:04.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:04.453+0000 7ff460e00640 1 Processor -- start 2024-05-09T08:08:04.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:04.454+0000 7ff460e00640 1 -- start start 2024-05-09T08:08:04.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:04.455+0000 7ff460e00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ff45c06de50 0x7ff45c163c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:08:04.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:04.455+0000 7ff460e00640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff45c158f20 con 0x7ff45c152550 2024-05-09T08:08:04.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:04.455+0000 7ff460e00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff45c06eab0 con 0x7ff45c0734e0 2024-05-09T08:08:04.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:04.455+0000 7ff460e00640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7ff45c06ed90 con 0x7ff45c06de50 2024-05-09T08:08:04.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:04.455+0000 7ff459e00640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7ff45c152550 0x7ff45c0723d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:45198/0 (socket says 172.21.15.1:45198) 2024-05-09T08:08:04.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:04.455+0000 7ff459e00640 1 -- 172.21.15.1:0/963827338 learned_addr learned my addr 172.21.15.1:0/963827338 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T08:08:04.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:04.455+0000 7ff45a800640 1 --2- 172.21.15.1:0/963827338 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ff45c06de50 0x7ff45c163c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:08:04.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:04.456+0000 7ff458a00640 1 -- 172.21.15.1:0/963827338 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3570139523 0 0) 0x7ff45c158f20 con 0x7ff45c152550 2024-05-09T08:08:04.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:04.456+0000 7ff458a00640 1 -- 172.21.15.1:0/963827338 --> v1:172.21.15.1:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff43c003520 con 0x7ff45c152550 2024-05-09T08:08:04.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:04.456+0000 7ff45a800640 1 -- 172.21.15.1:0/963827338 >> v1:172.21.15.7:6789/0 conn(0x7ff45c0734e0 legacy=0x7ff45c06d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:08:04.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:04.456+0000 7ff45a800640 1 -- 172.21.15.1:0/963827338 >> v1:172.21.15.1:6789/0 conn(0x7ff45c152550 legacy=0x7ff45c0723d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:08:04.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:04.456+0000 7ff458a00640 1 -- 172.21.15.1:0/963827338 <== mon.1 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3575602138 0 0) 0x7ff45c06eab0 con 0x7ff45c0734e0 2024-05-09T08:08:04.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:04.456+0000 7ff45a800640 1 -- 172.21.15.1:0/963827338 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff45c1591c0 con 0x7ff45c06de50 2024-05-09T08:08:04.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:04.457+0000 7ff45a800640 1 --2- 172.21.15.1:0/963827338 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ff45c06de50 0x7ff45c163c70 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7ff4540097d0 tx=0x7ff45402a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3b46e3e8691e9acf server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:08:04.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:04.457+0000 7ff458a00640 1 -- 172.21.15.1:0/963827338 <== mon.2 v2:172.21.15.7:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff454037070 con 0x7ff45c06de50 2024-05-09T08:08:04.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:04.457+0000 7ff458a00640 1 -- 172.21.15.1:0/963827338 <== mon.2 v2:172.21.15.7:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff454032440 con 0x7ff45c06de50 2024-05-09T08:08:04.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:04.457+0000 7ff458a00640 1 -- 172.21.15.1:0/963827338 <== mon.2 v2:172.21.15.7:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4540327f0 con 0x7ff45c06de50 2024-05-09T08:08:04.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:04.458+0000 7ff460e00640 1 -- 172.21.15.1:0/963827338 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ff45c06de50 msgr2=0x7ff45c163c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:08:04.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:04.458+0000 7ff460e00640 1 --2- 172.21.15.1:0/963827338 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ff45c06de50 0x7ff45c163c70 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7ff4540097d0 tx=0x7ff45402a180 comp rx=0 tx=0).stop 2024-05-09T08:08:04.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:04.458+0000 7ff460e00640 1 -- 172.21.15.1:0/963827338 shutdown_connections 2024-05-09T08:08:04.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:04.458+0000 7ff460e00640 1 --2- 172.21.15.1:0/963827338 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ff45c06de50 0x7ff45c163c70 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:04.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:04.458+0000 7ff460e00640 1 -- 172.21.15.1:0/963827338 >> 172.21.15.1:0/963827338 conn(0x7ff45c0c89b0 msgr2=0x7ff45c0c8dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:08:04.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:04.458+0000 7ff460e00640 1 -- 172.21.15.1:0/963827338 shutdown_connections 2024-05-09T08:08:04.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:04.458+0000 7ff460e00640 1 -- 172.21.15.1:0/963827338 wait complete. 2024-05-09T08:08:04.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:04.460+0000 7ff460e00640 1 Processor -- start 2024-05-09T08:08:04.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:04.460+0000 7ff460e00640 1 -- start start 2024-05-09T08:08:04.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:04.461+0000 7ff460e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff45c06de50 0x7ff45c148230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:08:04.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:04.461+0000 7ff460e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff45c0734e0 0x7ff45c0caf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:08:04.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:04.461+0000 7ff460e00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ff45c152550 0x7ff45c1f1150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:08:04.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:04.461+0000 7ff460e00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ff45c06e860 con 0x7ff45c0734e0 2024-05-09T08:08:04.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:04.461+0000 7ff460e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff45c071eb0 con 0x7ff45c06de50 2024-05-09T08:08:04.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:04.461+0000 7ff460e00640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7ff45c152930 con 0x7ff45c152550 2024-05-09T08:08:04.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:04.462+0000 7ff459e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff45c06de50 0x7ff45c148230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:08:04.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:04.462+0000 7ff459400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff45c0734e0 0x7ff45c0caf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:08:04.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:04.462+0000 7ff45a800640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ff45c152550 0x7ff45c1f1150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:08:04.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:04.462+0000 7ff459e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff45c06de50 0x7ff45c148230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.1:50322/0 (socket says 172.21.15.1:50322) 2024-05-09T08:08:04.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:04.462+0000 7ff459e00640 1 -- 172.21.15.1:0/3776694779 learned_addr learned my addr 172.21.15.1:0/3776694779 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:08:04.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:04.462+0000 7ff45a800640 1 -- 172.21.15.1:0/3776694779 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff45c06de50 msgr2=0x7ff45c148230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:08:04.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:04.462+0000 7ff45a800640 1 --2- 172.21.15.1:0/3776694779 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff45c06de50 0x7ff45c148230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:04.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:04.462+0000 7ff45a800640 1 -- 172.21.15.1:0/3776694779 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff45c0734e0 msgr2=0x7ff45c0caf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:08:04.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:04.462+0000 7ff45a800640 1 --2- 172.21.15.1:0/3776694779 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff45c0734e0 0x7ff45c0caf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:04.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:04.462+0000 7ff45a800640 1 -- 172.21.15.1:0/3776694779 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff45c148770 con 0x7ff45c152550 2024-05-09T08:08:04.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:04.463+0000 7ff45a800640 1 --2- 172.21.15.1:0/3776694779 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ff45c152550 0x7ff45c1f1150 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ff45c1590f0 tx=0x7ff4540046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:08:04.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:04.463+0000 7ff452000640 1 -- 172.21.15.1:0/3776694779 <== mon.2 v2:172.21.15.7:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff454041020 con 0x7ff45c152550 2024-05-09T08:08:04.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:04.463+0000 7ff452000640 1 -- 172.21.15.1:0/3776694779 <== mon.2 v2:172.21.15.7:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff45403d620 con 0x7ff45c152550 2024-05-09T08:08:04.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:04.463+0000 7ff452000640 1 -- 172.21.15.1:0/3776694779 <== mon.2 v2:172.21.15.7:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff454037040 con 0x7ff45c152550 2024-05-09T08:08:04.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:04.463+0000 7ff460e00640 1 -- 172.21.15.1:0/3776694779 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff45c143b20 con 0x7ff45c152550 2024-05-09T08:08:04.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:04.463+0000 7ff460e00640 1 -- 172.21.15.1:0/3776694779 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7ff45c143840 con 0x7ff45c152550 2024-05-09T08:08:04.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:04.464+0000 7ff460e00640 1 -- 172.21.15.1:0/3776694779 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff45c143310 con 0x7ff45c152550 2024-05-09T08:08:04.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:04.465+0000 7ff452000640 1 -- 172.21.15.1:0/3776694779 <== mon.2 v2:172.21.15.7:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff454044020 con 0x7ff45c152550 2024-05-09T08:08:04.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:04.466+0000 7ff452000640 1 --2- 172.21.15.1:0/3776694779 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7ff43806be30 0x7ff43806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:08:04.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:04.466+0000 7ff452000640 1 -- 172.21.15.1:0/3776694779 <== mon.2 v2:172.21.15.7:3301/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7ff4540ab940 con 0x7ff45c152550 2024-05-09T08:08:04.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:04.466+0000 7ff452000640 1 --2- 172.21.15.1:0/3776694779 >> [v2:172.21.15.7:6824/1442884138,v1:172.21.15.7:6825/1442884138] conn(0x7ff4380757c0 0x7ff438077c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:08:04.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:04.466+0000 7ff452000640 1 -- 172.21.15.1:0/3776694779 --> [v2:172.21.15.7:6824/1442884138,v1:172.21.15.7:6825/1442884138] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff454008eb0 con 0x7ff4380757c0 2024-05-09T08:08:04.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:04.466+0000 7ff452000640 1 -- 172.21.15.1:0/3776694779 <== mon.2 v2:172.21.15.7:3301/0 6 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff4540b02a0 con 0x7ff45c152550 2024-05-09T08:08:04.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:04.467+0000 7ff459e00640 1 --2- 172.21.15.1:0/3776694779 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7ff43806be30 0x7ff43806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:08:04.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:04.467+0000 7ff459400640 1 --2- 172.21.15.1:0/3776694779 >> [v2:172.21.15.7:6824/1442884138,v1:172.21.15.7:6825/1442884138] conn(0x7ff4380757c0 0x7ff438077c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:08:04.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:04.467+0000 7ff459e00640 1 --2- 172.21.15.1:0/3776694779 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7ff43806be30 0x7ff43806e2f0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7ff44c005880 tx=0x7ff44c009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:08:04.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:04.467+0000 7ff459400640 1 --2- 172.21.15.1:0/3776694779 >> [v2:172.21.15.7:6824/1442884138,v1:172.21.15.7:6825/1442884138] conn(0x7ff4380757c0 0x7ff438077c60 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:08:04.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:04.468+0000 7ff452000640 1 -- 172.21.15.1:0/3776694779 <== osd.7 v2:172.21.15.7:6824/1442884138 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff454008eb0 con 0x7ff4380757c0 2024-05-09T08:08:04.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:04.483+0000 7ff460e00640 1 -- 172.21.15.1:0/3776694779 --> [v2:172.21.15.7:6824/1442884138,v1:172.21.15.7:6825/1442884138] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7ff45c068520 con 0x7ff4380757c0 2024-05-09T08:08:04.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:04.484+0000 7ff452000640 1 -- 172.21.15.1:0/3776694779 <== osd.7 v2:172.21.15.7:6824/1442884138 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7ff45c068520 con 0x7ff4380757c0 2024-05-09T08:08:04.494 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-05-09T08:08:04.494 INFO:teuthology.orchestra.run.smithi001.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-09T08:08:04.494 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-05-09T08:08:04.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:04.484+0000 7ff460e00640 1 -- 172.21.15.1:0/3776694779 >> [v2:172.21.15.7:6824/1442884138,v1:172.21.15.7:6825/1442884138] conn(0x7ff4380757c0 msgr2=0x7ff438077c60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:08:04.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:04.484+0000 7ff460e00640 1 --2- 172.21.15.1:0/3776694779 >> [v2:172.21.15.7:6824/1442884138,v1:172.21.15.7:6825/1442884138] conn(0x7ff4380757c0 0x7ff438077c60 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:04.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:04.484+0000 7ff460e00640 1 -- 172.21.15.1:0/3776694779 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7ff43806be30 msgr2=0x7ff43806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:08:04.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:04.484+0000 7ff460e00640 1 --2- 172.21.15.1:0/3776694779 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7ff43806be30 0x7ff43806e2f0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7ff44c005880 tx=0x7ff44c009040 comp rx=0 tx=0).stop 2024-05-09T08:08:04.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:04.484+0000 7ff460e00640 1 -- 172.21.15.1:0/3776694779 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ff45c152550 msgr2=0x7ff45c1f1150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:08:04.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:04.484+0000 7ff460e00640 1 --2- 172.21.15.1:0/3776694779 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ff45c152550 0x7ff45c1f1150 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ff45c1590f0 tx=0x7ff4540046c0 comp rx=0 tx=0).stop 2024-05-09T08:08:04.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:04.484+0000 7ff45a800640 1 -- 172.21.15.1:0/3776694779 reap_dead start 2024-05-09T08:08:04.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:04.484+0000 7ff460e00640 1 -- 172.21.15.1:0/3776694779 shutdown_connections 2024-05-09T08:08:04.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:04.484+0000 7ff460e00640 1 -- 172.21.15.1:0/3776694779 >> 172.21.15.1:0/3776694779 conn(0x7ff45c0c89b0 msgr2=0x7ff45c0c8f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:08:04.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:04.484+0000 7ff460e00640 1 -- 172.21.15.1:0/3776694779 shutdown_connections 2024-05-09T08:08:04.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:04.484+0000 7ff460e00640 1 -- 172.21.15.1:0/3776694779 wait complete. 2024-05-09T08:08:04.513 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-05-09T08:08:04.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:04.696+0000 7f4748600640 1 Processor -- start 2024-05-09T08:08:04.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:04.696+0000 7f4748600640 1 -- start start 2024-05-09T08:08:04.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:04.697+0000 7f4748600640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f47400cb1f0 0x7f47400b3570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:08:04.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:04.697+0000 7f4748600640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f47400b4380 0x7f47400b47d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:08:04.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:04.697+0000 7f4748600640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f474015aa20 con 0x7f47400c9dc0 2024-05-09T08:08:04.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:04.697+0000 7f4748600640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f474015ace0 con 0x7f47400b4380 2024-05-09T08:08:04.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:04.697+0000 7f4748600640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f474015afc0 con 0x7f47400cb1f0 2024-05-09T08:08:04.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:04.697+0000 7f4746600640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f47400b4380 0x7f47400b47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:08:04.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:04.697+0000 7f4746600640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f47400b4380 0x7f47400b47d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:36920/0 (socket says 172.21.15.1:36920) 2024-05-09T08:08:04.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:04.698+0000 7f4746600640 1 -- 172.21.15.1:0/2251984220 learned_addr learned my addr 172.21.15.1:0/2251984220 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:08:04.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:04.697+0000 7f4745200640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f47400cb1f0 0x7f47400b3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:08:04.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:04.698+0000 7f4745c00640 1 --1- 172.21.15.1:0/2251984220 >> v1:172.21.15.7:6789/0 conn(0x7f47400c9dc0 0x7f47400d03a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.1:37364/0 (socket says 172.21.15.1:37364) 2024-05-09T08:08:04.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:04.698+0000 7f4745200640 1 -- 172.21.15.1:0/2251984220 >> v2:172.21.15.1:3300/0 conn(0x7f47400b4380 msgr2=0x7f47400b47d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:08:04.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:04.698+0000 7f4745200640 1 --2- 172.21.15.1:0/2251984220 >> v2:172.21.15.1:3300/0 conn(0x7f47400b4380 0x7f47400b47d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:04.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:04.698+0000 7f473fe00640 1 -- 172.21.15.1:0/2251984220 <== mon.1 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1305599087 0 0) 0x7f474015aa20 con 0x7f47400c9dc0 2024-05-09T08:08:04.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:04.698+0000 7f4745200640 1 -- 172.21.15.1:0/2251984220 >> v1:172.21.15.7:6789/0 conn(0x7f47400c9dc0 legacy=0x7f47400d03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:08:04.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:04.698+0000 7f4745200640 1 -- 172.21.15.1:0/2251984220 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f474015b2a0 con 0x7f47400cb1f0 2024-05-09T08:08:04.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:04.699+0000 7f4745200640 1 --2- 172.21.15.1:0/2251984220 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f47400cb1f0 0x7f47400b3570 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f472c0097a0 tx=0x7f472c02a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c58702a9af66dc99 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:08:04.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:04.699+0000 7f473fe00640 1 -- 172.21.15.1:0/2251984220 <== mon.2 v2:172.21.15.7:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f472c037070 con 0x7f47400cb1f0 2024-05-09T08:08:04.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:04.699+0000 7f473fe00640 1 -- 172.21.15.1:0/2251984220 <== mon.2 v2:172.21.15.7:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f472c032440 con 0x7f47400cb1f0 2024-05-09T08:08:04.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:04.699+0000 7f473fe00640 1 -- 172.21.15.1:0/2251984220 <== mon.2 v2:172.21.15.7:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f472c032810 con 0x7f47400cb1f0 2024-05-09T08:08:04.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:04.700+0000 7f4748600640 1 -- 172.21.15.1:0/2251984220 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f47400cb1f0 msgr2=0x7f47400b3570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:08:04.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:04.700+0000 7f4748600640 1 --2- 172.21.15.1:0/2251984220 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f47400cb1f0 0x7f47400b3570 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f472c0097a0 tx=0x7f472c02a180 comp rx=0 tx=0).stop 2024-05-09T08:08:04.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:04.700+0000 7f4748600640 1 -- 172.21.15.1:0/2251984220 shutdown_connections 2024-05-09T08:08:04.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:04.700+0000 7f4748600640 1 --2- 172.21.15.1:0/2251984220 >> v2:172.21.15.1:3300/0 conn(0x7f47400b4380 0x7f47400b47d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:04.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:04.700+0000 7f4748600640 1 --2- 172.21.15.1:0/2251984220 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f47400cb1f0 0x7f47400b3570 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:04.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:04.700+0000 7f4748600640 1 -- 172.21.15.1:0/2251984220 >> 172.21.15.1:0/2251984220 conn(0x7f474008c410 msgr2=0x7f474008c820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:08:04.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:04.700+0000 7f4748600640 1 -- 172.21.15.1:0/2251984220 shutdown_connections 2024-05-09T08:08:04.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:04.700+0000 7f4748600640 1 -- 172.21.15.1:0/2251984220 wait complete. 2024-05-09T08:08:04.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:04.702+0000 7f4748600640 1 Processor -- start 2024-05-09T08:08:04.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:04.702+0000 7f4748600640 1 -- start start 2024-05-09T08:08:04.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:04.703+0000 7f4748600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f47400b4380 0x7f47401f41a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:08:04.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:04.703+0000 7f4748600640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f47400c9dc0 0x7f4740153ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:08:04.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:04.703+0000 7f4748600640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f47400cb1f0 0x7f47400c85d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:08:04.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:04.703+0000 7f4748600640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f47400b3e60 con 0x7f47400c9dc0 2024-05-09T08:08:04.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:04.703+0000 7f4748600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f47400b3b80 con 0x7f47400b4380 2024-05-09T08:08:04.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:04.703+0000 7f4748600640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f474015a710 con 0x7f47400cb1f0 2024-05-09T08:08:04.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:04.703+0000 7f4745c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f47400b4380 0x7f47401f41a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:08:04.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:04.703+0000 7f4745200640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f47400c9dc0 0x7f4740153ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:08:04.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:04.703+0000 7f4745c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f47400b4380 0x7f47401f41a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.1:50338/0 (socket says 172.21.15.1:50338) 2024-05-09T08:08:04.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:04.703+0000 7f4745c00640 1 -- 172.21.15.1:0/1331482708 learned_addr learned my addr 172.21.15.1:0/1331482708 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:08:04.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:04.703+0000 7f4746600640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f47400cb1f0 0x7f47400c85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:08:04.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:04.704+0000 7f4745c00640 1 -- 172.21.15.1:0/1331482708 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f47400cb1f0 msgr2=0x7f47400c85d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:08:04.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:04.704+0000 7f4745c00640 1 --2- 172.21.15.1:0/1331482708 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f47400cb1f0 0x7f47400c85d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:04.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:04.704+0000 7f4745c00640 1 -- 172.21.15.1:0/1331482708 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f47400c9dc0 msgr2=0x7f4740153ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:08:04.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:04.704+0000 7f4745c00640 1 --2- 172.21.15.1:0/1331482708 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f47400c9dc0 0x7f4740153ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:04.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:04.704+0000 7f4745c00640 1 -- 172.21.15.1:0/1331482708 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47400c5ca0 con 0x7f47400b4380 2024-05-09T08:08:04.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:04.704+0000 7f4745c00640 1 --2- 172.21.15.1:0/1331482708 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f47400b4380 0x7f47401f41a0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f473000c8c0 tx=0x7f473000cd80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:08:04.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:04.704+0000 7f473d600640 1 -- 172.21.15.1:0/1331482708 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4730018070 con 0x7f47400b4380 2024-05-09T08:08:04.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:04.704+0000 7f473d600640 1 -- 172.21.15.1:0/1331482708 <== mon.1 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f473000a640 con 0x7f47400b4380 2024-05-09T08:08:04.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:04.704+0000 7f473d600640 1 -- 172.21.15.1:0/1331482708 <== mon.1 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f473000aa30 con 0x7f47400b4380 2024-05-09T08:08:04.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:04.704+0000 7f4748600640 1 -- 172.21.15.1:0/1331482708 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47400c5f80 con 0x7f47400b4380 2024-05-09T08:08:04.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:04.704+0000 7f4748600640 1 -- 172.21.15.1:0/1331482708 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47400c6540 con 0x7f47400b4380 2024-05-09T08:08:04.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:04.706+0000 7f4748600640 1 -- 172.21.15.1:0/1331482708 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47400ca1a0 con 0x7f47400b4380 2024-05-09T08:08:04.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:04.707+0000 7f473d600640 1 -- 172.21.15.1:0/1331482708 <== mon.1 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f473000abd0 con 0x7f47400b4380 2024-05-09T08:08:04.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:04.708+0000 7f473d600640 1 --2- 172.21.15.1:0/1331482708 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f472006be60 0x7f472006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:08:04.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:04.708+0000 7f473d600640 1 -- 172.21.15.1:0/1331482708 <== mon.1 v2:172.21.15.7:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7f473008a950 con 0x7f47400b4380 2024-05-09T08:08:04.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:04.708+0000 7f4745200640 1 --2- 172.21.15.1:0/1331482708 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f472006be60 0x7f472006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:08:04.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:04.709+0000 7f4745200640 1 --2- 172.21.15.1:0/1331482708 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f472006be60 0x7f472006e320 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f472c034000 tx=0x7f472c02e000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:08:04.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:04.712+0000 7f473d600640 1 -- 172.21.15.1:0/1331482708 <== mon.1 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f47400ca1a0 con 0x7f47400b4380 2024-05-09T08:08:04.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:04.919+0000 7f4748600640 1 -- 172.21.15.1:0/1331482708 --> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) -- 0x7f4740159790 con 0x7f472006be60 2024-05-09T08:08:04.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:04.920+0000 7f473d600640 1 -- 172.21.15.1:0/1331482708 <== mgr.4114 v2:172.21.15.7:6832/3562612464 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f4740159790 con 0x7f472006be60 2024-05-09T08:08:04.930 INFO:teuthology.orchestra.run.smithi001.stderr:instructed osd(s) 7 to deep-scrub 2024-05-09T08:08:04.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:04.923+0000 7f4748600640 1 -- 172.21.15.1:0/1331482708 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f472006be60 msgr2=0x7f472006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:08:04.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:04.923+0000 7f4748600640 1 --2- 172.21.15.1:0/1331482708 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f472006be60 0x7f472006e320 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f472c034000 tx=0x7f472c02e000 comp rx=0 tx=0).stop 2024-05-09T08:08:04.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:04.923+0000 7f4748600640 1 -- 172.21.15.1:0/1331482708 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f47400b4380 msgr2=0x7f47401f41a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:08:04.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:04.923+0000 7f4748600640 1 --2- 172.21.15.1:0/1331482708 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f47400b4380 0x7f47401f41a0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f473000c8c0 tx=0x7f473000cd80 comp rx=0 tx=0).stop 2024-05-09T08:08:04.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:04.923+0000 7f4748600640 1 -- 172.21.15.1:0/1331482708 shutdown_connections 2024-05-09T08:08:04.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:04.923+0000 7f4748600640 1 --2- 172.21.15.1:0/1331482708 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f472006be60 0x7f472006e320 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:04.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:04.923+0000 7f4748600640 1 --2- 172.21.15.1:0/1331482708 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f47400cb1f0 0x7f47400c85d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:04.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:04.923+0000 7f4748600640 1 --2- 172.21.15.1:0/1331482708 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f47400c9dc0 0x7f4740153ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:04.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:04.923+0000 7f4748600640 1 --2- 172.21.15.1:0/1331482708 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f47400b4380 0x7f47401f41a0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:04.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:04.923+0000 7f4748600640 1 -- 172.21.15.1:0/1331482708 >> 172.21.15.1:0/1331482708 conn(0x7f474008c410 msgr2=0x7f47400d0310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:08:04.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:04.923+0000 7f4748600640 1 -- 172.21.15.1:0/1331482708 shutdown_connections 2024-05-09T08:08:04.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:04.923+0000 7f4748600640 1 -- 172.21.15.1:0/1331482708 wait complete. 2024-05-09T08:08:04.953 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-09T08:08:05.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:05.155+0000 7efc89c00640 1 Processor -- start 2024-05-09T08:08:05.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:05.155+0000 7efc89c00640 1 -- start start 2024-05-09T08:08:05.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:05.157+0000 7efc89c00640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7efc841525e0 0x7efc8415c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:08:05.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:05.157+0000 7efc89c00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7efc840c7620 0x7efc840c9f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:08:05.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:05.157+0000 7efc89c00640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efc841529c0 con 0x7efc840cd040 2024-05-09T08:08:05.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:05.157+0000 7efc89c00640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7efc840c94c0 con 0x7efc841525e0 2024-05-09T08:08:05.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:05.157+0000 7efc89c00640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7efc8415c710 con 0x7efc840c7620 2024-05-09T08:08:05.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:05.157+0000 7efc83e00640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7efc841525e0 0x7efc8415c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:08:05.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:05.157+0000 7efc83e00640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7efc841525e0 0x7efc8415c1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:36930/0 (socket says 172.21.15.1:36930) 2024-05-09T08:08:05.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:05.157+0000 7efc83e00640 1 -- 172.21.15.1:0/1539575747 learned_addr learned my addr 172.21.15.1:0/1539575747 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:08:05.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:05.157+0000 7efc83400640 1 --2- 172.21.15.1:0/1539575747 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7efc840c7620 0x7efc840c9f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:08:05.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:05.158+0000 7efc83e00640 1 -- 172.21.15.1:0/1539575747 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7efc840c7620 msgr2=0x7efc840c9f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:08:05.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:05.158+0000 7efc83e00640 1 --2- 172.21.15.1:0/1539575747 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7efc840c7620 0x7efc840c9f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:05.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:05.158+0000 7efc83e00640 1 -- 172.21.15.1:0/1539575747 >> v1:172.21.15.1:6789/0 conn(0x7efc840cd040 legacy=0x7efc84158230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:08:05.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:05.158+0000 7efc83e00640 1 -- 172.21.15.1:0/1539575747 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc840c7ef0 con 0x7efc841525e0 2024-05-09T08:08:05.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:05.159+0000 7efc83e00640 1 --2- 172.21.15.1:0/1539575747 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efc841525e0 0x7efc8415c1d0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7efc74009e50 tx=0x7efc7402a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62f53ba8986c25f5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:08:05.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:05.159+0000 7efc82a00640 1 -- 172.21.15.1:0/1539575747 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc74036070 con 0x7efc841525e0 2024-05-09T08:08:05.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:05.159+0000 7efc82a00640 1 -- 172.21.15.1:0/1539575747 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc74031600 con 0x7efc841525e0 2024-05-09T08:08:05.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:05.159+0000 7efc82a00640 1 -- 172.21.15.1:0/1539575747 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc740319d0 con 0x7efc841525e0 2024-05-09T08:08:05.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:05.160+0000 7efc89c00640 1 -- 172.21.15.1:0/1539575747 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efc841525e0 msgr2=0x7efc8415c1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:08:05.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:05.160+0000 7efc89c00640 1 --2- 172.21.15.1:0/1539575747 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efc841525e0 0x7efc8415c1d0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7efc74009e50 tx=0x7efc7402a7a0 comp rx=0 tx=0).stop 2024-05-09T08:08:05.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:05.160+0000 7efc89c00640 1 -- 172.21.15.1:0/1539575747 shutdown_connections 2024-05-09T08:08:05.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:05.160+0000 7efc89c00640 1 --2- 172.21.15.1:0/1539575747 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7efc840c7620 0x7efc840c9f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:05.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:05.160+0000 7efc89c00640 1 --2- 172.21.15.1:0/1539575747 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efc841525e0 0x7efc8415c1d0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:05.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:05.160+0000 7efc89c00640 1 -- 172.21.15.1:0/1539575747 >> 172.21.15.1:0/1539575747 conn(0x7efc8406e850 msgr2=0x7efc8406ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:08:05.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:05.160+0000 7efc89c00640 1 -- 172.21.15.1:0/1539575747 shutdown_connections 2024-05-09T08:08:05.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:05.160+0000 7efc89c00640 1 -- 172.21.15.1:0/1539575747 wait complete. 2024-05-09T08:08:05.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:05.162+0000 7efc89c00640 1 Processor -- start 2024-05-09T08:08:05.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:05.162+0000 7efc89c00640 1 -- start start 2024-05-09T08:08:05.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:05.163+0000 7efc89c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc840c7620 0x7efc840caf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:08:05.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:05.163+0000 7efc89c00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efc840cd040 0x7efc841ee4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:08:05.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:05.163+0000 7efc89c00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7efc841525e0 0x7efc840c5a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:08:05.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:05.163+0000 7efc89c00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7efc840c9a40 con 0x7efc840cd040 2024-05-09T08:08:05.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:05.163+0000 7efc89c00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7efc840c9760 con 0x7efc840c7620 2024-05-09T08:08:05.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:05.163+0000 7efc89c00640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7efc840c84b0 con 0x7efc841525e0 2024-05-09T08:08:05.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:05.164+0000 7efc83400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efc840cd040 0x7efc841ee4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:08:05.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:05.164+0000 7efc83e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc840c7620 0x7efc840caf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:08:05.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:05.164+0000 7efc83e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc840c7620 0x7efc840caf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.1:50342/0 (socket says 172.21.15.1:50342) 2024-05-09T08:08:05.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:05.164+0000 7efc83e00640 1 -- 172.21.15.1:0/3912633666 learned_addr learned my addr 172.21.15.1:0/3912633666 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:08:05.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:05.164+0000 7efc89200640 1 --2- 172.21.15.1:0/3912633666 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7efc841525e0 0x7efc840c5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:08:05.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:05.164+0000 7efc83e00640 1 -- 172.21.15.1:0/3912633666 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7efc841525e0 msgr2=0x7efc840c5a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:08:05.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:05.164+0000 7efc83e00640 1 --2- 172.21.15.1:0/3912633666 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7efc841525e0 0x7efc840c5a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:05.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:05.164+0000 7efc83e00640 1 -- 172.21.15.1:0/3912633666 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efc840cd040 msgr2=0x7efc841ee4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:08:05.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:05.164+0000 7efc83e00640 1 --2- 172.21.15.1:0/3912633666 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efc840cd040 0x7efc841ee4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:05.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:05.164+0000 7efc83e00640 1 -- 172.21.15.1:0/3912633666 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc84155990 con 0x7efc840c7620 2024-05-09T08:08:05.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:05.164+0000 7efc83400640 1 --2- 172.21.15.1:0/3912633666 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efc840cd040 0x7efc841ee4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:08:05.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:05.164+0000 7efc83e00640 1 --2- 172.21.15.1:0/3912633666 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc840c7620 0x7efc840caf70 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7efc74000f80 tx=0x7efc74004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:08:05.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:05.165+0000 7efc7be00640 1 -- 172.21.15.1:0/3912633666 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc74041020 con 0x7efc840c7620 2024-05-09T08:08:05.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:05.165+0000 7efc7be00640 1 -- 172.21.15.1:0/3912633666 <== mon.1 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc740087b0 con 0x7efc840c7620 2024-05-09T08:08:05.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:05.165+0000 7efc89c00640 1 -- 172.21.15.1:0/3912633666 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc840cc260 con 0x7efc840c7620 2024-05-09T08:08:05.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:05.165+0000 7efc7be00640 1 -- 172.21.15.1:0/3912633666 <== mon.1 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc74036040 con 0x7efc840c7620 2024-05-09T08:08:05.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:05.165+0000 7efc89c00640 1 -- 172.21.15.1:0/3912633666 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc840cc820 con 0x7efc840c7620 2024-05-09T08:08:05.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:05.166+0000 7efc89c00640 1 -- 172.21.15.1:0/3912633666 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc8404f660 con 0x7efc840c7620 2024-05-09T08:08:05.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:05.167+0000 7efc7be00640 1 -- 172.21.15.1:0/3912633666 <== mon.1 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7efc74044020 con 0x7efc840c7620 2024-05-09T08:08:05.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:05.168+0000 7efc7be00640 1 --2- 172.21.15.1:0/3912633666 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7efc6006be30 0x7efc6006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:08:05.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:05.168+0000 7efc7be00640 1 -- 172.21.15.1:0/3912633666 <== mon.1 v2:172.21.15.7:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7efc740abc90 con 0x7efc840c7620 2024-05-09T08:08:05.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:05.168+0000 7efc83400640 1 --2- 172.21.15.1:0/3912633666 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7efc6006be30 0x7efc6006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:08:05.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:05.169+0000 7efc83400640 1 --2- 172.21.15.1:0/3912633666 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7efc6006be30 0x7efc6006e2f0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7efc840caf00 tx=0x7efc6c009190 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:08:05.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:05.173+0000 7efc7be00640 1 -- 172.21.15.1:0/3912633666 <== mon.1 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7efc7404b030 con 0x7efc840c7620 2024-05-09T08:08:05.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:05.370+0000 7efc89c00640 1 -- 172.21.15.1:0/3912633666 --> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7efc84155c30 con 0x7efc6006be30 2024-05-09T08:08:05.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:05.380+0000 7efc7be00640 1 -- 172.21.15.1:0/3912633666 <== mgr.4114 v2:172.21.15.7:6832/3562612464 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+330722 (secure 0 0 0) 0x7efc84155c30 con 0x7efc6006be30 2024-05-09T08:08:05.396 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T08:08:05.397 INFO:teuthology.orchestra.run.smithi001.stderr:dumped all 2024-05-09T08:08:05.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:05.387+0000 7efc89c00640 1 -- 172.21.15.1:0/3912633666 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7efc6006be30 msgr2=0x7efc6006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:08:05.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:05.387+0000 7efc89c00640 1 --2- 172.21.15.1:0/3912633666 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7efc6006be30 0x7efc6006e2f0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7efc840caf00 tx=0x7efc6c009190 comp rx=0 tx=0).stop 2024-05-09T08:08:05.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:05.387+0000 7efc89c00640 1 -- 172.21.15.1:0/3912633666 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc840c7620 msgr2=0x7efc840caf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:08:05.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:05.387+0000 7efc89c00640 1 --2- 172.21.15.1:0/3912633666 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc840c7620 0x7efc840caf70 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7efc74000f80 tx=0x7efc74004a40 comp rx=0 tx=0).stop 2024-05-09T08:08:05.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:05.387+0000 7efc89c00640 1 -- 172.21.15.1:0/3912633666 shutdown_connections 2024-05-09T08:08:05.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:05.387+0000 7efc89c00640 1 --2- 172.21.15.1:0/3912633666 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7efc6006be30 0x7efc6006e2f0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:05.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:05.387+0000 7efc89c00640 1 --2- 172.21.15.1:0/3912633666 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7efc841525e0 0x7efc840c5a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:05.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:05.387+0000 7efc89c00640 1 --2- 172.21.15.1:0/3912633666 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efc840cd040 0x7efc841ee4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:05.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:05.387+0000 7efc89c00640 1 --2- 172.21.15.1:0/3912633666 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc840c7620 0x7efc840caf70 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:05.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:05.387+0000 7efc89c00640 1 -- 172.21.15.1:0/3912633666 >> 172.21.15.1:0/3912633666 conn(0x7efc8406e850 msgr2=0x7efc8414fb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:08:05.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:05.387+0000 7efc89c00640 1 -- 172.21.15.1:0/3912633666 shutdown_connections 2024-05-09T08:08:05.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:05.388+0000 7efc89c00640 1 -- 172.21.15.1:0/3912633666 wait complete. 2024-05-09T08:08:05.417 INFO:teuthology.orchestra.run.smithi001.stdout:{"pg_ready":true,"pg_map":{"version":199,"stamp":"2024-05-09T08:08:05.187981+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1039379,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":121,"num_read_kb":105,"num_write":9657,"num_write_kb":25168860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":11648,"ondisk_log_size":11648,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":481240,"kb_used_data":4376,"kb_used_omap":12,"kb_used_meta":476787,"kb_avail":754493480,"statfs":{"total":773094113280,"available":772601323520,"internally_reserved":0,"allocated":4481024,"data_stored":3030510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":488230450},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":249,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003869"},"pg_stats":[{"pgid":"2.1f","version":"27'6","reported_seq":42,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:08:02.350512+0000","last_change":"2024-05-09T08:08:02.350541+0000","last_active":"2024-05-09T08:08:02.350512+0000","last_peered":"2024-05-09T08:08:02.350512+0000","last_clean":"2024-05-09T08:08:02.350512+0000","last_became_active":"2024-05-09T08:02:14.874841+0000","last_became_peered":"2024-05-09T08:02:14.874841+0000","last_unstale":"2024-05-09T08:08:02.350512+0000","last_undegraded":"2024-05-09T08:08:02.350512+0000","last_fullsized":"2024-05-09T08:08:02.350512+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'6","last_scrub_stamp":"2024-05-09T08:08:02.350436+0000","last_deep_scrub":"27'6","last_deep_scrub_stamp":"2024-05-09T08:08:02.350436+0000","last_clean_scrub_stamp":"2024-05-09T08:08:02.350436+0000","objects_scrubbed":2,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:32:34.124183+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"27'5","reported_seq":31,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:55.778875+0000","last_change":"2024-05-09T08:02:15.899818+0000","last_active":"2024-05-09T08:07:55.778875+0000","last_peered":"2024-05-09T08:07:55.778875+0000","last_clean":"2024-05-09T08:07:55.778875+0000","last_became_active":"2024-05-09T08:02:15.899479+0000","last_became_peered":"2024-05-09T08:02:15.899479+0000","last_unstale":"2024-05-09T08:07:55.778875+0000","last_undegraded":"2024-05-09T08:07:55.778875+0000","last_fullsized":"2024-05-09T08:07:55.778875+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"27'360","reported_seq":381,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:15.296220+0000","last_change":"2024-05-09T08:02:20.057944+0000","last_active":"2024-05-09T08:07:15.296220+0000","last_peered":"2024-05-09T08:07:15.296220+0000","last_clean":"2024-05-09T08:07:15.296220+0000","last_became_active":"2024-05-09T08:02:20.057016+0000","last_became_peered":"2024-05-09T08:02:20.057016+0000","last_unstale":"2024-05-09T08:07:15.296220+0000","last_undegraded":"2024-05-09T08:07:15.296220+0000","last_fullsized":"2024-05-09T08:07:15.296220+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":360,"log_dups_size":0,"ondisk_log_size":360,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":306,"num_write_kb":835584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"24'1","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:06:17.780562+0000","last_change":"2024-05-09T08:02:14.886137+0000","last_active":"2024-05-09T08:06:17.780562+0000","last_peered":"2024-05-09T08:06:17.780562+0000","last_clean":"2024-05-09T08:06:17.780562+0000","last_became_active":"2024-05-09T08:02:14.885830+0000","last_became_peered":"2024-05-09T08:02:14.885830+0000","last_unstale":"2024-05-09T08:06:17.780562+0000","last_undegraded":"2024-05-09T08:06:17.780562+0000","last_fullsized":"2024-05-09T08:06:17.780562+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"27'6","reported_seq":32,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:55.783091+0000","last_change":"2024-05-09T08:02:15.906280+0000","last_active":"2024-05-09T08:07:55.783091+0000","last_peered":"2024-05-09T08:07:55.783091+0000","last_clean":"2024-05-09T08:07:55.783091+0000","last_became_active":"2024-05-09T08:02:15.905590+0000","last_became_peered":"2024-05-09T08:02:15.905590+0000","last_unstale":"2024-05-09T08:07:55.783091+0000","last_undegraded":"2024-05-09T08:07:55.783091+0000","last_fullsized":"2024-05-09T08:07:55.783091+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"27'377","reported_seq":398,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:15.284099+0000","last_change":"2024-05-09T08:02:19.954548+0000","last_active":"2024-05-09T08:07:15.284099+0000","last_peered":"2024-05-09T08:07:15.284099+0000","last_clean":"2024-05-09T08:07:15.284099+0000","last_became_active":"2024-05-09T08:02:19.954161+0000","last_became_peered":"2024-05-09T08:02:19.954161+0000","last_unstale":"2024-05-09T08:07:15.284099+0000","last_undegraded":"2024-05-09T08:07:15.284099+0000","last_fullsized":"2024-05-09T08:07:15.284099+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":377,"log_dups_size":0,"ondisk_log_size":377,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":339,"num_write_kb":925696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"27'3","reported_seq":30,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:05:05.164895+0000","last_change":"2024-05-09T08:02:14.867128+0000","last_active":"2024-05-09T08:05:05.164895+0000","last_peered":"2024-05-09T08:05:05.164895+0000","last_clean":"2024-05-09T08:05:05.164895+0000","last_became_active":"2024-05-09T08:02:14.866809+0000","last_became_peered":"2024-05-09T08:02:14.866809+0000","last_unstale":"2024-05-09T08:05:05.164895+0000","last_undegraded":"2024-05-09T08:05:05.164895+0000","last_fullsized":"2024-05-09T08:05:05.164895+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:57:06.093718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"27'9","reported_seq":34,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:55.773651+0000","last_change":"2024-05-09T08:02:15.888431+0000","last_active":"2024-05-09T08:07:55.773651+0000","last_peered":"2024-05-09T08:07:55.773651+0000","last_clean":"2024-05-09T08:07:55.773651+0000","last_became_active":"2024-05-09T08:02:15.887509+0000","last_became_peered":"2024-05-09T08:02:15.887509+0000","last_unstale":"2024-05-09T08:07:55.773651+0000","last_undegraded":"2024-05-09T08:07:55.773651+0000","last_fullsized":"2024-05-09T08:07:55.773651+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:25:01.238816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"27'324","reported_seq":345,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:22.771050+0000","last_change":"2024-05-09T08:02:19.953600+0000","last_active":"2024-05-09T08:07:22.771050+0000","last_peered":"2024-05-09T08:07:22.771050+0000","last_clean":"2024-05-09T08:07:22.771050+0000","last_became_active":"2024-05-09T08:02:19.952275+0000","last_became_peered":"2024-05-09T08:02:19.952275+0000","last_unstale":"2024-05-09T08:07:22.771050+0000","last_undegraded":"2024-05-09T08:07:22.771050+0000","last_fullsized":"2024-05-09T08:07:22.771050+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":324,"log_dups_size":0,"ondisk_log_size":324,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":273,"num_write_kb":745472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"24'2","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:47.737301+0000","last_change":"2024-05-09T08:02:14.870819+0000","last_active":"2024-05-09T08:04:47.737301+0000","last_peered":"2024-05-09T08:04:47.737301+0000","last_clean":"2024-05-09T08:04:47.737301+0000","last_became_active":"2024-05-09T08:02:14.870476+0000","last_became_peered":"2024-05-09T08:02:14.870476+0000","last_unstale":"2024-05-09T08:04:47.737301+0000","last_undegraded":"2024-05-09T08:04:47.737301+0000","last_fullsized":"2024-05-09T08:04:47.737301+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:43:01.061801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"27'15","reported_seq":40,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:55.773757+0000","last_change":"2024-05-09T08:02:15.886471+0000","last_active":"2024-05-09T08:07:55.773757+0000","last_peered":"2024-05-09T08:07:55.773757+0000","last_clean":"2024-05-09T08:07:55.773757+0000","last_became_active":"2024-05-09T08:02:15.886168+0000","last_became_peered":"2024-05-09T08:02:15.886168+0000","last_unstale":"2024-05-09T08:07:55.773757+0000","last_undegraded":"2024-05-09T08:07:55.773757+0000","last_fullsized":"2024-05-09T08:07:55.773757+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:30:33.351158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"27'298","reported_seq":319,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:22.771013+0000","last_change":"2024-05-09T08:02:19.947347+0000","last_active":"2024-05-09T08:07:22.771013+0000","last_peered":"2024-05-09T08:07:22.771013+0000","last_clean":"2024-05-09T08:07:22.771013+0000","last_became_active":"2024-05-09T08:02:19.946906+0000","last_became_peered":"2024-05-09T08:02:19.946906+0000","last_unstale":"2024-05-09T08:07:22.771013+0000","last_undegraded":"2024-05-09T08:07:22.771013+0000","last_fullsized":"2024-05-09T08:07:22.771013+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":298,"log_dups_size":0,"ondisk_log_size":298,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":252,"num_write_kb":688128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"27'5","reported_seq":31,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:55.773997+0000","last_change":"2024-05-09T08:02:15.880765+0000","last_active":"2024-05-09T08:07:55.773997+0000","last_peered":"2024-05-09T08:07:55.773997+0000","last_clean":"2024-05-09T08:07:55.773997+0000","last_became_active":"2024-05-09T08:02:15.880439+0000","last_became_peered":"2024-05-09T08:02:15.880439+0000","last_unstale":"2024-05-09T08:07:55.773997+0000","last_undegraded":"2024-05-09T08:07:55.773997+0000","last_fullsized":"2024-05-09T08:07:55.773997+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:29:59.441314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"27'3","reported_seq":30,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:05:05.164991+0000","last_change":"2024-05-09T08:02:14.867587+0000","last_active":"2024-05-09T08:05:05.164991+0000","last_peered":"2024-05-09T08:05:05.164991+0000","last_clean":"2024-05-09T08:05:05.164991+0000","last_became_active":"2024-05-09T08:02:14.867291+0000","last_became_peered":"2024-05-09T08:02:14.867291+0000","last_unstale":"2024-05-09T08:05:05.164991+0000","last_undegraded":"2024-05-09T08:05:05.164991+0000","last_fullsized":"2024-05-09T08:05:05.164991+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:20:08.921752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"27'366","reported_seq":387,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:15.293713+0000","last_change":"2024-05-09T08:02:19.954902+0000","last_active":"2024-05-09T08:07:15.293713+0000","last_peered":"2024-05-09T08:07:15.293713+0000","last_clean":"2024-05-09T08:07:15.293713+0000","last_became_active":"2024-05-09T08:02:19.954007+0000","last_became_peered":"2024-05-09T08:02:19.954007+0000","last_unstale":"2024-05-09T08:07:15.293713+0000","last_undegraded":"2024-05-09T08:07:15.293713+0000","last_fullsized":"2024-05-09T08:07:15.293713+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":366,"log_dups_size":0,"ondisk_log_size":366,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":309,"num_write_kb":843776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"27'8","reported_seq":34,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:55.783616+0000","last_change":"2024-05-09T08:02:15.915155+0000","last_active":"2024-05-09T08:07:55.783616+0000","last_peered":"2024-05-09T08:07:55.783616+0000","last_clean":"2024-05-09T08:07:55.783616+0000","last_became_active":"2024-05-09T08:02:15.914852+0000","last_became_peered":"2024-05-09T08:02:15.914852+0000","last_unstale":"2024-05-09T08:07:55.783616+0000","last_undegraded":"2024-05-09T08:07:55.783616+0000","last_fullsized":"2024-05-09T08:07:55.783616+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":28,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:06:37.822275+0000","last_change":"2024-05-09T08:02:14.881770+0000","last_active":"2024-05-09T08:06:37.822275+0000","last_peered":"2024-05-09T08:06:37.822275+0000","last_clean":"2024-05-09T08:06:37.822275+0000","last_became_active":"2024-05-09T08:02:14.881437+0000","last_became_peered":"2024-05-09T08:02:14.881437+0000","last_unstale":"2024-05-09T08:06:37.822275+0000","last_undegraded":"2024-05-09T08:06:37.822275+0000","last_fullsized":"2024-05-09T08:06:37.822275+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"27'330","reported_seq":351,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:15.297681+0000","last_change":"2024-05-09T08:02:19.955928+0000","last_active":"2024-05-09T08:07:15.297681+0000","last_peered":"2024-05-09T08:07:15.297681+0000","last_clean":"2024-05-09T08:07:15.297681+0000","last_became_active":"2024-05-09T08:02:19.955432+0000","last_became_peered":"2024-05-09T08:02:19.955432+0000","last_unstale":"2024-05-09T08:07:15.297681+0000","last_undegraded":"2024-05-09T08:07:15.297681+0000","last_fullsized":"2024-05-09T08:07:15.297681+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":330,"log_dups_size":0,"ondisk_log_size":330,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":291,"num_write_kb":794624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"27'9","reported_seq":35,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:55.778625+0000","last_change":"2024-05-09T08:02:15.913311+0000","last_active":"2024-05-09T08:07:55.778625+0000","last_peered":"2024-05-09T08:07:55.778625+0000","last_clean":"2024-05-09T08:07:55.778625+0000","last_became_active":"2024-05-09T08:02:15.912484+0000","last_became_peered":"2024-05-09T08:02:15.912484+0000","last_unstale":"2024-05-09T08:07:55.778625+0000","last_undegraded":"2024-05-09T08:07:55.778625+0000","last_fullsized":"2024-05-09T08:07:55.778625+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":36,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:08:00.817741+0000","last_change":"2024-05-09T08:08:00.817783+0000","last_active":"2024-05-09T08:08:00.817741+0000","last_peered":"2024-05-09T08:08:00.817741+0000","last_clean":"2024-05-09T08:08:00.817741+0000","last_became_active":"2024-05-09T08:02:14.868553+0000","last_became_peered":"2024-05-09T08:02:14.868553+0000","last_unstale":"2024-05-09T08:08:00.817741+0000","last_undegraded":"2024-05-09T08:08:00.817741+0000","last_fullsized":"2024-05-09T08:08:00.817741+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:08:00.817653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:08:00.817653+0000","last_clean_scrub_stamp":"2024-05-09T08:08:00.817653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:53:20.120654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"27'321","reported_seq":336,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:32.799259+0000","last_change":"2024-05-09T08:03:02.609342+0000","last_active":"2024-05-09T08:07:32.799259+0000","last_peered":"2024-05-09T08:07:32.799259+0000","last_clean":"2024-05-09T08:07:32.799259+0000","last_became_active":"2024-05-09T08:03:02.607564+0000","last_became_peered":"2024-05-09T08:03:02.607564+0000","last_unstale":"2024-05-09T08:07:32.799259+0000","last_undegraded":"2024-05-09T08:07:32.799259+0000","last_fullsized":"2024-05-09T08:07:32.799259+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":321,"log_dups_size":0,"ondisk_log_size":321,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":267,"num_write_kb":729088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4,7],"acting":[2,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"27'2","reported_seq":28,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:55.770803+0000","last_change":"2024-05-09T08:02:15.891115+0000","last_active":"2024-05-09T08:07:55.770803+0000","last_peered":"2024-05-09T08:07:55.770803+0000","last_clean":"2024-05-09T08:07:55.770803+0000","last_became_active":"2024-05-09T08:02:15.890806+0000","last_became_peered":"2024-05-09T08:02:15.890806+0000","last_unstale":"2024-05-09T08:07:55.770803+0000","last_undegraded":"2024-05-09T08:07:55.770803+0000","last_fullsized":"2024-05-09T08:07:55.770803+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:27.735251+0000","last_change":"2024-05-09T08:02:14.874987+0000","last_active":"2024-05-09T08:04:27.735251+0000","last_peered":"2024-05-09T08:04:27.735251+0000","last_clean":"2024-05-09T08:04:27.735251+0000","last_became_active":"2024-05-09T08:02:14.874549+0000","last_became_peered":"2024-05-09T08:02:14.874549+0000","last_unstale":"2024-05-09T08:04:27.735251+0000","last_undegraded":"2024-05-09T08:04:27.735251+0000","last_fullsized":"2024-05-09T08:04:27.735251+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:57:10.338510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"27'339","reported_seq":367,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:15.296158+0000","last_change":"2024-05-09T08:03:02.516739+0000","last_active":"2024-05-09T08:07:15.296158+0000","last_peered":"2024-05-09T08:07:15.296158+0000","last_clean":"2024-05-09T08:07:15.296158+0000","last_became_active":"2024-05-09T08:03:02.516161+0000","last_became_peered":"2024-05-09T08:03:02.516161+0000","last_unstale":"2024-05-09T08:07:15.296158+0000","last_undegraded":"2024-05-09T08:07:15.296158+0000","last_fullsized":"2024-05-09T08:07:15.296158+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":339,"log_dups_size":0,"ondisk_log_size":339,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":288,"num_write_kb":786432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,2],"acting":[4,3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"27'7","reported_seq":33,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:55.783070+0000","last_change":"2024-05-09T08:02:15.913011+0000","last_active":"2024-05-09T08:07:55.783070+0000","last_peered":"2024-05-09T08:07:55.783070+0000","last_clean":"2024-05-09T08:07:55.783070+0000","last_became_active":"2024-05-09T08:02:15.912590+0000","last_became_peered":"2024-05-09T08:02:15.912590+0000","last_unstale":"2024-05-09T08:07:55.783070+0000","last_undegraded":"2024-05-09T08:07:55.783070+0000","last_fullsized":"2024-05-09T08:07:55.783070+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"27'3","reported_seq":31,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:06:37.749118+0000","last_change":"2024-05-09T08:02:14.864763+0000","last_active":"2024-05-09T08:06:37.749118+0000","last_peered":"2024-05-09T08:06:37.749118+0000","last_clean":"2024-05-09T08:06:37.749118+0000","last_became_active":"2024-05-09T08:02:14.864470+0000","last_became_peered":"2024-05-09T08:02:14.864470+0000","last_unstale":"2024-05-09T08:06:37.749118+0000","last_undegraded":"2024-05-09T08:06:37.749118+0000","last_fullsized":"2024-05-09T08:06:37.749118+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:00:40.599475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"27'306","reported_seq":327,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:15.291322+0000","last_change":"2024-05-09T08:02:20.054896+0000","last_active":"2024-05-09T08:07:15.291322+0000","last_peered":"2024-05-09T08:07:15.291322+0000","last_clean":"2024-05-09T08:07:15.291322+0000","last_became_active":"2024-05-09T08:02:20.054424+0000","last_became_peered":"2024-05-09T08:02:20.054424+0000","last_unstale":"2024-05-09T08:07:15.291322+0000","last_undegraded":"2024-05-09T08:07:15.291322+0000","last_fullsized":"2024-05-09T08:07:15.291322+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":306,"log_dups_size":0,"ondisk_log_size":306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":249,"num_write_kb":679936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"27'5","reported_seq":31,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:55.778748+0000","last_change":"2024-05-09T08:02:15.905448+0000","last_active":"2024-05-09T08:07:55.778748+0000","last_peered":"2024-05-09T08:07:55.778748+0000","last_clean":"2024-05-09T08:07:55.778748+0000","last_became_active":"2024-05-09T08:02:15.905131+0000","last_became_peered":"2024-05-09T08:02:15.905131+0000","last_unstale":"2024-05-09T08:07:55.778748+0000","last_undegraded":"2024-05-09T08:07:55.778748+0000","last_fullsized":"2024-05-09T08:07:55.778748+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"27'22","reported_seq":58,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:08:01.817574+0000","last_change":"2024-05-09T08:08:01.817598+0000","last_active":"2024-05-09T08:08:01.817574+0000","last_peered":"2024-05-09T08:08:01.817574+0000","last_clean":"2024-05-09T08:08:01.817574+0000","last_became_active":"2024-05-09T08:02:14.865621+0000","last_became_peered":"2024-05-09T08:02:14.865621+0000","last_unstale":"2024-05-09T08:08:01.817574+0000","last_undegraded":"2024-05-09T08:08:01.817574+0000","last_fullsized":"2024-05-09T08:08:01.817574+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'22","last_scrub_stamp":"2024-05-09T08:08:01.817506+0000","last_deep_scrub":"27'22","last_deep_scrub_stamp":"2024-05-09T08:08:01.817506+0000","last_clean_scrub_stamp":"2024-05-09T08:08:01.817506+0000","objects_scrubbed":1,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:11:55.649509+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":43,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"27'328","reported_seq":349,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:15.275763+0000","last_change":"2024-05-09T08:02:20.057603+0000","last_active":"2024-05-09T08:07:15.275763+0000","last_peered":"2024-05-09T08:07:15.275763+0000","last_clean":"2024-05-09T08:07:15.275763+0000","last_became_active":"2024-05-09T08:02:20.055365+0000","last_became_peered":"2024-05-09T08:02:20.055365+0000","last_unstale":"2024-05-09T08:07:15.275763+0000","last_undegraded":"2024-05-09T08:07:15.275763+0000","last_fullsized":"2024-05-09T08:07:15.275763+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":328,"log_dups_size":0,"ondisk_log_size":328,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":264,"num_write_kb":720896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"27'9","reported_seq":35,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:55.774253+0000","last_change":"2024-05-09T08:02:15.890980+0000","last_active":"2024-05-09T08:07:55.774253+0000","last_peered":"2024-05-09T08:07:55.774253+0000","last_clean":"2024-05-09T08:07:55.774253+0000","last_became_active":"2024-05-09T08:02:15.890346+0000","last_became_peered":"2024-05-09T08:02:15.890346+0000","last_unstale":"2024-05-09T08:07:55.774253+0000","last_undegraded":"2024-05-09T08:07:55.774253+0000","last_fullsized":"2024-05-09T08:07:55.774253+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:22.734773+0000","last_change":"2024-05-09T08:02:14.876206+0000","last_active":"2024-05-09T08:04:22.734773+0000","last_peered":"2024-05-09T08:04:22.734773+0000","last_clean":"2024-05-09T08:04:22.734773+0000","last_became_active":"2024-05-09T08:02:14.875894+0000","last_became_peered":"2024-05-09T08:02:14.875894+0000","last_unstale":"2024-05-09T08:04:22.734773+0000","last_undegraded":"2024-05-09T08:04:22.734773+0000","last_fullsized":"2024-05-09T08:04:22.734773+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:49:53.882513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"27'297","reported_seq":318,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:15.297485+0000","last_change":"2024-05-09T08:02:20.055222+0000","last_active":"2024-05-09T08:07:15.297485+0000","last_peered":"2024-05-09T08:07:15.297485+0000","last_clean":"2024-05-09T08:07:15.297485+0000","last_became_active":"2024-05-09T08:02:20.053802+0000","last_became_peered":"2024-05-09T08:02:20.053802+0000","last_unstale":"2024-05-09T08:07:15.297485+0000","last_undegraded":"2024-05-09T08:07:15.297485+0000","last_fullsized":"2024-05-09T08:07:15.297485+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":297,"log_dups_size":0,"ondisk_log_size":297,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":249,"num_write_kb":679936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"27'5","reported_seq":30,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:55.785453+0000","last_change":"2024-05-09T08:02:15.904528+0000","last_active":"2024-05-09T08:07:55.785453+0000","last_peered":"2024-05-09T08:07:55.785453+0000","last_clean":"2024-05-09T08:07:55.785453+0000","last_became_active":"2024-05-09T08:02:15.904049+0000","last_became_peered":"2024-05-09T08:02:15.904049+0000","last_unstale":"2024-05-09T08:07:55.785453+0000","last_undegraded":"2024-05-09T08:07:55.785453+0000","last_fullsized":"2024-05-09T08:07:55.785453+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:53:11.390395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":28,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:06:27.820966+0000","last_change":"2024-05-09T08:02:14.872897+0000","last_active":"2024-05-09T08:06:27.820966+0000","last_peered":"2024-05-09T08:06:27.820966+0000","last_clean":"2024-05-09T08:06:27.820966+0000","last_became_active":"2024-05-09T08:02:14.872096+0000","last_became_peered":"2024-05-09T08:02:14.872096+0000","last_unstale":"2024-05-09T08:06:27.820966+0000","last_undegraded":"2024-05-09T08:06:27.820966+0000","last_fullsized":"2024-05-09T08:06:27.820966+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"27'320","reported_seq":341,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:15.236654+0000","last_change":"2024-05-09T08:02:19.963716+0000","last_active":"2024-05-09T08:07:15.236654+0000","last_peered":"2024-05-09T08:07:15.236654+0000","last_clean":"2024-05-09T08:07:15.236654+0000","last_became_active":"2024-05-09T08:02:19.962922+0000","last_became_peered":"2024-05-09T08:02:19.962922+0000","last_unstale":"2024-05-09T08:07:15.236654+0000","last_undegraded":"2024-05-09T08:07:15.236654+0000","last_fullsized":"2024-05-09T08:07:15.236654+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":320,"log_dups_size":0,"ondisk_log_size":320,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":273,"num_write_kb":745472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"27'7","reported_seq":33,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:55.784148+0000","last_change":"2024-05-09T08:02:15.901450+0000","last_active":"2024-05-09T08:07:55.784148+0000","last_peered":"2024-05-09T08:07:55.784148+0000","last_clean":"2024-05-09T08:07:55.784148+0000","last_became_active":"2024-05-09T08:02:15.901180+0000","last_became_peered":"2024-05-09T08:02:15.901180+0000","last_unstale":"2024-05-09T08:07:55.784148+0000","last_undegraded":"2024-05-09T08:07:55.784148+0000","last_fullsized":"2024-05-09T08:07:55.784148+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:22.734997+0000","last_change":"2024-05-09T08:02:14.876518+0000","last_active":"2024-05-09T08:04:22.734997+0000","last_peered":"2024-05-09T08:04:22.734997+0000","last_clean":"2024-05-09T08:04:22.734997+0000","last_became_active":"2024-05-09T08:02:14.876175+0000","last_became_peered":"2024-05-09T08:02:14.876175+0000","last_unstale":"2024-05-09T08:04:22.734997+0000","last_undegraded":"2024-05-09T08:04:22.734997+0000","last_fullsized":"2024-05-09T08:04:22.734997+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:00:40.599475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"27'301","reported_seq":315,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:15.286265+0000","last_change":"2024-05-09T08:03:02.516595+0000","last_active":"2024-05-09T08:07:15.286265+0000","last_peered":"2024-05-09T08:07:15.286265+0000","last_clean":"2024-05-09T08:07:15.286265+0000","last_became_active":"2024-05-09T08:03:02.516104+0000","last_became_peered":"2024-05-09T08:03:02.516104+0000","last_unstale":"2024-05-09T08:07:15.286265+0000","last_undegraded":"2024-05-09T08:07:15.286265+0000","last_fullsized":"2024-05-09T08:07:15.286265+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":301,"log_dups_size":0,"ondisk_log_size":301,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:47:30.081323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":264,"num_write_kb":720896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3,6],"acting":[5,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"27'8","reported_seq":34,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:55.773773+0000","last_change":"2024-05-09T08:02:15.891774+0000","last_active":"2024-05-09T08:07:55.773773+0000","last_peered":"2024-05-09T08:07:55.773773+0000","last_clean":"2024-05-09T08:07:55.773773+0000","last_became_active":"2024-05-09T08:02:15.891407+0000","last_became_peered":"2024-05-09T08:02:15.891407+0000","last_unstale":"2024-05-09T08:07:55.773773+0000","last_undegraded":"2024-05-09T08:07:55.773773+0000","last_fullsized":"2024-05-09T08:07:55.773773+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"27'6","reported_seq":39,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:17.382365+0000","last_change":"2024-05-09T08:02:14.883655+0000","last_active":"2024-05-09T08:07:17.382365+0000","last_peered":"2024-05-09T08:07:17.382365+0000","last_clean":"2024-05-09T08:07:17.382365+0000","last_became_active":"2024-05-09T08:02:14.882359+0000","last_became_peered":"2024-05-09T08:02:14.882359+0000","last_unstale":"2024-05-09T08:07:17.382365+0000","last_undegraded":"2024-05-09T08:07:17.382365+0000","last_fullsized":"2024-05-09T08:07:17.382365+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":529,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"27'319","reported_seq":340,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:15.271903+0000","last_change":"2024-05-09T08:02:19.953319+0000","last_active":"2024-05-09T08:07:15.271903+0000","last_peered":"2024-05-09T08:07:15.271903+0000","last_clean":"2024-05-09T08:07:15.271903+0000","last_became_active":"2024-05-09T08:02:19.952740+0000","last_became_peered":"2024-05-09T08:02:19.952740+0000","last_unstale":"2024-05-09T08:07:15.271903+0000","last_undegraded":"2024-05-09T08:07:15.271903+0000","last_fullsized":"2024-05-09T08:07:15.271903+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":319,"log_dups_size":0,"ondisk_log_size":319,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":270,"num_write_kb":737280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"27'5","reported_seq":31,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:55.780987+0000","last_change":"2024-05-09T08:02:15.913828+0000","last_active":"2024-05-09T08:07:55.780987+0000","last_peered":"2024-05-09T08:07:55.780987+0000","last_clean":"2024-05-09T08:07:55.780987+0000","last_became_active":"2024-05-09T08:02:15.913313+0000","last_became_peered":"2024-05-09T08:02:15.913313+0000","last_unstale":"2024-05-09T08:07:55.780987+0000","last_undegraded":"2024-05-09T08:07:55.780987+0000","last_fullsized":"2024-05-09T08:07:55.780987+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:17.734196+0000","last_change":"2024-05-09T08:02:14.876438+0000","last_active":"2024-05-09T08:04:17.734196+0000","last_peered":"2024-05-09T08:04:17.734196+0000","last_clean":"2024-05-09T08:04:17.734196+0000","last_became_active":"2024-05-09T08:02:14.876146+0000","last_became_peered":"2024-05-09T08:02:14.876146+0000","last_unstale":"2024-05-09T08:04:17.734196+0000","last_undegraded":"2024-05-09T08:04:17.734196+0000","last_fullsized":"2024-05-09T08:04:17.734196+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:36:49.629991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"27'378","reported_seq":399,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:15.275038+0000","last_change":"2024-05-09T08:02:20.051067+0000","last_active":"2024-05-09T08:07:15.275038+0000","last_peered":"2024-05-09T08:07:15.275038+0000","last_clean":"2024-05-09T08:07:15.275038+0000","last_became_active":"2024-05-09T08:02:20.050593+0000","last_became_peered":"2024-05-09T08:02:20.050593+0000","last_unstale":"2024-05-09T08:07:15.275038+0000","last_undegraded":"2024-05-09T08:07:15.275038+0000","last_fullsized":"2024-05-09T08:07:15.275038+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":378,"log_dups_size":0,"ondisk_log_size":378,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":339,"num_write_kb":925696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"27'8","reported_seq":34,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:55.780001+0000","last_change":"2024-05-09T08:02:15.899137+0000","last_active":"2024-05-09T08:07:55.780001+0000","last_peered":"2024-05-09T08:07:55.780001+0000","last_clean":"2024-05-09T08:07:55.780001+0000","last_became_active":"2024-05-09T08:02:15.898766+0000","last_became_peered":"2024-05-09T08:02:15.898766+0000","last_unstale":"2024-05-09T08:07:55.780001+0000","last_undegraded":"2024-05-09T08:07:55.780001+0000","last_fullsized":"2024-05-09T08:07:55.780001+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:22.735011+0000","last_change":"2024-05-09T08:02:14.868850+0000","last_active":"2024-05-09T08:04:22.735011+0000","last_peered":"2024-05-09T08:04:22.735011+0000","last_clean":"2024-05-09T08:04:22.735011+0000","last_became_active":"2024-05-09T08:02:14.868384+0000","last_became_peered":"2024-05-09T08:02:14.868384+0000","last_unstale":"2024-05-09T08:04:22.735011+0000","last_undegraded":"2024-05-09T08:04:22.735011+0000","last_fullsized":"2024-05-09T08:04:22.735011+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:14:52.920972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":183,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:12.733624+0000","last_change":"2024-05-09T08:02:11.559804+0000","last_active":"2024-05-09T08:04:12.733624+0000","last_peered":"2024-05-09T08:04:12.733624+0000","last_clean":"2024-05-09T08:04:12.733624+0000","last_became_active":"2024-05-09T08:02:11.558761+0000","last_became_peered":"2024-05-09T08:02:11.558761+0000","last_unstale":"2024-05-09T08:04:12.733624+0000","last_undegraded":"2024-05-09T08:04:12.733624+0000","last_fullsized":"2024-05-09T08:04:12.733624+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:09.800143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:09.800143+0000","last_clean_scrub_stamp":"2024-05-09T08:02:09.800143+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:18:35.964149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"27'315","reported_seq":335,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:15.264268+0000","last_change":"2024-05-09T08:02:20.054746+0000","last_active":"2024-05-09T08:07:15.264268+0000","last_peered":"2024-05-09T08:07:15.264268+0000","last_clean":"2024-05-09T08:07:15.264268+0000","last_became_active":"2024-05-09T08:02:20.054186+0000","last_became_peered":"2024-05-09T08:02:20.054186+0000","last_unstale":"2024-05-09T08:07:15.264268+0000","last_undegraded":"2024-05-09T08:07:15.264268+0000","last_fullsized":"2024-05-09T08:07:15.264268+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":315,"log_dups_size":0,"ondisk_log_size":315,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:53:15.458120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":276,"num_write_kb":753664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"27'7","reported_seq":32,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:55.774548+0000","last_change":"2024-05-09T08:02:15.889657+0000","last_active":"2024-05-09T08:07:55.774548+0000","last_peered":"2024-05-09T08:07:55.774548+0000","last_clean":"2024-05-09T08:07:55.774548+0000","last_became_active":"2024-05-09T08:02:15.889381+0000","last_became_peered":"2024-05-09T08:02:15.889381+0000","last_unstale":"2024-05-09T08:07:55.774548+0000","last_undegraded":"2024-05-09T08:07:55.774548+0000","last_fullsized":"2024-05-09T08:07:55.774548+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:59:14.957806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":36,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:08:01.600999+0000","last_change":"2024-05-09T08:08:01.601071+0000","last_active":"2024-05-09T08:08:01.600999+0000","last_peered":"2024-05-09T08:08:01.600999+0000","last_clean":"2024-05-09T08:08:01.600999+0000","last_became_active":"2024-05-09T08:02:14.868513+0000","last_became_peered":"2024-05-09T08:02:14.868513+0000","last_unstale":"2024-05-09T08:08:01.600999+0000","last_undegraded":"2024-05-09T08:08:01.600999+0000","last_fullsized":"2024-05-09T08:08:01.600999+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:08:01.600849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:08:01.600849+0000","last_clean_scrub_stamp":"2024-05-09T08:08:01.600849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:14:22.308730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"27'338","reported_seq":358,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:15.275986+0000","last_change":"2024-05-09T08:02:20.053097+0000","last_active":"2024-05-09T08:07:15.275986+0000","last_peered":"2024-05-09T08:07:15.275986+0000","last_clean":"2024-05-09T08:07:15.275986+0000","last_became_active":"2024-05-09T08:02:20.052653+0000","last_became_peered":"2024-05-09T08:02:20.052653+0000","last_unstale":"2024-05-09T08:07:15.275986+0000","last_undegraded":"2024-05-09T08:07:15.275986+0000","last_fullsized":"2024-05-09T08:07:15.275986+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":338,"log_dups_size":0,"ondisk_log_size":338,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:25:05.306541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":279,"num_write_kb":761856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"27'6","reported_seq":31,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:55.785553+0000","last_change":"2024-05-09T08:02:15.899059+0000","last_active":"2024-05-09T08:07:55.785553+0000","last_peered":"2024-05-09T08:07:55.785553+0000","last_clean":"2024-05-09T08:07:55.785553+0000","last_became_active":"2024-05-09T08:02:15.898692+0000","last_became_peered":"2024-05-09T08:02:15.898692+0000","last_unstale":"2024-05-09T08:07:55.785553+0000","last_undegraded":"2024-05-09T08:07:55.785553+0000","last_fullsized":"2024-05-09T08:07:55.785553+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:52:36.461913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"24'1","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:06:47.823224+0000","last_change":"2024-05-09T08:02:14.873071+0000","last_active":"2024-05-09T08:06:47.823224+0000","last_peered":"2024-05-09T08:06:47.823224+0000","last_clean":"2024-05-09T08:06:47.823224+0000","last_became_active":"2024-05-09T08:02:14.872556+0000","last_became_peered":"2024-05-09T08:02:14.872556+0000","last_unstale":"2024-05-09T08:06:47.823224+0000","last_undegraded":"2024-05-09T08:06:47.823224+0000","last_fullsized":"2024-05-09T08:06:47.823224+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"27'299","reported_seq":320,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:15.292207+0000","last_change":"2024-05-09T08:02:19.960414+0000","last_active":"2024-05-09T08:07:15.292207+0000","last_peered":"2024-05-09T08:07:15.292207+0000","last_clean":"2024-05-09T08:07:15.292207+0000","last_became_active":"2024-05-09T08:02:19.957941+0000","last_became_peered":"2024-05-09T08:02:19.957941+0000","last_unstale":"2024-05-09T08:07:15.292207+0000","last_undegraded":"2024-05-09T08:07:15.292207+0000","last_fullsized":"2024-05-09T08:07:15.292207+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":299,"log_dups_size":0,"ondisk_log_size":299,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":261,"num_write_kb":712704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"27'9","reported_seq":34,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:55.770054+0000","last_change":"2024-05-09T08:02:15.884395+0000","last_active":"2024-05-09T08:07:55.770054+0000","last_peered":"2024-05-09T08:07:55.770054+0000","last_clean":"2024-05-09T08:07:55.770054+0000","last_became_active":"2024-05-09T08:02:15.883788+0000","last_became_peered":"2024-05-09T08:02:15.883788+0000","last_unstale":"2024-05-09T08:07:55.770054+0000","last_undegraded":"2024-05-09T08:07:55.770054+0000","last_fullsized":"2024-05-09T08:07:55.770054+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:48:34.215837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"24'2","reported_seq":30,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:06:47.785239+0000","last_change":"2024-05-09T08:02:14.885044+0000","last_active":"2024-05-09T08:06:47.785239+0000","last_peered":"2024-05-09T08:06:47.785239+0000","last_clean":"2024-05-09T08:06:47.785239+0000","last_became_active":"2024-05-09T08:02:14.884706+0000","last_became_peered":"2024-05-09T08:02:14.884706+0000","last_unstale":"2024-05-09T08:06:47.785239+0000","last_undegraded":"2024-05-09T08:06:47.785239+0000","last_fullsized":"2024-05-09T08:06:47.785239+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"27'365","reported_seq":385,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:15.276867+0000","last_change":"2024-05-09T08:02:20.050865+0000","last_active":"2024-05-09T08:07:15.276867+0000","last_peered":"2024-05-09T08:07:15.276867+0000","last_clean":"2024-05-09T08:07:15.276867+0000","last_became_active":"2024-05-09T08:02:20.050270+0000","last_became_peered":"2024-05-09T08:02:20.050270+0000","last_unstale":"2024-05-09T08:07:15.276867+0000","last_undegraded":"2024-05-09T08:07:15.276867+0000","last_fullsized":"2024-05-09T08:07:15.276867+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":365,"log_dups_size":0,"ondisk_log_size":365,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:48:38.283562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":318,"num_write_kb":868352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"27'5","reported_seq":30,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:55.768555+0000","last_change":"2024-05-09T08:02:15.884737+0000","last_active":"2024-05-09T08:07:55.768555+0000","last_peered":"2024-05-09T08:07:55.768555+0000","last_clean":"2024-05-09T08:07:55.768555+0000","last_became_active":"2024-05-09T08:02:15.884465+0000","last_became_peered":"2024-05-09T08:02:15.884465+0000","last_unstale":"2024-05-09T08:07:55.768555+0000","last_undegraded":"2024-05-09T08:07:55.768555+0000","last_fullsized":"2024-05-09T08:07:55.768555+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:52:36.461913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"27'2","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:08:02.773744+0000","last_change":"2024-05-09T08:08:02.773775+0000","last_active":"2024-05-09T08:08:02.773744+0000","last_peered":"2024-05-09T08:08:02.773744+0000","last_clean":"2024-05-09T08:08:02.773744+0000","last_became_active":"2024-05-09T08:02:14.874665+0000","last_became_peered":"2024-05-09T08:02:14.874665+0000","last_unstale":"2024-05-09T08:08:02.773744+0000","last_undegraded":"2024-05-09T08:08:02.773744+0000","last_fullsized":"2024-05-09T08:08:02.773744+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2","last_scrub_stamp":"2024-05-09T08:08:02.773645+0000","last_deep_scrub":"27'2","last_deep_scrub_stamp":"2024-05-09T08:08:02.773645+0000","last_clean_scrub_stamp":"2024-05-09T08:08:02.773645+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:05:19.799951+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"27'302","reported_seq":323,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:15.294545+0000","last_change":"2024-05-09T08:02:19.952692+0000","last_active":"2024-05-09T08:07:15.294545+0000","last_peered":"2024-05-09T08:07:15.294545+0000","last_clean":"2024-05-09T08:07:15.294545+0000","last_became_active":"2024-05-09T08:02:19.952224+0000","last_became_peered":"2024-05-09T08:02:19.952224+0000","last_unstale":"2024-05-09T08:07:15.294545+0000","last_undegraded":"2024-05-09T08:07:15.294545+0000","last_fullsized":"2024-05-09T08:07:15.294545+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":302,"log_dups_size":0,"ondisk_log_size":302,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":258,"num_write_kb":704512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"27'6","reported_seq":31,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:55.761882+0000","last_change":"2024-05-09T08:02:15.883408+0000","last_active":"2024-05-09T08:07:55.761882+0000","last_peered":"2024-05-09T08:07:55.761882+0000","last_clean":"2024-05-09T08:07:55.761882+0000","last_became_active":"2024-05-09T08:02:15.883103+0000","last_became_peered":"2024-05-09T08:02:15.883103+0000","last_unstale":"2024-05-09T08:07:55.761882+0000","last_undegraded":"2024-05-09T08:07:55.761882+0000","last_fullsized":"2024-05-09T08:07:55.761882+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:53:11.390395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"27'13","reported_seq":49,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:08:02.642638+0000","last_change":"2024-05-09T08:08:02.642716+0000","last_active":"2024-05-09T08:08:02.642638+0000","last_peered":"2024-05-09T08:08:02.642638+0000","last_clean":"2024-05-09T08:08:02.642638+0000","last_became_active":"2024-05-09T08:02:14.871055+0000","last_became_peered":"2024-05-09T08:02:14.871055+0000","last_unstale":"2024-05-09T08:08:02.642638+0000","last_undegraded":"2024-05-09T08:08:02.642638+0000","last_fullsized":"2024-05-09T08:08:02.642638+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'13","last_scrub_stamp":"2024-05-09T08:08:02.642463+0000","last_deep_scrub":"27'13","last_deep_scrub_stamp":"2024-05-09T08:08:02.642463+0000","last_clean_scrub_stamp":"2024-05-09T08:08:02.642463+0000","objects_scrubbed":4,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:37:49.509704+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"27'416","reported_seq":437,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:15.288107+0000","last_change":"2024-05-09T08:02:19.950823+0000","last_active":"2024-05-09T08:07:15.288107+0000","last_peered":"2024-05-09T08:07:15.288107+0000","last_clean":"2024-05-09T08:07:15.288107+0000","last_became_active":"2024-05-09T08:02:19.950352+0000","last_became_peered":"2024-05-09T08:02:19.950352+0000","last_unstale":"2024-05-09T08:07:15.288107+0000","last_undegraded":"2024-05-09T08:07:15.288107+0000","last_fullsized":"2024-05-09T08:07:15.288107+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":416,"log_dups_size":0,"ondisk_log_size":416,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":360,"num_write_kb":983040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"27'6","reported_seq":32,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:55.770019+0000","last_change":"2024-05-09T08:02:15.899115+0000","last_active":"2024-05-09T08:07:55.770019+0000","last_peered":"2024-05-09T08:07:55.770019+0000","last_clean":"2024-05-09T08:07:55.770019+0000","last_became_active":"2024-05-09T08:02:15.898811+0000","last_became_peered":"2024-05-09T08:02:15.898811+0000","last_unstale":"2024-05-09T08:07:55.770019+0000","last_undegraded":"2024-05-09T08:07:55.770019+0000","last_fullsized":"2024-05-09T08:07:55.770019+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:43:06.938904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":28,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:37.736636+0000","last_change":"2024-05-09T08:02:14.860818+0000","last_active":"2024-05-09T08:04:37.736636+0000","last_peered":"2024-05-09T08:04:37.736636+0000","last_clean":"2024-05-09T08:04:37.736636+0000","last_became_active":"2024-05-09T08:02:14.860518+0000","last_became_peered":"2024-05-09T08:02:14.860518+0000","last_unstale":"2024-05-09T08:04:37.736636+0000","last_undegraded":"2024-05-09T08:04:37.736636+0000","last_fullsized":"2024-05-09T08:04:37.736636+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:25:23.306827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"27'318","reported_seq":339,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:15.295941+0000","last_change":"2024-05-09T08:02:20.053251+0000","last_active":"2024-05-09T08:07:15.295941+0000","last_peered":"2024-05-09T08:07:15.295941+0000","last_clean":"2024-05-09T08:07:15.295941+0000","last_became_active":"2024-05-09T08:02:20.052783+0000","last_became_peered":"2024-05-09T08:02:20.052783+0000","last_unstale":"2024-05-09T08:07:15.295941+0000","last_undegraded":"2024-05-09T08:07:15.295941+0000","last_fullsized":"2024-05-09T08:07:15.295941+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":318,"log_dups_size":0,"ondisk_log_size":318,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":273,"num_write_kb":745472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"27'15","reported_seq":40,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:55.769859+0000","last_change":"2024-05-09T08:02:15.883972+0000","last_active":"2024-05-09T08:07:55.769859+0000","last_peered":"2024-05-09T08:07:55.769859+0000","last_clean":"2024-05-09T08:07:55.769859+0000","last_became_active":"2024-05-09T08:02:15.883613+0000","last_became_peered":"2024-05-09T08:02:15.883613+0000","last_unstale":"2024-05-09T08:07:55.769859+0000","last_undegraded":"2024-05-09T08:07:55.769859+0000","last_fullsized":"2024-05-09T08:07:55.769859+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:41:27.402821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:37.736743+0000","last_change":"2024-05-09T08:02:14.883366+0000","last_active":"2024-05-09T08:04:37.736743+0000","last_peered":"2024-05-09T08:04:37.736743+0000","last_clean":"2024-05-09T08:04:37.736743+0000","last_became_active":"2024-05-09T08:02:14.883135+0000","last_became_peered":"2024-05-09T08:02:14.883135+0000","last_unstale":"2024-05-09T08:04:37.736743+0000","last_undegraded":"2024-05-09T08:04:37.736743+0000","last_fullsized":"2024-05-09T08:04:37.736743+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:32:48.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"27'316","reported_seq":337,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:15.297404+0000","last_change":"2024-05-09T08:02:20.050888+0000","last_active":"2024-05-09T08:07:15.297404+0000","last_peered":"2024-05-09T08:07:15.297404+0000","last_clean":"2024-05-09T08:07:15.297404+0000","last_became_active":"2024-05-09T08:02:20.050390+0000","last_became_peered":"2024-05-09T08:02:20.050390+0000","last_unstale":"2024-05-09T08:07:15.297404+0000","last_undegraded":"2024-05-09T08:07:15.297404+0000","last_fullsized":"2024-05-09T08:07:15.297404+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":316,"log_dups_size":0,"ondisk_log_size":316,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":273,"num_write_kb":745472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"27'10","reported_seq":36,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:55.781813+0000","last_change":"2024-05-09T08:02:15.899096+0000","last_active":"2024-05-09T08:07:55.781813+0000","last_peered":"2024-05-09T08:07:55.781813+0000","last_clean":"2024-05-09T08:07:55.781813+0000","last_became_active":"2024-05-09T08:02:15.898811+0000","last_became_peered":"2024-05-09T08:02:15.898811+0000","last_unstale":"2024-05-09T08:07:55.781813+0000","last_undegraded":"2024-05-09T08:07:55.781813+0000","last_fullsized":"2024-05-09T08:07:55.781813+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"27'4","reported_seq":40,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:08:01.342238+0000","last_change":"2024-05-09T08:08:01.342274+0000","last_active":"2024-05-09T08:08:01.342238+0000","last_peered":"2024-05-09T08:08:01.342238+0000","last_clean":"2024-05-09T08:08:01.342238+0000","last_became_active":"2024-05-09T08:02:14.868385+0000","last_became_peered":"2024-05-09T08:02:14.868385+0000","last_unstale":"2024-05-09T08:08:01.342238+0000","last_undegraded":"2024-05-09T08:08:01.342238+0000","last_fullsized":"2024-05-09T08:08:01.342238+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4","last_scrub_stamp":"2024-05-09T08:08:01.342147+0000","last_deep_scrub":"27'4","last_deep_scrub_stamp":"2024-05-09T08:08:01.342147+0000","last_clean_scrub_stamp":"2024-05-09T08:08:01.342147+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:11:55.174150+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"27'378","reported_seq":398,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:15.279263+0000","last_change":"2024-05-09T08:02:20.053155+0000","last_active":"2024-05-09T08:07:15.279263+0000","last_peered":"2024-05-09T08:07:15.279263+0000","last_clean":"2024-05-09T08:07:15.279263+0000","last_became_active":"2024-05-09T08:02:20.052629+0000","last_became_peered":"2024-05-09T08:02:20.052629+0000","last_unstale":"2024-05-09T08:07:15.279263+0000","last_undegraded":"2024-05-09T08:07:15.279263+0000","last_fullsized":"2024-05-09T08:07:15.279263+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":378,"log_dups_size":0,"ondisk_log_size":378,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:35:54.628874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":327,"num_write_kb":892928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"27'6","reported_seq":31,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:55.769230+0000","last_change":"2024-05-09T08:02:15.905406+0000","last_active":"2024-05-09T08:07:55.769230+0000","last_peered":"2024-05-09T08:07:55.769230+0000","last_clean":"2024-05-09T08:07:55.769230+0000","last_became_active":"2024-05-09T08:02:15.905076+0000","last_became_peered":"2024-05-09T08:02:15.905076+0000","last_unstale":"2024-05-09T08:07:55.769230+0000","last_undegraded":"2024-05-09T08:07:55.769230+0000","last_fullsized":"2024-05-09T08:07:55.769230+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:25:01.238816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"27'192","reported_seq":220,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:55.723052+0000","last_change":"2024-05-09T08:02:14.880410+0000","last_active":"2024-05-09T08:07:55.723052+0000","last_peered":"2024-05-09T08:07:55.723052+0000","last_clean":"2024-05-09T08:07:55.723052+0000","last_became_active":"2024-05-09T08:02:14.880045+0000","last_became_peered":"2024-05-09T08:02:14.880045+0000","last_unstale":"2024-05-09T08:07:55.723052+0000","last_undegraded":"2024-05-09T08:07:55.723052+0000","last_fullsized":"2024-05-09T08:07:55.723052+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":192,"log_dups_size":0,"ondisk_log_size":192,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:19:56.315972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":447684,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":191,"num_write_kb":585,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"27'332","reported_seq":353,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:15.248695+0000","last_change":"2024-05-09T08:02:19.963378+0000","last_active":"2024-05-09T08:07:15.248695+0000","last_peered":"2024-05-09T08:07:15.248695+0000","last_clean":"2024-05-09T08:07:15.248695+0000","last_became_active":"2024-05-09T08:02:19.958964+0000","last_became_peered":"2024-05-09T08:02:19.958964+0000","last_unstale":"2024-05-09T08:07:15.248695+0000","last_undegraded":"2024-05-09T08:07:15.248695+0000","last_fullsized":"2024-05-09T08:07:15.248695+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":332,"log_dups_size":0,"ondisk_log_size":332,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":288,"num_write_kb":786432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"27'11","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:55.773581+0000","last_change":"2024-05-09T08:02:15.888428+0000","last_active":"2024-05-09T08:07:55.773581+0000","last_peered":"2024-05-09T08:07:55.773581+0000","last_clean":"2024-05-09T08:07:55.773581+0000","last_became_active":"2024-05-09T08:02:15.888071+0000","last_became_peered":"2024-05-09T08:02:15.888071+0000","last_unstale":"2024-05-09T08:07:55.773581+0000","last_undegraded":"2024-05-09T08:07:55.773581+0000","last_fullsized":"2024-05-09T08:07:55.773581+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"27'10","reported_seq":46,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:08:02.825312+0000","last_change":"2024-05-09T08:08:02.825383+0000","last_active":"2024-05-09T08:08:02.825312+0000","last_peered":"2024-05-09T08:08:02.825312+0000","last_clean":"2024-05-09T08:08:02.825312+0000","last_became_active":"2024-05-09T08:02:14.871945+0000","last_became_peered":"2024-05-09T08:02:14.871945+0000","last_unstale":"2024-05-09T08:08:02.825312+0000","last_undegraded":"2024-05-09T08:08:02.825312+0000","last_fullsized":"2024-05-09T08:08:02.825312+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'10","last_scrub_stamp":"2024-05-09T08:08:02.825166+0000","last_deep_scrub":"27'10","last_deep_scrub_stamp":"2024-05-09T08:08:02.825166+0000","last_clean_scrub_stamp":"2024-05-09T08:08:02.825166+0000","objects_scrubbed":2,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:56:58.698469+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"27'353","reported_seq":374,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:15.288405+0000","last_change":"2024-05-09T08:02:19.953635+0000","last_active":"2024-05-09T08:07:15.288405+0000","last_peered":"2024-05-09T08:07:15.288405+0000","last_clean":"2024-05-09T08:07:15.288405+0000","last_became_active":"2024-05-09T08:02:19.953166+0000","last_became_peered":"2024-05-09T08:02:19.953166+0000","last_unstale":"2024-05-09T08:07:15.288405+0000","last_undegraded":"2024-05-09T08:07:15.288405+0000","last_fullsized":"2024-05-09T08:07:15.288405+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":353,"log_dups_size":0,"ondisk_log_size":353,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":297,"num_write_kb":811008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"27'9","reported_seq":35,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:55.784245+0000","last_change":"2024-05-09T08:02:15.905285+0000","last_active":"2024-05-09T08:07:55.784245+0000","last_peered":"2024-05-09T08:07:55.784245+0000","last_clean":"2024-05-09T08:07:55.784245+0000","last_became_active":"2024-05-09T08:02:15.904898+0000","last_became_peered":"2024-05-09T08:02:15.904898+0000","last_unstale":"2024-05-09T08:07:55.784245+0000","last_undegraded":"2024-05-09T08:07:55.784245+0000","last_fullsized":"2024-05-09T08:07:55.784245+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:42.737369+0000","last_change":"2024-05-09T08:02:14.868682+0000","last_active":"2024-05-09T08:04:42.737369+0000","last_peered":"2024-05-09T08:04:42.737369+0000","last_clean":"2024-05-09T08:04:42.737369+0000","last_became_active":"2024-05-09T08:02:14.868294+0000","last_became_peered":"2024-05-09T08:02:14.868294+0000","last_unstale":"2024-05-09T08:04:42.737369+0000","last_undegraded":"2024-05-09T08:04:42.737369+0000","last_fullsized":"2024-05-09T08:04:42.737369+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:57:06.093718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"27'344","reported_seq":365,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:15.290108+0000","last_change":"2024-05-09T08:02:19.956115+0000","last_active":"2024-05-09T08:07:15.290108+0000","last_peered":"2024-05-09T08:07:15.290108+0000","last_clean":"2024-05-09T08:07:15.290108+0000","last_became_active":"2024-05-09T08:02:19.955662+0000","last_became_peered":"2024-05-09T08:02:19.955662+0000","last_unstale":"2024-05-09T08:07:15.290108+0000","last_undegraded":"2024-05-09T08:07:15.290108+0000","last_fullsized":"2024-05-09T08:07:15.290108+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":344,"log_dups_size":0,"ondisk_log_size":344,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":297,"num_write_kb":811008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"27'10","reported_seq":35,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:55.774692+0000","last_change":"2024-05-09T08:02:15.888406+0000","last_active":"2024-05-09T08:07:55.774692+0000","last_peered":"2024-05-09T08:07:55.774692+0000","last_clean":"2024-05-09T08:07:55.774692+0000","last_became_active":"2024-05-09T08:02:15.888090+0000","last_became_peered":"2024-05-09T08:02:15.888090+0000","last_unstale":"2024-05-09T08:07:55.774692+0000","last_undegraded":"2024-05-09T08:07:55.774692+0000","last_fullsized":"2024-05-09T08:07:55.774692+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:53:23.178610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"27'1","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:05:05.164756+0000","last_change":"2024-05-09T08:02:14.862463+0000","last_active":"2024-05-09T08:05:05.164756+0000","last_peered":"2024-05-09T08:05:05.164756+0000","last_clean":"2024-05-09T08:05:05.164756+0000","last_became_active":"2024-05-09T08:02:14.862104+0000","last_became_peered":"2024-05-09T08:02:14.862104+0000","last_unstale":"2024-05-09T08:05:05.164756+0000","last_undegraded":"2024-05-09T08:05:05.164756+0000","last_fullsized":"2024-05-09T08:05:05.164756+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:32:48.832871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"27'379","reported_seq":400,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:15.262910+0000","last_change":"2024-05-09T08:02:20.053600+0000","last_active":"2024-05-09T08:07:15.262910+0000","last_peered":"2024-05-09T08:07:15.262910+0000","last_clean":"2024-05-09T08:07:15.262910+0000","last_became_active":"2024-05-09T08:02:20.052485+0000","last_became_peered":"2024-05-09T08:02:20.052485+0000","last_unstale":"2024-05-09T08:07:15.262910+0000","last_undegraded":"2024-05-09T08:07:15.262910+0000","last_fullsized":"2024-05-09T08:07:15.262910+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":379,"log_dups_size":0,"ondisk_log_size":379,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":327,"num_write_kb":892928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"27'324","reported_seq":345,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:15.292294+0000","last_change":"2024-05-09T08:02:20.054016+0000","last_active":"2024-05-09T08:07:15.292294+0000","last_peered":"2024-05-09T08:07:15.292294+0000","last_clean":"2024-05-09T08:07:15.292294+0000","last_became_active":"2024-05-09T08:02:20.053250+0000","last_became_peered":"2024-05-09T08:02:20.053250+0000","last_unstale":"2024-05-09T08:07:15.292294+0000","last_undegraded":"2024-05-09T08:07:15.292294+0000","last_fullsized":"2024-05-09T08:07:15.292294+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":324,"log_dups_size":0,"ondisk_log_size":324,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":273,"num_write_kb":745472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"27'11","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:55.775272+0000","last_change":"2024-05-09T08:02:15.907080+0000","last_active":"2024-05-09T08:07:55.775272+0000","last_peered":"2024-05-09T08:07:55.775272+0000","last_clean":"2024-05-09T08:07:55.775272+0000","last_became_active":"2024-05-09T08:02:15.906600+0000","last_became_peered":"2024-05-09T08:02:15.906600+0000","last_unstale":"2024-05-09T08:07:55.775272+0000","last_undegraded":"2024-05-09T08:07:55.775272+0000","last_fullsized":"2024-05-09T08:07:55.775272+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:42.737302+0000","last_change":"2024-05-09T08:02:14.870990+0000","last_active":"2024-05-09T08:04:42.737302+0000","last_peered":"2024-05-09T08:04:42.737302+0000","last_clean":"2024-05-09T08:04:42.737302+0000","last_became_active":"2024-05-09T08:02:14.869818+0000","last_became_peered":"2024-05-09T08:02:14.869818+0000","last_unstale":"2024-05-09T08:04:42.737302+0000","last_undegraded":"2024-05-09T08:04:42.737302+0000","last_fullsized":"2024-05-09T08:04:42.737302+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:25:23.306827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"27'313","reported_seq":333,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:15.240636+0000","last_change":"2024-05-09T08:02:19.952582+0000","last_active":"2024-05-09T08:07:15.240636+0000","last_peered":"2024-05-09T08:07:15.240636+0000","last_clean":"2024-05-09T08:07:15.240636+0000","last_became_active":"2024-05-09T08:02:19.952117+0000","last_became_peered":"2024-05-09T08:02:19.952117+0000","last_unstale":"2024-05-09T08:07:15.240636+0000","last_undegraded":"2024-05-09T08:07:15.240636+0000","last_fullsized":"2024-05-09T08:07:15.240636+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":313,"log_dups_size":0,"ondisk_log_size":313,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:42:00.939587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":267,"num_write_kb":729088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"27'8","reported_seq":34,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:55.784725+0000","last_change":"2024-05-09T08:02:15.913109+0000","last_active":"2024-05-09T08:07:55.784725+0000","last_peered":"2024-05-09T08:07:55.784725+0000","last_clean":"2024-05-09T08:07:55.784725+0000","last_became_active":"2024-05-09T08:02:15.912883+0000","last_became_peered":"2024-05-09T08:02:15.912883+0000","last_unstale":"2024-05-09T08:07:55.784725+0000","last_undegraded":"2024-05-09T08:07:55.784725+0000","last_fullsized":"2024-05-09T08:07:55.784725+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":28,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:07.825413+0000","last_change":"2024-05-09T08:02:14.885256+0000","last_active":"2024-05-09T08:07:07.825413+0000","last_peered":"2024-05-09T08:07:07.825413+0000","last_clean":"2024-05-09T08:07:07.825413+0000","last_became_active":"2024-05-09T08:02:14.884965+0000","last_became_peered":"2024-05-09T08:02:14.884965+0000","last_unstale":"2024-05-09T08:07:07.825413+0000","last_undegraded":"2024-05-09T08:07:07.825413+0000","last_fullsized":"2024-05-09T08:07:07.825413+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"24'1","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:04:47.737722+0000","last_change":"2024-05-09T08:02:14.860339+0000","last_active":"2024-05-09T08:04:47.737722+0000","last_peered":"2024-05-09T08:04:47.737722+0000","last_clean":"2024-05-09T08:04:47.737722+0000","last_became_active":"2024-05-09T08:02:14.859917+0000","last_became_peered":"2024-05-09T08:02:14.859917+0000","last_unstale":"2024-05-09T08:04:47.737722+0000","last_undegraded":"2024-05-09T08:04:47.737722+0000","last_fullsized":"2024-05-09T08:04:47.737722+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:14:52.920972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"27'8","reported_seq":34,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:55.784672+0000","last_change":"2024-05-09T08:02:15.897001+0000","last_active":"2024-05-09T08:07:55.784672+0000","last_peered":"2024-05-09T08:07:55.784672+0000","last_clean":"2024-05-09T08:07:55.784672+0000","last_became_active":"2024-05-09T08:02:15.896708+0000","last_became_peered":"2024-05-09T08:02:15.896708+0000","last_unstale":"2024-05-09T08:07:55.784672+0000","last_undegraded":"2024-05-09T08:07:55.784672+0000","last_fullsized":"2024-05-09T08:07:55.784672+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"27'338","reported_seq":359,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:22.797510+0000","last_change":"2024-05-09T08:02:19.955369+0000","last_active":"2024-05-09T08:07:22.797510+0000","last_peered":"2024-05-09T08:07:22.797510+0000","last_clean":"2024-05-09T08:07:22.797510+0000","last_became_active":"2024-05-09T08:02:19.954829+0000","last_became_peered":"2024-05-09T08:02:19.954829+0000","last_unstale":"2024-05-09T08:07:22.797510+0000","last_undegraded":"2024-05-09T08:07:22.797510+0000","last_fullsized":"2024-05-09T08:07:22.797510+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":338,"log_dups_size":0,"ondisk_log_size":338,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":297,"num_write_kb":811008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"24'1","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:17.849924+0000","last_change":"2024-05-09T08:02:14.883856+0000","last_active":"2024-05-09T08:07:17.849924+0000","last_peered":"2024-05-09T08:07:17.849924+0000","last_clean":"2024-05-09T08:07:17.849924+0000","last_became_active":"2024-05-09T08:02:14.882610+0000","last_became_peered":"2024-05-09T08:02:14.882610+0000","last_unstale":"2024-05-09T08:07:17.849924+0000","last_undegraded":"2024-05-09T08:07:17.849924+0000","last_fullsized":"2024-05-09T08:07:17.849924+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:13.827313+0000","last_clean_scrub_stamp":"2024-05-09T08:02:13.827313+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"27'7","reported_seq":33,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:55.776266+0000","last_change":"2024-05-09T08:02:15.889974+0000","last_active":"2024-05-09T08:07:55.776266+0000","last_peered":"2024-05-09T08:07:55.776266+0000","last_clean":"2024-05-09T08:07:55.776266+0000","last_became_active":"2024-05-09T08:02:15.889650+0000","last_became_peered":"2024-05-09T08:02:15.889650+0000","last_unstale":"2024-05-09T08:07:55.776266+0000","last_undegraded":"2024-05-09T08:07:55.776266+0000","last_fullsized":"2024-05-09T08:07:55.776266+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"27'361","reported_seq":382,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:22.850833+0000","last_change":"2024-05-09T08:02:19.955362+0000","last_active":"2024-05-09T08:07:22.850833+0000","last_peered":"2024-05-09T08:07:22.850833+0000","last_clean":"2024-05-09T08:07:22.850833+0000","last_became_active":"2024-05-09T08:02:19.954947+0000","last_became_peered":"2024-05-09T08:02:19.954947+0000","last_unstale":"2024-05-09T08:07:22.850833+0000","last_undegraded":"2024-05-09T08:07:22.850833+0000","last_fullsized":"2024-05-09T08:07:22.850833+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:18.908679+0000","last_clean_scrub_stamp":"2024-05-09T08:02:18.908679+0000","objects_scrubbed":0,"log_size":361,"log_dups_size":0,"ondisk_log_size":361,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":312,"num_write_kb":851968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"27'9","reported_seq":35,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:55.782257+0000","last_change":"2024-05-09T08:02:15.909379+0000","last_active":"2024-05-09T08:07:55.782257+0000","last_peered":"2024-05-09T08:07:55.782257+0000","last_clean":"2024-05-09T08:07:55.782257+0000","last_became_active":"2024-05-09T08:02:15.909086+0000","last_became_peered":"2024-05-09T08:02:15.909086+0000","last_unstale":"2024-05-09T08:07:55.782257+0000","last_undegraded":"2024-05-09T08:07:55.782257+0000","last_fullsized":"2024-05-09T08:07:55.782257+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:05:02.181367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"27'12","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:55.777487+0000","last_change":"2024-05-09T08:02:15.891242+0000","last_active":"2024-05-09T08:07:55.777487+0000","last_peered":"2024-05-09T08:07:55.777487+0000","last_clean":"2024-05-09T08:07:55.777487+0000","last_became_active":"2024-05-09T08:02:15.890640+0000","last_became_peered":"2024-05-09T08:02:15.890640+0000","last_unstale":"2024-05-09T08:07:55.777487+0000","last_undegraded":"2024-05-09T08:07:55.777487+0000","last_fullsized":"2024-05-09T08:07:55.777487+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"27'8","reported_seq":33,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:55.785206+0000","last_change":"2024-05-09T08:02:15.899001+0000","last_active":"2024-05-09T08:07:55.785206+0000","last_peered":"2024-05-09T08:07:55.785206+0000","last_clean":"2024-05-09T08:07:55.785206+0000","last_became_active":"2024-05-09T08:02:15.898695+0000","last_became_peered":"2024-05-09T08:02:15.898695+0000","last_unstale":"2024-05-09T08:07:55.785206+0000","last_undegraded":"2024-05-09T08:07:55.785206+0000","last_fullsized":"2024-05-09T08:07:55.785206+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:41:27.402821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"27'7","reported_seq":33,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:55.785341+0000","last_change":"2024-05-09T08:02:15.907374+0000","last_active":"2024-05-09T08:07:55.785341+0000","last_peered":"2024-05-09T08:07:55.785341+0000","last_clean":"2024-05-09T08:07:55.785341+0000","last_became_active":"2024-05-09T08:02:15.906767+0000","last_became_peered":"2024-05-09T08:02:15.906767+0000","last_unstale":"2024-05-09T08:07:55.785341+0000","last_undegraded":"2024-05-09T08:07:55.785341+0000","last_fullsized":"2024-05-09T08:07:55.785341+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"27'4","reported_seq":30,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:55.774898+0000","last_change":"2024-05-09T08:02:15.905326+0000","last_active":"2024-05-09T08:07:55.774898+0000","last_peered":"2024-05-09T08:07:55.774898+0000","last_clean":"2024-05-09T08:07:55.774898+0000","last_became_active":"2024-05-09T08:02:15.905083+0000","last_became_peered":"2024-05-09T08:02:15.905083+0000","last_unstale":"2024-05-09T08:07:55.774898+0000","last_undegraded":"2024-05-09T08:07:55.774898+0000","last_fullsized":"2024-05-09T08:07:55.774898+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"27'6","reported_seq":32,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:55.768144+0000","last_change":"2024-05-09T08:02:15.907753+0000","last_active":"2024-05-09T08:07:55.768144+0000","last_peered":"2024-05-09T08:07:55.768144+0000","last_clean":"2024-05-09T08:07:55.768144+0000","last_became_active":"2024-05-09T08:02:15.907480+0000","last_became_peered":"2024-05-09T08:02:15.907480+0000","last_unstale":"2024-05-09T08:07:55.768144+0000","last_undegraded":"2024-05-09T08:07:55.768144+0000","last_fullsized":"2024-05-09T08:07:55.768144+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"27'15","reported_seq":41,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:55.769742+0000","last_change":"2024-05-09T08:02:15.888605+0000","last_active":"2024-05-09T08:07:55.769742+0000","last_peered":"2024-05-09T08:07:55.769742+0000","last_clean":"2024-05-09T08:07:55.769742+0000","last_became_active":"2024-05-09T08:02:15.887981+0000","last_became_peered":"2024-05-09T08:02:15.887981+0000","last_unstale":"2024-05-09T08:07:55.769742+0000","last_undegraded":"2024-05-09T08:07:55.769742+0000","last_fullsized":"2024-05-09T08:07:55.769742+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:47:32.605288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"27'2","reported_seq":28,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:55.774429+0000","last_change":"2024-05-09T08:02:15.904744+0000","last_active":"2024-05-09T08:07:55.774429+0000","last_peered":"2024-05-09T08:07:55.774429+0000","last_clean":"2024-05-09T08:07:55.774429+0000","last_became_active":"2024-05-09T08:02:15.904334+0000","last_became_peered":"2024-05-09T08:02:15.904334+0000","last_unstale":"2024-05-09T08:07:55.774429+0000","last_undegraded":"2024-05-09T08:07:55.774429+0000","last_fullsized":"2024-05-09T08:07:55.774429+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"27'6","reported_seq":31,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:55.785699+0000","last_change":"2024-05-09T08:02:15.904835+0000","last_active":"2024-05-09T08:07:55.785699+0000","last_peered":"2024-05-09T08:07:55.785699+0000","last_clean":"2024-05-09T08:07:55.785699+0000","last_became_active":"2024-05-09T08:02:15.904416+0000","last_became_peered":"2024-05-09T08:02:15.904416+0000","last_unstale":"2024-05-09T08:07:55.785699+0000","last_undegraded":"2024-05-09T08:07:55.785699+0000","last_fullsized":"2024-05-09T08:07:55.785699+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:47:32.605288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"27'8","reported_seq":34,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:55.778342+0000","last_change":"2024-05-09T08:02:15.898324+0000","last_active":"2024-05-09T08:07:55.778342+0000","last_peered":"2024-05-09T08:07:55.778342+0000","last_clean":"2024-05-09T08:07:55.778342+0000","last_became_active":"2024-05-09T08:02:15.897981+0000","last_became_peered":"2024-05-09T08:02:15.897981+0000","last_unstale":"2024-05-09T08:07:55.778342+0000","last_undegraded":"2024-05-09T08:07:55.778342+0000","last_fullsized":"2024-05-09T08:07:55.778342+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"27'10","reported_seq":35,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:55.774577+0000","last_change":"2024-05-09T08:02:15.890945+0000","last_active":"2024-05-09T08:07:55.774577+0000","last_peered":"2024-05-09T08:07:55.774577+0000","last_clean":"2024-05-09T08:07:55.774577+0000","last_became_active":"2024-05-09T08:02:15.890647+0000","last_became_peered":"2024-05-09T08:02:15.890647+0000","last_unstale":"2024-05-09T08:07:55.774577+0000","last_undegraded":"2024-05-09T08:07:55.774577+0000","last_fullsized":"2024-05-09T08:07:55.774577+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:41:56.871863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"27'11","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:55.783821+0000","last_change":"2024-05-09T08:02:15.913728+0000","last_active":"2024-05-09T08:07:55.783821+0000","last_peered":"2024-05-09T08:07:55.783821+0000","last_clean":"2024-05-09T08:07:55.783821+0000","last_became_active":"2024-05-09T08:02:15.909818+0000","last_became_peered":"2024-05-09T08:02:15.909818+0000","last_unstale":"2024-05-09T08:07:55.783821+0000","last_undegraded":"2024-05-09T08:07:55.783821+0000","last_fullsized":"2024-05-09T08:07:55.783821+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"27'8","reported_seq":34,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:55.769565+0000","last_change":"2024-05-09T08:02:15.885025+0000","last_active":"2024-05-09T08:07:55.769565+0000","last_peered":"2024-05-09T08:07:55.769565+0000","last_clean":"2024-05-09T08:07:55.769565+0000","last_became_active":"2024-05-09T08:02:15.884517+0000","last_became_peered":"2024-05-09T08:02:15.884517+0000","last_unstale":"2024-05-09T08:07:55.769565+0000","last_undegraded":"2024-05-09T08:07:55.769565+0000","last_fullsized":"2024-05-09T08:07:55.769565+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:01:50.341133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"27'7","reported_seq":32,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:55.769555+0000","last_change":"2024-05-09T08:02:15.890306+0000","last_active":"2024-05-09T08:07:55.769555+0000","last_peered":"2024-05-09T08:07:55.769555+0000","last_clean":"2024-05-09T08:07:55.769555+0000","last_became_active":"2024-05-09T08:02:15.889998+0000","last_became_peered":"2024-05-09T08:02:15.889998+0000","last_unstale":"2024-05-09T08:07:55.769555+0000","last_undegraded":"2024-05-09T08:07:55.769555+0000","last_fullsized":"2024-05-09T08:07:55.769555+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:16:57.446444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"27'10","reported_seq":36,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:55.769874+0000","last_change":"2024-05-09T08:02:15.886417+0000","last_active":"2024-05-09T08:07:55.769874+0000","last_peered":"2024-05-09T08:07:55.769874+0000","last_clean":"2024-05-09T08:07:55.769874+0000","last_became_active":"2024-05-09T08:02:15.885525+0000","last_became_peered":"2024-05-09T08:02:15.885525+0000","last_unstale":"2024-05-09T08:07:55.769874+0000","last_undegraded":"2024-05-09T08:07:55.769874+0000","last_fullsized":"2024-05-09T08:07:55.769874+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"27'8","reported_seq":33,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:55.774936+0000","last_change":"2024-05-09T08:02:15.893172+0000","last_active":"2024-05-09T08:07:55.774936+0000","last_peered":"2024-05-09T08:07:55.774936+0000","last_clean":"2024-05-09T08:07:55.774936+0000","last_became_active":"2024-05-09T08:02:15.892886+0000","last_became_peered":"2024-05-09T08:02:15.892886+0000","last_unstale":"2024-05-09T08:07:55.774936+0000","last_undegraded":"2024-05-09T08:07:55.774936+0000","last_fullsized":"2024-05-09T08:07:55.774936+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:59:14.957806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"27'9","reported_seq":34,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:55.782783+0000","last_change":"2024-05-09T08:02:15.908659+0000","last_active":"2024-05-09T08:07:55.782783+0000","last_peered":"2024-05-09T08:07:55.782783+0000","last_clean":"2024-05-09T08:07:55.782783+0000","last_became_active":"2024-05-09T08:02:15.908307+0000","last_became_peered":"2024-05-09T08:02:15.908307+0000","last_unstale":"2024-05-09T08:07:55.782783+0000","last_undegraded":"2024-05-09T08:07:55.782783+0000","last_fullsized":"2024-05-09T08:07:55.782783+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:20:07.368327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"27'7","reported_seq":33,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:55.774433+0000","last_change":"2024-05-09T08:02:15.885800+0000","last_active":"2024-05-09T08:07:55.774433+0000","last_peered":"2024-05-09T08:07:55.774433+0000","last_clean":"2024-05-09T08:07:55.774433+0000","last_became_active":"2024-05-09T08:02:15.885185+0000","last_became_peered":"2024-05-09T08:02:15.885185+0000","last_unstale":"2024-05-09T08:07:55.774433+0000","last_undegraded":"2024-05-09T08:07:55.774433+0000","last_fullsized":"2024-05-09T08:07:55.774433+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"27'8","reported_seq":42,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:08:00.299276+0000","last_change":"2024-05-09T08:08:00.299335+0000","last_active":"2024-05-09T08:08:00.299276+0000","last_peered":"2024-05-09T08:08:00.299276+0000","last_clean":"2024-05-09T08:08:00.299276+0000","last_became_active":"2024-05-09T08:02:15.911142+0000","last_became_peered":"2024-05-09T08:02:15.911142+0000","last_unstale":"2024-05-09T08:08:00.299276+0000","last_undegraded":"2024-05-09T08:08:00.299276+0000","last_fullsized":"2024-05-09T08:08:00.299276+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'8","last_scrub_stamp":"2024-05-09T08:08:00.299108+0000","last_deep_scrub":"27'8","last_deep_scrub_stamp":"2024-05-09T08:08:00.299108+0000","last_clean_scrub_stamp":"2024-05-09T08:08:00.299108+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:53:19.602110+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"27'10","reported_seq":36,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:55.773634+0000","last_change":"2024-05-09T08:02:15.898847+0000","last_active":"2024-05-09T08:07:55.773634+0000","last_peered":"2024-05-09T08:07:55.773634+0000","last_clean":"2024-05-09T08:07:55.773634+0000","last_became_active":"2024-05-09T08:02:15.898480+0000","last_became_peered":"2024-05-09T08:02:15.898480+0000","last_unstale":"2024-05-09T08:07:55.773634+0000","last_undegraded":"2024-05-09T08:07:55.773634+0000","last_fullsized":"2024-05-09T08:07:55.773634+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"27'7","reported_seq":33,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:55.772744+0000","last_change":"2024-05-09T08:02:15.883117+0000","last_active":"2024-05-09T08:07:55.772744+0000","last_peered":"2024-05-09T08:07:55.772744+0000","last_clean":"2024-05-09T08:07:55.772744+0000","last_became_active":"2024-05-09T08:02:15.882141+0000","last_became_peered":"2024-05-09T08:02:15.882141+0000","last_unstale":"2024-05-09T08:07:55.772744+0000","last_undegraded":"2024-05-09T08:07:55.772744+0000","last_fullsized":"2024-05-09T08:07:55.772744+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"27'6","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:55.768725+0000","last_change":"2024-05-09T08:03:02.322462+0000","last_active":"2024-05-09T08:07:55.768725+0000","last_peered":"2024-05-09T08:07:55.768725+0000","last_clean":"2024-05-09T08:07:55.768725+0000","last_became_active":"2024-05-09T08:03:02.322073+0000","last_became_peered":"2024-05-09T08:03:02.322073+0000","last_unstale":"2024-05-09T08:07:55.768725+0000","last_undegraded":"2024-05-09T08:07:55.768725+0000","last_fullsized":"2024-05-09T08:07:55.768725+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:16:37.795617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"27'16","reported_seq":42,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:55.773145+0000","last_change":"2024-05-09T08:02:15.886482+0000","last_active":"2024-05-09T08:07:55.773145+0000","last_peered":"2024-05-09T08:07:55.773145+0000","last_clean":"2024-05-09T08:07:55.773145+0000","last_became_active":"2024-05-09T08:02:15.886183+0000","last_became_peered":"2024-05-09T08:02:15.886183+0000","last_unstale":"2024-05-09T08:07:55.773145+0000","last_undegraded":"2024-05-09T08:07:55.773145+0000","last_fullsized":"2024-05-09T08:07:55.773145+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:23:40.248821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"27'11","reported_seq":36,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:55.772619+0000","last_change":"2024-05-09T08:02:15.887506+0000","last_active":"2024-05-09T08:07:55.772619+0000","last_peered":"2024-05-09T08:07:55.772619+0000","last_clean":"2024-05-09T08:07:55.772619+0000","last_became_active":"2024-05-09T08:02:15.887225+0000","last_became_peered":"2024-05-09T08:02:15.887225+0000","last_unstale":"2024-05-09T08:07:55.772619+0000","last_undegraded":"2024-05-09T08:07:55.772619+0000","last_fullsized":"2024-05-09T08:07:55.772619+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:25:24.736323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"27'9","reported_seq":35,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:55.781747+0000","last_change":"2024-05-09T08:02:15.902065+0000","last_active":"2024-05-09T08:07:55.781747+0000","last_peered":"2024-05-09T08:07:55.781747+0000","last_clean":"2024-05-09T08:07:55.781747+0000","last_became_active":"2024-05-09T08:02:15.901766+0000","last_became_peered":"2024-05-09T08:02:15.901766+0000","last_unstale":"2024-05-09T08:07:55.781747+0000","last_undegraded":"2024-05-09T08:07:55.781747+0000","last_fullsized":"2024-05-09T08:07:55.781747+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"27'5","reported_seq":30,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:55.779373+0000","last_change":"2024-05-09T08:02:15.911058+0000","last_active":"2024-05-09T08:07:55.779373+0000","last_peered":"2024-05-09T08:07:55.779373+0000","last_clean":"2024-05-09T08:07:55.779373+0000","last_became_active":"2024-05-09T08:02:15.910716+0000","last_became_peered":"2024-05-09T08:02:15.910716+0000","last_unstale":"2024-05-09T08:07:55.779373+0000","last_undegraded":"2024-05-09T08:07:55.779373+0000","last_fullsized":"2024-05-09T08:07:55.779373+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:41:56.871863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"27'6","reported_seq":31,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:55.782144+0000","last_change":"2024-05-09T08:02:15.908653+0000","last_active":"2024-05-09T08:07:55.782144+0000","last_peered":"2024-05-09T08:07:55.782144+0000","last_clean":"2024-05-09T08:07:55.782144+0000","last_became_active":"2024-05-09T08:02:15.908290+0000","last_became_peered":"2024-05-09T08:02:15.908290+0000","last_unstale":"2024-05-09T08:07:55.782144+0000","last_undegraded":"2024-05-09T08:07:55.782144+0000","last_fullsized":"2024-05-09T08:07:55.782144+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:18:36.309627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"27'4","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:55.768769+0000","last_change":"2024-05-09T08:02:15.896652+0000","last_active":"2024-05-09T08:07:55.768769+0000","last_peered":"2024-05-09T08:07:55.768769+0000","last_clean":"2024-05-09T08:07:55.768769+0000","last_became_active":"2024-05-09T08:02:15.896384+0000","last_became_peered":"2024-05-09T08:02:15.896384+0000","last_unstale":"2024-05-09T08:07:55.768769+0000","last_undegraded":"2024-05-09T08:07:55.768769+0000","last_fullsized":"2024-05-09T08:07:55.768769+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:18:36.309627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"27'8","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:55.777300+0000","last_change":"2024-05-09T08:03:02.615230+0000","last_active":"2024-05-09T08:07:55.777300+0000","last_peered":"2024-05-09T08:07:55.777300+0000","last_clean":"2024-05-09T08:07:55.777300+0000","last_became_active":"2024-05-09T08:03:02.614387+0000","last_became_peered":"2024-05-09T08:03:02.614387+0000","last_unstale":"2024-05-09T08:07:55.777300+0000","last_undegraded":"2024-05-09T08:07:55.777300+0000","last_fullsized":"2024-05-09T08:07:55.777300+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3d","version":"27'7","reported_seq":33,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:55.782051+0000","last_change":"2024-05-09T08:02:15.905078+0000","last_active":"2024-05-09T08:07:55.782051+0000","last_peered":"2024-05-09T08:07:55.782051+0000","last_clean":"2024-05-09T08:07:55.782051+0000","last_became_active":"2024-05-09T08:02:15.904736+0000","last_became_peered":"2024-05-09T08:02:15.904736+0000","last_unstale":"2024-05-09T08:07:55.782051+0000","last_undegraded":"2024-05-09T08:07:55.782051+0000","last_fullsized":"2024-05-09T08:07:55.782051+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"27'4","reported_seq":30,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:55.779639+0000","last_change":"2024-05-09T08:02:15.897743+0000","last_active":"2024-05-09T08:07:55.779639+0000","last_peered":"2024-05-09T08:07:55.779639+0000","last_clean":"2024-05-09T08:07:55.779639+0000","last_became_active":"2024-05-09T08:02:15.897367+0000","last_became_peered":"2024-05-09T08:02:15.897367+0000","last_unstale":"2024-05-09T08:07:55.779639+0000","last_undegraded":"2024-05-09T08:07:55.779639+0000","last_fullsized":"2024-05-09T08:07:55.779639+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"27'8","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:07:55.769375+0000","last_change":"2024-05-09T08:03:02.323519+0000","last_active":"2024-05-09T08:07:55.769375+0000","last_peered":"2024-05-09T08:07:55.769375+0000","last_clean":"2024-05-09T08:07:55.769375+0000","last_became_active":"2024-05-09T08:03:02.322397+0000","last_became_peered":"2024-05-09T08:03:02.322397+0000","last_unstale":"2024-05-09T08:07:55.769375+0000","last_undegraded":"2024-05-09T08:07:55.769375+0000","last_fullsized":"2024-05-09T08:07:55.769375+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:02:14.840955+0000","last_clean_scrub_stamp":"2024-05-09T08:02:14.840955+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9216,"num_write_kb":25165824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10755,"ondisk_log_size":10755,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":503,"ondisk_log_size":503,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":449011,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":282,"num_write_kb":620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":966656,"data_stored":898022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":273,"ondisk_log_size":273,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574921,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50092,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321748,"statfs":{"total":96636764160,"available":96585469952,"internally_reserved":0,"allocated":888832,"data_stored":709919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574921,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":76260,"kb_used_data":732,"kb_used_omap":1,"kb_used_meta":75518,"kb_avail":94295580,"statfs":{"total":96636764160,"available":96558673920,"internally_reserved":0,"allocated":749568,"data_stored":566653,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":77330886},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574921,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45352,"kb_used_data":288,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94326488,"statfs":{"total":96636764160,"available":96590323712,"internally_reserved":0,"allocated":294912,"data_stored":119059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574922,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":53680,"kb_used_data":296,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94318160,"statfs":{"total":96636764160,"available":96581795840,"internally_reserved":0,"allocated":303104,"data_stored":119095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":54655431},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574922,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":76264,"kb_used_data":736,"kb_used_omap":1,"kb_used_meta":75518,"kb_avail":94295576,"statfs":{"total":96636764160,"available":96558669824,"internally_reserved":0,"allocated":753664,"data_stored":567182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":77330887},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574922,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":57832,"kb_used_data":288,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94314008,"statfs":{"total":96636764160,"available":96577544192,"internally_reserved":0,"allocated":294912,"data_stored":119498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574923,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":75820,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":75518,"kb_avail":94296020,"statfs":{"total":96636764160,"available":96559124480,"internally_reserved":0,"allocated":299008,"data_stored":119129,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77330885},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574922,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45940,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325900,"statfs":{"total":96636764160,"available":96589721600,"internally_reserved":0,"allocated":897024,"data_stored":709975,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":462848,"data_stored":448213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":458752,"data_stored":447684,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T08:08:05.425 INFO:tasks.ceph:pgid 3.1e last_scrub_stamp 2024-05-09T08:02:14.840955+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=2, tm_sec=14, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=7, tm_sec=58, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:08:05.425 INFO:tasks.ceph:pgid 4.19 last_scrub_stamp 2024-05-09T08:02:18.908679+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=2, tm_sec=18, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=7, tm_sec=58, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:08:05.426 INFO:tasks.ceph:pgid 2.1e last_scrub_stamp 2024-05-09T08:02:13.827313+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=2, tm_sec=13, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=7, tm_sec=58, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:08:05.426 INFO:tasks.ceph:pgid 3.1f last_scrub_stamp 2024-05-09T08:02:14.840955+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=2, tm_sec=14, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=7, tm_sec=58, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:08:05.426 INFO:tasks.ceph:pgid 4.18 last_scrub_stamp 2024-05-09T08:02:18.908679+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=2, tm_sec=18, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=7, tm_sec=58, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:08:05.426 INFO:tasks.ceph:pgid 2.1d last_scrub_stamp 2024-05-09T08:02:13.827313+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=2, tm_sec=13, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=7, tm_sec=58, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:08:05.426 INFO:tasks.ceph:pgid 3.1c last_scrub_stamp 2024-05-09T08:02:14.840955+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=2, tm_sec=14, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=7, tm_sec=58, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:08:05.426 INFO:tasks.ceph:pgid 4.1b last_scrub_stamp 2024-05-09T08:02:18.908679+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=2, tm_sec=18, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=7, tm_sec=58, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:08:05.426 INFO:tasks.ceph:pgid 2.1c last_scrub_stamp 2024-05-09T08:02:13.827313+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=2, tm_sec=13, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=7, tm_sec=58, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:08:05.426 INFO:tasks.ceph:pgid 3.1d last_scrub_stamp 2024-05-09T08:02:14.840955+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=2, tm_sec=14, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=7, tm_sec=58, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:08:05.426 INFO:tasks.ceph:pgid 4.1a last_scrub_stamp 2024-05-09T08:02:18.908679+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=2, tm_sec=18, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=7, tm_sec=58, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:08:05.427 INFO:tasks.ceph:pgid 3.a last_scrub_stamp 2024-05-09T08:02:14.840955+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=2, tm_sec=14, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=7, tm_sec=58, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:08:05.427 INFO:tasks.ceph:pgid 2.b last_scrub_stamp 2024-05-09T08:02:13.827313+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=2, tm_sec=13, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=7, tm_sec=58, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:08:05.427 INFO:tasks.ceph:pgid 4.d last_scrub_stamp 2024-05-09T08:02:18.908679+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=2, tm_sec=18, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=7, tm_sec=58, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:08:05.427 INFO:tasks.ceph:pgid 3.b last_scrub_stamp 2024-05-09T08:02:14.840955+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=2, tm_sec=14, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=7, tm_sec=58, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:08:05.427 INFO:tasks.ceph:pgid 2.a last_scrub_stamp 2024-05-09T08:02:13.827313+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=2, tm_sec=13, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=7, tm_sec=58, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:08:05.427 INFO:tasks.ceph:pgid 4.c last_scrub_stamp 2024-05-09T08:02:18.908679+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=2, tm_sec=18, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=7, tm_sec=58, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:08:05.427 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2024-05-09T08:02:14.840955+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=2, tm_sec=14, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=7, tm_sec=58, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:08:05.427 INFO:tasks.ceph:pgid 4.f last_scrub_stamp 2024-05-09T08:02:18.908679+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=2, tm_sec=18, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=7, tm_sec=58, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:08:05.427 INFO:tasks.ceph:pgid 3.9 last_scrub_stamp 2024-05-09T08:02:14.840955+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=2, tm_sec=14, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=7, tm_sec=58, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:08:05.427 INFO:tasks.ceph:pgid 2.8 last_scrub_stamp 2024-05-09T08:02:13.827313+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=2, tm_sec=13, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=7, tm_sec=58, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:08:05.428 INFO:tasks.ceph:pgid 4.e last_scrub_stamp 2024-05-09T08:02:18.908679+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=2, tm_sec=18, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=7, tm_sec=58, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:08:05.428 INFO:tasks.ceph:pgid 3.6 last_scrub_stamp 2024-05-09T08:02:14.840955+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=2, tm_sec=14, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=7, tm_sec=58, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:08:05.428 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2024-05-09T08:02:13.827313+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=2, tm_sec=13, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=7, tm_sec=58, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:08:05.428 INFO:tasks.ceph:pgid 4.1 last_scrub_stamp 2024-05-09T08:02:18.908679+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=2, tm_sec=18, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=7, tm_sec=58, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:08:05.428 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2024-05-09T08:02:14.840955+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=2, tm_sec=14, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=7, tm_sec=58, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:08:05.428 INFO:tasks.ceph:pgid 4.0 last_scrub_stamp 2024-05-09T08:02:18.908679+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=2, tm_sec=18, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=7, tm_sec=58, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:08:05.428 INFO:tasks.ceph:pgid 3.4 last_scrub_stamp 2024-05-09T08:02:14.840955+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=2, tm_sec=14, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=7, tm_sec=58, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:08:05.428 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-05-09T08:02:13.827313+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=2, tm_sec=13, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=7, tm_sec=58, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:08:05.428 INFO:tasks.ceph:pgid 4.3 last_scrub_stamp 2024-05-09T08:02:18.908679+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=2, tm_sec=18, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=7, tm_sec=58, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:08:05.428 INFO:tasks.ceph:pgid 3.5 last_scrub_stamp 2024-05-09T08:02:14.840955+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=2, tm_sec=14, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=7, tm_sec=58, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:08:05.429 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2024-05-09T08:02:13.827313+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=2, tm_sec=13, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=7, tm_sec=58, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:08:05.429 INFO:tasks.ceph:pgid 4.2 last_scrub_stamp 2024-05-09T08:02:18.908679+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=2, tm_sec=18, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=7, tm_sec=58, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:08:05.429 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2024-05-09T08:02:14.840955+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=2, tm_sec=14, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=7, tm_sec=58, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:08:05.429 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2024-05-09T08:02:13.827313+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=2, tm_sec=13, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=7, tm_sec=58, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:08:05.429 INFO:tasks.ceph:pgid 4.4 last_scrub_stamp 2024-05-09T08:02:18.908679+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=2, tm_sec=18, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=7, tm_sec=58, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:08:05.429 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2024-05-09T08:02:14.840955+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=2, tm_sec=14, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=7, tm_sec=58, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:08:05.429 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2024-05-09T08:02:13.827313+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=2, tm_sec=13, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=7, tm_sec=58, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:08:05.429 INFO:tasks.ceph:pgid 4.7 last_scrub_stamp 2024-05-09T08:02:18.908679+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=2, tm_sec=18, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=7, tm_sec=58, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:08:05.429 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2024-05-09T08:02:14.840955+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=2, tm_sec=14, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=7, tm_sec=58, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:08:05.429 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-05-09T08:02:13.827313+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=2, tm_sec=13, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=7, tm_sec=58, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:08:05.430 INFO:tasks.ceph:pgid 4.6 last_scrub_stamp 2024-05-09T08:02:18.908679+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=2, tm_sec=18, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=7, tm_sec=58, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:08:05.430 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2024-05-09T08:02:14.840955+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=2, tm_sec=14, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=7, tm_sec=58, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:08:05.430 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2024-05-09T08:02:13.827313+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=2, tm_sec=13, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=7, tm_sec=58, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:08:05.430 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-05-09T08:02:09.800143+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=2, tm_sec=9, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=7, tm_sec=58, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:08:05.430 INFO:tasks.ceph:pgid 4.5 last_scrub_stamp 2024-05-09T08:02:18.908679+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=2, tm_sec=18, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=7, tm_sec=58, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:08:05.430 INFO:tasks.ceph:pgid 3.d last_scrub_stamp 2024-05-09T08:02:14.840955+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=2, tm_sec=14, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=7, tm_sec=58, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:08:05.430 INFO:tasks.ceph:pgid 4.a last_scrub_stamp 2024-05-09T08:02:18.908679+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=2, tm_sec=18, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=7, tm_sec=58, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:08:05.430 INFO:tasks.ceph:pgid 3.c last_scrub_stamp 2024-05-09T08:02:14.840955+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=2, tm_sec=14, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=7, tm_sec=58, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:08:05.430 INFO:tasks.ceph:pgid 2.d last_scrub_stamp 2024-05-09T08:02:13.827313+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=2, tm_sec=13, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=7, tm_sec=58, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:08:05.431 INFO:tasks.ceph:pgid 4.b last_scrub_stamp 2024-05-09T08:02:18.908679+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=2, tm_sec=18, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=7, tm_sec=58, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:08:05.431 INFO:tasks.ceph:pgid 3.f last_scrub_stamp 2024-05-09T08:02:14.840955+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=2, tm_sec=14, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=7, tm_sec=58, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:08:05.431 INFO:tasks.ceph:pgid 2.e last_scrub_stamp 2024-05-09T08:02:13.827313+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=2, tm_sec=13, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=7, tm_sec=58, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:08:05.431 INFO:tasks.ceph:pgid 4.8 last_scrub_stamp 2024-05-09T08:02:18.908679+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=2, tm_sec=18, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=7, tm_sec=58, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:08:05.431 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2024-05-09T08:02:14.840955+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=2, tm_sec=14, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=7, tm_sec=58, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:08:05.431 INFO:tasks.ceph:pgid 4.9 last_scrub_stamp 2024-05-09T08:02:18.908679+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=2, tm_sec=18, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=7, tm_sec=58, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:08:05.431 INFO:tasks.ceph:pgid 3.11 last_scrub_stamp 2024-05-09T08:02:14.840955+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=2, tm_sec=14, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=7, tm_sec=58, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:08:05.431 INFO:tasks.ceph:pgid 4.16 last_scrub_stamp 2024-05-09T08:02:18.908679+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=2, tm_sec=18, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=7, tm_sec=58, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:08:05.431 INFO:tasks.ceph:pgid 3.10 last_scrub_stamp 2024-05-09T08:02:14.840955+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=2, tm_sec=14, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=7, tm_sec=58, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:08:05.432 INFO:tasks.ceph:pgid 2.11 last_scrub_stamp 2024-05-09T08:02:13.827313+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=2, tm_sec=13, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=7, tm_sec=58, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:08:05.432 INFO:tasks.ceph:pgid 4.17 last_scrub_stamp 2024-05-09T08:02:18.908679+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=2, tm_sec=18, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=7, tm_sec=58, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:08:05.432 INFO:tasks.ceph:pgid 3.13 last_scrub_stamp 2024-05-09T08:02:14.840955+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=2, tm_sec=14, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=7, tm_sec=58, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:08:05.432 INFO:tasks.ceph:pgid 2.12 last_scrub_stamp 2024-05-09T08:02:13.827313+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=2, tm_sec=13, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=7, tm_sec=58, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:08:05.432 INFO:tasks.ceph:pgid 4.14 last_scrub_stamp 2024-05-09T08:02:18.908679+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=2, tm_sec=18, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=7, tm_sec=58, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:08:05.432 INFO:tasks.ceph:pgid 3.12 last_scrub_stamp 2024-05-09T08:02:14.840955+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=2, tm_sec=14, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=7, tm_sec=58, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:08:05.432 INFO:tasks.ceph:pgid 4.15 last_scrub_stamp 2024-05-09T08:02:18.908679+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=2, tm_sec=18, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=7, tm_sec=58, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:08:05.432 INFO:tasks.ceph:pgid 3.15 last_scrub_stamp 2024-05-09T08:02:14.840955+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=2, tm_sec=14, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=7, tm_sec=58, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:08:05.432 INFO:tasks.ceph:pgid 2.14 last_scrub_stamp 2024-05-09T08:02:13.827313+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=2, tm_sec=13, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=7, tm_sec=58, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:08:05.432 INFO:tasks.ceph:pgid 4.12 last_scrub_stamp 2024-05-09T08:02:18.908679+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=2, tm_sec=18, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=7, tm_sec=58, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:08:05.433 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2024-05-09T08:02:14.840955+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=2, tm_sec=14, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=7, tm_sec=58, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:08:05.433 INFO:tasks.ceph:pgid 4.13 last_scrub_stamp 2024-05-09T08:02:18.908679+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=2, tm_sec=18, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=7, tm_sec=58, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:08:05.433 INFO:tasks.ceph:pgid 3.17 last_scrub_stamp 2024-05-09T08:02:14.840955+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=2, tm_sec=14, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=7, tm_sec=58, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:08:05.433 INFO:tasks.ceph:pgid 2.16 last_scrub_stamp 2024-05-09T08:02:13.827313+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=2, tm_sec=13, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=7, tm_sec=58, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:08:05.433 INFO:tasks.ceph:pgid 4.10 last_scrub_stamp 2024-05-09T08:02:18.908679+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=2, tm_sec=18, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=7, tm_sec=58, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:08:05.433 INFO:tasks.ceph:pgid 3.16 last_scrub_stamp 2024-05-09T08:02:14.840955+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=2, tm_sec=14, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=7, tm_sec=58, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:08:05.433 INFO:tasks.ceph:pgid 2.17 last_scrub_stamp 2024-05-09T08:02:13.827313+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=2, tm_sec=13, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=7, tm_sec=58, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:08:05.433 INFO:tasks.ceph:pgid 4.11 last_scrub_stamp 2024-05-09T08:02:18.908679+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=2, tm_sec=18, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=7, tm_sec=58, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:08:05.434 INFO:tasks.ceph:pgid 4.1e last_scrub_stamp 2024-05-09T08:02:18.908679+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=2, tm_sec=18, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=7, tm_sec=58, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:08:05.434 INFO:tasks.ceph:pgid 3.19 last_scrub_stamp 2024-05-09T08:02:14.840955+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=2, tm_sec=14, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=7, tm_sec=58, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:08:05.434 INFO:tasks.ceph:pgid 2.18 last_scrub_stamp 2024-05-09T08:02:13.827313+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=2, tm_sec=13, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=7, tm_sec=58, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:08:05.434 INFO:tasks.ceph:pgid 4.1f last_scrub_stamp 2024-05-09T08:02:18.908679+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=2, tm_sec=18, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=7, tm_sec=58, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:08:05.434 INFO:tasks.ceph:pgid 3.18 last_scrub_stamp 2024-05-09T08:02:14.840955+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=2, tm_sec=14, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=7, tm_sec=58, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:08:05.434 INFO:tasks.ceph:pgid 2.19 last_scrub_stamp 2024-05-09T08:02:13.827313+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=2, tm_sec=13, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=7, tm_sec=58, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:08:05.434 INFO:tasks.ceph:pgid 2.1a last_scrub_stamp 2024-05-09T08:02:13.827313+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=2, tm_sec=13, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=7, tm_sec=58, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:08:05.434 INFO:tasks.ceph:pgid 3.1b last_scrub_stamp 2024-05-09T08:02:14.840955+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=2, tm_sec=14, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=7, tm_sec=58, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:08:05.434 INFO:tasks.ceph:pgid 4.1c last_scrub_stamp 2024-05-09T08:02:18.908679+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=2, tm_sec=18, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=7, tm_sec=58, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:08:05.434 INFO:tasks.ceph:pgid 2.1b last_scrub_stamp 2024-05-09T08:02:13.827313+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=2, tm_sec=13, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=7, tm_sec=58, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:08:05.435 INFO:tasks.ceph:pgid 3.1a last_scrub_stamp 2024-05-09T08:02:14.840955+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=2, tm_sec=14, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=7, tm_sec=58, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:08:05.435 INFO:tasks.ceph:pgid 4.1d last_scrub_stamp 2024-05-09T08:02:18.908679+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=2, tm_sec=18, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=7, tm_sec=58, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:08:05.435 INFO:tasks.ceph:pgid 3.20 last_scrub_stamp 2024-05-09T08:02:14.840955+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=2, tm_sec=14, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=7, tm_sec=58, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:08:05.435 INFO:tasks.ceph:pgid 3.21 last_scrub_stamp 2024-05-09T08:02:14.840955+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=2, tm_sec=14, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=7, tm_sec=58, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:08:05.435 INFO:tasks.ceph:pgid 3.22 last_scrub_stamp 2024-05-09T08:02:14.840955+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=2, tm_sec=14, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=7, tm_sec=58, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:08:05.435 INFO:tasks.ceph:pgid 3.23 last_scrub_stamp 2024-05-09T08:02:14.840955+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=2, tm_sec=14, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=7, tm_sec=58, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:08:05.435 INFO:tasks.ceph:pgid 3.24 last_scrub_stamp 2024-05-09T08:02:14.840955+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=2, tm_sec=14, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=7, tm_sec=58, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:08:05.435 INFO:tasks.ceph:pgid 3.25 last_scrub_stamp 2024-05-09T08:02:14.840955+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=2, tm_sec=14, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=7, tm_sec=58, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:08:05.435 INFO:tasks.ceph:pgid 3.26 last_scrub_stamp 2024-05-09T08:02:14.840955+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=2, tm_sec=14, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=7, tm_sec=58, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:08:05.436 INFO:tasks.ceph:pgid 3.27 last_scrub_stamp 2024-05-09T08:02:14.840955+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=2, tm_sec=14, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=7, tm_sec=58, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:08:05.436 INFO:tasks.ceph:pgid 3.28 last_scrub_stamp 2024-05-09T08:02:14.840955+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=2, tm_sec=14, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=7, tm_sec=58, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:08:05.436 INFO:tasks.ceph:pgid 3.29 last_scrub_stamp 2024-05-09T08:02:14.840955+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=2, tm_sec=14, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=7, tm_sec=58, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:08:05.436 INFO:tasks.ceph:pgid 3.2a last_scrub_stamp 2024-05-09T08:02:14.840955+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=2, tm_sec=14, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=7, tm_sec=58, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:08:05.436 INFO:tasks.ceph:pgid 3.2b last_scrub_stamp 2024-05-09T08:02:14.840955+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=2, tm_sec=14, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=7, tm_sec=58, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:08:05.436 INFO:tasks.ceph:pgid 3.2c last_scrub_stamp 2024-05-09T08:02:14.840955+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=2, tm_sec=14, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=7, tm_sec=58, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:08:05.436 INFO:tasks.ceph:pgid 3.2d last_scrub_stamp 2024-05-09T08:02:14.840955+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=2, tm_sec=14, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=7, tm_sec=58, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:08:05.436 INFO:tasks.ceph:pgid 3.2e last_scrub_stamp 2024-05-09T08:02:14.840955+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=2, tm_sec=14, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=7, tm_sec=58, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:08:05.436 INFO:tasks.ceph:pgid 3.2f last_scrub_stamp 2024-05-09T08:02:14.840955+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=2, tm_sec=14, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=7, tm_sec=58, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:08:05.437 INFO:tasks.ceph:pgid 3.30 last_scrub_stamp 2024-05-09T08:02:14.840955+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=2, tm_sec=14, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=7, tm_sec=58, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:08:05.437 INFO:tasks.ceph:pgid 3.31 last_scrub_stamp 2024-05-09T08:02:14.840955+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=2, tm_sec=14, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=7, tm_sec=58, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:08:05.437 INFO:tasks.ceph:pgid 3.33 last_scrub_stamp 2024-05-09T08:02:14.840955+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=2, tm_sec=14, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=7, tm_sec=58, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:08:05.437 INFO:tasks.ceph:pgid 3.34 last_scrub_stamp 2024-05-09T08:02:14.840955+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=2, tm_sec=14, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=7, tm_sec=58, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:08:05.437 INFO:tasks.ceph:pgid 3.35 last_scrub_stamp 2024-05-09T08:02:14.840955+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=2, tm_sec=14, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=7, tm_sec=58, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:08:05.437 INFO:tasks.ceph:pgid 3.36 last_scrub_stamp 2024-05-09T08:02:14.840955+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=2, tm_sec=14, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=7, tm_sec=58, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:08:05.437 INFO:tasks.ceph:pgid 3.37 last_scrub_stamp 2024-05-09T08:02:14.840955+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=2, tm_sec=14, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=7, tm_sec=58, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:08:05.437 INFO:tasks.ceph:pgid 3.38 last_scrub_stamp 2024-05-09T08:02:14.840955+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=2, tm_sec=14, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=7, tm_sec=58, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:08:05.437 INFO:tasks.ceph:pgid 3.39 last_scrub_stamp 2024-05-09T08:02:14.840955+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=2, tm_sec=14, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=7, tm_sec=58, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:08:05.437 INFO:tasks.ceph:pgid 3.3a last_scrub_stamp 2024-05-09T08:02:14.840955+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=2, tm_sec=14, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=7, tm_sec=58, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:08:05.438 INFO:tasks.ceph:pgid 3.3b last_scrub_stamp 2024-05-09T08:02:14.840955+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=2, tm_sec=14, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=7, tm_sec=58, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:08:05.438 INFO:tasks.ceph:pgid 3.3c last_scrub_stamp 2024-05-09T08:02:14.840955+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=2, tm_sec=14, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=7, tm_sec=58, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:08:05.438 INFO:tasks.ceph:pgid 3.3d last_scrub_stamp 2024-05-09T08:02:14.840955+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=2, tm_sec=14, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=7, tm_sec=58, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:08:05.438 INFO:tasks.ceph:pgid 3.3e last_scrub_stamp 2024-05-09T08:02:14.840955+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=2, tm_sec=14, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=7, tm_sec=58, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:08:05.438 INFO:tasks.ceph:pgid 3.3f last_scrub_stamp 2024-05-09T08:02:14.840955+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=2, tm_sec=14, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=7, tm_sec=58, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:08:05.438 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-05-09T08:08:21.826 DEBUG:teuthology.orchestra.run.smithi001:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T08:08:21.835 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T08:08:25.439 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-09T08:08:25.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:25.654+0000 7f8316600640 1 Processor -- start 2024-05-09T08:08:25.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:25.655+0000 7f8316600640 1 -- start start 2024-05-09T08:08:25.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:25.656+0000 7f8316600640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f83100c8180 0x7f83100cc730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:08:25.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:25.656+0000 7f8316600640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f83100c7660 0x7f83100c7ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:08:25.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:25.656+0000 7f8316600640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f831015c790 con 0x7f8310152500 2024-05-09T08:08:25.667 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:25.656+0000 7f8316600640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f83100c5e50 con 0x7f83100c8180 2024-05-09T08:08:25.667 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:25.656+0000 7f8316600640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f83100c22d0 con 0x7f83100c7660 2024-05-09T08:08:25.667 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:25.657+0000 7f8315200640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f8310152500 0x7f831015c080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:38794/0 (socket says 172.21.15.1:38794) 2024-05-09T08:08:25.667 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:25.657+0000 7f8315200640 1 -- 172.21.15.1:0/1673807973 learned_addr learned my addr 172.21.15.1:0/1673807973 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T08:08:25.667 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:25.657+0000 7f8315c00640 1 --2- 172.21.15.1:0/1673807973 >> v2:172.21.15.7:3300/0 conn(0x7f83100c7660 0x7f83100c7ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:08:25.667 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:25.657+0000 7f830fe00640 1 --2- 172.21.15.1:0/1673807973 >> v2:172.21.15.1:3300/0 conn(0x7f83100c8180 0x7f83100cc730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:08:25.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:25.657+0000 7f830fe00640 1 -- 172.21.15.1:0/1673807973 >> v2:172.21.15.7:3300/0 conn(0x7f83100c7660 msgr2=0x7f83100c7ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:08:25.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:25.657+0000 7f830fe00640 1 --2- 172.21.15.1:0/1673807973 >> v2:172.21.15.7:3300/0 conn(0x7f83100c7660 0x7f83100c7ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:25.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:25.657+0000 7f830fe00640 1 -- 172.21.15.1:0/1673807973 >> v1:172.21.15.1:6789/0 conn(0x7f8310152500 legacy=0x7f831015c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:08:25.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:25.657+0000 7f830fe00640 1 -- 172.21.15.1:0/1673807973 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83100c6a30 con 0x7f83100c8180 2024-05-09T08:08:25.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:25.657+0000 7f8315c00640 1 --2- 172.21.15.1:0/1673807973 >> v2:172.21.15.7:3300/0 conn(0x7f83100c7660 0x7f83100c7ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:08:25.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:25.658+0000 7f830fe00640 1 --2- 172.21.15.1:0/1673807973 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f83100c8180 0x7f83100cc730 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f831006eaf0 tx=0x7f82fc02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23d1168858f8c359 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:08:25.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:25.659+0000 7f830f400640 1 -- 172.21.15.1:0/1673807973 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82fc037070 con 0x7f83100c8180 2024-05-09T08:08:25.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:25.659+0000 7f830f400640 1 -- 172.21.15.1:0/1673807973 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82fc032440 con 0x7f83100c8180 2024-05-09T08:08:25.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:25.659+0000 7f830f400640 1 -- 172.21.15.1:0/1673807973 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82fc032810 con 0x7f83100c8180 2024-05-09T08:08:25.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:25.660+0000 7f8316600640 1 -- 172.21.15.1:0/1673807973 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f83100c8180 msgr2=0x7f83100cc730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:08:25.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:25.660+0000 7f8316600640 1 --2- 172.21.15.1:0/1673807973 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f83100c8180 0x7f83100cc730 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f831006eaf0 tx=0x7f82fc02a1a0 comp rx=0 tx=0).stop 2024-05-09T08:08:25.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:25.660+0000 7f8316600640 1 -- 172.21.15.1:0/1673807973 shutdown_connections 2024-05-09T08:08:25.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:25.660+0000 7f8316600640 1 --2- 172.21.15.1:0/1673807973 >> v2:172.21.15.7:3300/0 conn(0x7f83100c7660 0x7f83100c7ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:25.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:25.660+0000 7f8316600640 1 --2- 172.21.15.1:0/1673807973 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f83100c8180 0x7f83100cc730 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:25.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:25.660+0000 7f8316600640 1 -- 172.21.15.1:0/1673807973 >> 172.21.15.1:0/1673807973 conn(0x7f83100cbc10 msgr2=0x7f83100cc020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:08:25.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:25.660+0000 7f8316600640 1 -- 172.21.15.1:0/1673807973 shutdown_connections 2024-05-09T08:08:25.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:25.660+0000 7f8316600640 1 -- 172.21.15.1:0/1673807973 wait complete. 2024-05-09T08:08:25.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:25.662+0000 7f8316600640 1 Processor -- start 2024-05-09T08:08:25.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:25.662+0000 7f8316600640 1 -- start start 2024-05-09T08:08:25.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:25.664+0000 7f8316600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f83100c7660 0x7f831014a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:08:25.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:25.664+0000 7f8316600640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f83100c8180 0x7f8310148c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:08:25.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:25.664+0000 7f8316600640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f8310152500 0x7f831015aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:08:25.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:25.664+0000 7f8316600640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f83100c72b0 con 0x7f83100c8180 2024-05-09T08:08:25.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:25.664+0000 7f8316600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f83100c6fd0 con 0x7f83100c7660 2024-05-09T08:08:25.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:25.664+0000 7f8316600640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f831015b880 con 0x7f8310152500 2024-05-09T08:08:25.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:25.664+0000 7f830fe00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f83100c8180 0x7f8310148c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:08:25.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:25.664+0000 7f830fe00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f83100c8180 0x7f8310148c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49194/0 (socket says 172.21.15.1:49194) 2024-05-09T08:08:25.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:25.664+0000 7f830fe00640 1 -- 172.21.15.1:0/2523781313 learned_addr learned my addr 172.21.15.1:0/2523781313 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:08:25.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:25.664+0000 7f8315200640 1 --2- 172.21.15.1:0/2523781313 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f83100c7660 0x7f831014a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:08:25.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:25.664+0000 7f8315c00640 1 --2- 172.21.15.1:0/2523781313 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f8310152500 0x7f831015aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:08:25.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:25.665+0000 7f830fe00640 1 -- 172.21.15.1:0/2523781313 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f8310152500 msgr2=0x7f831015aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:08:25.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:25.665+0000 7f830fe00640 1 --2- 172.21.15.1:0/2523781313 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f8310152500 0x7f831015aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:25.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:25.665+0000 7f830fe00640 1 -- 172.21.15.1:0/2523781313 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f83100c7660 msgr2=0x7f831014a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:08:25.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:25.665+0000 7f830fe00640 1 --2- 172.21.15.1:0/2523781313 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f83100c7660 0x7f831014a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:25.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:25.665+0000 7f830fe00640 1 -- 172.21.15.1:0/2523781313 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8310157910 con 0x7f83100c8180 2024-05-09T08:08:25.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:25.665+0000 7f830fe00640 1 --2- 172.21.15.1:0/2523781313 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f83100c8180 0x7f8310148c60 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f82fc009890 tx=0x7f82fc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:08:25.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:25.665+0000 7f830cc00640 1 -- 172.21.15.1:0/2523781313 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82fc037040 con 0x7f83100c8180 2024-05-09T08:08:25.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:25.665+0000 7f830cc00640 1 -- 172.21.15.1:0/2523781313 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82fc042070 con 0x7f83100c8180 2024-05-09T08:08:25.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:25.665+0000 7f830cc00640 1 -- 172.21.15.1:0/2523781313 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82fc0308a0 con 0x7f83100c8180 2024-05-09T08:08:25.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:25.665+0000 7f8316600640 1 -- 172.21.15.1:0/2523781313 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83100c3af0 con 0x7f83100c8180 2024-05-09T08:08:25.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:25.666+0000 7f8316600640 1 -- 172.21.15.1:0/2523781313 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8310157d00 con 0x7f83100c8180 2024-05-09T08:08:25.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:25.667+0000 7f8316600640 1 -- 172.21.15.1:0/2523781313 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f831004f660 con 0x7f83100c8180 2024-05-09T08:08:25.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:25.668+0000 7f830cc00640 1 -- 172.21.15.1:0/2523781313 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f82fc030a40 con 0x7f83100c8180 2024-05-09T08:08:25.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:25.669+0000 7f830cc00640 1 --2- 172.21.15.1:0/2523781313 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f82f006be30 0x7f82f006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:08:25.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:25.669+0000 7f830cc00640 1 -- 172.21.15.1:0/2523781313 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7f82fc0abab0 con 0x7f83100c8180 2024-05-09T08:08:25.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:25.669+0000 7f8315200640 1 --2- 172.21.15.1:0/2523781313 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f82f006be30 0x7f82f006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:08:25.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:25.670+0000 7f8315200640 1 --2- 172.21.15.1:0/2523781313 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f82f006be30 0x7f82f006e2f0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f8300007fd0 tx=0x7f8300007e80 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:08:25.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:25.674+0000 7f830cc00640 1 -- 172.21.15.1:0/2523781313 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f82fc07a060 con 0x7f83100c8180 2024-05-09T08:08:25.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:25.873+0000 7f8316600640 1 -- 172.21.15.1:0/2523781313 --> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f83100c9a10 con 0x7f82f006be30 2024-05-09T08:08:25.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:25.883+0000 7f830cc00640 1 -- 172.21.15.1:0/2523781313 <== mgr.4114 v2:172.21.15.7:6832/3562612464 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+333672 (secure 0 0 0) 0x7f83100c9a10 con 0x7f82f006be30 2024-05-09T08:08:25.898 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T08:08:25.900 INFO:teuthology.orchestra.run.smithi001.stderr:dumped all 2024-05-09T08:08:25.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:25.890+0000 7f8316600640 1 -- 172.21.15.1:0/2523781313 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f82f006be30 msgr2=0x7f82f006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:08:25.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:25.890+0000 7f8316600640 1 --2- 172.21.15.1:0/2523781313 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f82f006be30 0x7f82f006e2f0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f8300007fd0 tx=0x7f8300007e80 comp rx=0 tx=0).stop 2024-05-09T08:08:25.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:25.890+0000 7f8316600640 1 -- 172.21.15.1:0/2523781313 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f83100c8180 msgr2=0x7f8310148c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:08:25.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:25.890+0000 7f8316600640 1 --2- 172.21.15.1:0/2523781313 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f83100c8180 0x7f8310148c60 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f82fc009890 tx=0x7f82fc0046c0 comp rx=0 tx=0).stop 2024-05-09T08:08:25.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:25.890+0000 7f8316600640 1 -- 172.21.15.1:0/2523781313 shutdown_connections 2024-05-09T08:08:25.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:25.890+0000 7f8316600640 1 --2- 172.21.15.1:0/2523781313 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f82f006be30 0x7f82f006e2f0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:25.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:25.890+0000 7f8316600640 1 --2- 172.21.15.1:0/2523781313 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f8310152500 0x7f831015aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:25.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:25.890+0000 7f8316600640 1 --2- 172.21.15.1:0/2523781313 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f83100c8180 0x7f8310148c60 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:25.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:25.890+0000 7f8316600640 1 --2- 172.21.15.1:0/2523781313 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f83100c7660 0x7f831014a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:25.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:25.890+0000 7f8316600640 1 -- 172.21.15.1:0/2523781313 >> 172.21.15.1:0/2523781313 conn(0x7f83100cbc10 msgr2=0x7f83100cbff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:08:25.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:25.890+0000 7f8316600640 1 -- 172.21.15.1:0/2523781313 shutdown_connections 2024-05-09T08:08:25.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:25.890+0000 7f8316600640 1 -- 172.21.15.1:0/2523781313 wait complete. 2024-05-09T08:08:25.917 INFO:teuthology.orchestra.run.smithi001.stdout:{"pg_ready":true,"pg_map":{"version":209,"stamp":"2024-05-09T08:08:25.193732+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1039379,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":121,"num_read_kb":105,"num_write":9657,"num_write_kb":25168860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":11659,"ondisk_log_size":11659,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":481240,"kb_used_data":4376,"kb_used_omap":12,"kb_used_meta":476787,"kb_avail":754493480,"statfs":{"total":773094113280,"available":772601323520,"internally_reserved":0,"allocated":4481024,"data_stored":3030510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":488230450},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003509"},"pg_stats":[{"pgid":"2.1f","version":"27'6","reported_seq":42,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:08:02.350512+0000","last_change":"2024-05-09T08:08:02.350541+0000","last_active":"2024-05-09T08:08:02.350512+0000","last_peered":"2024-05-09T08:08:02.350512+0000","last_clean":"2024-05-09T08:08:02.350512+0000","last_became_active":"2024-05-09T08:02:14.874841+0000","last_became_peered":"2024-05-09T08:02:14.874841+0000","last_unstale":"2024-05-09T08:08:02.350512+0000","last_undegraded":"2024-05-09T08:08:02.350512+0000","last_fullsized":"2024-05-09T08:08:02.350512+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'6","last_scrub_stamp":"2024-05-09T08:08:02.350436+0000","last_deep_scrub":"27'6","last_deep_scrub_stamp":"2024-05-09T08:08:02.350436+0000","last_clean_scrub_stamp":"2024-05-09T08:08:02.350436+0000","objects_scrubbed":2,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:32:34.124183+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"27'5","reported_seq":40,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:08:13.883560+0000","last_change":"2024-05-09T08:08:13.883560+0000","last_active":"2024-05-09T08:08:13.883560+0000","last_peered":"2024-05-09T08:08:13.883560+0000","last_clean":"2024-05-09T08:08:13.883560+0000","last_became_active":"2024-05-09T08:02:15.899479+0000","last_became_peered":"2024-05-09T08:02:15.899479+0000","last_unstale":"2024-05-09T08:08:13.883560+0000","last_undegraded":"2024-05-09T08:08:13.883560+0000","last_fullsized":"2024-05-09T08:08:13.883560+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'5","last_scrub_stamp":"2024-05-09T08:08:13.883240+0000","last_deep_scrub":"27'5","last_deep_scrub_stamp":"2024-05-09T08:08:13.883240+0000","last_clean_scrub_stamp":"2024-05-09T08:08:13.883240+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:07:56.177414+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"27'360","reported_seq":389,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:08:22.720812+0000","last_change":"2024-05-09T08:08:22.720934+0000","last_active":"2024-05-09T08:08:22.720812+0000","last_peered":"2024-05-09T08:08:22.720812+0000","last_clean":"2024-05-09T08:08:22.720812+0000","last_became_active":"2024-05-09T08:02:20.057016+0000","last_became_peered":"2024-05-09T08:02:20.057016+0000","last_unstale":"2024-05-09T08:08:22.720812+0000","last_undegraded":"2024-05-09T08:08:22.720812+0000","last_fullsized":"2024-05-09T08:08:22.720812+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'360","last_scrub_stamp":"2024-05-09T08:08:22.720597+0000","last_deep_scrub":"27'360","last_deep_scrub_stamp":"2024-05-09T08:08:22.720597+0000","last_clean_scrub_stamp":"2024-05-09T08:08:22.720597+0000","objects_scrubbed":0,"log_size":360,"log_dups_size":0,"ondisk_log_size":360,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:12:16.516079+0000","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":306,"num_write_kb":835584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"24'1","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:08:09.900908+0000","last_change":"2024-05-09T08:08:09.900908+0000","last_active":"2024-05-09T08:08:09.900908+0000","last_peered":"2024-05-09T08:08:09.900908+0000","last_clean":"2024-05-09T08:08:09.900908+0000","last_became_active":"2024-05-09T08:02:14.885830+0000","last_became_peered":"2024-05-09T08:02:14.885830+0000","last_unstale":"2024-05-09T08:08:09.900908+0000","last_undegraded":"2024-05-09T08:08:09.900908+0000","last_fullsized":"2024-05-09T08:08:09.900908+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"24'1","last_scrub_stamp":"2024-05-09T08:08:09.900569+0000","last_deep_scrub":"24'1","last_deep_scrub_stamp":"2024-05-09T08:08:09.900569+0000","last_clean_scrub_stamp":"2024-05-09T08:08:09.900569+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:28:51.428287+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"27'6","reported_seq":40,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:08:07.284173+0000","last_change":"2024-05-09T08:08:07.284286+0000","last_active":"2024-05-09T08:08:07.284173+0000","last_peered":"2024-05-09T08:08:07.284173+0000","last_clean":"2024-05-09T08:08:07.284173+0000","last_became_active":"2024-05-09T08:02:15.905590+0000","last_became_peered":"2024-05-09T08:02:15.905590+0000","last_unstale":"2024-05-09T08:08:07.284173+0000","last_undegraded":"2024-05-09T08:08:07.284173+0000","last_fullsized":"2024-05-09T08:08:07.284173+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'6","last_scrub_stamp":"2024-05-09T08:08:07.283893+0000","last_deep_scrub":"27'6","last_deep_scrub_stamp":"2024-05-09T08:08:07.283893+0000","last_clean_scrub_stamp":"2024-05-09T08:08:07.283893+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:55:47.536284+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"27'377","reported_seq":406,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:08:12.693470+0000","last_change":"2024-05-09T08:08:12.693501+0000","last_active":"2024-05-09T08:08:12.693470+0000","last_peered":"2024-05-09T08:08:12.693470+0000","last_clean":"2024-05-09T08:08:12.693470+0000","last_became_active":"2024-05-09T08:02:19.954161+0000","last_became_peered":"2024-05-09T08:02:19.954161+0000","last_unstale":"2024-05-09T08:08:12.693470+0000","last_undegraded":"2024-05-09T08:08:12.693470+0000","last_fullsized":"2024-05-09T08:08:12.693470+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'377","last_scrub_stamp":"2024-05-09T08:08:12.693360+0000","last_deep_scrub":"27'377","last_deep_scrub_stamp":"2024-05-09T08:08:12.693360+0000","last_clean_scrub_stamp":"2024-05-09T08:08:12.693360+0000","objects_scrubbed":0,"log_size":377,"log_dups_size":0,"ondisk_log_size":377,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:13:58.753057+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":339,"num_write_kb":925696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"27'4","reported_seq":40,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:08:08.467672+0000","last_change":"2024-05-09T08:08:08.467701+0000","last_active":"2024-05-09T08:08:08.467672+0000","last_peered":"2024-05-09T08:08:08.467672+0000","last_clean":"2024-05-09T08:08:08.467672+0000","last_became_active":"2024-05-09T08:02:14.866809+0000","last_became_peered":"2024-05-09T08:02:14.866809+0000","last_unstale":"2024-05-09T08:08:08.467672+0000","last_undegraded":"2024-05-09T08:08:08.467672+0000","last_fullsized":"2024-05-09T08:08:08.467672+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4","last_scrub_stamp":"2024-05-09T08:08:08.467590+0000","last_deep_scrub":"27'4","last_deep_scrub_stamp":"2024-05-09T08:08:08.467590+0000","last_clean_scrub_stamp":"2024-05-09T08:08:08.467590+0000","objects_scrubbed":1,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:09:39.327167+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"27'9","reported_seq":43,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:08:13.673547+0000","last_change":"2024-05-09T08:08:13.673593+0000","last_active":"2024-05-09T08:08:13.673547+0000","last_peered":"2024-05-09T08:08:13.673547+0000","last_clean":"2024-05-09T08:08:13.673547+0000","last_became_active":"2024-05-09T08:02:15.887509+0000","last_became_peered":"2024-05-09T08:02:15.887509+0000","last_unstale":"2024-05-09T08:08:13.673547+0000","last_undegraded":"2024-05-09T08:08:13.673547+0000","last_fullsized":"2024-05-09T08:08:13.673547+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'9","last_scrub_stamp":"2024-05-09T08:08:13.673442+0000","last_deep_scrub":"27'9","last_deep_scrub_stamp":"2024-05-09T08:08:13.673442+0000","last_clean_scrub_stamp":"2024-05-09T08:08:13.673442+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:43:24.193848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"27'324","reported_seq":353,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:08:10.722904+0000","last_change":"2024-05-09T08:08:10.722937+0000","last_active":"2024-05-09T08:08:10.722904+0000","last_peered":"2024-05-09T08:08:10.722904+0000","last_clean":"2024-05-09T08:08:10.722904+0000","last_became_active":"2024-05-09T08:02:19.952275+0000","last_became_peered":"2024-05-09T08:02:19.952275+0000","last_unstale":"2024-05-09T08:08:10.722904+0000","last_undegraded":"2024-05-09T08:08:10.722904+0000","last_fullsized":"2024-05-09T08:08:10.722904+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'324","last_scrub_stamp":"2024-05-09T08:08:10.722785+0000","last_deep_scrub":"27'324","last_deep_scrub_stamp":"2024-05-09T08:08:10.722785+0000","last_clean_scrub_stamp":"2024-05-09T08:08:10.722785+0000","objects_scrubbed":0,"log_size":324,"log_dups_size":0,"ondisk_log_size":324,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:15:42.451000+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":273,"num_write_kb":745472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"27'3","reported_seq":39,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:08:09.419841+0000","last_change":"2024-05-09T08:08:09.419875+0000","last_active":"2024-05-09T08:08:09.419841+0000","last_peered":"2024-05-09T08:08:09.419841+0000","last_clean":"2024-05-09T08:08:09.419841+0000","last_became_active":"2024-05-09T08:02:14.870476+0000","last_became_peered":"2024-05-09T08:02:14.870476+0000","last_unstale":"2024-05-09T08:08:09.419841+0000","last_undegraded":"2024-05-09T08:08:09.419841+0000","last_fullsized":"2024-05-09T08:08:09.419841+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'3","last_scrub_stamp":"2024-05-09T08:08:09.419748+0000","last_deep_scrub":"27'3","last_deep_scrub_stamp":"2024-05-09T08:08:09.419748+0000","last_clean_scrub_stamp":"2024-05-09T08:08:09.419748+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:15:41.147962+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"27'15","reported_seq":49,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:08:11.624654+0000","last_change":"2024-05-09T08:08:11.624680+0000","last_active":"2024-05-09T08:08:11.624654+0000","last_peered":"2024-05-09T08:08:11.624654+0000","last_clean":"2024-05-09T08:08:11.624654+0000","last_became_active":"2024-05-09T08:02:15.886168+0000","last_became_peered":"2024-05-09T08:02:15.886168+0000","last_unstale":"2024-05-09T08:08:11.624654+0000","last_undegraded":"2024-05-09T08:08:11.624654+0000","last_fullsized":"2024-05-09T08:08:11.624654+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'15","last_scrub_stamp":"2024-05-09T08:08:11.624554+0000","last_deep_scrub":"27'15","last_deep_scrub_stamp":"2024-05-09T08:08:11.624554+0000","last_clean_scrub_stamp":"2024-05-09T08:08:11.624554+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:01:17.040420+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"27'298","reported_seq":327,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:08:08.728659+0000","last_change":"2024-05-09T08:08:08.728685+0000","last_active":"2024-05-09T08:08:08.728659+0000","last_peered":"2024-05-09T08:08:08.728659+0000","last_clean":"2024-05-09T08:08:08.728659+0000","last_became_active":"2024-05-09T08:02:19.946906+0000","last_became_peered":"2024-05-09T08:02:19.946906+0000","last_unstale":"2024-05-09T08:08:08.728659+0000","last_undegraded":"2024-05-09T08:08:08.728659+0000","last_fullsized":"2024-05-09T08:08:08.728659+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'298","last_scrub_stamp":"2024-05-09T08:08:08.728563+0000","last_deep_scrub":"27'298","last_deep_scrub_stamp":"2024-05-09T08:08:08.728563+0000","last_clean_scrub_stamp":"2024-05-09T08:08:08.728563+0000","objects_scrubbed":0,"log_size":298,"log_dups_size":0,"ondisk_log_size":298,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:50:17.339475+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":252,"num_write_kb":688128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"27'5","reported_seq":40,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:08:09.796391+0000","last_change":"2024-05-09T08:08:09.796416+0000","last_active":"2024-05-09T08:08:09.796391+0000","last_peered":"2024-05-09T08:08:09.796391+0000","last_clean":"2024-05-09T08:08:09.796391+0000","last_became_active":"2024-05-09T08:02:15.880439+0000","last_became_peered":"2024-05-09T08:02:15.880439+0000","last_unstale":"2024-05-09T08:08:09.796391+0000","last_undegraded":"2024-05-09T08:08:09.796391+0000","last_fullsized":"2024-05-09T08:08:09.796391+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'5","last_scrub_stamp":"2024-05-09T08:08:09.796309+0000","last_deep_scrub":"27'5","last_deep_scrub_stamp":"2024-05-09T08:08:09.796309+0000","last_clean_scrub_stamp":"2024-05-09T08:08:09.796309+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:52:46.662507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"27'4","reported_seq":40,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:08:07.444952+0000","last_change":"2024-05-09T08:08:07.444981+0000","last_active":"2024-05-09T08:08:07.444952+0000","last_peered":"2024-05-09T08:08:07.444952+0000","last_clean":"2024-05-09T08:08:07.444952+0000","last_became_active":"2024-05-09T08:02:14.867291+0000","last_became_peered":"2024-05-09T08:02:14.867291+0000","last_unstale":"2024-05-09T08:08:07.444952+0000","last_undegraded":"2024-05-09T08:08:07.444952+0000","last_fullsized":"2024-05-09T08:08:07.444952+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4","last_scrub_stamp":"2024-05-09T08:08:07.444871+0000","last_deep_scrub":"27'4","last_deep_scrub_stamp":"2024-05-09T08:08:07.444871+0000","last_clean_scrub_stamp":"2024-05-09T08:08:07.444871+0000","objects_scrubbed":1,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:50:16.055783+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"27'366","reported_seq":395,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:08:13.686598+0000","last_change":"2024-05-09T08:08:13.686625+0000","last_active":"2024-05-09T08:08:13.686598+0000","last_peered":"2024-05-09T08:08:13.686598+0000","last_clean":"2024-05-09T08:08:13.686598+0000","last_became_active":"2024-05-09T08:02:19.954007+0000","last_became_peered":"2024-05-09T08:02:19.954007+0000","last_unstale":"2024-05-09T08:08:13.686598+0000","last_undegraded":"2024-05-09T08:08:13.686598+0000","last_fullsized":"2024-05-09T08:08:13.686598+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'366","last_scrub_stamp":"2024-05-09T08:08:13.686503+0000","last_deep_scrub":"27'366","last_deep_scrub_stamp":"2024-05-09T08:08:13.686503+0000","last_clean_scrub_stamp":"2024-05-09T08:08:13.686503+0000","objects_scrubbed":0,"log_size":366,"log_dups_size":0,"ondisk_log_size":366,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:10:57.857999+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":309,"num_write_kb":843776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"27'8","reported_seq":42,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:08:15.823320+0000","last_change":"2024-05-09T08:08:15.823384+0000","last_active":"2024-05-09T08:08:15.823320+0000","last_peered":"2024-05-09T08:08:15.823320+0000","last_clean":"2024-05-09T08:08:15.823320+0000","last_became_active":"2024-05-09T08:02:15.914852+0000","last_became_peered":"2024-05-09T08:02:15.914852+0000","last_unstale":"2024-05-09T08:08:15.823320+0000","last_undegraded":"2024-05-09T08:08:15.823320+0000","last_fullsized":"2024-05-09T08:08:15.823320+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'8","last_scrub_stamp":"2024-05-09T08:08:15.823148+0000","last_deep_scrub":"27'8","last_deep_scrub_stamp":"2024-05-09T08:08:15.823148+0000","last_clean_scrub_stamp":"2024-05-09T08:08:15.823148+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:07:24.190912+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":36,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:08:05.894811+0000","last_change":"2024-05-09T08:08:05.894867+0000","last_active":"2024-05-09T08:08:05.894811+0000","last_peered":"2024-05-09T08:08:05.894811+0000","last_clean":"2024-05-09T08:08:05.894811+0000","last_became_active":"2024-05-09T08:02:14.881437+0000","last_became_peered":"2024-05-09T08:02:14.881437+0000","last_unstale":"2024-05-09T08:08:05.894811+0000","last_undegraded":"2024-05-09T08:08:05.894811+0000","last_fullsized":"2024-05-09T08:08:05.894811+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:08:05.894645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:08:05.894645+0000","last_clean_scrub_stamp":"2024-05-09T08:08:05.894645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:50:14.505557+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"27'330","reported_seq":359,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:08:14.715529+0000","last_change":"2024-05-09T08:08:14.715559+0000","last_active":"2024-05-09T08:08:14.715529+0000","last_peered":"2024-05-09T08:08:14.715529+0000","last_clean":"2024-05-09T08:08:14.715529+0000","last_became_active":"2024-05-09T08:02:19.955432+0000","last_became_peered":"2024-05-09T08:02:19.955432+0000","last_unstale":"2024-05-09T08:08:14.715529+0000","last_undegraded":"2024-05-09T08:08:14.715529+0000","last_fullsized":"2024-05-09T08:08:14.715529+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'330","last_scrub_stamp":"2024-05-09T08:08:14.715416+0000","last_deep_scrub":"27'330","last_deep_scrub_stamp":"2024-05-09T08:08:14.715416+0000","last_clean_scrub_stamp":"2024-05-09T08:08:14.715416+0000","objects_scrubbed":0,"log_size":330,"log_dups_size":0,"ondisk_log_size":330,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:08:33.079853+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":291,"num_write_kb":794624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"27'9","reported_seq":43,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:08:05.982144+0000","last_change":"2024-05-09T08:08:05.982203+0000","last_active":"2024-05-09T08:08:05.982144+0000","last_peered":"2024-05-09T08:08:05.982144+0000","last_clean":"2024-05-09T08:08:05.982144+0000","last_became_active":"2024-05-09T08:02:15.912484+0000","last_became_peered":"2024-05-09T08:02:15.912484+0000","last_unstale":"2024-05-09T08:08:05.982144+0000","last_undegraded":"2024-05-09T08:08:05.982144+0000","last_fullsized":"2024-05-09T08:08:05.982144+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'9","last_scrub_stamp":"2024-05-09T08:08:05.981934+0000","last_deep_scrub":"27'9","last_deep_scrub_stamp":"2024-05-09T08:08:05.981934+0000","last_clean_scrub_stamp":"2024-05-09T08:08:05.981934+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:15:37.710149+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":36,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:08:00.817741+0000","last_change":"2024-05-09T08:08:00.817783+0000","last_active":"2024-05-09T08:08:00.817741+0000","last_peered":"2024-05-09T08:08:00.817741+0000","last_clean":"2024-05-09T08:08:00.817741+0000","last_became_active":"2024-05-09T08:02:14.868553+0000","last_became_peered":"2024-05-09T08:02:14.868553+0000","last_unstale":"2024-05-09T08:08:00.817741+0000","last_undegraded":"2024-05-09T08:08:00.817741+0000","last_fullsized":"2024-05-09T08:08:00.817741+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:08:00.817653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:08:00.817653+0000","last_clean_scrub_stamp":"2024-05-09T08:08:00.817653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:53:20.120654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"27'321","reported_seq":344,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:08:09.718208+0000","last_change":"2024-05-09T08:08:09.718276+0000","last_active":"2024-05-09T08:08:09.718208+0000","last_peered":"2024-05-09T08:08:09.718208+0000","last_clean":"2024-05-09T08:08:09.718208+0000","last_became_active":"2024-05-09T08:03:02.607564+0000","last_became_peered":"2024-05-09T08:03:02.607564+0000","last_unstale":"2024-05-09T08:08:09.718208+0000","last_undegraded":"2024-05-09T08:08:09.718208+0000","last_fullsized":"2024-05-09T08:08:09.718208+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'321","last_scrub_stamp":"2024-05-09T08:08:09.717973+0000","last_deep_scrub":"27'321","last_deep_scrub_stamp":"2024-05-09T08:08:09.717973+0000","last_clean_scrub_stamp":"2024-05-09T08:08:09.717973+0000","objects_scrubbed":0,"log_size":321,"log_dups_size":0,"ondisk_log_size":321,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:56:59.965416+0000","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":267,"num_write_kb":729088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4,7],"acting":[2,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"27'2","reported_seq":36,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:08:06.761264+0000","last_change":"2024-05-09T08:08:06.761289+0000","last_active":"2024-05-09T08:08:06.761264+0000","last_peered":"2024-05-09T08:08:06.761264+0000","last_clean":"2024-05-09T08:08:06.761264+0000","last_became_active":"2024-05-09T08:02:15.890806+0000","last_became_peered":"2024-05-09T08:02:15.890806+0000","last_unstale":"2024-05-09T08:08:06.761264+0000","last_undegraded":"2024-05-09T08:08:06.761264+0000","last_fullsized":"2024-05-09T08:08:06.761264+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2","last_scrub_stamp":"2024-05-09T08:08:06.761174+0000","last_deep_scrub":"27'2","last_deep_scrub_stamp":"2024-05-09T08:08:06.761174+0000","last_clean_scrub_stamp":"2024-05-09T08:08:06.761174+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:16:36.082602+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":36,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:08:10.444899+0000","last_change":"2024-05-09T08:08:10.444925+0000","last_active":"2024-05-09T08:08:10.444899+0000","last_peered":"2024-05-09T08:08:10.444899+0000","last_clean":"2024-05-09T08:08:10.444899+0000","last_became_active":"2024-05-09T08:02:14.874549+0000","last_became_peered":"2024-05-09T08:02:14.874549+0000","last_unstale":"2024-05-09T08:08:10.444899+0000","last_undegraded":"2024-05-09T08:08:10.444899+0000","last_fullsized":"2024-05-09T08:08:10.444899+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:08:10.444802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:08:10.444802+0000","last_clean_scrub_stamp":"2024-05-09T08:08:10.444802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:55:40.049274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"27'339","reported_seq":375,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:08:16.787007+0000","last_change":"2024-05-09T08:08:16.787037+0000","last_active":"2024-05-09T08:08:16.787007+0000","last_peered":"2024-05-09T08:08:16.787007+0000","last_clean":"2024-05-09T08:08:16.787007+0000","last_became_active":"2024-05-09T08:03:02.516161+0000","last_became_peered":"2024-05-09T08:03:02.516161+0000","last_unstale":"2024-05-09T08:08:16.787007+0000","last_undegraded":"2024-05-09T08:08:16.787007+0000","last_fullsized":"2024-05-09T08:08:16.787007+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'339","last_scrub_stamp":"2024-05-09T08:08:16.786895+0000","last_deep_scrub":"27'339","last_deep_scrub_stamp":"2024-05-09T08:08:16.786895+0000","last_clean_scrub_stamp":"2024-05-09T08:08:16.786895+0000","objects_scrubbed":0,"log_size":339,"log_dups_size":0,"ondisk_log_size":339,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:06:06.127926+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":288,"num_write_kb":786432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,2],"acting":[4,3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"27'7","reported_seq":41,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:08:09.268584+0000","last_change":"2024-05-09T08:08:09.268649+0000","last_active":"2024-05-09T08:08:09.268584+0000","last_peered":"2024-05-09T08:08:09.268584+0000","last_clean":"2024-05-09T08:08:09.268584+0000","last_became_active":"2024-05-09T08:02:15.912590+0000","last_became_peered":"2024-05-09T08:02:15.912590+0000","last_unstale":"2024-05-09T08:08:09.268584+0000","last_undegraded":"2024-05-09T08:08:09.268584+0000","last_fullsized":"2024-05-09T08:08:09.268584+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'7","last_scrub_stamp":"2024-05-09T08:08:09.268403+0000","last_deep_scrub":"27'7","last_deep_scrub_stamp":"2024-05-09T08:08:09.268403+0000","last_clean_scrub_stamp":"2024-05-09T08:08:09.268403+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:17:37.982877+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"27'4","reported_seq":41,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:08:07.875252+0000","last_change":"2024-05-09T08:08:07.875281+0000","last_active":"2024-05-09T08:08:07.875252+0000","last_peered":"2024-05-09T08:08:07.875252+0000","last_clean":"2024-05-09T08:08:07.875252+0000","last_became_active":"2024-05-09T08:02:14.864470+0000","last_became_peered":"2024-05-09T08:02:14.864470+0000","last_unstale":"2024-05-09T08:08:07.875252+0000","last_undegraded":"2024-05-09T08:08:07.875252+0000","last_fullsized":"2024-05-09T08:08:07.875252+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4","last_scrub_stamp":"2024-05-09T08:08:07.875166+0000","last_deep_scrub":"27'4","last_deep_scrub_stamp":"2024-05-09T08:08:07.875166+0000","last_clean_scrub_stamp":"2024-05-09T08:08:07.875166+0000","objects_scrubbed":1,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:57:03.748469+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"27'306","reported_seq":335,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:08:15.749403+0000","last_change":"2024-05-09T08:08:15.749433+0000","last_active":"2024-05-09T08:08:15.749403+0000","last_peered":"2024-05-09T08:08:15.749403+0000","last_clean":"2024-05-09T08:08:15.749403+0000","last_became_active":"2024-05-09T08:02:20.054424+0000","last_became_peered":"2024-05-09T08:02:20.054424+0000","last_unstale":"2024-05-09T08:08:15.749403+0000","last_undegraded":"2024-05-09T08:08:15.749403+0000","last_fullsized":"2024-05-09T08:08:15.749403+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'306","last_scrub_stamp":"2024-05-09T08:08:15.749285+0000","last_deep_scrub":"27'306","last_deep_scrub_stamp":"2024-05-09T08:08:15.749285+0000","last_clean_scrub_stamp":"2024-05-09T08:08:15.749285+0000","objects_scrubbed":0,"log_size":306,"log_dups_size":0,"ondisk_log_size":306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:01:21.165151+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":249,"num_write_kb":679936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"27'5","reported_seq":39,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:08:02.953492+0000","last_change":"2024-05-09T08:08:02.953612+0000","last_active":"2024-05-09T08:08:02.953492+0000","last_peered":"2024-05-09T08:08:02.953492+0000","last_clean":"2024-05-09T08:08:02.953492+0000","last_became_active":"2024-05-09T08:02:15.905131+0000","last_became_peered":"2024-05-09T08:02:15.905131+0000","last_unstale":"2024-05-09T08:08:02.953492+0000","last_undegraded":"2024-05-09T08:08:02.953492+0000","last_fullsized":"2024-05-09T08:08:02.953492+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'5","last_scrub_stamp":"2024-05-09T08:08:02.953285+0000","last_deep_scrub":"27'5","last_deep_scrub_stamp":"2024-05-09T08:08:02.953285+0000","last_clean_scrub_stamp":"2024-05-09T08:08:02.953285+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:52:39.819483+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"27'22","reported_seq":58,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:08:01.817574+0000","last_change":"2024-05-09T08:08:01.817598+0000","last_active":"2024-05-09T08:08:01.817574+0000","last_peered":"2024-05-09T08:08:01.817574+0000","last_clean":"2024-05-09T08:08:01.817574+0000","last_became_active":"2024-05-09T08:02:14.865621+0000","last_became_peered":"2024-05-09T08:02:14.865621+0000","last_unstale":"2024-05-09T08:08:01.817574+0000","last_undegraded":"2024-05-09T08:08:01.817574+0000","last_fullsized":"2024-05-09T08:08:01.817574+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'22","last_scrub_stamp":"2024-05-09T08:08:01.817506+0000","last_deep_scrub":"27'22","last_deep_scrub_stamp":"2024-05-09T08:08:01.817506+0000","last_clean_scrub_stamp":"2024-05-09T08:08:01.817506+0000","objects_scrubbed":1,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:11:55.649509+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":43,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"27'328","reported_seq":358,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:08:21.733793+0000","last_change":"2024-05-09T08:08:21.733793+0000","last_active":"2024-05-09T08:08:21.733793+0000","last_peered":"2024-05-09T08:08:21.733793+0000","last_clean":"2024-05-09T08:08:21.733793+0000","last_became_active":"2024-05-09T08:02:20.055365+0000","last_became_peered":"2024-05-09T08:02:20.055365+0000","last_unstale":"2024-05-09T08:08:21.733793+0000","last_undegraded":"2024-05-09T08:08:21.733793+0000","last_fullsized":"2024-05-09T08:08:21.733793+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'328","last_scrub_stamp":"2024-05-09T08:08:21.733466+0000","last_deep_scrub":"27'328","last_deep_scrub_stamp":"2024-05-09T08:08:21.733466+0000","last_clean_scrub_stamp":"2024-05-09T08:08:21.733466+0000","objects_scrubbed":0,"log_size":328,"log_dups_size":0,"ondisk_log_size":328,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:56:48.631066+0000","scrub_duration":21,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":264,"num_write_kb":720896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"27'9","reported_seq":43,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:08:07.805691+0000","last_change":"2024-05-09T08:08:07.805806+0000","last_active":"2024-05-09T08:08:07.805691+0000","last_peered":"2024-05-09T08:08:07.805691+0000","last_clean":"2024-05-09T08:08:07.805691+0000","last_became_active":"2024-05-09T08:02:15.890346+0000","last_became_peered":"2024-05-09T08:02:15.890346+0000","last_unstale":"2024-05-09T08:08:07.805691+0000","last_undegraded":"2024-05-09T08:08:07.805691+0000","last_fullsized":"2024-05-09T08:08:07.805691+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'9","last_scrub_stamp":"2024-05-09T08:08:07.805491+0000","last_deep_scrub":"27'9","last_deep_scrub_stamp":"2024-05-09T08:08:07.805491+0000","last_clean_scrub_stamp":"2024-05-09T08:08:07.805491+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:15:39.533705+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":36,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:08:11.486082+0000","last_change":"2024-05-09T08:08:11.486114+0000","last_active":"2024-05-09T08:08:11.486082+0000","last_peered":"2024-05-09T08:08:11.486082+0000","last_clean":"2024-05-09T08:08:11.486082+0000","last_became_active":"2024-05-09T08:02:14.875894+0000","last_became_peered":"2024-05-09T08:02:14.875894+0000","last_unstale":"2024-05-09T08:08:11.486082+0000","last_undegraded":"2024-05-09T08:08:11.486082+0000","last_fullsized":"2024-05-09T08:08:11.486082+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:08:11.485768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:08:11.485768+0000","last_clean_scrub_stamp":"2024-05-09T08:08:11.485768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:13:57.545465+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"27'297","reported_seq":326,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:08:16.248069+0000","last_change":"2024-05-09T08:08:16.248135+0000","last_active":"2024-05-09T08:08:16.248069+0000","last_peered":"2024-05-09T08:08:16.248069+0000","last_clean":"2024-05-09T08:08:16.248069+0000","last_became_active":"2024-05-09T08:02:20.053802+0000","last_became_peered":"2024-05-09T08:02:20.053802+0000","last_unstale":"2024-05-09T08:08:16.248069+0000","last_undegraded":"2024-05-09T08:08:16.248069+0000","last_fullsized":"2024-05-09T08:08:16.248069+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'297","last_scrub_stamp":"2024-05-09T08:08:16.247848+0000","last_deep_scrub":"27'297","last_deep_scrub_stamp":"2024-05-09T08:08:16.247848+0000","last_clean_scrub_stamp":"2024-05-09T08:08:16.247848+0000","objects_scrubbed":0,"log_size":297,"log_dups_size":0,"ondisk_log_size":297,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:20:43.055995+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":249,"num_write_kb":679936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"27'5","reported_seq":39,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:08:15.636550+0000","last_change":"2024-05-09T08:08:15.636577+0000","last_active":"2024-05-09T08:08:15.636550+0000","last_peered":"2024-05-09T08:08:15.636550+0000","last_clean":"2024-05-09T08:08:15.636550+0000","last_became_active":"2024-05-09T08:02:15.904049+0000","last_became_peered":"2024-05-09T08:02:15.904049+0000","last_unstale":"2024-05-09T08:08:15.636550+0000","last_undegraded":"2024-05-09T08:08:15.636550+0000","last_fullsized":"2024-05-09T08:08:15.636550+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'5","last_scrub_stamp":"2024-05-09T08:08:15.636468+0000","last_deep_scrub":"27'5","last_deep_scrub_stamp":"2024-05-09T08:08:15.636468+0000","last_clean_scrub_stamp":"2024-05-09T08:08:15.636468+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:20:42.444615+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":36,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:08:03.872125+0000","last_change":"2024-05-09T08:08:03.872157+0000","last_active":"2024-05-09T08:08:03.872125+0000","last_peered":"2024-05-09T08:08:03.872125+0000","last_clean":"2024-05-09T08:08:03.872125+0000","last_became_active":"2024-05-09T08:02:14.872096+0000","last_became_peered":"2024-05-09T08:02:14.872096+0000","last_unstale":"2024-05-09T08:08:03.872125+0000","last_undegraded":"2024-05-09T08:08:03.872125+0000","last_fullsized":"2024-05-09T08:08:03.872125+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:08:03.871997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:08:03.871997+0000","last_clean_scrub_stamp":"2024-05-09T08:08:03.871997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:13:39.508871+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"27'320","reported_seq":349,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:08:14.721721+0000","last_change":"2024-05-09T08:08:14.721841+0000","last_active":"2024-05-09T08:08:14.721721+0000","last_peered":"2024-05-09T08:08:14.721721+0000","last_clean":"2024-05-09T08:08:14.721721+0000","last_became_active":"2024-05-09T08:02:19.962922+0000","last_became_peered":"2024-05-09T08:02:19.962922+0000","last_unstale":"2024-05-09T08:08:14.721721+0000","last_undegraded":"2024-05-09T08:08:14.721721+0000","last_fullsized":"2024-05-09T08:08:14.721721+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'320","last_scrub_stamp":"2024-05-09T08:08:14.721424+0000","last_deep_scrub":"27'320","last_deep_scrub_stamp":"2024-05-09T08:08:14.721424+0000","last_clean_scrub_stamp":"2024-05-09T08:08:14.721424+0000","objects_scrubbed":0,"log_size":320,"log_dups_size":0,"ondisk_log_size":320,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:09:57.894458+0000","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":273,"num_write_kb":745472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"27'7","reported_seq":41,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:08:07.755834+0000","last_change":"2024-05-09T08:08:07.755859+0000","last_active":"2024-05-09T08:08:07.755834+0000","last_peered":"2024-05-09T08:08:07.755834+0000","last_clean":"2024-05-09T08:08:07.755834+0000","last_became_active":"2024-05-09T08:02:15.901180+0000","last_became_peered":"2024-05-09T08:02:15.901180+0000","last_unstale":"2024-05-09T08:08:07.755834+0000","last_undegraded":"2024-05-09T08:08:07.755834+0000","last_fullsized":"2024-05-09T08:08:07.755834+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'7","last_scrub_stamp":"2024-05-09T08:08:07.755741+0000","last_deep_scrub":"27'7","last_deep_scrub_stamp":"2024-05-09T08:08:07.755741+0000","last_clean_scrub_stamp":"2024-05-09T08:08:07.755741+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:52:44.621939+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":36,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:08:09.584725+0000","last_change":"2024-05-09T08:08:09.584776+0000","last_active":"2024-05-09T08:08:09.584725+0000","last_peered":"2024-05-09T08:08:09.584725+0000","last_clean":"2024-05-09T08:08:09.584725+0000","last_became_active":"2024-05-09T08:02:14.876175+0000","last_became_peered":"2024-05-09T08:02:14.876175+0000","last_unstale":"2024-05-09T08:08:09.584725+0000","last_undegraded":"2024-05-09T08:08:09.584725+0000","last_fullsized":"2024-05-09T08:08:09.584725+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:08:09.584607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:08:09.584607+0000","last_clean_scrub_stamp":"2024-05-09T08:08:09.584607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:10:53.756103+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"27'301","reported_seq":324,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:08:21.770579+0000","last_change":"2024-05-09T08:08:21.770606+0000","last_active":"2024-05-09T08:08:21.770579+0000","last_peered":"2024-05-09T08:08:21.770579+0000","last_clean":"2024-05-09T08:08:21.770579+0000","last_became_active":"2024-05-09T08:03:02.516104+0000","last_became_peered":"2024-05-09T08:03:02.516104+0000","last_unstale":"2024-05-09T08:08:21.770579+0000","last_undegraded":"2024-05-09T08:08:21.770579+0000","last_fullsized":"2024-05-09T08:08:21.770579+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'301","last_scrub_stamp":"2024-05-09T08:08:21.770480+0000","last_deep_scrub":"27'301","last_deep_scrub_stamp":"2024-05-09T08:08:21.770480+0000","last_clean_scrub_stamp":"2024-05-09T08:08:21.770480+0000","objects_scrubbed":0,"log_size":301,"log_dups_size":0,"ondisk_log_size":301,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:54:04.496604+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":264,"num_write_kb":720896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3,6],"acting":[5,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"27'8","reported_seq":43,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:08:08.779002+0000","last_change":"2024-05-09T08:08:08.779002+0000","last_active":"2024-05-09T08:08:08.779002+0000","last_peered":"2024-05-09T08:08:08.779002+0000","last_clean":"2024-05-09T08:08:08.779002+0000","last_became_active":"2024-05-09T08:02:15.891407+0000","last_became_peered":"2024-05-09T08:02:15.891407+0000","last_unstale":"2024-05-09T08:08:08.779002+0000","last_undegraded":"2024-05-09T08:08:08.779002+0000","last_fullsized":"2024-05-09T08:08:08.779002+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'8","last_scrub_stamp":"2024-05-09T08:08:08.778697+0000","last_deep_scrub":"27'8","last_deep_scrub_stamp":"2024-05-09T08:08:08.778697+0000","last_clean_scrub_stamp":"2024-05-09T08:08:08.778697+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:55:38.383169+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"27'7","reported_seq":48,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:08:03.642944+0000","last_change":"2024-05-09T08:08:03.643010+0000","last_active":"2024-05-09T08:08:03.642944+0000","last_peered":"2024-05-09T08:08:03.642944+0000","last_clean":"2024-05-09T08:08:03.642944+0000","last_became_active":"2024-05-09T08:02:14.882359+0000","last_became_peered":"2024-05-09T08:02:14.882359+0000","last_unstale":"2024-05-09T08:08:03.642944+0000","last_undegraded":"2024-05-09T08:08:03.642944+0000","last_fullsized":"2024-05-09T08:08:03.642944+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'7","last_scrub_stamp":"2024-05-09T08:08:03.642764+0000","last_deep_scrub":"27'7","last_deep_scrub_stamp":"2024-05-09T08:08:03.642764+0000","last_clean_scrub_stamp":"2024-05-09T08:08:03.642764+0000","objects_scrubbed":1,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:50:36.255395+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":529,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"27'319","reported_seq":348,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:08:12.739420+0000","last_change":"2024-05-09T08:08:12.739485+0000","last_active":"2024-05-09T08:08:12.739420+0000","last_peered":"2024-05-09T08:08:12.739420+0000","last_clean":"2024-05-09T08:08:12.739420+0000","last_became_active":"2024-05-09T08:02:19.952740+0000","last_became_peered":"2024-05-09T08:02:19.952740+0000","last_unstale":"2024-05-09T08:08:12.739420+0000","last_undegraded":"2024-05-09T08:08:12.739420+0000","last_fullsized":"2024-05-09T08:08:12.739420+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'319","last_scrub_stamp":"2024-05-09T08:08:12.739224+0000","last_deep_scrub":"27'319","last_deep_scrub_stamp":"2024-05-09T08:08:12.739224+0000","last_clean_scrub_stamp":"2024-05-09T08:08:12.739224+0000","objects_scrubbed":0,"log_size":319,"log_dups_size":0,"ondisk_log_size":319,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:05:48.129947+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":270,"num_write_kb":737280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"27'5","reported_seq":39,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:08:11.210741+0000","last_change":"2024-05-09T08:08:11.210801+0000","last_active":"2024-05-09T08:08:11.210741+0000","last_peered":"2024-05-09T08:08:11.210741+0000","last_clean":"2024-05-09T08:08:11.210741+0000","last_became_active":"2024-05-09T08:02:15.913313+0000","last_became_peered":"2024-05-09T08:02:15.913313+0000","last_unstale":"2024-05-09T08:08:11.210741+0000","last_undegraded":"2024-05-09T08:08:11.210741+0000","last_fullsized":"2024-05-09T08:08:11.210741+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'5","last_scrub_stamp":"2024-05-09T08:08:11.210567+0000","last_deep_scrub":"27'5","last_deep_scrub_stamp":"2024-05-09T08:08:11.210567+0000","last_clean_scrub_stamp":"2024-05-09T08:08:11.210567+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:28:52.738285+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:08:12.441844+0000","last_change":"2024-05-09T08:08:12.441844+0000","last_active":"2024-05-09T08:08:12.441844+0000","last_peered":"2024-05-09T08:08:12.441844+0000","last_clean":"2024-05-09T08:08:12.441844+0000","last_became_active":"2024-05-09T08:02:14.876146+0000","last_became_peered":"2024-05-09T08:02:14.876146+0000","last_unstale":"2024-05-09T08:08:12.441844+0000","last_undegraded":"2024-05-09T08:08:12.441844+0000","last_fullsized":"2024-05-09T08:08:12.441844+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:08:12.441710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:08:12.441710+0000","last_clean_scrub_stamp":"2024-05-09T08:08:12.441710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:26:51.027984+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"27'378","reported_seq":407,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:08:14.254716+0000","last_change":"2024-05-09T08:08:14.254782+0000","last_active":"2024-05-09T08:08:14.254716+0000","last_peered":"2024-05-09T08:08:14.254716+0000","last_clean":"2024-05-09T08:08:14.254716+0000","last_became_active":"2024-05-09T08:02:20.050593+0000","last_became_peered":"2024-05-09T08:02:20.050593+0000","last_unstale":"2024-05-09T08:08:14.254716+0000","last_undegraded":"2024-05-09T08:08:14.254716+0000","last_fullsized":"2024-05-09T08:08:14.254716+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'378","last_scrub_stamp":"2024-05-09T08:08:14.254508+0000","last_deep_scrub":"27'378","last_deep_scrub_stamp":"2024-05-09T08:08:14.254508+0000","last_clean_scrub_stamp":"2024-05-09T08:08:14.254508+0000","objects_scrubbed":0,"log_size":378,"log_dups_size":0,"ondisk_log_size":378,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:43:24.774915+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":339,"num_write_kb":925696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"27'8","reported_seq":42,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:08:12.916746+0000","last_change":"2024-05-09T08:08:12.916813+0000","last_active":"2024-05-09T08:08:12.916746+0000","last_peered":"2024-05-09T08:08:12.916746+0000","last_clean":"2024-05-09T08:08:12.916746+0000","last_became_active":"2024-05-09T08:02:15.898766+0000","last_became_peered":"2024-05-09T08:02:15.898766+0000","last_unstale":"2024-05-09T08:08:12.916746+0000","last_undegraded":"2024-05-09T08:08:12.916746+0000","last_fullsized":"2024-05-09T08:08:12.916746+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'8","last_scrub_stamp":"2024-05-09T08:08:12.916520+0000","last_deep_scrub":"27'8","last_deep_scrub_stamp":"2024-05-09T08:08:12.916520+0000","last_clean_scrub_stamp":"2024-05-09T08:08:12.916520+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:43:23.436926+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":36,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:08:07.537627+0000","last_change":"2024-05-09T08:08:07.537654+0000","last_active":"2024-05-09T08:08:07.537627+0000","last_peered":"2024-05-09T08:08:07.537627+0000","last_clean":"2024-05-09T08:08:07.537627+0000","last_became_active":"2024-05-09T08:02:14.868384+0000","last_became_peered":"2024-05-09T08:02:14.868384+0000","last_unstale":"2024-05-09T08:08:07.537627+0000","last_undegraded":"2024-05-09T08:08:07.537627+0000","last_fullsized":"2024-05-09T08:08:07.537627+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:08:07.537531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:08:07.537531+0000","last_clean_scrub_stamp":"2024-05-09T08:08:07.537531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:45:14.776633+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"27'119","reported_seq":195,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:08:06.463188+0000","last_change":"2024-05-09T08:08:06.463188+0000","last_active":"2024-05-09T08:08:06.463188+0000","last_peered":"2024-05-09T08:08:06.463188+0000","last_clean":"2024-05-09T08:08:06.463188+0000","last_became_active":"2024-05-09T08:02:11.558761+0000","last_became_peered":"2024-05-09T08:02:11.558761+0000","last_unstale":"2024-05-09T08:08:06.463188+0000","last_undegraded":"2024-05-09T08:08:06.463188+0000","last_fullsized":"2024-05-09T08:08:06.463188+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'119","last_scrub_stamp":"2024-05-09T08:08:06.463008+0000","last_deep_scrub":"27'119","last_deep_scrub_stamp":"2024-05-09T08:08:06.463008+0000","last_clean_scrub_stamp":"2024-05-09T08:08:06.463008+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:43:54.366039+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"27'315","reported_seq":345,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:08:17.763452+0000","last_change":"2024-05-09T08:08:17.763506+0000","last_active":"2024-05-09T08:08:17.763452+0000","last_peered":"2024-05-09T08:08:17.763452+0000","last_clean":"2024-05-09T08:08:17.763452+0000","last_became_active":"2024-05-09T08:02:20.054186+0000","last_became_peered":"2024-05-09T08:02:20.054186+0000","last_unstale":"2024-05-09T08:08:17.763452+0000","last_undegraded":"2024-05-09T08:08:17.763452+0000","last_fullsized":"2024-05-09T08:08:17.763452+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'315","last_scrub_stamp":"2024-05-09T08:08:17.763326+0000","last_deep_scrub":"27'315","last_deep_scrub_stamp":"2024-05-09T08:08:17.763326+0000","last_clean_scrub_stamp":"2024-05-09T08:08:17.763326+0000","objects_scrubbed":0,"log_size":315,"log_dups_size":0,"ondisk_log_size":315,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:05:53.154048+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":276,"num_write_kb":753664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"27'7","reported_seq":41,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:08:17.412327+0000","last_change":"2024-05-09T08:08:17.412354+0000","last_active":"2024-05-09T08:08:17.412327+0000","last_peered":"2024-05-09T08:08:17.412327+0000","last_clean":"2024-05-09T08:08:17.412327+0000","last_became_active":"2024-05-09T08:02:15.889381+0000","last_became_peered":"2024-05-09T08:02:15.889381+0000","last_unstale":"2024-05-09T08:08:17.412327+0000","last_undegraded":"2024-05-09T08:08:17.412327+0000","last_fullsized":"2024-05-09T08:08:17.412327+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'7","last_scrub_stamp":"2024-05-09T08:08:17.412242+0000","last_deep_scrub":"27'7","last_deep_scrub_stamp":"2024-05-09T08:08:17.412242+0000","last_clean_scrub_stamp":"2024-05-09T08:08:17.412242+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:58:56.458579+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":36,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:08:01.600999+0000","last_change":"2024-05-09T08:08:01.601071+0000","last_active":"2024-05-09T08:08:01.600999+0000","last_peered":"2024-05-09T08:08:01.600999+0000","last_clean":"2024-05-09T08:08:01.600999+0000","last_became_active":"2024-05-09T08:02:14.868513+0000","last_became_peered":"2024-05-09T08:02:14.868513+0000","last_unstale":"2024-05-09T08:08:01.600999+0000","last_undegraded":"2024-05-09T08:08:01.600999+0000","last_fullsized":"2024-05-09T08:08:01.600999+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:08:01.600849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:08:01.600849+0000","last_clean_scrub_stamp":"2024-05-09T08:08:01.600849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:14:22.308730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"27'338","reported_seq":367,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:08:16.798690+0000","last_change":"2024-05-09T08:08:16.798720+0000","last_active":"2024-05-09T08:08:16.798690+0000","last_peered":"2024-05-09T08:08:16.798690+0000","last_clean":"2024-05-09T08:08:16.798690+0000","last_became_active":"2024-05-09T08:02:20.052653+0000","last_became_peered":"2024-05-09T08:02:20.052653+0000","last_unstale":"2024-05-09T08:08:16.798690+0000","last_undegraded":"2024-05-09T08:08:16.798690+0000","last_fullsized":"2024-05-09T08:08:16.798690+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'338","last_scrub_stamp":"2024-05-09T08:08:16.798562+0000","last_deep_scrub":"27'338","last_deep_scrub_stamp":"2024-05-09T08:08:16.798562+0000","last_clean_scrub_stamp":"2024-05-09T08:08:16.798562+0000","objects_scrubbed":0,"log_size":338,"log_dups_size":0,"ondisk_log_size":338,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:08:35.162999+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":279,"num_write_kb":761856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"27'6","reported_seq":40,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:08:14.627269+0000","last_change":"2024-05-09T08:08:14.627294+0000","last_active":"2024-05-09T08:08:14.627269+0000","last_peered":"2024-05-09T08:08:14.627269+0000","last_clean":"2024-05-09T08:08:14.627269+0000","last_became_active":"2024-05-09T08:02:15.898692+0000","last_became_peered":"2024-05-09T08:02:15.898692+0000","last_unstale":"2024-05-09T08:08:14.627269+0000","last_undegraded":"2024-05-09T08:08:14.627269+0000","last_fullsized":"2024-05-09T08:08:14.627269+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'6","last_scrub_stamp":"2024-05-09T08:08:14.627175+0000","last_deep_scrub":"27'6","last_deep_scrub_stamp":"2024-05-09T08:08:14.627175+0000","last_clean_scrub_stamp":"2024-05-09T08:08:14.627175+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:38:24.574847+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"27'2","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:08:04.857872+0000","last_change":"2024-05-09T08:08:04.857917+0000","last_active":"2024-05-09T08:08:04.857872+0000","last_peered":"2024-05-09T08:08:04.857872+0000","last_clean":"2024-05-09T08:08:04.857872+0000","last_became_active":"2024-05-09T08:02:14.872556+0000","last_became_peered":"2024-05-09T08:02:14.872556+0000","last_unstale":"2024-05-09T08:08:04.857872+0000","last_undegraded":"2024-05-09T08:08:04.857872+0000","last_fullsized":"2024-05-09T08:08:04.857872+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2","last_scrub_stamp":"2024-05-09T08:08:04.857767+0000","last_deep_scrub":"27'2","last_deep_scrub_stamp":"2024-05-09T08:08:04.857767+0000","last_clean_scrub_stamp":"2024-05-09T08:08:04.857767+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:43:52.760798+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"27'299","reported_seq":328,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:08:13.264650+0000","last_change":"2024-05-09T08:08:13.264714+0000","last_active":"2024-05-09T08:08:13.264650+0000","last_peered":"2024-05-09T08:08:13.264650+0000","last_clean":"2024-05-09T08:08:13.264650+0000","last_became_active":"2024-05-09T08:02:19.957941+0000","last_became_peered":"2024-05-09T08:02:19.957941+0000","last_unstale":"2024-05-09T08:08:13.264650+0000","last_undegraded":"2024-05-09T08:08:13.264650+0000","last_fullsized":"2024-05-09T08:08:13.264650+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'299","last_scrub_stamp":"2024-05-09T08:08:13.264459+0000","last_deep_scrub":"27'299","last_deep_scrub_stamp":"2024-05-09T08:08:13.264459+0000","last_clean_scrub_stamp":"2024-05-09T08:08:13.264459+0000","objects_scrubbed":0,"log_size":299,"log_dups_size":0,"ondisk_log_size":299,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:06:02.605490+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":261,"num_write_kb":712704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"27'9","reported_seq":43,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:08:14.444979+0000","last_change":"2024-05-09T08:08:14.445042+0000","last_active":"2024-05-09T08:08:14.444979+0000","last_peered":"2024-05-09T08:08:14.444979+0000","last_clean":"2024-05-09T08:08:14.444979+0000","last_became_active":"2024-05-09T08:02:15.883788+0000","last_became_peered":"2024-05-09T08:02:15.883788+0000","last_unstale":"2024-05-09T08:08:14.444979+0000","last_undegraded":"2024-05-09T08:08:14.444979+0000","last_fullsized":"2024-05-09T08:08:14.444979+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'9","last_scrub_stamp":"2024-05-09T08:08:14.444862+0000","last_deep_scrub":"27'9","last_deep_scrub_stamp":"2024-05-09T08:08:14.444862+0000","last_clean_scrub_stamp":"2024-05-09T08:08:14.444862+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:05:49.835584+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"27'3","reported_seq":39,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:08:04.635794+0000","last_change":"2024-05-09T08:08:04.635878+0000","last_active":"2024-05-09T08:08:04.635794+0000","last_peered":"2024-05-09T08:08:04.635794+0000","last_clean":"2024-05-09T08:08:04.635794+0000","last_became_active":"2024-05-09T08:02:14.884706+0000","last_became_peered":"2024-05-09T08:02:14.884706+0000","last_unstale":"2024-05-09T08:08:04.635794+0000","last_undegraded":"2024-05-09T08:08:04.635794+0000","last_fullsized":"2024-05-09T08:08:04.635794+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'3","last_scrub_stamp":"2024-05-09T08:08:04.635624+0000","last_deep_scrub":"27'3","last_deep_scrub_stamp":"2024-05-09T08:08:04.635624+0000","last_clean_scrub_stamp":"2024-05-09T08:08:04.635624+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:43:41.015149+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"27'365","reported_seq":394,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:08:19.710231+0000","last_change":"2024-05-09T08:08:19.710281+0000","last_active":"2024-05-09T08:08:19.710231+0000","last_peered":"2024-05-09T08:08:19.710231+0000","last_clean":"2024-05-09T08:08:19.710231+0000","last_became_active":"2024-05-09T08:02:20.050270+0000","last_became_peered":"2024-05-09T08:02:20.050270+0000","last_unstale":"2024-05-09T08:08:19.710231+0000","last_undegraded":"2024-05-09T08:08:19.710231+0000","last_fullsized":"2024-05-09T08:08:19.710231+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'365","last_scrub_stamp":"2024-05-09T08:08:19.710106+0000","last_deep_scrub":"27'365","last_deep_scrub_stamp":"2024-05-09T08:08:19.710106+0000","last_clean_scrub_stamp":"2024-05-09T08:08:19.710106+0000","objects_scrubbed":0,"log_size":365,"log_dups_size":0,"ondisk_log_size":365,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:51:02.399482+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":318,"num_write_kb":868352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"27'5","reported_seq":39,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:08:15.443667+0000","last_change":"2024-05-09T08:08:15.443692+0000","last_active":"2024-05-09T08:08:15.443667+0000","last_peered":"2024-05-09T08:08:15.443667+0000","last_clean":"2024-05-09T08:08:15.443667+0000","last_became_active":"2024-05-09T08:02:15.884465+0000","last_became_peered":"2024-05-09T08:02:15.884465+0000","last_unstale":"2024-05-09T08:08:15.443667+0000","last_undegraded":"2024-05-09T08:08:15.443667+0000","last_fullsized":"2024-05-09T08:08:15.443667+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'5","last_scrub_stamp":"2024-05-09T08:08:15.443569+0000","last_deep_scrub":"27'5","last_deep_scrub_stamp":"2024-05-09T08:08:15.443569+0000","last_clean_scrub_stamp":"2024-05-09T08:08:15.443569+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:46:40.162416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"27'2","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:08:02.773744+0000","last_change":"2024-05-09T08:08:02.773775+0000","last_active":"2024-05-09T08:08:02.773744+0000","last_peered":"2024-05-09T08:08:02.773744+0000","last_clean":"2024-05-09T08:08:02.773744+0000","last_became_active":"2024-05-09T08:02:14.874665+0000","last_became_peered":"2024-05-09T08:02:14.874665+0000","last_unstale":"2024-05-09T08:08:02.773744+0000","last_undegraded":"2024-05-09T08:08:02.773744+0000","last_fullsized":"2024-05-09T08:08:02.773744+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2","last_scrub_stamp":"2024-05-09T08:08:02.773645+0000","last_deep_scrub":"27'2","last_deep_scrub_stamp":"2024-05-09T08:08:02.773645+0000","last_clean_scrub_stamp":"2024-05-09T08:08:02.773645+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:05:19.799951+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"27'302","reported_seq":331,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:08:09.715210+0000","last_change":"2024-05-09T08:08:09.715239+0000","last_active":"2024-05-09T08:08:09.715210+0000","last_peered":"2024-05-09T08:08:09.715210+0000","last_clean":"2024-05-09T08:08:09.715210+0000","last_became_active":"2024-05-09T08:02:19.952224+0000","last_became_peered":"2024-05-09T08:02:19.952224+0000","last_unstale":"2024-05-09T08:08:09.715210+0000","last_undegraded":"2024-05-09T08:08:09.715210+0000","last_fullsized":"2024-05-09T08:08:09.715210+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'302","last_scrub_stamp":"2024-05-09T08:08:09.715109+0000","last_deep_scrub":"27'302","last_deep_scrub_stamp":"2024-05-09T08:08:09.715109+0000","last_clean_scrub_stamp":"2024-05-09T08:08:09.715109+0000","objects_scrubbed":0,"log_size":302,"log_dups_size":0,"ondisk_log_size":302,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:07:50.106575+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":258,"num_write_kb":704512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"27'6","reported_seq":40,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:08:13.487101+0000","last_change":"2024-05-09T08:08:13.487159+0000","last_active":"2024-05-09T08:08:13.487101+0000","last_peered":"2024-05-09T08:08:13.487101+0000","last_clean":"2024-05-09T08:08:13.487101+0000","last_became_active":"2024-05-09T08:02:15.883103+0000","last_became_peered":"2024-05-09T08:02:15.883103+0000","last_unstale":"2024-05-09T08:08:13.487101+0000","last_undegraded":"2024-05-09T08:08:13.487101+0000","last_fullsized":"2024-05-09T08:08:13.487101+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'6","last_scrub_stamp":"2024-05-09T08:08:13.487004+0000","last_deep_scrub":"27'6","last_deep_scrub_stamp":"2024-05-09T08:08:13.487004+0000","last_clean_scrub_stamp":"2024-05-09T08:08:13.487004+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:08:31.851440+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"27'13","reported_seq":49,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:08:02.642638+0000","last_change":"2024-05-09T08:08:02.642716+0000","last_active":"2024-05-09T08:08:02.642638+0000","last_peered":"2024-05-09T08:08:02.642638+0000","last_clean":"2024-05-09T08:08:02.642638+0000","last_became_active":"2024-05-09T08:02:14.871055+0000","last_became_peered":"2024-05-09T08:02:14.871055+0000","last_unstale":"2024-05-09T08:08:02.642638+0000","last_undegraded":"2024-05-09T08:08:02.642638+0000","last_fullsized":"2024-05-09T08:08:02.642638+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'13","last_scrub_stamp":"2024-05-09T08:08:02.642463+0000","last_deep_scrub":"27'13","last_deep_scrub_stamp":"2024-05-09T08:08:02.642463+0000","last_clean_scrub_stamp":"2024-05-09T08:08:02.642463+0000","objects_scrubbed":4,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:37:49.509704+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"27'416","reported_seq":445,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:08:12.264354+0000","last_change":"2024-05-09T08:08:12.264481+0000","last_active":"2024-05-09T08:08:12.264354+0000","last_peered":"2024-05-09T08:08:12.264354+0000","last_clean":"2024-05-09T08:08:12.264354+0000","last_became_active":"2024-05-09T08:02:19.950352+0000","last_became_peered":"2024-05-09T08:02:19.950352+0000","last_unstale":"2024-05-09T08:08:12.264354+0000","last_undegraded":"2024-05-09T08:08:12.264354+0000","last_fullsized":"2024-05-09T08:08:12.264354+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'416","last_scrub_stamp":"2024-05-09T08:08:12.264122+0000","last_deep_scrub":"27'416","last_deep_scrub_stamp":"2024-05-09T08:08:12.264122+0000","last_clean_scrub_stamp":"2024-05-09T08:08:12.264122+0000","objects_scrubbed":0,"log_size":416,"log_dups_size":0,"ondisk_log_size":416,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:01:17.679987+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":360,"num_write_kb":983040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"27'6","reported_seq":41,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:08:13.838202+0000","last_change":"2024-05-09T08:08:13.838257+0000","last_active":"2024-05-09T08:08:13.838202+0000","last_peered":"2024-05-09T08:08:13.838202+0000","last_clean":"2024-05-09T08:08:13.838202+0000","last_became_active":"2024-05-09T08:02:15.898811+0000","last_became_peered":"2024-05-09T08:02:15.898811+0000","last_unstale":"2024-05-09T08:08:13.838202+0000","last_undegraded":"2024-05-09T08:08:13.838202+0000","last_fullsized":"2024-05-09T08:08:13.838202+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'6","last_scrub_stamp":"2024-05-09T08:08:13.838082+0000","last_deep_scrub":"27'6","last_deep_scrub_stamp":"2024-05-09T08:08:13.838082+0000","last_clean_scrub_stamp":"2024-05-09T08:08:13.838082+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:55:43.442554+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:08:05.895125+0000","last_change":"2024-05-09T08:08:05.895151+0000","last_active":"2024-05-09T08:08:05.895125+0000","last_peered":"2024-05-09T08:08:05.895125+0000","last_clean":"2024-05-09T08:08:05.895125+0000","last_became_active":"2024-05-09T08:02:14.860518+0000","last_became_peered":"2024-05-09T08:02:14.860518+0000","last_unstale":"2024-05-09T08:08:05.895125+0000","last_undegraded":"2024-05-09T08:08:05.895125+0000","last_fullsized":"2024-05-09T08:08:05.895125+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:08:05.895042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:08:05.895042+0000","last_clean_scrub_stamp":"2024-05-09T08:08:05.895042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:53:25.198043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"27'318","reported_seq":347,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:08:19.820179+0000","last_change":"2024-05-09T08:08:19.820252+0000","last_active":"2024-05-09T08:08:19.820179+0000","last_peered":"2024-05-09T08:08:19.820179+0000","last_clean":"2024-05-09T08:08:19.820179+0000","last_became_active":"2024-05-09T08:02:20.052783+0000","last_became_peered":"2024-05-09T08:02:20.052783+0000","last_unstale":"2024-05-09T08:08:19.820179+0000","last_undegraded":"2024-05-09T08:08:19.820179+0000","last_fullsized":"2024-05-09T08:08:19.820179+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'318","last_scrub_stamp":"2024-05-09T08:08:19.819949+0000","last_deep_scrub":"27'318","last_deep_scrub_stamp":"2024-05-09T08:08:19.819949+0000","last_clean_scrub_stamp":"2024-05-09T08:08:19.819949+0000","objects_scrubbed":0,"log_size":318,"log_dups_size":0,"ondisk_log_size":318,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:17:21.240678+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":273,"num_write_kb":745472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"27'15","reported_seq":49,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:08:05.497953+0000","last_change":"2024-05-09T08:08:05.497985+0000","last_active":"2024-05-09T08:08:05.497953+0000","last_peered":"2024-05-09T08:08:05.497953+0000","last_clean":"2024-05-09T08:08:05.497953+0000","last_became_active":"2024-05-09T08:02:15.883613+0000","last_became_peered":"2024-05-09T08:02:15.883613+0000","last_unstale":"2024-05-09T08:08:05.497953+0000","last_undegraded":"2024-05-09T08:08:05.497953+0000","last_fullsized":"2024-05-09T08:08:05.497953+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'15","last_scrub_stamp":"2024-05-09T08:08:05.497806+0000","last_deep_scrub":"27'15","last_deep_scrub_stamp":"2024-05-09T08:08:05.497806+0000","last_clean_scrub_stamp":"2024-05-09T08:08:05.497806+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:13:41.134680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":36,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:08:10.605262+0000","last_change":"2024-05-09T08:08:10.605289+0000","last_active":"2024-05-09T08:08:10.605262+0000","last_peered":"2024-05-09T08:08:10.605262+0000","last_clean":"2024-05-09T08:08:10.605262+0000","last_became_active":"2024-05-09T08:02:14.883135+0000","last_became_peered":"2024-05-09T08:02:14.883135+0000","last_unstale":"2024-05-09T08:08:10.605262+0000","last_undegraded":"2024-05-09T08:08:10.605262+0000","last_fullsized":"2024-05-09T08:08:10.605262+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:08:10.605171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:08:10.605171+0000","last_clean_scrub_stamp":"2024-05-09T08:08:10.605171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:08:28.969607+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"27'316","reported_seq":345,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:08:18.864064+0000","last_change":"2024-05-09T08:08:18.864191+0000","last_active":"2024-05-09T08:08:18.864064+0000","last_peered":"2024-05-09T08:08:18.864064+0000","last_clean":"2024-05-09T08:08:18.864064+0000","last_became_active":"2024-05-09T08:02:20.050390+0000","last_became_peered":"2024-05-09T08:02:20.050390+0000","last_unstale":"2024-05-09T08:08:18.864064+0000","last_undegraded":"2024-05-09T08:08:18.864064+0000","last_fullsized":"2024-05-09T08:08:18.864064+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'316","last_scrub_stamp":"2024-05-09T08:08:18.863746+0000","last_deep_scrub":"27'316","last_deep_scrub_stamp":"2024-05-09T08:08:18.863746+0000","last_clean_scrub_stamp":"2024-05-09T08:08:18.863746+0000","objects_scrubbed":0,"log_size":316,"log_dups_size":0,"ondisk_log_size":316,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:40:13.045393+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":273,"num_write_kb":745472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"27'10","reported_seq":44,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:08:04.262987+0000","last_change":"2024-05-09T08:08:04.263022+0000","last_active":"2024-05-09T08:08:04.262987+0000","last_peered":"2024-05-09T08:08:04.262987+0000","last_clean":"2024-05-09T08:08:04.262987+0000","last_became_active":"2024-05-09T08:02:15.898811+0000","last_became_peered":"2024-05-09T08:02:15.898811+0000","last_unstale":"2024-05-09T08:08:04.262987+0000","last_undegraded":"2024-05-09T08:08:04.262987+0000","last_fullsized":"2024-05-09T08:08:04.262987+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'10","last_scrub_stamp":"2024-05-09T08:08:04.262893+0000","last_deep_scrub":"27'10","last_deep_scrub_stamp":"2024-05-09T08:08:04.262893+0000","last_clean_scrub_stamp":"2024-05-09T08:08:04.262893+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:43:52.165924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"27'4","reported_seq":40,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:08:01.342238+0000","last_change":"2024-05-09T08:08:01.342274+0000","last_active":"2024-05-09T08:08:01.342238+0000","last_peered":"2024-05-09T08:08:01.342238+0000","last_clean":"2024-05-09T08:08:01.342238+0000","last_became_active":"2024-05-09T08:02:14.868385+0000","last_became_peered":"2024-05-09T08:02:14.868385+0000","last_unstale":"2024-05-09T08:08:01.342238+0000","last_undegraded":"2024-05-09T08:08:01.342238+0000","last_fullsized":"2024-05-09T08:08:01.342238+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4","last_scrub_stamp":"2024-05-09T08:08:01.342147+0000","last_deep_scrub":"27'4","last_deep_scrub_stamp":"2024-05-09T08:08:01.342147+0000","last_clean_scrub_stamp":"2024-05-09T08:08:01.342147+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:11:55.174150+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"27'378","reported_seq":407,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:08:20.742610+0000","last_change":"2024-05-09T08:08:20.742639+0000","last_active":"2024-05-09T08:08:20.742610+0000","last_peered":"2024-05-09T08:08:20.742610+0000","last_clean":"2024-05-09T08:08:20.742610+0000","last_became_active":"2024-05-09T08:02:20.052629+0000","last_became_peered":"2024-05-09T08:02:20.052629+0000","last_unstale":"2024-05-09T08:08:20.742610+0000","last_undegraded":"2024-05-09T08:08:20.742610+0000","last_fullsized":"2024-05-09T08:08:20.742610+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'378","last_scrub_stamp":"2024-05-09T08:08:20.742511+0000","last_deep_scrub":"27'378","last_deep_scrub_stamp":"2024-05-09T08:08:20.742511+0000","last_clean_scrub_stamp":"2024-05-09T08:08:20.742511+0000","objects_scrubbed":0,"log_size":378,"log_dups_size":0,"ondisk_log_size":378,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:17:22.163241+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":327,"num_write_kb":892928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"27'6","reported_seq":40,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:08:20.387471+0000","last_change":"2024-05-09T08:08:20.387498+0000","last_active":"2024-05-09T08:08:20.387471+0000","last_peered":"2024-05-09T08:08:20.387471+0000","last_clean":"2024-05-09T08:08:20.387471+0000","last_became_active":"2024-05-09T08:02:15.905076+0000","last_became_peered":"2024-05-09T08:02:15.905076+0000","last_unstale":"2024-05-09T08:08:20.387471+0000","last_undegraded":"2024-05-09T08:08:20.387471+0000","last_fullsized":"2024-05-09T08:08:20.387471+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'6","last_scrub_stamp":"2024-05-09T08:08:20.387368+0000","last_deep_scrub":"27'6","last_deep_scrub_stamp":"2024-05-09T08:08:20.387368+0000","last_clean_scrub_stamp":"2024-05-09T08:08:20.387368+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:37:22.279357+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"27'194","reported_seq":231,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:08:08.833210+0000","last_change":"2024-05-09T08:08:08.833238+0000","last_active":"2024-05-09T08:08:08.833210+0000","last_peered":"2024-05-09T08:08:08.833210+0000","last_clean":"2024-05-09T08:08:08.833210+0000","last_became_active":"2024-05-09T08:02:14.880045+0000","last_became_peered":"2024-05-09T08:02:14.880045+0000","last_unstale":"2024-05-09T08:08:08.833210+0000","last_undegraded":"2024-05-09T08:08:08.833210+0000","last_fullsized":"2024-05-09T08:08:08.833210+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'194","last_scrub_stamp":"2024-05-09T08:08:08.833130+0000","last_deep_scrub":"27'194","last_deep_scrub_stamp":"2024-05-09T08:08:08.833130+0000","last_clean_scrub_stamp":"2024-05-09T08:08:08.833130+0000","objects_scrubbed":4,"log_size":194,"log_dups_size":0,"ondisk_log_size":194,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:16:38.154558+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":447684,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":191,"num_write_kb":585,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"27'332","reported_seq":361,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:08:13.711839+0000","last_change":"2024-05-09T08:08:13.711903+0000","last_active":"2024-05-09T08:08:13.711839+0000","last_peered":"2024-05-09T08:08:13.711839+0000","last_clean":"2024-05-09T08:08:13.711839+0000","last_became_active":"2024-05-09T08:02:19.958964+0000","last_became_peered":"2024-05-09T08:02:19.958964+0000","last_unstale":"2024-05-09T08:08:13.711839+0000","last_undegraded":"2024-05-09T08:08:13.711839+0000","last_fullsized":"2024-05-09T08:08:13.711839+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'332","last_scrub_stamp":"2024-05-09T08:08:13.711649+0000","last_deep_scrub":"27'332","last_deep_scrub_stamp":"2024-05-09T08:08:13.711649+0000","last_clean_scrub_stamp":"2024-05-09T08:08:13.711649+0000","objects_scrubbed":0,"log_size":332,"log_dups_size":0,"ondisk_log_size":332,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:46:38.430496+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":288,"num_write_kb":786432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"27'11","reported_seq":45,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:08:04.788564+0000","last_change":"2024-05-09T08:08:04.788595+0000","last_active":"2024-05-09T08:08:04.788564+0000","last_peered":"2024-05-09T08:08:04.788564+0000","last_clean":"2024-05-09T08:08:04.788564+0000","last_became_active":"2024-05-09T08:02:15.888071+0000","last_became_peered":"2024-05-09T08:02:15.888071+0000","last_unstale":"2024-05-09T08:08:04.788564+0000","last_undegraded":"2024-05-09T08:08:04.788564+0000","last_fullsized":"2024-05-09T08:08:04.788564+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'11","last_scrub_stamp":"2024-05-09T08:08:04.788469+0000","last_deep_scrub":"27'11","last_deep_scrub_stamp":"2024-05-09T08:08:04.788469+0000","last_clean_scrub_stamp":"2024-05-09T08:08:04.788469+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:59:50.017294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"27'10","reported_seq":46,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:08:02.825312+0000","last_change":"2024-05-09T08:08:02.825383+0000","last_active":"2024-05-09T08:08:02.825312+0000","last_peered":"2024-05-09T08:08:02.825312+0000","last_clean":"2024-05-09T08:08:02.825312+0000","last_became_active":"2024-05-09T08:02:14.871945+0000","last_became_peered":"2024-05-09T08:02:14.871945+0000","last_unstale":"2024-05-09T08:08:02.825312+0000","last_undegraded":"2024-05-09T08:08:02.825312+0000","last_fullsized":"2024-05-09T08:08:02.825312+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'10","last_scrub_stamp":"2024-05-09T08:08:02.825166+0000","last_deep_scrub":"27'10","last_deep_scrub_stamp":"2024-05-09T08:08:02.825166+0000","last_clean_scrub_stamp":"2024-05-09T08:08:02.825166+0000","objects_scrubbed":2,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:56:58.698469+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"27'353","reported_seq":382,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:08:11.716047+0000","last_change":"2024-05-09T08:08:11.716077+0000","last_active":"2024-05-09T08:08:11.716047+0000","last_peered":"2024-05-09T08:08:11.716047+0000","last_clean":"2024-05-09T08:08:11.716047+0000","last_became_active":"2024-05-09T08:02:19.953166+0000","last_became_peered":"2024-05-09T08:02:19.953166+0000","last_unstale":"2024-05-09T08:08:11.716047+0000","last_undegraded":"2024-05-09T08:08:11.716047+0000","last_fullsized":"2024-05-09T08:08:11.716047+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'353","last_scrub_stamp":"2024-05-09T08:08:11.715936+0000","last_deep_scrub":"27'353","last_deep_scrub_stamp":"2024-05-09T08:08:11.715936+0000","last_clean_scrub_stamp":"2024-05-09T08:08:11.715936+0000","objects_scrubbed":0,"log_size":353,"log_dups_size":0,"ondisk_log_size":353,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:55:41.320408+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":297,"num_write_kb":811008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"27'9","reported_seq":43,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:08:06.319552+0000","last_change":"2024-05-09T08:08:06.319613+0000","last_active":"2024-05-09T08:08:06.319552+0000","last_peered":"2024-05-09T08:08:06.319552+0000","last_clean":"2024-05-09T08:08:06.319552+0000","last_became_active":"2024-05-09T08:02:15.904898+0000","last_became_peered":"2024-05-09T08:02:15.904898+0000","last_unstale":"2024-05-09T08:08:06.319552+0000","last_undegraded":"2024-05-09T08:08:06.319552+0000","last_fullsized":"2024-05-09T08:08:06.319552+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'9","last_scrub_stamp":"2024-05-09T08:08:06.319360+0000","last_deep_scrub":"27'9","last_deep_scrub_stamp":"2024-05-09T08:08:06.319360+0000","last_clean_scrub_stamp":"2024-05-09T08:08:06.319360+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:07:46.710826+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":36,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:08:06.542942+0000","last_change":"2024-05-09T08:08:06.542964+0000","last_active":"2024-05-09T08:08:06.542942+0000","last_peered":"2024-05-09T08:08:06.542942+0000","last_clean":"2024-05-09T08:08:06.542942+0000","last_became_active":"2024-05-09T08:02:14.868294+0000","last_became_peered":"2024-05-09T08:02:14.868294+0000","last_unstale":"2024-05-09T08:08:06.542942+0000","last_undegraded":"2024-05-09T08:08:06.542942+0000","last_fullsized":"2024-05-09T08:08:06.542942+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:08:06.542860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:08:06.542860+0000","last_clean_scrub_stamp":"2024-05-09T08:08:06.542860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:09:37.402437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"27'344","reported_seq":374,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:08:17.866647+0000","last_change":"2024-05-09T08:08:17.866709+0000","last_active":"2024-05-09T08:08:17.866647+0000","last_peered":"2024-05-09T08:08:17.866647+0000","last_clean":"2024-05-09T08:08:17.866647+0000","last_became_active":"2024-05-09T08:02:19.955662+0000","last_became_peered":"2024-05-09T08:02:19.955662+0000","last_unstale":"2024-05-09T08:08:17.866647+0000","last_undegraded":"2024-05-09T08:08:17.866647+0000","last_fullsized":"2024-05-09T08:08:17.866647+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'344","last_scrub_stamp":"2024-05-09T08:08:17.866447+0000","last_deep_scrub":"27'344","last_deep_scrub_stamp":"2024-05-09T08:08:17.866447+0000","last_clean_scrub_stamp":"2024-05-09T08:08:17.866447+0000","objects_scrubbed":0,"log_size":344,"log_dups_size":0,"ondisk_log_size":344,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:03:20.303453+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":297,"num_write_kb":811008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"27'10","reported_seq":44,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:08:12.645165+0000","last_change":"2024-05-09T08:08:12.645194+0000","last_active":"2024-05-09T08:08:12.645165+0000","last_peered":"2024-05-09T08:08:12.645165+0000","last_clean":"2024-05-09T08:08:12.645165+0000","last_became_active":"2024-05-09T08:02:15.888090+0000","last_became_peered":"2024-05-09T08:02:15.888090+0000","last_unstale":"2024-05-09T08:08:12.645165+0000","last_undegraded":"2024-05-09T08:08:12.645165+0000","last_fullsized":"2024-05-09T08:08:12.645165+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'10","last_scrub_stamp":"2024-05-09T08:08:12.645074+0000","last_deep_scrub":"27'10","last_deep_scrub_stamp":"2024-05-09T08:08:12.645074+0000","last_clean_scrub_stamp":"2024-05-09T08:08:12.645074+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:06:01.986104+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"27'1","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:08:06.872206+0000","last_change":"2024-05-09T08:08:06.872258+0000","last_active":"2024-05-09T08:08:06.872206+0000","last_peered":"2024-05-09T08:08:06.872206+0000","last_clean":"2024-05-09T08:08:06.872206+0000","last_became_active":"2024-05-09T08:02:14.862104+0000","last_became_peered":"2024-05-09T08:02:14.862104+0000","last_unstale":"2024-05-09T08:08:06.872206+0000","last_undegraded":"2024-05-09T08:08:06.872206+0000","last_fullsized":"2024-05-09T08:08:06.872206+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'1","last_scrub_stamp":"2024-05-09T08:08:06.872103+0000","last_deep_scrub":"27'1","last_deep_scrub_stamp":"2024-05-09T08:08:06.872103+0000","last_clean_scrub_stamp":"2024-05-09T08:08:06.872103+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:59:52.100928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"27'379","reported_seq":408,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:08:20.769777+0000","last_change":"2024-05-09T08:08:20.769839+0000","last_active":"2024-05-09T08:08:20.769777+0000","last_peered":"2024-05-09T08:08:20.769777+0000","last_clean":"2024-05-09T08:08:20.769777+0000","last_became_active":"2024-05-09T08:02:20.052485+0000","last_became_peered":"2024-05-09T08:02:20.052485+0000","last_unstale":"2024-05-09T08:08:20.769777+0000","last_undegraded":"2024-05-09T08:08:20.769777+0000","last_fullsized":"2024-05-09T08:08:20.769777+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'379","last_scrub_stamp":"2024-05-09T08:08:20.769608+0000","last_deep_scrub":"27'379","last_deep_scrub_stamp":"2024-05-09T08:08:20.769608+0000","last_clean_scrub_stamp":"2024-05-09T08:08:20.769608+0000","objects_scrubbed":0,"log_size":379,"log_dups_size":0,"ondisk_log_size":379,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:23:08.250035+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":327,"num_write_kb":892928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"27'324","reported_seq":354,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:08:15.215049+0000","last_change":"2024-05-09T08:08:15.215049+0000","last_active":"2024-05-09T08:08:15.215049+0000","last_peered":"2024-05-09T08:08:15.215049+0000","last_clean":"2024-05-09T08:08:15.215049+0000","last_became_active":"2024-05-09T08:02:20.053250+0000","last_became_peered":"2024-05-09T08:02:20.053250+0000","last_unstale":"2024-05-09T08:08:15.215049+0000","last_undegraded":"2024-05-09T08:08:15.215049+0000","last_fullsized":"2024-05-09T08:08:15.215049+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'324","last_scrub_stamp":"2024-05-09T08:08:15.214758+0000","last_deep_scrub":"27'324","last_deep_scrub_stamp":"2024-05-09T08:08:15.214758+0000","last_clean_scrub_stamp":"2024-05-09T08:08:15.214758+0000","objects_scrubbed":0,"log_size":324,"log_dups_size":0,"ondisk_log_size":324,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:07:57.508931+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":273,"num_write_kb":745472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"27'11","reported_seq":45,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:08:10.724187+0000","last_change":"2024-05-09T08:08:10.724246+0000","last_active":"2024-05-09T08:08:10.724187+0000","last_peered":"2024-05-09T08:08:10.724187+0000","last_clean":"2024-05-09T08:08:10.724187+0000","last_became_active":"2024-05-09T08:02:15.906600+0000","last_became_peered":"2024-05-09T08:02:15.906600+0000","last_unstale":"2024-05-09T08:08:10.724187+0000","last_undegraded":"2024-05-09T08:08:10.724187+0000","last_fullsized":"2024-05-09T08:08:10.724187+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'11","last_scrub_stamp":"2024-05-09T08:08:10.723976+0000","last_deep_scrub":"27'11","last_deep_scrub_stamp":"2024-05-09T08:08:10.723976+0000","last_clean_scrub_stamp":"2024-05-09T08:08:10.723976+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:26:49.310250+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":36,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:08:08.574131+0000","last_change":"2024-05-09T08:08:08.574153+0000","last_active":"2024-05-09T08:08:08.574131+0000","last_peered":"2024-05-09T08:08:08.574131+0000","last_clean":"2024-05-09T08:08:08.574131+0000","last_became_active":"2024-05-09T08:02:14.869818+0000","last_became_peered":"2024-05-09T08:02:14.869818+0000","last_unstale":"2024-05-09T08:08:08.574131+0000","last_undegraded":"2024-05-09T08:08:08.574131+0000","last_fullsized":"2024-05-09T08:08:08.574131+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:08:08.574045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:08:08.574045+0000","last_clean_scrub_stamp":"2024-05-09T08:08:08.574045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:13:54.633742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"27'313","reported_seq":342,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:08:15.847923+0000","last_change":"2024-05-09T08:08:15.847950+0000","last_active":"2024-05-09T08:08:15.847923+0000","last_peered":"2024-05-09T08:08:15.847923+0000","last_clean":"2024-05-09T08:08:15.847923+0000","last_became_active":"2024-05-09T08:02:19.952117+0000","last_became_peered":"2024-05-09T08:02:19.952117+0000","last_unstale":"2024-05-09T08:08:15.847923+0000","last_undegraded":"2024-05-09T08:08:15.847923+0000","last_fullsized":"2024-05-09T08:08:15.847923+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'313","last_scrub_stamp":"2024-05-09T08:08:15.847806+0000","last_deep_scrub":"27'313","last_deep_scrub_stamp":"2024-05-09T08:08:15.847806+0000","last_clean_scrub_stamp":"2024-05-09T08:08:15.847806+0000","objects_scrubbed":0,"log_size":313,"log_dups_size":0,"ondisk_log_size":313,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:26:54.434080+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":267,"num_write_kb":729088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"27'8","reported_seq":42,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:08:03.949385+0000","last_change":"2024-05-09T08:08:03.949448+0000","last_active":"2024-05-09T08:08:03.949385+0000","last_peered":"2024-05-09T08:08:03.949385+0000","last_clean":"2024-05-09T08:08:03.949385+0000","last_became_active":"2024-05-09T08:02:15.912883+0000","last_became_peered":"2024-05-09T08:02:15.912883+0000","last_unstale":"2024-05-09T08:08:03.949385+0000","last_undegraded":"2024-05-09T08:08:03.949385+0000","last_fullsized":"2024-05-09T08:08:03.949385+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'8","last_scrub_stamp":"2024-05-09T08:08:03.949201+0000","last_deep_scrub":"27'8","last_deep_scrub_stamp":"2024-05-09T08:08:03.949201+0000","last_clean_scrub_stamp":"2024-05-09T08:08:03.949201+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:50:12.560113+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":36,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:08:08.885117+0000","last_change":"2024-05-09T08:08:08.885216+0000","last_active":"2024-05-09T08:08:08.885117+0000","last_peered":"2024-05-09T08:08:08.885117+0000","last_clean":"2024-05-09T08:08:08.885117+0000","last_became_active":"2024-05-09T08:02:14.884965+0000","last_became_peered":"2024-05-09T08:02:14.884965+0000","last_unstale":"2024-05-09T08:08:08.885117+0000","last_undegraded":"2024-05-09T08:08:08.885117+0000","last_fullsized":"2024-05-09T08:08:08.885117+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:08:08.884835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:08:08.884835+0000","last_clean_scrub_stamp":"2024-05-09T08:08:08.884835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:10:53.056331+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"24'1","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:08:04.881782+0000","last_change":"2024-05-09T08:08:04.881812+0000","last_active":"2024-05-09T08:08:04.881782+0000","last_peered":"2024-05-09T08:08:04.881782+0000","last_clean":"2024-05-09T08:08:04.881782+0000","last_became_active":"2024-05-09T08:02:14.859917+0000","last_became_peered":"2024-05-09T08:02:14.859917+0000","last_unstale":"2024-05-09T08:08:04.881782+0000","last_undegraded":"2024-05-09T08:08:04.881782+0000","last_fullsized":"2024-05-09T08:08:04.881782+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"24'1","last_scrub_stamp":"2024-05-09T08:08:04.881686+0000","last_deep_scrub":"24'1","last_deep_scrub_stamp":"2024-05-09T08:08:04.881686+0000","last_clean_scrub_stamp":"2024-05-09T08:08:04.881686+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:05:21.907992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"27'8","reported_seq":42,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:08:03.310212+0000","last_change":"2024-05-09T08:08:03.310278+0000","last_active":"2024-05-09T08:08:03.310212+0000","last_peered":"2024-05-09T08:08:03.310212+0000","last_clean":"2024-05-09T08:08:03.310212+0000","last_became_active":"2024-05-09T08:02:15.896708+0000","last_became_peered":"2024-05-09T08:02:15.896708+0000","last_unstale":"2024-05-09T08:08:03.310212+0000","last_undegraded":"2024-05-09T08:08:03.310212+0000","last_fullsized":"2024-05-09T08:08:03.310212+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'8","last_scrub_stamp":"2024-05-09T08:08:03.309989+0000","last_deep_scrub":"27'8","last_deep_scrub_stamp":"2024-05-09T08:08:03.309989+0000","last_clean_scrub_stamp":"2024-05-09T08:08:03.309989+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:13:38.946864+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"27'338","reported_seq":367,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:08:07.654180+0000","last_change":"2024-05-09T08:08:07.654249+0000","last_active":"2024-05-09T08:08:07.654180+0000","last_peered":"2024-05-09T08:08:07.654180+0000","last_clean":"2024-05-09T08:08:07.654180+0000","last_became_active":"2024-05-09T08:02:19.954829+0000","last_became_peered":"2024-05-09T08:02:19.954829+0000","last_unstale":"2024-05-09T08:08:07.654180+0000","last_undegraded":"2024-05-09T08:08:07.654180+0000","last_fullsized":"2024-05-09T08:08:07.654180+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'338","last_scrub_stamp":"2024-05-09T08:08:07.653938+0000","last_deep_scrub":"27'338","last_deep_scrub_stamp":"2024-05-09T08:08:07.653938+0000","last_clean_scrub_stamp":"2024-05-09T08:08:07.653938+0000","objects_scrubbed":0,"log_size":338,"log_dups_size":0,"ondisk_log_size":338,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:05:41.846898+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":297,"num_write_kb":811008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"24'1","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:08:07.925000+0000","last_change":"2024-05-09T08:08:07.925097+0000","last_active":"2024-05-09T08:08:07.925000+0000","last_peered":"2024-05-09T08:08:07.925000+0000","last_clean":"2024-05-09T08:08:07.925000+0000","last_became_active":"2024-05-09T08:02:14.882610+0000","last_became_peered":"2024-05-09T08:02:14.882610+0000","last_unstale":"2024-05-09T08:08:07.925000+0000","last_undegraded":"2024-05-09T08:08:07.925000+0000","last_fullsized":"2024-05-09T08:08:07.925000+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"24'1","last_scrub_stamp":"2024-05-09T08:08:07.924823+0000","last_deep_scrub":"24'1","last_deep_scrub_stamp":"2024-05-09T08:08:07.924823+0000","last_clean_scrub_stamp":"2024-05-09T08:08:07.924823+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:13:53.984520+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"27'7","reported_seq":41,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:08:05.802282+0000","last_change":"2024-05-09T08:08:05.802334+0000","last_active":"2024-05-09T08:08:05.802282+0000","last_peered":"2024-05-09T08:08:05.802282+0000","last_clean":"2024-05-09T08:08:05.802282+0000","last_became_active":"2024-05-09T08:02:15.889650+0000","last_became_peered":"2024-05-09T08:02:15.889650+0000","last_unstale":"2024-05-09T08:08:05.802282+0000","last_undegraded":"2024-05-09T08:08:05.802282+0000","last_fullsized":"2024-05-09T08:08:05.802282+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'7","last_scrub_stamp":"2024-05-09T08:08:05.802180+0000","last_deep_scrub":"27'7","last_deep_scrub_stamp":"2024-05-09T08:08:05.802180+0000","last_clean_scrub_stamp":"2024-05-09T08:08:05.802180+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:57:01.675483+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"27'361","reported_seq":390,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:08:16.872608+0000","last_change":"2024-05-09T08:08:16.872672+0000","last_active":"2024-05-09T08:08:16.872608+0000","last_peered":"2024-05-09T08:08:16.872608+0000","last_clean":"2024-05-09T08:08:16.872608+0000","last_became_active":"2024-05-09T08:02:19.954947+0000","last_became_peered":"2024-05-09T08:02:19.954947+0000","last_unstale":"2024-05-09T08:08:16.872608+0000","last_undegraded":"2024-05-09T08:08:16.872608+0000","last_fullsized":"2024-05-09T08:08:16.872608+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'361","last_scrub_stamp":"2024-05-09T08:08:16.872411+0000","last_deep_scrub":"27'361","last_deep_scrub_stamp":"2024-05-09T08:08:16.872411+0000","last_clean_scrub_stamp":"2024-05-09T08:08:16.872411+0000","objects_scrubbed":0,"log_size":361,"log_dups_size":0,"ondisk_log_size":361,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:37:18.764400+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":312,"num_write_kb":851968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"27'9","reported_seq":44,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:08:14.837125+0000","last_change":"2024-05-09T08:08:14.837183+0000","last_active":"2024-05-09T08:08:14.837125+0000","last_peered":"2024-05-09T08:08:14.837125+0000","last_clean":"2024-05-09T08:08:14.837125+0000","last_became_active":"2024-05-09T08:02:15.909086+0000","last_became_peered":"2024-05-09T08:02:15.909086+0000","last_unstale":"2024-05-09T08:08:14.837125+0000","last_undegraded":"2024-05-09T08:08:14.837125+0000","last_fullsized":"2024-05-09T08:08:14.837125+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'9","last_scrub_stamp":"2024-05-09T08:08:14.837003+0000","last_deep_scrub":"27'9","last_deep_scrub_stamp":"2024-05-09T08:08:14.837003+0000","last_clean_scrub_stamp":"2024-05-09T08:08:14.837003+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:14:00.896700+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"27'12","reported_seq":46,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:08:06.645108+0000","last_change":"2024-05-09T08:08:06.645223+0000","last_active":"2024-05-09T08:08:06.645108+0000","last_peered":"2024-05-09T08:08:06.645108+0000","last_clean":"2024-05-09T08:08:06.645108+0000","last_became_active":"2024-05-09T08:02:15.890640+0000","last_became_peered":"2024-05-09T08:02:15.890640+0000","last_unstale":"2024-05-09T08:08:06.645108+0000","last_undegraded":"2024-05-09T08:08:06.645108+0000","last_fullsized":"2024-05-09T08:08:06.645108+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'12","last_scrub_stamp":"2024-05-09T08:08:06.644859+0000","last_deep_scrub":"27'12","last_deep_scrub_stamp":"2024-05-09T08:08:06.644859+0000","last_clean_scrub_stamp":"2024-05-09T08:08:06.644859+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:20:59.171436+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"27'8","reported_seq":42,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:08:04.523416+0000","last_change":"2024-05-09T08:08:04.523443+0000","last_active":"2024-05-09T08:08:04.523416+0000","last_peered":"2024-05-09T08:08:04.523416+0000","last_clean":"2024-05-09T08:08:04.523416+0000","last_became_active":"2024-05-09T08:02:15.898695+0000","last_became_peered":"2024-05-09T08:02:15.898695+0000","last_unstale":"2024-05-09T08:08:04.523416+0000","last_undegraded":"2024-05-09T08:08:04.523416+0000","last_fullsized":"2024-05-09T08:08:04.523416+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'8","last_scrub_stamp":"2024-05-09T08:08:04.523331+0000","last_deep_scrub":"27'8","last_deep_scrub_stamp":"2024-05-09T08:08:04.523331+0000","last_clean_scrub_stamp":"2024-05-09T08:08:04.523331+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:37:58.047576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"27'7","reported_seq":41,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:08:08.242624+0000","last_change":"2024-05-09T08:08:08.242689+0000","last_active":"2024-05-09T08:08:08.242624+0000","last_peered":"2024-05-09T08:08:08.242624+0000","last_clean":"2024-05-09T08:08:08.242624+0000","last_became_active":"2024-05-09T08:02:15.906767+0000","last_became_peered":"2024-05-09T08:02:15.906767+0000","last_unstale":"2024-05-09T08:08:08.242624+0000","last_undegraded":"2024-05-09T08:08:08.242624+0000","last_fullsized":"2024-05-09T08:08:08.242624+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'7","last_scrub_stamp":"2024-05-09T08:08:08.242449+0000","last_deep_scrub":"27'7","last_deep_scrub_stamp":"2024-05-09T08:08:08.242449+0000","last_clean_scrub_stamp":"2024-05-09T08:08:08.242449+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:45:15.481551+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"27'4","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:08:04.954296+0000","last_change":"2024-05-09T08:08:04.954357+0000","last_active":"2024-05-09T08:08:04.954296+0000","last_peered":"2024-05-09T08:08:04.954296+0000","last_clean":"2024-05-09T08:08:04.954296+0000","last_became_active":"2024-05-09T08:02:15.905083+0000","last_became_peered":"2024-05-09T08:02:15.905083+0000","last_unstale":"2024-05-09T08:08:04.954296+0000","last_undegraded":"2024-05-09T08:08:04.954296+0000","last_fullsized":"2024-05-09T08:08:04.954296+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4","last_scrub_stamp":"2024-05-09T08:08:04.954176+0000","last_deep_scrub":"27'4","last_deep_scrub_stamp":"2024-05-09T08:08:04.954176+0000","last_clean_scrub_stamp":"2024-05-09T08:08:04.954176+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:07:45.345643+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"27'6","reported_seq":40,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:08:11.773949+0000","last_change":"2024-05-09T08:08:11.774007+0000","last_active":"2024-05-09T08:08:11.773949+0000","last_peered":"2024-05-09T08:08:11.773949+0000","last_clean":"2024-05-09T08:08:11.773949+0000","last_became_active":"2024-05-09T08:02:15.907480+0000","last_became_peered":"2024-05-09T08:02:15.907480+0000","last_unstale":"2024-05-09T08:08:11.773949+0000","last_undegraded":"2024-05-09T08:08:11.773949+0000","last_fullsized":"2024-05-09T08:08:11.773949+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'6","last_scrub_stamp":"2024-05-09T08:08:11.773770+0000","last_deep_scrub":"27'6","last_deep_scrub_stamp":"2024-05-09T08:08:11.773770+0000","last_clean_scrub_stamp":"2024-05-09T08:08:11.773770+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:08:30.138207+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"27'15","reported_seq":50,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:08:12.799135+0000","last_change":"2024-05-09T08:08:12.799163+0000","last_active":"2024-05-09T08:08:12.799135+0000","last_peered":"2024-05-09T08:08:12.799135+0000","last_clean":"2024-05-09T08:08:12.799135+0000","last_became_active":"2024-05-09T08:02:15.887981+0000","last_became_peered":"2024-05-09T08:02:15.887981+0000","last_unstale":"2024-05-09T08:08:12.799135+0000","last_undegraded":"2024-05-09T08:08:12.799135+0000","last_fullsized":"2024-05-09T08:08:12.799135+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'15","last_scrub_stamp":"2024-05-09T08:08:12.799026+0000","last_deep_scrub":"27'15","last_deep_scrub_stamp":"2024-05-09T08:08:12.799026+0000","last_clean_scrub_stamp":"2024-05-09T08:08:12.799026+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:15:44.527240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"27'2","reported_seq":36,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:08:06.959163+0000","last_change":"2024-05-09T08:08:06.959226+0000","last_active":"2024-05-09T08:08:06.959163+0000","last_peered":"2024-05-09T08:08:06.959163+0000","last_clean":"2024-05-09T08:08:06.959163+0000","last_became_active":"2024-05-09T08:02:15.904334+0000","last_became_peered":"2024-05-09T08:02:15.904334+0000","last_unstale":"2024-05-09T08:08:06.959163+0000","last_undegraded":"2024-05-09T08:08:06.959163+0000","last_fullsized":"2024-05-09T08:08:06.959163+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2","last_scrub_stamp":"2024-05-09T08:08:06.958956+0000","last_deep_scrub":"27'2","last_deep_scrub_stamp":"2024-05-09T08:08:06.958956+0000","last_clean_scrub_stamp":"2024-05-09T08:08:06.958956+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:55:36.563428+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"27'6","reported_seq":40,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:08:16.683498+0000","last_change":"2024-05-09T08:08:16.683526+0000","last_active":"2024-05-09T08:08:16.683498+0000","last_peered":"2024-05-09T08:08:16.683498+0000","last_clean":"2024-05-09T08:08:16.683498+0000","last_became_active":"2024-05-09T08:02:15.904416+0000","last_became_peered":"2024-05-09T08:02:15.904416+0000","last_unstale":"2024-05-09T08:08:16.683498+0000","last_undegraded":"2024-05-09T08:08:16.683498+0000","last_fullsized":"2024-05-09T08:08:16.683498+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'6","last_scrub_stamp":"2024-05-09T08:08:16.683410+0000","last_deep_scrub":"27'6","last_deep_scrub_stamp":"2024-05-09T08:08:16.683410+0000","last_clean_scrub_stamp":"2024-05-09T08:08:16.683410+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:07:25.051175+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"27'8","reported_seq":42,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:08:10.950355+0000","last_change":"2024-05-09T08:08:10.950413+0000","last_active":"2024-05-09T08:08:10.950355+0000","last_peered":"2024-05-09T08:08:10.950355+0000","last_clean":"2024-05-09T08:08:10.950355+0000","last_became_active":"2024-05-09T08:02:15.897981+0000","last_became_peered":"2024-05-09T08:02:15.897981+0000","last_unstale":"2024-05-09T08:08:10.950355+0000","last_undegraded":"2024-05-09T08:08:10.950355+0000","last_fullsized":"2024-05-09T08:08:10.950355+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'8","last_scrub_stamp":"2024-05-09T08:08:10.950185+0000","last_deep_scrub":"27'8","last_deep_scrub_stamp":"2024-05-09T08:08:10.950185+0000","last_clean_scrub_stamp":"2024-05-09T08:08:10.950185+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:01:16.366051+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"27'10","reported_seq":44,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:08:05.512173+0000","last_change":"2024-05-09T08:08:05.512199+0000","last_active":"2024-05-09T08:08:05.512173+0000","last_peered":"2024-05-09T08:08:05.512173+0000","last_clean":"2024-05-09T08:08:05.512173+0000","last_became_active":"2024-05-09T08:02:15.890647+0000","last_became_peered":"2024-05-09T08:02:15.890647+0000","last_unstale":"2024-05-09T08:08:05.512173+0000","last_undegraded":"2024-05-09T08:08:05.512173+0000","last_fullsized":"2024-05-09T08:08:05.512173+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'10","last_scrub_stamp":"2024-05-09T08:08:05.512092+0000","last_deep_scrub":"27'10","last_deep_scrub_stamp":"2024-05-09T08:08:05.512092+0000","last_clean_scrub_stamp":"2024-05-09T08:08:05.512092+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:13:13.151852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"27'11","reported_seq":45,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:08:10.255202+0000","last_change":"2024-05-09T08:08:10.255262+0000","last_active":"2024-05-09T08:08:10.255202+0000","last_peered":"2024-05-09T08:08:10.255202+0000","last_clean":"2024-05-09T08:08:10.255202+0000","last_became_active":"2024-05-09T08:02:15.909818+0000","last_became_peered":"2024-05-09T08:02:15.909818+0000","last_unstale":"2024-05-09T08:08:10.255202+0000","last_undegraded":"2024-05-09T08:08:10.255202+0000","last_fullsized":"2024-05-09T08:08:10.255202+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'11","last_scrub_stamp":"2024-05-09T08:08:10.254999+0000","last_deep_scrub":"27'11","last_deep_scrub_stamp":"2024-05-09T08:08:10.254999+0000","last_clean_scrub_stamp":"2024-05-09T08:08:10.254999+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:10:54.426495+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"27'8","reported_seq":43,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:08:10.752056+0000","last_change":"2024-05-09T08:08:10.752083+0000","last_active":"2024-05-09T08:08:10.752056+0000","last_peered":"2024-05-09T08:08:10.752056+0000","last_clean":"2024-05-09T08:08:10.752056+0000","last_became_active":"2024-05-09T08:02:15.884517+0000","last_became_peered":"2024-05-09T08:02:15.884517+0000","last_unstale":"2024-05-09T08:08:10.752056+0000","last_undegraded":"2024-05-09T08:08:10.752056+0000","last_fullsized":"2024-05-09T08:08:10.752056+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'8","last_scrub_stamp":"2024-05-09T08:08:10.751966+0000","last_deep_scrub":"27'8","last_deep_scrub_stamp":"2024-05-09T08:08:10.751966+0000","last_clean_scrub_stamp":"2024-05-09T08:08:10.751966+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:50:19.362878+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"27'7","reported_seq":41,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:08:18.410598+0000","last_change":"2024-05-09T08:08:18.410626+0000","last_active":"2024-05-09T08:08:18.410598+0000","last_peered":"2024-05-09T08:08:18.410598+0000","last_clean":"2024-05-09T08:08:18.410598+0000","last_became_active":"2024-05-09T08:02:15.889998+0000","last_became_peered":"2024-05-09T08:02:15.889998+0000","last_unstale":"2024-05-09T08:08:18.410598+0000","last_undegraded":"2024-05-09T08:08:18.410598+0000","last_fullsized":"2024-05-09T08:08:18.410598+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'7","last_scrub_stamp":"2024-05-09T08:08:18.410505+0000","last_deep_scrub":"27'7","last_deep_scrub_stamp":"2024-05-09T08:08:18.410505+0000","last_clean_scrub_stamp":"2024-05-09T08:08:18.410505+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:13:36.341532+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"27'10","reported_seq":44,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:08:05.661525+0000","last_change":"2024-05-09T08:08:05.661585+0000","last_active":"2024-05-09T08:08:05.661525+0000","last_peered":"2024-05-09T08:08:05.661525+0000","last_clean":"2024-05-09T08:08:05.661525+0000","last_became_active":"2024-05-09T08:02:15.885525+0000","last_became_peered":"2024-05-09T08:02:15.885525+0000","last_unstale":"2024-05-09T08:08:05.661525+0000","last_undegraded":"2024-05-09T08:08:05.661525+0000","last_fullsized":"2024-05-09T08:08:05.661525+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'10","last_scrub_stamp":"2024-05-09T08:08:05.661350+0000","last_deep_scrub":"27'10","last_deep_scrub_stamp":"2024-05-09T08:08:05.661350+0000","last_clean_scrub_stamp":"2024-05-09T08:08:05.661350+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:36:44.481965+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"27'8","reported_seq":42,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:08:03.553964+0000","last_change":"2024-05-09T08:08:03.553992+0000","last_active":"2024-05-09T08:08:03.553964+0000","last_peered":"2024-05-09T08:08:03.553964+0000","last_clean":"2024-05-09T08:08:03.553964+0000","last_became_active":"2024-05-09T08:02:15.892886+0000","last_became_peered":"2024-05-09T08:02:15.892886+0000","last_unstale":"2024-05-09T08:08:03.553964+0000","last_undegraded":"2024-05-09T08:08:03.553964+0000","last_fullsized":"2024-05-09T08:08:03.553964+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'8","last_scrub_stamp":"2024-05-09T08:08:03.553866+0000","last_deep_scrub":"27'8","last_deep_scrub_stamp":"2024-05-09T08:08:03.553866+0000","last_clean_scrub_stamp":"2024-05-09T08:08:03.553866+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:16:32.875294+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"27'9","reported_seq":43,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:08:18.685902+0000","last_change":"2024-05-09T08:08:18.685930+0000","last_active":"2024-05-09T08:08:18.685902+0000","last_peered":"2024-05-09T08:08:18.685902+0000","last_clean":"2024-05-09T08:08:18.685902+0000","last_became_active":"2024-05-09T08:02:15.908307+0000","last_became_peered":"2024-05-09T08:02:15.908307+0000","last_unstale":"2024-05-09T08:08:18.685902+0000","last_undegraded":"2024-05-09T08:08:18.685902+0000","last_fullsized":"2024-05-09T08:08:18.685902+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'9","last_scrub_stamp":"2024-05-09T08:08:18.685806+0000","last_deep_scrub":"27'9","last_deep_scrub_stamp":"2024-05-09T08:08:18.685806+0000","last_clean_scrub_stamp":"2024-05-09T08:08:18.685806+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:03:21.122812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"27'7","reported_seq":41,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:08:06.851303+0000","last_change":"2024-05-09T08:08:06.851361+0000","last_active":"2024-05-09T08:08:06.851303+0000","last_peered":"2024-05-09T08:08:06.851303+0000","last_clean":"2024-05-09T08:08:06.851303+0000","last_became_active":"2024-05-09T08:02:15.885185+0000","last_became_peered":"2024-05-09T08:02:15.885185+0000","last_unstale":"2024-05-09T08:08:06.851303+0000","last_undegraded":"2024-05-09T08:08:06.851303+0000","last_fullsized":"2024-05-09T08:08:06.851303+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'7","last_scrub_stamp":"2024-05-09T08:08:06.851125+0000","last_deep_scrub":"27'7","last_deep_scrub_stamp":"2024-05-09T08:08:06.851125+0000","last_clean_scrub_stamp":"2024-05-09T08:08:06.851125+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:09:37.710702+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"27'8","reported_seq":42,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:08:00.299276+0000","last_change":"2024-05-09T08:08:00.299335+0000","last_active":"2024-05-09T08:08:00.299276+0000","last_peered":"2024-05-09T08:08:00.299276+0000","last_clean":"2024-05-09T08:08:00.299276+0000","last_became_active":"2024-05-09T08:02:15.911142+0000","last_became_peered":"2024-05-09T08:02:15.911142+0000","last_unstale":"2024-05-09T08:08:00.299276+0000","last_undegraded":"2024-05-09T08:08:00.299276+0000","last_fullsized":"2024-05-09T08:08:00.299276+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'8","last_scrub_stamp":"2024-05-09T08:08:00.299108+0000","last_deep_scrub":"27'8","last_deep_scrub_stamp":"2024-05-09T08:08:00.299108+0000","last_clean_scrub_stamp":"2024-05-09T08:08:00.299108+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:53:19.602110+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"27'10","reported_seq":44,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:08:11.952852+0000","last_change":"2024-05-09T08:08:11.952962+0000","last_active":"2024-05-09T08:08:11.952852+0000","last_peered":"2024-05-09T08:08:11.952852+0000","last_clean":"2024-05-09T08:08:11.952852+0000","last_became_active":"2024-05-09T08:02:15.898480+0000","last_became_peered":"2024-05-09T08:02:15.898480+0000","last_unstale":"2024-05-09T08:08:11.952852+0000","last_undegraded":"2024-05-09T08:08:11.952852+0000","last_fullsized":"2024-05-09T08:08:11.952852+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'10","last_scrub_stamp":"2024-05-09T08:08:11.952587+0000","last_deep_scrub":"27'10","last_deep_scrub_stamp":"2024-05-09T08:08:11.952587+0000","last_clean_scrub_stamp":"2024-05-09T08:08:11.952587+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:06:01.293618+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"27'7","reported_seq":41,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:08:03.751670+0000","last_change":"2024-05-09T08:08:03.751700+0000","last_active":"2024-05-09T08:08:03.751670+0000","last_peered":"2024-05-09T08:08:03.751670+0000","last_clean":"2024-05-09T08:08:03.751670+0000","last_became_active":"2024-05-09T08:02:15.882141+0000","last_became_peered":"2024-05-09T08:02:15.882141+0000","last_unstale":"2024-05-09T08:08:03.751670+0000","last_undegraded":"2024-05-09T08:08:03.751670+0000","last_fullsized":"2024-05-09T08:08:03.751670+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'7","last_scrub_stamp":"2024-05-09T08:08:03.751575+0000","last_deep_scrub":"27'7","last_deep_scrub_stamp":"2024-05-09T08:08:03.751575+0000","last_clean_scrub_stamp":"2024-05-09T08:08:03.751575+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:53:23.054577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"27'6","reported_seq":46,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:08:18.708248+0000","last_change":"2024-05-09T08:08:18.708271+0000","last_active":"2024-05-09T08:08:18.708248+0000","last_peered":"2024-05-09T08:08:18.708248+0000","last_clean":"2024-05-09T08:08:18.708248+0000","last_became_active":"2024-05-09T08:03:02.322073+0000","last_became_peered":"2024-05-09T08:03:02.322073+0000","last_unstale":"2024-05-09T08:08:18.708248+0000","last_undegraded":"2024-05-09T08:08:18.708248+0000","last_fullsized":"2024-05-09T08:08:18.708248+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'6","last_scrub_stamp":"2024-05-09T08:08:18.708165+0000","last_deep_scrub":"27'6","last_deep_scrub_stamp":"2024-05-09T08:08:18.708165+0000","last_clean_scrub_stamp":"2024-05-09T08:08:18.708165+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:46:43.427012+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"27'16","reported_seq":51,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:08:11.790517+0000","last_change":"2024-05-09T08:08:11.790545+0000","last_active":"2024-05-09T08:08:11.790517+0000","last_peered":"2024-05-09T08:08:11.790517+0000","last_clean":"2024-05-09T08:08:11.790517+0000","last_became_active":"2024-05-09T08:02:15.886183+0000","last_became_peered":"2024-05-09T08:02:15.886183+0000","last_unstale":"2024-05-09T08:08:11.790517+0000","last_undegraded":"2024-05-09T08:08:11.790517+0000","last_fullsized":"2024-05-09T08:08:11.790517+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'16","last_scrub_stamp":"2024-05-09T08:08:11.790430+0000","last_deep_scrub":"27'16","last_deep_scrub_stamp":"2024-05-09T08:08:11.790430+0000","last_clean_scrub_stamp":"2024-05-09T08:08:11.790430+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:09:42.650007+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"27'11","reported_seq":45,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:08:16.448536+0000","last_change":"2024-05-09T08:08:16.448563+0000","last_active":"2024-05-09T08:08:16.448536+0000","last_peered":"2024-05-09T08:08:16.448536+0000","last_clean":"2024-05-09T08:08:16.448536+0000","last_became_active":"2024-05-09T08:02:15.887225+0000","last_became_peered":"2024-05-09T08:02:15.887225+0000","last_unstale":"2024-05-09T08:08:16.448536+0000","last_undegraded":"2024-05-09T08:08:16.448536+0000","last_fullsized":"2024-05-09T08:08:16.448536+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'11","last_scrub_stamp":"2024-05-09T08:08:16.448451+0000","last_deep_scrub":"27'11","last_deep_scrub_stamp":"2024-05-09T08:08:16.448451+0000","last_clean_scrub_stamp":"2024-05-09T08:08:16.448451+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:38:26.396123+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"27'9","reported_seq":43,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:08:14.844513+0000","last_change":"2024-05-09T08:08:14.844575+0000","last_active":"2024-05-09T08:08:14.844513+0000","last_peered":"2024-05-09T08:08:14.844513+0000","last_clean":"2024-05-09T08:08:14.844513+0000","last_became_active":"2024-05-09T08:02:15.901766+0000","last_became_peered":"2024-05-09T08:02:15.901766+0000","last_unstale":"2024-05-09T08:08:14.844513+0000","last_undegraded":"2024-05-09T08:08:14.844513+0000","last_fullsized":"2024-05-09T08:08:14.844513+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'9","last_scrub_stamp":"2024-05-09T08:08:14.844329+0000","last_deep_scrub":"27'9","last_deep_scrub_stamp":"2024-05-09T08:08:14.844329+0000","last_clean_scrub_stamp":"2024-05-09T08:08:14.844329+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:20:41.652476+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"27'5","reported_seq":39,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:08:21.421750+0000","last_change":"2024-05-09T08:08:21.421777+0000","last_active":"2024-05-09T08:08:21.421750+0000","last_peered":"2024-05-09T08:08:21.421750+0000","last_clean":"2024-05-09T08:08:21.421750+0000","last_became_active":"2024-05-09T08:02:15.910716+0000","last_became_peered":"2024-05-09T08:02:15.910716+0000","last_unstale":"2024-05-09T08:08:21.421750+0000","last_undegraded":"2024-05-09T08:08:21.421750+0000","last_fullsized":"2024-05-09T08:08:21.421750+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'5","last_scrub_stamp":"2024-05-09T08:08:21.421657+0000","last_deep_scrub":"27'5","last_deep_scrub_stamp":"2024-05-09T08:08:21.421657+0000","last_clean_scrub_stamp":"2024-05-09T08:08:21.421657+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:03:23.858662+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"27'6","reported_seq":40,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:08:17.695076+0000","last_change":"2024-05-09T08:08:17.695104+0000","last_active":"2024-05-09T08:08:17.695076+0000","last_peered":"2024-05-09T08:08:17.695076+0000","last_clean":"2024-05-09T08:08:17.695076+0000","last_became_active":"2024-05-09T08:02:15.908290+0000","last_became_peered":"2024-05-09T08:02:15.908290+0000","last_unstale":"2024-05-09T08:08:17.695076+0000","last_undegraded":"2024-05-09T08:08:17.695076+0000","last_fullsized":"2024-05-09T08:08:17.695076+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'6","last_scrub_stamp":"2024-05-09T08:08:17.694983+0000","last_deep_scrub":"27'6","last_deep_scrub_stamp":"2024-05-09T08:08:17.694983+0000","last_clean_scrub_stamp":"2024-05-09T08:08:17.694983+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:58:12.492229+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"27'4","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:08:19.430276+0000","last_change":"2024-05-09T08:08:19.430305+0000","last_active":"2024-05-09T08:08:19.430276+0000","last_peered":"2024-05-09T08:08:19.430276+0000","last_clean":"2024-05-09T08:08:19.430276+0000","last_became_active":"2024-05-09T08:02:15.896384+0000","last_became_peered":"2024-05-09T08:02:15.896384+0000","last_unstale":"2024-05-09T08:08:19.430276+0000","last_undegraded":"2024-05-09T08:08:19.430276+0000","last_fullsized":"2024-05-09T08:08:19.430276+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4","last_scrub_stamp":"2024-05-09T08:08:19.430165+0000","last_deep_scrub":"27'4","last_deep_scrub_stamp":"2024-05-09T08:08:19.430165+0000","last_clean_scrub_stamp":"2024-05-09T08:08:19.430165+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:56:34.141323+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"27'8","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:08:10.749876+0000","last_change":"2024-05-09T08:08:10.749936+0000","last_active":"2024-05-09T08:08:10.749876+0000","last_peered":"2024-05-09T08:08:10.749876+0000","last_clean":"2024-05-09T08:08:10.749876+0000","last_became_active":"2024-05-09T08:03:02.614387+0000","last_became_peered":"2024-05-09T08:03:02.614387+0000","last_unstale":"2024-05-09T08:08:10.749876+0000","last_undegraded":"2024-05-09T08:08:10.749876+0000","last_fullsized":"2024-05-09T08:08:10.749876+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'8","last_scrub_stamp":"2024-05-09T08:08:10.749689+0000","last_deep_scrub":"27'8","last_deep_scrub_stamp":"2024-05-09T08:08:10.749689+0000","last_clean_scrub_stamp":"2024-05-09T08:08:10.749689+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:45:45.879393+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3d","version":"27'7","reported_seq":41,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:08:05.280075+0000","last_change":"2024-05-09T08:08:05.280134+0000","last_active":"2024-05-09T08:08:05.280075+0000","last_peered":"2024-05-09T08:08:05.280075+0000","last_clean":"2024-05-09T08:08:05.280075+0000","last_became_active":"2024-05-09T08:02:15.904736+0000","last_became_peered":"2024-05-09T08:02:15.904736+0000","last_unstale":"2024-05-09T08:08:05.280075+0000","last_undegraded":"2024-05-09T08:08:05.280075+0000","last_fullsized":"2024-05-09T08:08:05.280075+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'7","last_scrub_stamp":"2024-05-09T08:08:05.279869+0000","last_deep_scrub":"27'7","last_deep_scrub_stamp":"2024-05-09T08:08:05.279869+0000","last_clean_scrub_stamp":"2024-05-09T08:08:05.279869+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:37:58.804114+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"27'4","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:08:09.737422+0000","last_change":"2024-05-09T08:08:09.737484+0000","last_active":"2024-05-09T08:08:09.737422+0000","last_peered":"2024-05-09T08:08:09.737422+0000","last_clean":"2024-05-09T08:08:09.737422+0000","last_became_active":"2024-05-09T08:02:15.897367+0000","last_became_peered":"2024-05-09T08:02:15.897367+0000","last_unstale":"2024-05-09T08:08:09.737422+0000","last_undegraded":"2024-05-09T08:08:09.737422+0000","last_fullsized":"2024-05-09T08:08:09.737422+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4","last_scrub_stamp":"2024-05-09T08:08:09.737239+0000","last_deep_scrub":"27'4","last_deep_scrub_stamp":"2024-05-09T08:08:09.737239+0000","last_clean_scrub_stamp":"2024-05-09T08:08:09.737239+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:13:55.796936+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"27'8","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T08:08:08.660598+0000","last_change":"2024-05-09T08:08:08.660659+0000","last_active":"2024-05-09T08:08:08.660598+0000","last_peered":"2024-05-09T08:08:08.660598+0000","last_clean":"2024-05-09T08:08:08.660598+0000","last_became_active":"2024-05-09T08:03:02.322397+0000","last_became_peered":"2024-05-09T08:03:02.322397+0000","last_unstale":"2024-05-09T08:08:08.660598+0000","last_undegraded":"2024-05-09T08:08:08.660598+0000","last_fullsized":"2024-05-09T08:08:08.660598+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'8","last_scrub_stamp":"2024-05-09T08:08:08.660417+0000","last_deep_scrub":"27'8","last_deep_scrub_stamp":"2024-05-09T08:08:08.660417+0000","last_clean_scrub_stamp":"2024-05-09T08:08:08.660417+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:40:32.548565+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9216,"num_write_kb":25165824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10755,"ondisk_log_size":10755,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":503,"ondisk_log_size":503,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":449011,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":282,"num_write_kb":620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":966656,"data_stored":898022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":282,"ondisk_log_size":282,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574925,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50092,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321748,"statfs":{"total":96636764160,"available":96585469952,"internally_reserved":0,"allocated":888832,"data_stored":709919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574925,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":76260,"kb_used_data":732,"kb_used_omap":1,"kb_used_meta":75518,"kb_avail":94295580,"statfs":{"total":96636764160,"available":96558673920,"internally_reserved":0,"allocated":749568,"data_stored":566653,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":77330886},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574925,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45352,"kb_used_data":288,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94326488,"statfs":{"total":96636764160,"available":96590323712,"internally_reserved":0,"allocated":294912,"data_stored":119059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574926,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":53680,"kb_used_data":296,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94318160,"statfs":{"total":96636764160,"available":96581795840,"internally_reserved":0,"allocated":303104,"data_stored":119095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":54655431},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574926,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":76264,"kb_used_data":736,"kb_used_omap":1,"kb_used_meta":75518,"kb_avail":94295576,"statfs":{"total":96636764160,"available":96558669824,"internally_reserved":0,"allocated":753664,"data_stored":567182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":77330887},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574926,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":57832,"kb_used_data":288,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94314008,"statfs":{"total":96636764160,"available":96577544192,"internally_reserved":0,"allocated":294912,"data_stored":119498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574927,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":75820,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":75518,"kb_avail":94296020,"statfs":{"total":96636764160,"available":96559124480,"internally_reserved":0,"allocated":299008,"data_stored":119129,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77330885},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574926,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45940,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325900,"statfs":{"total":96636764160,"available":96589721600,"internally_reserved":0,"allocated":897024,"data_stored":709975,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":462848,"data_stored":448213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":458752,"data_stored":447684,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T08:08:25.924 DEBUG:teuthology.orchestra.run.smithi001:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-05-09T08:08:26.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:26.152+0000 7fbccb200640 1 Processor -- start 2024-05-09T08:08:26.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:26.153+0000 7fbccb200640 1 -- start start 2024-05-09T08:08:26.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:26.154+0000 7fbccb200640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fbccc1513a0 0x7fbccc0c3070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:08:26.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:26.154+0000 7fbccb200640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbccc0c5350 con 0x7fbccc0c5df0 2024-05-09T08:08:26.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:26.154+0000 7fbccb200640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbccc0c3670 con 0x7fbccc0c7b10 2024-05-09T08:08:26.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:26.154+0000 7fbccb200640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fbccc0c3950 con 0x7fbccc1513a0 2024-05-09T08:08:26.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:26.155+0000 7fbcca800640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fbccc0c7b10 0x7fbccc0c8600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.1:38922/0 (socket says 172.21.15.1:38922) 2024-05-09T08:08:26.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:26.155+0000 7fbcca800640 1 -- 172.21.15.1:0/1846439202 learned_addr learned my addr 172.21.15.1:0/1846439202 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T08:08:26.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:26.155+0000 7fbcc9e00640 1 --2- 172.21.15.1:0/1846439202 >> v2:172.21.15.7:3300/0 conn(0x7fbccc1513a0 0x7fbccc0c3070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:08:26.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:26.155+0000 7fbcc9400640 1 -- 172.21.15.1:0/1846439202 <== mon.1 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 451777528 0 0) 0x7fbccc0c3670 con 0x7fbccc0c7b10 2024-05-09T08:08:26.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:26.155+0000 7fbcc9400640 1 -- 172.21.15.1:0/1846439202 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbcb0003520 con 0x7fbccc0c7b10 2024-05-09T08:08:26.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:26.155+0000 7fbcc9e00640 1 -- 172.21.15.1:0/1846439202 >> v1:172.21.15.7:6789/0 conn(0x7fbccc0c7b10 legacy=0x7fbccc0c8600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:08:26.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:26.155+0000 7fbcc9e00640 1 -- 172.21.15.1:0/1846439202 >> v1:172.21.15.1:6789/0 conn(0x7fbccc0c5df0 legacy=0x7fbccc157240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:08:26.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:26.155+0000 7fbcc9e00640 1 -- 172.21.15.1:0/1846439202 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbccc0c55f0 con 0x7fbccc1513a0 2024-05-09T08:08:26.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:26.157+0000 7fbcc9e00640 1 --2- 172.21.15.1:0/1846439202 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbccc1513a0 0x7fbccc0c3070 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc0097a0 tx=0x7fbcbc02a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5cf0d9041cad6ebc server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:08:26.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:26.157+0000 7fbcc9400640 1 -- 172.21.15.1:0/1846439202 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcbc037070 con 0x7fbccc1513a0 2024-05-09T08:08:26.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:26.157+0000 7fbcc9400640 1 -- 172.21.15.1:0/1846439202 <== mon.1 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbcbc032440 con 0x7fbccc1513a0 2024-05-09T08:08:26.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:26.157+0000 7fbcc9400640 1 -- 172.21.15.1:0/1846439202 <== mon.1 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcbc032810 con 0x7fbccc1513a0 2024-05-09T08:08:26.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:26.157+0000 7fbccb200640 1 -- 172.21.15.1:0/1846439202 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbccc1513a0 msgr2=0x7fbccc0c3070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:08:26.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:26.157+0000 7fbccb200640 1 --2- 172.21.15.1:0/1846439202 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbccc1513a0 0x7fbccc0c3070 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc0097a0 tx=0x7fbcbc02a230 comp rx=0 tx=0).stop 2024-05-09T08:08:26.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:26.158+0000 7fbccb200640 1 -- 172.21.15.1:0/1846439202 shutdown_connections 2024-05-09T08:08:26.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:26.158+0000 7fbccb200640 1 --2- 172.21.15.1:0/1846439202 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbccc1513a0 0x7fbccc0c3070 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:26.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:26.158+0000 7fbccb200640 1 -- 172.21.15.1:0/1846439202 >> 172.21.15.1:0/1846439202 conn(0x7fbccc06b250 msgr2=0x7fbccc06b630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:08:26.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:26.158+0000 7fbccb200640 1 -- 172.21.15.1:0/1846439202 shutdown_connections 2024-05-09T08:08:26.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:26.158+0000 7fbccb200640 1 -- 172.21.15.1:0/1846439202 wait complete. 2024-05-09T08:08:26.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:26.159+0000 7fbccb200640 1 Processor -- start 2024-05-09T08:08:26.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:26.160+0000 7fbccb200640 1 -- start start 2024-05-09T08:08:26.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:26.160+0000 7fbccb200640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbccc0c5df0 0x7fbccc1f41f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:08:26.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:26.161+0000 7fbccb200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbccc0c7b10 0x7fbccc1e6ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:08:26.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:26.161+0000 7fbccb200640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fbccc1513a0 0x7fbccc0afa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:08:26.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:26.161+0000 7fbccb200640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fbccc0c8e90 con 0x7fbccc0c5df0 2024-05-09T08:08:26.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:26.161+0000 7fbccb200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fbccc0c8d10 con 0x7fbccc0c7b10 2024-05-09T08:08:26.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:26.161+0000 7fbccb200640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7fbccc151780 con 0x7fbccc1513a0 2024-05-09T08:08:26.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:26.161+0000 7fbcc9e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbccc0c5df0 0x7fbccc1f41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:08:26.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:26.161+0000 7fbcca800640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fbccc1513a0 0x7fbccc0afa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:08:26.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:26.161+0000 7fbcc9e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbccc0c5df0 0x7fbccc1f41f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49198/0 (socket says 172.21.15.1:49198) 2024-05-09T08:08:26.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:26.161+0000 7fbcc3e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbccc0c7b10 0x7fbccc1e6ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:08:26.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:26.161+0000 7fbcca800640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fbccc1513a0 0x7fbccc0afa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3301/0 says I am v2:172.21.15.1:40436/0 (socket says 172.21.15.1:40436) 2024-05-09T08:08:26.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:26.161+0000 7fbcc9e00640 1 -- 172.21.15.1:0/3354961163 learned_addr learned my addr 172.21.15.1:0/3354961163 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:08:26.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:26.162+0000 7fbcc9e00640 1 -- 172.21.15.1:0/3354961163 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fbccc1513a0 msgr2=0x7fbccc0afa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:08:26.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:26.162+0000 7fbcc9e00640 1 --2- 172.21.15.1:0/3354961163 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fbccc1513a0 0x7fbccc0afa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:26.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:26.162+0000 7fbcc9e00640 1 -- 172.21.15.1:0/3354961163 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbccc0c7b10 msgr2=0x7fbccc1e6ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:08:26.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:26.162+0000 7fbcc9e00640 1 --2- 172.21.15.1:0/3354961163 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbccc0c7b10 0x7fbccc1e6ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:26.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:26.162+0000 7fbcc9e00640 1 -- 172.21.15.1:0/3354961163 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbccc1f05c0 con 0x7fbccc0c5df0 2024-05-09T08:08:26.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:26.162+0000 7fbcca800640 1 --2- 172.21.15.1:0/3354961163 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fbccc1513a0 0x7fbccc0afa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:08:26.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:26.162+0000 7fbcc9e00640 1 --2- 172.21.15.1:0/3354961163 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbccc0c5df0 0x7fbccc1f41f0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc0098c0 tx=0x7fbcbc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:08:26.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:26.162+0000 7fbcc2000640 1 -- 172.21.15.1:0/3354961163 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcbc037040 con 0x7fbccc0c5df0 2024-05-09T08:08:26.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:26.162+0000 7fbcc2000640 1 -- 172.21.15.1:0/3354961163 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbcbc030500 con 0x7fbccc0c5df0 2024-05-09T08:08:26.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:26.162+0000 7fbcc2000640 1 -- 172.21.15.1:0/3354961163 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcbc0308d0 con 0x7fbccc0c5df0 2024-05-09T08:08:26.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:26.162+0000 7fbccb200640 1 -- 172.21.15.1:0/3354961163 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbccc1f08a0 con 0x7fbccc0c5df0 2024-05-09T08:08:26.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:26.162+0000 7fbccb200640 1 -- 172.21.15.1:0/3354961163 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbccc1f0e60 con 0x7fbccc0c5df0 2024-05-09T08:08:26.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:26.163+0000 7fbccb200640 1 -- 172.21.15.1:0/3354961163 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbccc04f660 con 0x7fbccc0c5df0 2024-05-09T08:08:26.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:26.164+0000 7fbcc2000640 1 -- 172.21.15.1:0/3354961163 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fbcbc044020 con 0x7fbccc0c5df0 2024-05-09T08:08:26.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:26.165+0000 7fbcc2000640 1 --2- 172.21.15.1:0/3354961163 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fbca806be30 0x7fbca806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:08:26.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:26.166+0000 7fbcc2000640 1 -- 172.21.15.1:0/3354961163 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7fbcbc0ab4d0 con 0x7fbccc0c5df0 2024-05-09T08:08:26.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:26.166+0000 7fbcc3e00640 1 --2- 172.21.15.1:0/3354961163 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fbca806be30 0x7fbca806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:08:26.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:26.167+0000 7fbcc3e00640 1 --2- 172.21.15.1:0/3354961163 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fbca806be30 0x7fbca806e2f0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fbccc1edea0 tx=0x7fbcb4009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:08:26.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:26.170+0000 7fbcc2000640 1 -- 172.21.15.1:0/3354961163 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fbcbc079310 con 0x7fbccc0c5df0 2024-05-09T08:08:26.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:26.366+0000 7fbccb200640 1 -- 172.21.15.1:0/3354961163 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) -- 0x7fbccc14e850 con 0x7fbccc0c5df0 2024-05-09T08:08:26.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:26.373+0000 7fbcc2000640 1 -- 172.21.15.1:0/3354961163 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v10)=0 v10) ==== 123+0+0 (secure 0 0 0) 0x7fbcbc07d610 con 0x7fbccc0c5df0 2024-05-09T08:08:26.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:26.375+0000 7fbcc2000640 1 -- 172.21.15.1:0/3354961163 <== mon.0 v2:172.21.15.1:3300/0 8 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fbcbc081ed0 con 0x7fbccc0c5df0 2024-05-09T08:08:26.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:26.377+0000 7fbccb200640 1 -- 172.21.15.1:0/3354961163 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fbca806be30 msgr2=0x7fbca806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:08:26.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:26.377+0000 7fbccb200640 1 --2- 172.21.15.1:0/3354961163 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fbca806be30 0x7fbca806e2f0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fbccc1edea0 tx=0x7fbcb4009040 comp rx=0 tx=0).stop 2024-05-09T08:08:26.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:26.377+0000 7fbccb200640 1 -- 172.21.15.1:0/3354961163 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbccc0c5df0 msgr2=0x7fbccc1f41f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:08:26.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:26.377+0000 7fbccb200640 1 --2- 172.21.15.1:0/3354961163 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbccc0c5df0 0x7fbccc1f41f0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc0098c0 tx=0x7fbcbc0046c0 comp rx=0 tx=0).stop 2024-05-09T08:08:26.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:26.377+0000 7fbccb200640 1 -- 172.21.15.1:0/3354961163 shutdown_connections 2024-05-09T08:08:26.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:26.377+0000 7fbccb200640 1 --2- 172.21.15.1:0/3354961163 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fbca806be30 0x7fbca806e2f0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:26.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:26.377+0000 7fbccb200640 1 --2- 172.21.15.1:0/3354961163 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbccc0c7b10 0x7fbccc1e6ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:26.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:26.377+0000 7fbccb200640 1 --2- 172.21.15.1:0/3354961163 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fbccc1513a0 0x7fbccc0afa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:26.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:26.377+0000 7fbccb200640 1 --2- 172.21.15.1:0/3354961163 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbccc0c5df0 0x7fbccc1f41f0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:26.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:26.377+0000 7fbccb200640 1 -- 172.21.15.1:0/3354961163 >> 172.21.15.1:0/3354961163 conn(0x7fbccc06b250 msgr2=0x7fbccc06b630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:08:26.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:26.377+0000 7fbccb200640 1 -- 172.21.15.1:0/3354961163 shutdown_connections 2024-05-09T08:08:26.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:26.377+0000 7fbccb200640 1 -- 172.21.15.1:0/3354961163 wait complete. 2024-05-09T08:08:26.404 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-09T08:08:26.404 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T08:08:26.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:26.606+0000 7f2811e00640 1 Processor -- start 2024-05-09T08:08:26.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:26.606+0000 7f2811e00640 1 -- start start 2024-05-09T08:08:26.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:26.608+0000 7f2811e00640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f280c0cc730 0x7f280c163bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:08:26.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:26.608+0000 7f2811e00640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f280c15c790 con 0x7f280c0c8180 2024-05-09T08:08:26.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:26.608+0000 7f2811e00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f280c0c6a30 con 0x7f280c152500 2024-05-09T08:08:26.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:26.608+0000 7f2811e00640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f280c0ccb80 con 0x7f280c0cc730 2024-05-09T08:08:26.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:26.608+0000 7f2811400640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f280c0cc730 0x7f280c163bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:08:26.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:26.608+0000 7f2810a00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f280c152500 0x7f280c15c080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.1:38926/0 (socket says 172.21.15.1:38926) 2024-05-09T08:08:26.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:26.608+0000 7f2810a00640 1 -- 172.21.15.1:0/706396227 learned_addr learned my addr 172.21.15.1:0/706396227 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T08:08:26.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:26.609+0000 7f280b400640 1 -- 172.21.15.1:0/706396227 <== mon.1 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3312082661 0 0) 0x7f280c0c6a30 con 0x7f280c152500 2024-05-09T08:08:26.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:26.609+0000 7f2811400640 1 -- 172.21.15.1:0/706396227 >> v1:172.21.15.7:6789/0 conn(0x7f280c152500 legacy=0x7f280c15c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:08:26.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:26.609+0000 7f2811400640 1 -- 172.21.15.1:0/706396227 >> v1:172.21.15.1:6789/0 conn(0x7f280c0c8180 legacy=0x7f280c0c7660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:08:26.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:26.609+0000 7f2811400640 1 -- 172.21.15.1:0/706396227 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f280c0c6cf0 con 0x7f280c0cc730 2024-05-09T08:08:26.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:26.610+0000 7f2811400640 1 --2- 172.21.15.1:0/706396227 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f280c0cc730 0x7f280c163bc0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f2804009d90 tx=0x7f280402a780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c50e68a955e3111 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:08:26.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:26.610+0000 7f280b400640 1 -- 172.21.15.1:0/706396227 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2804037070 con 0x7f280c0cc730 2024-05-09T08:08:26.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:26.610+0000 7f280b400640 1 -- 172.21.15.1:0/706396227 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f2804007740 con 0x7f280c0cc730 2024-05-09T08:08:26.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:26.611+0000 7f280b400640 1 -- 172.21.15.1:0/706396227 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2804007b30 con 0x7f280c0cc730 2024-05-09T08:08:26.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:26.611+0000 7f2811e00640 1 -- 172.21.15.1:0/706396227 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f280c0cc730 msgr2=0x7f280c163bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:08:26.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:26.611+0000 7f2811e00640 1 --2- 172.21.15.1:0/706396227 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f280c0cc730 0x7f280c163bc0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f2804009d90 tx=0x7f280402a780 comp rx=0 tx=0).stop 2024-05-09T08:08:26.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:26.611+0000 7f2811e00640 1 -- 172.21.15.1:0/706396227 shutdown_connections 2024-05-09T08:08:26.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:26.611+0000 7f2811e00640 1 --2- 172.21.15.1:0/706396227 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f280c0cc730 0x7f280c163bc0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:26.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:26.611+0000 7f2811e00640 1 -- 172.21.15.1:0/706396227 >> 172.21.15.1:0/706396227 conn(0x7f280c0cbc10 msgr2=0x7f280c0cc020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:08:26.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:26.611+0000 7f2811e00640 1 -- 172.21.15.1:0/706396227 shutdown_connections 2024-05-09T08:08:26.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:26.611+0000 7f2811e00640 1 -- 172.21.15.1:0/706396227 wait complete. 2024-05-09T08:08:26.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:26.613+0000 7f2811e00640 1 Processor -- start 2024-05-09T08:08:26.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:26.613+0000 7f2811e00640 1 -- start start 2024-05-09T08:08:26.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:26.614+0000 7f2811e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f280c0c8180 0x7f280c1e3b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:08:26.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:26.614+0000 7f2811e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f280c0cc730 0x7f280c1e7140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:08:26.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:26.614+0000 7f2811e00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f280c152500 0x7f280c14d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:08:26.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:26.614+0000 7f2811e00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f280c0c22d0 con 0x7f280c0c8180 2024-05-09T08:08:26.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:26.614+0000 7f2811e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f280c0c72b0 con 0x7f280c0cc730 2024-05-09T08:08:26.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:26.614+0000 7f2811e00640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f280c15b880 con 0x7f280c152500 2024-05-09T08:08:26.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:26.614+0000 7f2810a00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f280c0c8180 0x7f280c1e3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:08:26.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:26.614+0000 7f2810a00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f280c0c8180 0x7f280c1e3b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49210/0 (socket says 172.21.15.1:49210) 2024-05-09T08:08:26.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:26.614+0000 7f2810a00640 1 -- 172.21.15.1:0/2407793725 learned_addr learned my addr 172.21.15.1:0/2407793725 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:08:26.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:26.614+0000 7f280be00640 1 --2- 172.21.15.1:0/2407793725 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f280c0cc730 0x7f280c1e7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:08:26.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:26.615+0000 7f2811400640 1 --2- 172.21.15.1:0/2407793725 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f280c152500 0x7f280c14d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:08:26.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:26.615+0000 7f2810a00640 1 -- 172.21.15.1:0/2407793725 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f280c152500 msgr2=0x7f280c14d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:08:26.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:26.615+0000 7f2810a00640 1 --2- 172.21.15.1:0/2407793725 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f280c152500 0x7f280c14d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:26.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:26.615+0000 7f2810a00640 1 -- 172.21.15.1:0/2407793725 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f280c0cc730 msgr2=0x7f280c1e7140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:08:26.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:26.615+0000 7f2810a00640 1 --2- 172.21.15.1:0/2407793725 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f280c0cc730 0x7f280c1e7140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:26.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:26.615+0000 7f2810a00640 1 -- 172.21.15.1:0/2407793725 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f280c1e43e0 con 0x7f280c0c8180 2024-05-09T08:08:26.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:26.615+0000 7f2810a00640 1 --2- 172.21.15.1:0/2407793725 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f280c0c8180 0x7f280c1e3b80 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f280000c4a0 tx=0x7f280000c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:08:26.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:26.616+0000 7f2808c00640 1 -- 172.21.15.1:0/2407793725 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2800018070 con 0x7f280c0c8180 2024-05-09T08:08:26.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:26.616+0000 7f2808c00640 1 -- 172.21.15.1:0/2407793725 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f28000074e0 con 0x7f280c0c8180 2024-05-09T08:08:26.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:26.616+0000 7f2808c00640 1 -- 172.21.15.1:0/2407793725 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2800004a40 con 0x7f280c0c8180 2024-05-09T08:08:26.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:26.616+0000 7f2811e00640 1 -- 172.21.15.1:0/2407793725 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f280c15a770 con 0x7f280c0c8180 2024-05-09T08:08:26.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:26.616+0000 7f2811e00640 1 -- 172.21.15.1:0/2407793725 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f280c15ad30 con 0x7f280c0c8180 2024-05-09T08:08:26.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:26.617+0000 7f2811e00640 1 -- 172.21.15.1:0/2407793725 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f280c157f20 con 0x7f280c0c8180 2024-05-09T08:08:26.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:26.618+0000 7f2808c00640 1 -- 172.21.15.1:0/2407793725 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f280001b070 con 0x7f280c0c8180 2024-05-09T08:08:26.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:26.619+0000 7f2808c00640 1 --2- 172.21.15.1:0/2407793725 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f27ec06be30 0x7f27ec06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:08:26.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:26.619+0000 7f2808c00640 1 -- 172.21.15.1:0/2407793725 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7f280008a5e0 con 0x7f280c0c8180 2024-05-09T08:08:26.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:26.619+0000 7f280be00640 1 --2- 172.21.15.1:0/2407793725 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f27ec06be30 0x7f27ec06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:08:26.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:26.620+0000 7f280be00640 1 --2- 172.21.15.1:0/2407793725 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f27ec06be30 0x7f27ec06e2f0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f280c1e79f0 tx=0x7f27f8009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:08:26.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:26.623+0000 7f2808c00640 1 -- 172.21.15.1:0/2407793725 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f280c157f20 con 0x7f280c0c8180 2024-05-09T08:08:26.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:26.867+0000 7f2811e00640 1 -- 172.21.15.1:0/2407793725 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f280c06f1c0 con 0x7f280c0c8180 2024-05-09T08:08:26.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:26.868+0000 7f2808c00640 1 -- 172.21.15.1:0/2407793725 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+6807 (secure 0 0 0) 0x7f280c06f1c0 con 0x7f280c0c8180 2024-05-09T08:08:26.878 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T08:08:26.878 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":10,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4220,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/1157980939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":1157980939},{"type":"v1","addr":"172.21.15.7:6837","nonce":1157980939}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4235,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6833/3269798795","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6832","nonce":3269798795},{"type":"v1","addr":"172.21.15.1:6833","nonce":3269798795}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4241,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/725115499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6835","nonce":725115499},{"type":"v1","addr":"172.21.15.1:6837","nonce":725115499}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4276,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6838/1827115274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1827115274},{"type":"v1","addr":"172.21.15.7:6838","nonce":1827115274}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/1032767352","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":1032767352},{"type":"v1","addr":"172.21.15.7:6839","nonce":1032767352}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":10,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-09T08:02:15.332696+0000","modified":"2024-05-09T08:02:49.734561+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4294},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4294":{"gid":4294,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":6,"addr":"172.21.15.1:6836/1509134635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":1509134635},{"type":"v1","addr":"172.21.15.1:6836","nonce":1509134635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4294,"qdb_cluster":[4294]},"id":1}]} 2024-05-09T08:08:26.878 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 10 2024-05-09T08:08:26.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:26.870+0000 7f2811e00640 1 -- 172.21.15.1:0/2407793725 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f27ec06be30 msgr2=0x7f27ec06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:08:26.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:26.870+0000 7f2811e00640 1 --2- 172.21.15.1:0/2407793725 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f27ec06be30 0x7f27ec06e2f0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f280c1e79f0 tx=0x7f27f8009040 comp rx=0 tx=0).stop 2024-05-09T08:08:26.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:26.870+0000 7f2811e00640 1 -- 172.21.15.1:0/2407793725 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f280c0c8180 msgr2=0x7f280c1e3b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:08:26.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:26.870+0000 7f2811e00640 1 --2- 172.21.15.1:0/2407793725 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f280c0c8180 0x7f280c1e3b80 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f280000c4a0 tx=0x7f280000c960 comp rx=0 tx=0).stop 2024-05-09T08:08:26.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:26.870+0000 7f2811e00640 1 -- 172.21.15.1:0/2407793725 shutdown_connections 2024-05-09T08:08:26.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:26.871+0000 7f2811e00640 1 --2- 172.21.15.1:0/2407793725 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f27ec06be30 0x7f27ec06e2f0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:26.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:26.871+0000 7f2811e00640 1 --2- 172.21.15.1:0/2407793725 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f280c0cc730 0x7f280c1e7140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:26.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:26.871+0000 7f2811e00640 1 --2- 172.21.15.1:0/2407793725 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f280c152500 0x7f280c14d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:26.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:26.871+0000 7f2811e00640 1 --2- 172.21.15.1:0/2407793725 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f280c0c8180 0x7f280c1e3b80 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:26.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:26.871+0000 7f2811e00640 1 -- 172.21.15.1:0/2407793725 >> 172.21.15.1:0/2407793725 conn(0x7f280c0cbc10 msgr2=0x7f280c157810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:08:26.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:26.871+0000 7f2811e00640 1 -- 172.21.15.1:0/2407793725 shutdown_connections 2024-05-09T08:08:26.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:26.871+0000 7f2811e00640 1 -- 172.21.15.1:0/2407793725 wait complete. 2024-05-09T08:08:26.901 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T08:08:27.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:27.105+0000 7f19d1200640 1 Processor -- start 2024-05-09T08:08:27.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:27.105+0000 7f19d1200640 1 -- start start 2024-05-09T08:08:27.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:27.107+0000 7f19d1200640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f19cc06dca0 0x7f19cc072900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:08:27.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:27.107+0000 7f19d1200640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f19cc0c6ea0 0x7f19cc159010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:08:27.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:27.107+0000 7f19d1200640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f19cc073200 con 0x7f19cc06ece0 2024-05-09T08:08:27.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:27.107+0000 7f19d1200640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f19cc072e40 con 0x7f19cc0c6ea0 2024-05-09T08:08:27.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:27.107+0000 7f19d1200640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f19cc0737c0 con 0x7f19cc06dca0 2024-05-09T08:08:27.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:27.107+0000 7f19c9400640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f19cc0c6ea0 0x7f19cc159010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:08:27.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:27.107+0000 7f19ca800640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f19cc06ece0 0x7f19cc06e080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.1:38940/0 (socket says 172.21.15.1:38940) 2024-05-09T08:08:27.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:27.107+0000 7f19c9e00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f19cc06dca0 0x7f19cc072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:08:27.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:27.107+0000 7f19ca800640 1 -- 172.21.15.1:0/2813235824 learned_addr learned my addr 172.21.15.1:0/2813235824 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T08:08:27.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:27.107+0000 7f19c9400640 1 --2- 172.21.15.1:0/2813235824 >> v2:172.21.15.1:3300/0 conn(0x7f19cc0c6ea0 0x7f19cc159010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49220/0 (socket says 172.21.15.1:49220) 2024-05-09T08:08:27.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:27.108+0000 7f19c8a00640 1 -- 172.21.15.1:0/2813235824 <== mon.1 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2758452969 0 0) 0x7f19cc073200 con 0x7f19cc06ece0 2024-05-09T08:08:27.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:27.108+0000 7f19c9400640 1 -- 172.21.15.1:0/2813235824 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f19cc06dca0 msgr2=0x7f19cc072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:08:27.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:27.108+0000 7f19c9400640 1 --2- 172.21.15.1:0/2813235824 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f19cc06dca0 0x7f19cc072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:27.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:27.108+0000 7f19c9400640 1 -- 172.21.15.1:0/2813235824 >> v1:172.21.15.7:6789/0 conn(0x7f19cc06ece0 legacy=0x7f19cc06e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:08:27.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:27.108+0000 7f19c9400640 1 -- 172.21.15.1:0/2813235824 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19cc0734e0 con 0x7f19cc0c6ea0 2024-05-09T08:08:27.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:27.108+0000 7f19c9e00640 1 --2- 172.21.15.1:0/2813235824 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f19cc06dca0 0x7f19cc072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:08:27.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:27.109+0000 7f19c9400640 1 --2- 172.21.15.1:0/2813235824 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f19cc0c6ea0 0x7f19cc159010 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f19b80097a0 tx=0x7f19b802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7054f839f3d1607 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:08:27.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:27.109+0000 7f19c8a00640 1 -- 172.21.15.1:0/2813235824 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19b8037070 con 0x7f19cc0c6ea0 2024-05-09T08:08:27.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:27.109+0000 7f19c8a00640 1 -- 172.21.15.1:0/2813235824 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f19b8032440 con 0x7f19cc0c6ea0 2024-05-09T08:08:27.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:27.109+0000 7f19c8a00640 1 -- 172.21.15.1:0/2813235824 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19b8032830 con 0x7f19cc0c6ea0 2024-05-09T08:08:27.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:27.110+0000 7f19d1200640 1 -- 172.21.15.1:0/2813235824 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f19cc0c6ea0 msgr2=0x7f19cc159010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:08:27.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:27.110+0000 7f19d1200640 1 --2- 172.21.15.1:0/2813235824 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f19cc0c6ea0 0x7f19cc159010 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f19b80097a0 tx=0x7f19b802a210 comp rx=0 tx=0).stop 2024-05-09T08:08:27.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:27.110+0000 7f19d1200640 1 -- 172.21.15.1:0/2813235824 shutdown_connections 2024-05-09T08:08:27.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:27.110+0000 7f19d1200640 1 --2- 172.21.15.1:0/2813235824 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f19cc0c6ea0 0x7f19cc159010 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:27.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:27.110+0000 7f19d1200640 1 --2- 172.21.15.1:0/2813235824 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f19cc06dca0 0x7f19cc072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:27.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:27.110+0000 7f19d1200640 1 -- 172.21.15.1:0/2813235824 >> 172.21.15.1:0/2813235824 conn(0x7f19cc0cc070 msgr2=0x7f19cc0cc480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:08:27.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:27.110+0000 7f19d1200640 1 -- 172.21.15.1:0/2813235824 shutdown_connections 2024-05-09T08:08:27.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:27.110+0000 7f19d1200640 1 -- 172.21.15.1:0/2813235824 wait complete. 2024-05-09T08:08:27.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:27.112+0000 7f19d1200640 1 Processor -- start 2024-05-09T08:08:27.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:27.112+0000 7f19d1200640 1 -- start start 2024-05-09T08:08:27.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:27.113+0000 7f19d1200640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f19cc06dca0 0x7f19cc150f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:08:27.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:27.113+0000 7f19d1200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f19cc06ece0 0x7f19cc0c9f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:08:27.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:27.113+0000 7f19d1200640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f19cc0c6ea0 0x7f19cc0ca470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:08:27.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:27.113+0000 7f19d1200640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f19cc072470 con 0x7f19cc06dca0 2024-05-09T08:08:27.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:27.113+0000 7f19d1200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f19cc159550 con 0x7f19cc06ece0 2024-05-09T08:08:27.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:27.113+0000 7f19d1200640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f19cc072100 con 0x7f19cc0c6ea0 2024-05-09T08:08:27.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:27.113+0000 7f19c9e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f19cc06dca0 0x7f19cc150f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:08:27.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:27.113+0000 7f19c9e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f19cc06dca0 0x7f19cc150f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49222/0 (socket says 172.21.15.1:49222) 2024-05-09T08:08:27.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:27.113+0000 7f19c9e00640 1 -- 172.21.15.1:0/3635516186 learned_addr learned my addr 172.21.15.1:0/3635516186 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:08:27.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:27.114+0000 7f19c9400640 1 --2- 172.21.15.1:0/3635516186 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f19cc06ece0 0x7f19cc0c9f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:08:27.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:27.114+0000 7f19ca800640 1 --2- 172.21.15.1:0/3635516186 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f19cc0c6ea0 0x7f19cc0ca470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:08:27.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:27.114+0000 7f19c9e00640 1 -- 172.21.15.1:0/3635516186 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f19cc0c6ea0 msgr2=0x7f19cc0ca470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:08:27.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:27.114+0000 7f19c9e00640 1 --2- 172.21.15.1:0/3635516186 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f19cc0c6ea0 0x7f19cc0ca470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:27.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:27.114+0000 7f19c9e00640 1 -- 172.21.15.1:0/3635516186 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f19cc06ece0 msgr2=0x7f19cc0c9f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:08:27.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:27.114+0000 7f19c9e00640 1 --2- 172.21.15.1:0/3635516186 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f19cc06ece0 0x7f19cc0c9f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:27.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:27.114+0000 7f19c9e00640 1 -- 172.21.15.1:0/3635516186 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19cc0c46b0 con 0x7f19cc06dca0 2024-05-09T08:08:27.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:27.114+0000 7f19c9e00640 1 --2- 172.21.15.1:0/3635516186 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f19cc06dca0 0x7f19cc150f30 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f19bc00ebe0 tx=0x7f19bc00c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:08:27.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:27.114+0000 7f19c2000640 1 -- 172.21.15.1:0/3635516186 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19bc017070 con 0x7f19cc06dca0 2024-05-09T08:08:27.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:27.115+0000 7f19c2000640 1 -- 172.21.15.1:0/3635516186 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f19bc00cc30 con 0x7f19cc06dca0 2024-05-09T08:08:27.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:27.115+0000 7f19c2000640 1 -- 172.21.15.1:0/3635516186 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19bc013470 con 0x7f19cc06dca0 2024-05-09T08:08:27.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:27.115+0000 7f19d1200640 1 -- 172.21.15.1:0/3635516186 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19cc0c4920 con 0x7f19cc06dca0 2024-05-09T08:08:27.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:27.115+0000 7f19d1200640 1 -- 172.21.15.1:0/3635516186 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19cc0c4ee0 con 0x7f19cc06dca0 2024-05-09T08:08:27.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:27.116+0000 7f19d1200640 1 -- 172.21.15.1:0/3635516186 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19cc0711a0 con 0x7f19cc06dca0 2024-05-09T08:08:27.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:27.117+0000 7f19c2000640 1 -- 172.21.15.1:0/3635516186 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f19bc01a070 con 0x7f19cc06dca0 2024-05-09T08:08:27.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:27.118+0000 7f19c2000640 1 --2- 172.21.15.1:0/3635516186 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f19ac06be30 0x7f19ac06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:08:27.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:27.118+0000 7f19c2000640 1 -- 172.21.15.1:0/3635516186 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7f19bc08b6a0 con 0x7f19cc06dca0 2024-05-09T08:08:27.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:27.118+0000 7f19c9400640 1 --2- 172.21.15.1:0/3635516186 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f19ac06be30 0x7f19ac06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:08:27.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:27.119+0000 7f19c9400640 1 --2- 172.21.15.1:0/3635516186 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f19ac06be30 0x7f19ac06e2f0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f19b8034000 tx=0x7f19b802e000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:08:27.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:27.123+0000 7f19c2000640 1 -- 172.21.15.1:0/3635516186 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f19cc0711a0 con 0x7f19cc06dca0 2024-05-09T08:08:27.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:27.368+0000 7f19d1200640 1 -- 172.21.15.1:0/3635516186 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f19cc06d760 con 0x7f19cc06dca0 2024-05-09T08:08:27.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:27.370+0000 7f19c2000640 1 -- 172.21.15.1:0/3635516186 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+6807 (secure 0 0 0) 0x7f19cc06d760 con 0x7f19cc06dca0 2024-05-09T08:08:27.379 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T08:08:27.379 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":10,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4220,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/1157980939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":1157980939},{"type":"v1","addr":"172.21.15.7:6837","nonce":1157980939}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4235,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6833/3269798795","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6832","nonce":3269798795},{"type":"v1","addr":"172.21.15.1:6833","nonce":3269798795}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4241,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/725115499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6835","nonce":725115499},{"type":"v1","addr":"172.21.15.1:6837","nonce":725115499}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4276,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6838/1827115274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1827115274},{"type":"v1","addr":"172.21.15.7:6838","nonce":1827115274}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4279,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/1032767352","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":1032767352},{"type":"v1","addr":"172.21.15.7:6839","nonce":1032767352}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":10,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-09T08:02:15.332696+0000","modified":"2024-05-09T08:02:49.734561+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4294},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4294":{"gid":4294,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":6,"addr":"172.21.15.1:6836/1509134635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":1509134635},{"type":"v1","addr":"172.21.15.1:6836","nonce":1509134635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4294,"qdb_cluster":[4294]},"id":1}]} 2024-05-09T08:08:27.380 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 10 2024-05-09T08:08:27.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:27.373+0000 7f19d1200640 1 -- 172.21.15.1:0/3635516186 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f19ac06be30 msgr2=0x7f19ac06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:08:27.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:27.373+0000 7f19d1200640 1 --2- 172.21.15.1:0/3635516186 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f19ac06be30 0x7f19ac06e2f0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f19b8034000 tx=0x7f19b802e000 comp rx=0 tx=0).stop 2024-05-09T08:08:27.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:27.373+0000 7f19d1200640 1 -- 172.21.15.1:0/3635516186 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f19cc06dca0 msgr2=0x7f19cc150f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:08:27.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:27.373+0000 7f19d1200640 1 --2- 172.21.15.1:0/3635516186 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f19cc06dca0 0x7f19cc150f30 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f19bc00ebe0 tx=0x7f19bc00c520 comp rx=0 tx=0).stop 2024-05-09T08:08:27.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:27.373+0000 7f19d1200640 1 -- 172.21.15.1:0/3635516186 shutdown_connections 2024-05-09T08:08:27.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:27.373+0000 7f19d1200640 1 --2- 172.21.15.1:0/3635516186 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f19ac06be30 0x7f19ac06e2f0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:27.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:27.373+0000 7f19d1200640 1 --2- 172.21.15.1:0/3635516186 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f19cc06ece0 0x7f19cc0c9f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:27.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:27.373+0000 7f19d1200640 1 --2- 172.21.15.1:0/3635516186 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f19cc0c6ea0 0x7f19cc0ca470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:27.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:27.373+0000 7f19d1200640 1 --2- 172.21.15.1:0/3635516186 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f19cc06dca0 0x7f19cc150f30 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:27.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:27.373+0000 7f19d1200640 1 -- 172.21.15.1:0/3635516186 >> 172.21.15.1:0/3635516186 conn(0x7f19cc0cc070 msgr2=0x7f19cc1e2940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:08:27.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:27.373+0000 7f19d1200640 1 -- 172.21.15.1:0/3635516186 shutdown_connections 2024-05-09T08:08:27.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:27.373+0000 7f19d1200640 1 -- 172.21.15.1:0/3635516186 wait complete. 2024-05-09T08:08:27.402 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T08:08:27.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:27.608+0000 7ff901600640 1 Processor -- start 2024-05-09T08:08:27.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:27.608+0000 7ff901600640 1 -- start start 2024-05-09T08:08:27.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:27.610+0000 7ff901600640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff8fc157b30 0x7ff8fc0cd0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:08:27.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:27.610+0000 7ff901600640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff8fc0c9750 con 0x7ff8fc15e1e0 2024-05-09T08:08:27.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:27.610+0000 7ff901600640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff8fc0c9a30 con 0x7ff8fc0c7620 2024-05-09T08:08:27.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:27.610+0000 7ff901600640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7ff8fc157f10 con 0x7ff8fc157b30 2024-05-09T08:08:27.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:27.610+0000 7ff8fbe00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff8fc157b30 0x7ff8fc0cd0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:08:27.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:27.610+0000 7ff8f3400640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7ff8fc0c7620 0x7ff8fc15dad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.1:38946/0 (socket says 172.21.15.1:38946) 2024-05-09T08:08:27.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:27.610+0000 7ff8f3400640 1 -- 172.21.15.1:0/355636141 learned_addr learned my addr 172.21.15.1:0/355636141 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T08:08:27.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:27.610+0000 7ff900c00640 1 --1- 172.21.15.1:0/355636141 >> v1:172.21.15.1:6789/0 conn(0x7ff8fc15e1e0 0x7ff8fc164d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:38810/0 (socket says 172.21.15.1:38810) 2024-05-09T08:08:27.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:27.611+0000 7ff8fbe00640 1 -- 172.21.15.1:0/355636141 >> v1:172.21.15.7:6789/0 conn(0x7ff8fc0c7620 legacy=0x7ff8fc15dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:08:27.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:27.611+0000 7ff8fbe00640 1 -- 172.21.15.1:0/355636141 >> v1:172.21.15.1:6789/0 conn(0x7ff8fc15e1e0 legacy=0x7ff8fc164d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:08:27.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:27.611+0000 7ff8fbe00640 1 -- 172.21.15.1:0/355636141 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8fc0c9d10 con 0x7ff8fc157b30 2024-05-09T08:08:27.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:27.612+0000 7ff8fbe00640 1 --2- 172.21.15.1:0/355636141 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff8fc157b30 0x7ff8fc0cd0e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ff8fc158090 tx=0x7ff8ec02a780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7ae1b25d03852de server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:08:27.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:27.613+0000 7ff8fb400640 1 -- 172.21.15.1:0/355636141 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8ec037070 con 0x7ff8fc157b30 2024-05-09T08:08:27.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:27.613+0000 7ff8fb400640 1 -- 172.21.15.1:0/355636141 <== mon.1 v2:172.21.15.7:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7ff8ec007740 con 0x7ff8fc157b30 2024-05-09T08:08:27.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:27.613+0000 7ff8fb400640 1 -- 172.21.15.1:0/355636141 <== mon.1 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8ec007b30 con 0x7ff8fc157b30 2024-05-09T08:08:27.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:27.614+0000 7ff901600640 1 -- 172.21.15.1:0/355636141 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff8fc157b30 msgr2=0x7ff8fc0cd0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:08:27.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:27.614+0000 7ff901600640 1 --2- 172.21.15.1:0/355636141 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff8fc157b30 0x7ff8fc0cd0e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ff8fc158090 tx=0x7ff8ec02a780 comp rx=0 tx=0).stop 2024-05-09T08:08:27.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:27.614+0000 7ff901600640 1 -- 172.21.15.1:0/355636141 shutdown_connections 2024-05-09T08:08:27.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:27.614+0000 7ff901600640 1 --2- 172.21.15.1:0/355636141 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff8fc157b30 0x7ff8fc0cd0e0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:27.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:27.614+0000 7ff901600640 1 -- 172.21.15.1:0/355636141 >> 172.21.15.1:0/355636141 conn(0x7ff8fc06e850 msgr2=0x7ff8fc06ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:08:27.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:27.614+0000 7ff901600640 1 -- 172.21.15.1:0/355636141 shutdown_connections 2024-05-09T08:08:27.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:27.614+0000 7ff901600640 1 -- 172.21.15.1:0/355636141 wait complete. 2024-05-09T08:08:27.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:27.616+0000 7ff901600640 1 Processor -- start 2024-05-09T08:08:27.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:27.616+0000 7ff901600640 1 -- start start 2024-05-09T08:08:27.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:27.617+0000 7ff901600640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff8fc0c7620 0x7ff8fc147ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:08:27.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:27.617+0000 7ff901600640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ff8fc157b30 0x7ff8fc15c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:08:27.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:27.617+0000 7ff901600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff8fc15e1e0 0x7ff8fc15cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:08:27.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:27.617+0000 7ff901600640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ff8fc0c7b80 con 0x7ff8fc0c7620 2024-05-09T08:08:27.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:27.617+0000 7ff901600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff8fc0ca2d0 con 0x7ff8fc15e1e0 2024-05-09T08:08:27.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:27.617+0000 7ff901600640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7ff8fc06c620 con 0x7ff8fc157b30 2024-05-09T08:08:27.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:27.618+0000 7ff8fbe00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff8fc0c7620 0x7ff8fc147ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:08:27.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:27.618+0000 7ff8fbe00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff8fc0c7620 0x7ff8fc147ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49228/0 (socket says 172.21.15.1:49228) 2024-05-09T08:08:27.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:27.618+0000 7ff8f3400640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ff8fc157b30 0x7ff8fc15c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:08:27.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:27.618+0000 7ff8fbe00640 1 -- 172.21.15.1:0/844589480 learned_addr learned my addr 172.21.15.1:0/844589480 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:08:27.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:27.618+0000 7ff900c00640 1 --2- 172.21.15.1:0/844589480 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff8fc15e1e0 0x7ff8fc15cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:08:27.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:27.618+0000 7ff8fbe00640 1 -- 172.21.15.1:0/844589480 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ff8fc157b30 msgr2=0x7ff8fc15c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:08:27.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:27.618+0000 7ff8fbe00640 1 --2- 172.21.15.1:0/844589480 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ff8fc157b30 0x7ff8fc15c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:27.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:27.618+0000 7ff8fbe00640 1 -- 172.21.15.1:0/844589480 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff8fc15e1e0 msgr2=0x7ff8fc15cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:08:27.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:27.618+0000 7ff8fbe00640 1 --2- 172.21.15.1:0/844589480 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff8fc15e1e0 0x7ff8fc15cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:27.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:27.618+0000 7ff8fbe00640 1 -- 172.21.15.1:0/844589480 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8fc0730e0 con 0x7ff8fc0c7620 2024-05-09T08:08:27.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:27.618+0000 7ff8f3400640 1 --2- 172.21.15.1:0/844589480 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ff8fc157b30 0x7ff8fc15c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:08:27.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:27.619+0000 7ff8fbe00640 1 --2- 172.21.15.1:0/844589480 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff8fc0c7620 0x7ff8fc147ac0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7ff8ec02ad70 tx=0x7ff8ec004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:08:27.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:27.619+0000 7ff8f8c00640 1 -- 172.21.15.1:0/844589480 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8ec042020 con 0x7ff8fc0c7620 2024-05-09T08:08:27.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:27.619+0000 7ff8f8c00640 1 -- 172.21.15.1:0/844589480 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7ff8ec032a90 con 0x7ff8fc0c7620 2024-05-09T08:08:27.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:27.619+0000 7ff8f8c00640 1 -- 172.21.15.1:0/844589480 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8ec037040 con 0x7ff8fc0c7620 2024-05-09T08:08:27.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:27.619+0000 7ff901600640 1 -- 172.21.15.1:0/844589480 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8fc15c4d0 con 0x7ff8fc0c7620 2024-05-09T08:08:27.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:27.619+0000 7ff901600640 1 -- 172.21.15.1:0/844589480 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8fc1f3c30 con 0x7ff8fc0c7620 2024-05-09T08:08:27.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:27.620+0000 7ff901600640 1 -- 172.21.15.1:0/844589480 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8fc15c6a0 con 0x7ff8fc0c7620 2024-05-09T08:08:27.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:27.621+0000 7ff8f8c00640 1 -- 172.21.15.1:0/844589480 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff8ec045020 con 0x7ff8fc0c7620 2024-05-09T08:08:27.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:27.622+0000 7ff8f8c00640 1 --2- 172.21.15.1:0/844589480 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7ff8d806be30 0x7ff8d806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:08:27.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:27.622+0000 7ff8f8c00640 1 -- 172.21.15.1:0/844589480 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7ff8ec0ac360 con 0x7ff8fc0c7620 2024-05-09T08:08:27.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:27.623+0000 7ff8f3400640 1 --2- 172.21.15.1:0/844589480 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7ff8d806be30 0x7ff8d806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:08:27.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:27.623+0000 7ff8f3400640 1 --2- 172.21.15.1:0/844589480 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7ff8d806be30 0x7ff8d806e2f0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7ff8fc071200 tx=0x7ff8e4009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:08:27.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:27.627+0000 7ff8f8c00640 1 -- 172.21.15.1:0/844589480 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ff8ec04c070 con 0x7ff8fc0c7620 2024-05-09T08:08:27.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:27.832+0000 7ff901600640 1 -- 172.21.15.1:0/844589480 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff8fc0c7dd0 con 0x7ff8fc0c7620 2024-05-09T08:08:27.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:27.835+0000 7ff8f8c00640 1 -- 172.21.15.1:0/844589480 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18741 (secure 0 0 0) 0x7ff8ec07a1a0 con 0x7ff8fc0c7620 2024-05-09T08:08:27.845 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-09T08:08:27.845 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":27,"fsid":"54329ec8-3270-4043-9a82-c6cd7420aa7f","created":"2024-05-09T08:01:56.666970+0000","modified":"2024-05-09T08:03:02.304156+0000","last_up_change":"2024-05-09T08:02:07.146082+0000","last_in_change":"2024-05-09T08:01:59.008225+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T08:02:09.165511+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T08:02:12.925485+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T08:02:14.367455+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-09T08:02:17.967064+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c064454e-a9b5-4a24-9a77-17f0a4085952","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6800","nonce":1513349547},{"type":"v1","addr":"172.21.15.1:6801","nonce":1513349547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":1513349547},{"type":"v1","addr":"172.21.15.1:6803","nonce":1513349547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":1513349547},{"type":"v1","addr":"172.21.15.1:6807","nonce":1513349547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":1513349547},{"type":"v1","addr":"172.21.15.1:6805","nonce":1513349547}]},"public_addr":"172.21.15.1:6801/1513349547","cluster_addr":"172.21.15.1:6803/1513349547","heartbeat_back_addr":"172.21.15.1:6807/1513349547","heartbeat_front_addr":"172.21.15.1:6805/1513349547","state":["exists","up"]},{"osd":1,"uuid":"1a9ea687-2ff6-407a-949b-0e7e1083af21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":2942255259},{"type":"v1","addr":"172.21.15.1:6817","nonce":2942255259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":2942255259},{"type":"v1","addr":"172.21.15.1:6819","nonce":2942255259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":2942255259},{"type":"v1","addr":"172.21.15.1:6823","nonce":2942255259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6820","nonce":2942255259},{"type":"v1","addr":"172.21.15.1:6821","nonce":2942255259}]},"public_addr":"172.21.15.1:6817/2942255259","cluster_addr":"172.21.15.1:6819/2942255259","heartbeat_back_addr":"172.21.15.1:6823/2942255259","heartbeat_front_addr":"172.21.15.1:6821/2942255259","state":["exists","up"]},{"osd":2,"uuid":"c88af08b-7c9a-4dd8-9640-44138239addb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":2925031811},{"type":"v1","addr":"172.21.15.1:6809","nonce":2925031811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":2925031811},{"type":"v1","addr":"172.21.15.1:6811","nonce":2925031811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":2925031811},{"type":"v1","addr":"172.21.15.1:6815","nonce":2925031811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":2925031811},{"type":"v1","addr":"172.21.15.1:6813","nonce":2925031811}]},"public_addr":"172.21.15.1:6809/2925031811","cluster_addr":"172.21.15.1:6811/2925031811","heartbeat_back_addr":"172.21.15.1:6815/2925031811","heartbeat_front_addr":"172.21.15.1:6813/2925031811","state":["exists","up"]},{"osd":3,"uuid":"c087b7e3-a41d-4333-9365-32e3bd8c8e6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6824","nonce":2497776654},{"type":"v1","addr":"172.21.15.1:6825","nonce":2497776654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":2497776654},{"type":"v1","addr":"172.21.15.1:6827","nonce":2497776654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":2497776654},{"type":"v1","addr":"172.21.15.1:6831","nonce":2497776654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6828","nonce":2497776654},{"type":"v1","addr":"172.21.15.1:6829","nonce":2497776654}]},"public_addr":"172.21.15.1:6825/2497776654","cluster_addr":"172.21.15.1:6827/2497776654","heartbeat_back_addr":"172.21.15.1:6831/2497776654","heartbeat_front_addr":"172.21.15.1:6829/2497776654","state":["exists","up"]},{"osd":4,"uuid":"b4576b9c-871d-47a8-8d36-65af525d0200","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":2957026670},{"type":"v1","addr":"172.21.15.7:6801","nonce":2957026670}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2957026670},{"type":"v1","addr":"172.21.15.7:6803","nonce":2957026670}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":2957026670},{"type":"v1","addr":"172.21.15.7:6807","nonce":2957026670}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2957026670},{"type":"v1","addr":"172.21.15.7:6805","nonce":2957026670}]},"public_addr":"172.21.15.7:6801/2957026670","cluster_addr":"172.21.15.7:6803/2957026670","heartbeat_back_addr":"172.21.15.7:6807/2957026670","heartbeat_front_addr":"172.21.15.7:6805/2957026670","state":["exists","up"]},{"osd":5,"uuid":"902b3d94-72f3-4499-b21a-e32453ef5144","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1670295242},{"type":"v1","addr":"172.21.15.7:6809","nonce":1670295242}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1670295242},{"type":"v1","addr":"172.21.15.7:6811","nonce":1670295242}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1670295242},{"type":"v1","addr":"172.21.15.7:6815","nonce":1670295242}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1670295242},{"type":"v1","addr":"172.21.15.7:6813","nonce":1670295242}]},"public_addr":"172.21.15.7:6809/1670295242","cluster_addr":"172.21.15.7:6811/1670295242","heartbeat_back_addr":"172.21.15.7:6815/1670295242","heartbeat_front_addr":"172.21.15.7:6813/1670295242","state":["exists","up"]},{"osd":6,"uuid":"ead1d771-1728-4f85-adb4-26be21cedff3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":2852086880},{"type":"v1","addr":"172.21.15.7:6817","nonce":2852086880}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":2852086880},{"type":"v1","addr":"172.21.15.7:6819","nonce":2852086880}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":2852086880},{"type":"v1","addr":"172.21.15.7:6823","nonce":2852086880}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":2852086880},{"type":"v1","addr":"172.21.15.7:6821","nonce":2852086880}]},"public_addr":"172.21.15.7:6817/2852086880","cluster_addr":"172.21.15.7:6819/2852086880","heartbeat_back_addr":"172.21.15.7:6823/2852086880","heartbeat_front_addr":"172.21.15.7:6821/2852086880","state":["exists","up"]},{"osd":7,"uuid":"3ab5a201-9394-48a8-8918-e95197893669","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":1442884138},{"type":"v1","addr":"172.21.15.7:6825","nonce":1442884138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1442884138},{"type":"v1","addr":"172.21.15.7:6827","nonce":1442884138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1442884138},{"type":"v1","addr":"172.21.15.7:6831","nonce":1442884138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1442884138},{"type":"v1","addr":"172.21.15.7:6829","nonce":1442884138}]},"public_addr":"172.21.15.7:6825/1442884138","cluster_addr":"172.21.15.7:6827/1442884138","heartbeat_back_addr":"172.21.15.7:6831/1442884138","heartbeat_front_addr":"172.21.15.7:6829/1442884138","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:02:04.615675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:02:04.385974+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:02:04.619217+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:02:04.401681+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:02:04.097081+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:02:04.129981+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:02:04.225327+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:02:04.193351+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.35","mappings":[{"from":5,"to":2}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2},{"from":3,"to":1}]},{"pgid":"3.3f","mappings":[{"from":7,"to":2}]},{"pgid":"4.4","mappings":[{"from":1,"to":5}]},{"pgid":"4.e","mappings":[{"from":1,"to":2}]},{"pgid":"4.f","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T08:08:27.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:27.838+0000 7ff901600640 1 -- 172.21.15.1:0/844589480 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7ff8d806be30 msgr2=0x7ff8d806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:08:27.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:27.838+0000 7ff901600640 1 --2- 172.21.15.1:0/844589480 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7ff8d806be30 0x7ff8d806e2f0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7ff8fc071200 tx=0x7ff8e4009040 comp rx=0 tx=0).stop 2024-05-09T08:08:27.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:27.838+0000 7ff901600640 1 -- 172.21.15.1:0/844589480 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff8fc0c7620 msgr2=0x7ff8fc147ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:08:27.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:27.838+0000 7ff901600640 1 --2- 172.21.15.1:0/844589480 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff8fc0c7620 0x7ff8fc147ac0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7ff8ec02ad70 tx=0x7ff8ec004a40 comp rx=0 tx=0).stop 2024-05-09T08:08:27.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:27.838+0000 7ff901600640 1 -- 172.21.15.1:0/844589480 shutdown_connections 2024-05-09T08:08:27.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:27.838+0000 7ff901600640 1 --2- 172.21.15.1:0/844589480 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7ff8d806be30 0x7ff8d806e2f0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:27.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:27.838+0000 7ff901600640 1 --2- 172.21.15.1:0/844589480 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff8fc15e1e0 0x7ff8fc15cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:27.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:27.838+0000 7ff901600640 1 --2- 172.21.15.1:0/844589480 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7ff8fc157b30 0x7ff8fc15c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:27.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:27.838+0000 7ff901600640 1 --2- 172.21.15.1:0/844589480 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff8fc0c7620 0x7ff8fc147ac0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:27.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:27.838+0000 7ff901600640 1 -- 172.21.15.1:0/844589480 >> 172.21.15.1:0/844589480 conn(0x7ff8fc06e850 msgr2=0x7ff8fc070140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:08:27.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:27.838+0000 7ff901600640 1 -- 172.21.15.1:0/844589480 shutdown_connections 2024-05-09T08:08:27.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:27.838+0000 7ff901600640 1 -- 172.21.15.1:0/844589480 wait complete. 2024-05-09T08:08:27.868 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-09T08:08:28.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:28.074+0000 7f986fe00640 1 Processor -- start 2024-05-09T08:08:28.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:28.074+0000 7f986fe00640 1 -- start start 2024-05-09T08:08:28.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:28.076+0000 7f986fe00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f98700ccd10 0x7f98700cf1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:08:28.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:28.076+0000 7f986fe00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f987015a690 0x7f98700cf730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:08:28.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:28.076+0000 7f986fe00640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f987015abd0 con 0x7f98700ce1f0 2024-05-09T08:08:28.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:28.076+0000 7f986fe00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f9870154e50 con 0x7f98700ccd10 2024-05-09T08:08:28.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:28.076+0000 7f986fe00640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f98700cda80 con 0x7f987015a690 2024-05-09T08:08:28.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:28.076+0000 7f986e000640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f987015a690 0x7f98700cf730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:08:28.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:28.076+0000 7f986ea00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f98700ccd10 0x7f98700cf1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:08:28.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:28.076+0000 7f986f400640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7f98700ce1f0 0x7f98700ce680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:38826/0 (socket says 172.21.15.1:38826) 2024-05-09T08:08:28.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:28.076+0000 7f986f400640 1 -- 172.21.15.1:0/944164622 learned_addr learned my addr 172.21.15.1:0/944164622 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T08:08:28.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:28.077+0000 7f986ea00640 1 -- 172.21.15.1:0/944164622 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f987015a690 msgr2=0x7f98700cf730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:08:28.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:28.077+0000 7f986ea00640 1 --2- 172.21.15.1:0/944164622 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f987015a690 0x7f98700cf730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:28.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:28.077+0000 7f986ea00640 1 -- 172.21.15.1:0/944164622 >> v1:172.21.15.1:6789/0 conn(0x7f98700ce1f0 legacy=0x7f98700ce680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:08:28.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:28.077+0000 7f986ea00640 1 -- 172.21.15.1:0/944164622 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f987015aeb0 con 0x7f98700ccd10 2024-05-09T08:08:28.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:28.077+0000 7f986d600640 1 -- 172.21.15.1:0/944164622 <== mon.0 v1:172.21.15.1:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 966263825 0 0) 0x7f987015abd0 con 0x7f98700ce1f0 2024-05-09T08:08:28.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:28.078+0000 7f986ea00640 1 --2- 172.21.15.1:0/944164622 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98700ccd10 0x7f98700cf1f0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f98600097a0 tx=0x7f986002a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=104dd4458a1f3ef7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:08:28.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:28.079+0000 7f986d600640 1 -- 172.21.15.1:0/944164622 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9860037070 con 0x7f98700ccd10 2024-05-09T08:08:28.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:28.079+0000 7f986d600640 1 -- 172.21.15.1:0/944164622 <== mon.1 v2:172.21.15.7:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f9860032440 con 0x7f98700ccd10 2024-05-09T08:08:28.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:28.079+0000 7f986d600640 1 -- 172.21.15.1:0/944164622 <== mon.1 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9860032830 con 0x7f98700ccd10 2024-05-09T08:08:28.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:28.079+0000 7f986fe00640 1 -- 172.21.15.1:0/944164622 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98700ccd10 msgr2=0x7f98700cf1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:08:28.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:28.079+0000 7f986fe00640 1 --2- 172.21.15.1:0/944164622 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98700ccd10 0x7f98700cf1f0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f98600097a0 tx=0x7f986002a180 comp rx=0 tx=0).stop 2024-05-09T08:08:28.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:28.079+0000 7f986fe00640 1 -- 172.21.15.1:0/944164622 shutdown_connections 2024-05-09T08:08:28.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:28.079+0000 7f986fe00640 1 --2- 172.21.15.1:0/944164622 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f987015a690 0x7f98700cf730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:28.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:28.079+0000 7f986fe00640 1 --2- 172.21.15.1:0/944164622 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98700ccd10 0x7f98700cf1f0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:28.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:28.079+0000 7f986fe00640 1 -- 172.21.15.1:0/944164622 >> 172.21.15.1:0/944164622 conn(0x7f98700b1790 msgr2=0x7f98700b1ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:08:28.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:28.079+0000 7f986fe00640 1 -- 172.21.15.1:0/944164622 shutdown_connections 2024-05-09T08:08:28.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:28.080+0000 7f986fe00640 1 -- 172.21.15.1:0/944164622 wait complete. 2024-05-09T08:08:28.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:28.081+0000 7f986fe00640 1 Processor -- start 2024-05-09T08:08:28.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:28.082+0000 7f986fe00640 1 -- start start 2024-05-09T08:08:28.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:28.082+0000 7f986fe00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f98700ccd10 0x7f9870153630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:08:28.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:28.083+0000 7f986fe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98700ce1f0 0x7f98700c4c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:08:28.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:28.083+0000 7f986fe00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f987015a690 0x7f98701f0160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:08:28.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:28.083+0000 7f986fe00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f987015b470 con 0x7f98700ccd10 2024-05-09T08:08:28.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:28.083+0000 7f986fe00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f98700cdd60 con 0x7f98700ce1f0 2024-05-09T08:08:28.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:28.083+0000 7f986fe00640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f98700ca600 con 0x7f987015a690 2024-05-09T08:08:28.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:28.083+0000 7f986ea00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f98700ccd10 0x7f9870153630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:08:28.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:28.083+0000 7f986ea00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f98700ccd10 0x7f9870153630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49236/0 (socket says 172.21.15.1:49236) 2024-05-09T08:08:28.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:28.083+0000 7f986ea00640 1 -- 172.21.15.1:0/1933274549 learned_addr learned my addr 172.21.15.1:0/1933274549 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:08:28.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:28.083+0000 7f986e000640 1 --2- 172.21.15.1:0/1933274549 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98700ce1f0 0x7f98700c4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:08:28.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:28.083+0000 7f986f400640 1 --2- 172.21.15.1:0/1933274549 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f987015a690 0x7f98701f0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:08:28.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:28.083+0000 7f986ea00640 1 -- 172.21.15.1:0/1933274549 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f987015a690 msgr2=0x7f98701f0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:08:28.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:28.083+0000 7f986ea00640 1 --2- 172.21.15.1:0/1933274549 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f987015a690 0x7f98701f0160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:28.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:28.083+0000 7f986ea00640 1 -- 172.21.15.1:0/1933274549 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98700ce1f0 msgr2=0x7f98700c4c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:08:28.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:28.083+0000 7f986ea00640 1 --2- 172.21.15.1:0/1933274549 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98700ce1f0 0x7f98700c4c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:28.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:28.083+0000 7f986ea00640 1 -- 172.21.15.1:0/1933274549 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98700d0840 con 0x7f98700ccd10 2024-05-09T08:08:28.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:28.084+0000 7f986ea00640 1 --2- 172.21.15.1:0/1933274549 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f98700ccd10 0x7f9870153630 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f9860035040 tx=0x7f98600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:08:28.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:28.084+0000 7f9866a00640 1 -- 172.21.15.1:0/1933274549 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9860041030 con 0x7f98700ccd10 2024-05-09T08:08:28.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:28.084+0000 7f9866a00640 1 -- 172.21.15.1:0/1933274549 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f9860030540 con 0x7f98700ccd10 2024-05-09T08:08:28.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:28.084+0000 7f9866a00640 1 -- 172.21.15.1:0/1933274549 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9860037040 con 0x7f98700ccd10 2024-05-09T08:08:28.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:28.084+0000 7f986fe00640 1 -- 172.21.15.1:0/1933274549 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98700c5aa0 con 0x7f98700ccd10 2024-05-09T08:08:28.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:28.084+0000 7f986fe00640 1 -- 172.21.15.1:0/1933274549 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98700c6060 con 0x7f98700ccd10 2024-05-09T08:08:28.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:28.086+0000 7f986fe00640 1 -- 172.21.15.1:0/1933274549 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f987004f660 con 0x7f98700ccd10 2024-05-09T08:08:28.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:28.087+0000 7f9866a00640 1 -- 172.21.15.1:0/1933274549 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9860044070 con 0x7f98700ccd10 2024-05-09T08:08:28.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:28.088+0000 7f9866a00640 1 --2- 172.21.15.1:0/1933274549 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f985006be60 0x7f985006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:08:28.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:28.088+0000 7f9866a00640 1 -- 172.21.15.1:0/1933274549 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7f98600ab960 con 0x7f98700ccd10 2024-05-09T08:08:28.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:28.088+0000 7f986e000640 1 --2- 172.21.15.1:0/1933274549 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f985006be60 0x7f985006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:08:28.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:28.089+0000 7f986e000640 1 --2- 172.21.15.1:0/1933274549 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f985006be60 0x7f985006e320 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f98700c6cb0 tx=0x7f985c009190 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:08:28.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:28.093+0000 7f9866a00640 1 -- 172.21.15.1:0/1933274549 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f9860079830 con 0x7f98700ccd10 2024-05-09T08:08:28.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:28.337+0000 7f986fe00640 1 -- 172.21.15.1:0/1933274549 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f987006dfc0 con 0x7f98700ccd10 2024-05-09T08:08:28.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:28.341+0000 7f9866a00640 1 -- 172.21.15.1:0/1933274549 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v11) ==== 161+0+0 (secure 0 0 0) 0x7f98600a9e20 con 0x7f98700ccd10 2024-05-09T08:08:28.350 INFO:teuthology.orchestra.run.smithi001.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-09T08:08:28.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:28.345+0000 7f986fe00640 1 -- 172.21.15.1:0/1933274549 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f985006be60 msgr2=0x7f985006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:08:28.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:28.345+0000 7f986fe00640 1 --2- 172.21.15.1:0/1933274549 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f985006be60 0x7f985006e320 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f98700c6cb0 tx=0x7f985c009190 comp rx=0 tx=0).stop 2024-05-09T08:08:28.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:28.345+0000 7f986fe00640 1 -- 172.21.15.1:0/1933274549 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f98700ccd10 msgr2=0x7f9870153630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:08:28.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:28.345+0000 7f986fe00640 1 --2- 172.21.15.1:0/1933274549 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f98700ccd10 0x7f9870153630 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f9860035040 tx=0x7f98600046c0 comp rx=0 tx=0).stop 2024-05-09T08:08:28.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:28.345+0000 7f986fe00640 1 -- 172.21.15.1:0/1933274549 shutdown_connections 2024-05-09T08:08:28.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:28.345+0000 7f986fe00640 1 --2- 172.21.15.1:0/1933274549 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f985006be60 0x7f985006e320 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:28.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:28.345+0000 7f986fe00640 1 --2- 172.21.15.1:0/1933274549 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98700ce1f0 0x7f98700c4c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:28.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:28.345+0000 7f986fe00640 1 --2- 172.21.15.1:0/1933274549 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f987015a690 0x7f98701f0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:28.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:28.345+0000 7f986fe00640 1 --2- 172.21.15.1:0/1933274549 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f98700ccd10 0x7f9870153630 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:28.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:28.345+0000 7f986fe00640 1 -- 172.21.15.1:0/1933274549 >> 172.21.15.1:0/1933274549 conn(0x7f98700b1790 msgr2=0x7f98700ce680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:08:28.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:28.345+0000 7f986fe00640 1 -- 172.21.15.1:0/1933274549 shutdown_connections 2024-05-09T08:08:28.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:28.346+0000 7f986fe00640 1 -- 172.21.15.1:0/1933274549 wait complete. 2024-05-09T08:08:28.373 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-05-09T08:08:28.442 INFO:tasks.ceph.mds.c.smithi001.stdout:starting mds.c at 2024-05-09T08:08:28.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:28.557+0000 7f840a000640 1 Processor -- start 2024-05-09T08:08:28.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:28.557+0000 7f840a000640 1 -- start start 2024-05-09T08:08:28.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:28.559+0000 7f840a000640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f84040718e0 0x7f840415b050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:08:28.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:28.559+0000 7f840a000640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f8404072940 0x7f8404072dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:08:28.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:28.559+0000 7f840a000640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8404073310 con 0x7f84040c5890 2024-05-09T08:08:28.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:28.559+0000 7f840a000640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f840415b590 con 0x7f84040718e0 2024-05-09T08:08:28.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:28.559+0000 7f840a000640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f840406f060 con 0x7f8404072940 2024-05-09T08:08:28.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:28.559+0000 7f8409600640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f8404072940 0x7f8404072dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:08:28.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:28.559+0000 7f8403e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f84040718e0 0x7f840415b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:08:28.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:28.559+0000 7f8408c00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f84040c5890 0x7f84040c69e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.1:38972/0 (socket says 172.21.15.1:38972) 2024-05-09T08:08:28.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:28.559+0000 7f8408c00640 1 -- 172.21.15.1:0/2546217177 learned_addr learned my addr 172.21.15.1:0/2546217177 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T08:08:28.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:28.560+0000 7f8403400640 1 -- 172.21.15.1:0/2546217177 <== mon.1 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3574426558 0 0) 0x7f8404073310 con 0x7f84040c5890 2024-05-09T08:08:28.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:28.560+0000 7f8403e00640 1 -- 172.21.15.1:0/2546217177 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f8404072940 msgr2=0x7f8404072dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:08:28.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:28.560+0000 7f8403e00640 1 --2- 172.21.15.1:0/2546217177 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f8404072940 0x7f8404072dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:28.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:28.560+0000 7f8403e00640 1 -- 172.21.15.1:0/2546217177 >> v1:172.21.15.7:6789/0 conn(0x7f84040c5890 legacy=0x7f84040c69e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:08:28.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:28.560+0000 7f8403e00640 1 -- 172.21.15.1:0/2546217177 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8404072150 con 0x7f84040718e0 2024-05-09T08:08:28.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:28.561+0000 7f8403e00640 1 --2- 172.21.15.1:0/2546217177 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84040718e0 0x7f840415b050 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f83f00097a0 tx=0x7f83f002a210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bb8ae5a049aa4f75 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:08:28.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:28.561+0000 7f8403400640 1 -- 172.21.15.1:0/2546217177 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83f0037070 con 0x7f84040718e0 2024-05-09T08:08:28.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:28.561+0000 7f8403400640 1 -- 172.21.15.1:0/2546217177 <== mon.1 v2:172.21.15.7:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f83f0032440 con 0x7f84040718e0 2024-05-09T08:08:28.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:28.561+0000 7f8403400640 1 -- 172.21.15.1:0/2546217177 <== mon.1 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83f0032830 con 0x7f84040718e0 2024-05-09T08:08:28.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:28.562+0000 7f840a000640 1 -- 172.21.15.1:0/2546217177 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84040718e0 msgr2=0x7f840415b050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:08:28.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:28.562+0000 7f840a000640 1 --2- 172.21.15.1:0/2546217177 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84040718e0 0x7f840415b050 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f83f00097a0 tx=0x7f83f002a210 comp rx=0 tx=0).stop 2024-05-09T08:08:28.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:28.562+0000 7f840a000640 1 -- 172.21.15.1:0/2546217177 shutdown_connections 2024-05-09T08:08:28.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:28.562+0000 7f840a000640 1 --2- 172.21.15.1:0/2546217177 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f8404072940 0x7f8404072dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:28.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:28.562+0000 7f840a000640 1 --2- 172.21.15.1:0/2546217177 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84040718e0 0x7f840415b050 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:28.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:28.562+0000 7f840a000640 1 -- 172.21.15.1:0/2546217177 >> 172.21.15.1:0/2546217177 conn(0x7f840406b250 msgr2=0x7f840406b630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:08:28.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:28.562+0000 7f840a000640 1 -- 172.21.15.1:0/2546217177 shutdown_connections 2024-05-09T08:08:28.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:28.562+0000 7f840a000640 1 -- 172.21.15.1:0/2546217177 wait complete. 2024-05-09T08:08:28.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:28.564+0000 7f840a000640 1 Processor -- start 2024-05-09T08:08:28.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:28.564+0000 7f840a000640 1 -- start start 2024-05-09T08:08:28.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:28.565+0000 7f840a000640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f84040718e0 0x7f84041e4480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:08:28.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:28.565+0000 7f840a000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8404072940 0x7f84040bfce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:08:28.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:28.565+0000 7f840a000640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f84040c5890 0x7f84040c0d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:08:28.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:28.565+0000 7f840a000640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f840406f5e0 con 0x7f84040718e0 2024-05-09T08:08:28.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:28.565+0000 7f840a000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f840406f300 con 0x7f8404072940 2024-05-09T08:08:28.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:28.565+0000 7f840a000640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f840406f8c0 con 0x7f84040c5890 2024-05-09T08:08:28.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:28.565+0000 7f8408c00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f84040718e0 0x7f84041e4480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:08:28.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:28.565+0000 7f8408c00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f84040718e0 0x7f84041e4480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49264/0 (socket says 172.21.15.1:49264) 2024-05-09T08:08:28.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:28.565+0000 7f8408c00640 1 -- 172.21.15.1:0/3252021166 learned_addr learned my addr 172.21.15.1:0/3252021166 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:08:28.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:28.565+0000 7f8409600640 1 --2- 172.21.15.1:0/3252021166 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f84040c5890 0x7f84040c0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:08:28.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:28.565+0000 7f8403e00640 1 --2- 172.21.15.1:0/3252021166 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8404072940 0x7f84040bfce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:08:28.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:28.566+0000 7f8409600640 1 -- 172.21.15.1:0/3252021166 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8404072940 msgr2=0x7f84040bfce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:08:28.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:28.566+0000 7f8409600640 1 --2- 172.21.15.1:0/3252021166 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8404072940 0x7f84040bfce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:28.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:28.566+0000 7f8409600640 1 -- 172.21.15.1:0/3252021166 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f84040718e0 msgr2=0x7f84041e4480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:08:28.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:28.566+0000 7f8409600640 1 --2- 172.21.15.1:0/3252021166 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f84040718e0 0x7f84041e4480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:28.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:28.566+0000 7f8409600640 1 -- 172.21.15.1:0/3252021166 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84041f01c0 con 0x7f84040c5890 2024-05-09T08:08:28.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:28.566+0000 7f8403e00640 1 --2- 172.21.15.1:0/3252021166 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8404072940 0x7f84040bfce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:08:28.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:28.566+0000 7f8408c00640 1 --2- 172.21.15.1:0/3252021166 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f84040718e0 0x7f84041e4480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:08:28.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:28.566+0000 7f8409600640 1 --2- 172.21.15.1:0/3252021166 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f84040c5890 0x7f84040c0d30 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f83fc00efc0 tx=0x7f83fc00c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:08:28.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:28.567+0000 7f8400c00640 1 -- 172.21.15.1:0/3252021166 <== mon.2 v2:172.21.15.7:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83fc018020 con 0x7f84040c5890 2024-05-09T08:08:28.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:28.567+0000 7f8400c00640 1 -- 172.21.15.1:0/3252021166 <== mon.2 v2:172.21.15.7:3301/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f83fc014480 con 0x7f84040c5890 2024-05-09T08:08:28.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:28.567+0000 7f8400c00640 1 -- 172.21.15.1:0/3252021166 <== mon.2 v2:172.21.15.7:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83fc014890 con 0x7f84040c5890 2024-05-09T08:08:28.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:28.567+0000 7f840a000640 1 -- 172.21.15.1:0/3252021166 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84041f04a0 con 0x7f84040c5890 2024-05-09T08:08:28.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:28.567+0000 7f840a000640 1 -- 172.21.15.1:0/3252021166 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f84041f0a60 con 0x7f84040c5890 2024-05-09T08:08:28.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:28.568+0000 7f840a000640 1 -- 172.21.15.1:0/3252021166 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f840404f660 con 0x7f84040c5890 2024-05-09T08:08:28.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:28.569+0000 7f8400c00640 1 -- 172.21.15.1:0/3252021166 <== mon.2 v2:172.21.15.7:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f83fc01b020 con 0x7f84040c5890 2024-05-09T08:08:28.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:28.570+0000 7f8400c00640 1 --2- 172.21.15.1:0/3252021166 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f83e406be30 0x7f83e406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:08:28.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:28.570+0000 7f8400c00640 1 -- 172.21.15.1:0/3252021166 <== mon.2 v2:172.21.15.7:3301/0 5 ==== osd_map(28..28 src has 1..28) ==== 6800+0+0 (secure 0 0 0) 0x7f83fc08bfe0 con 0x7f84040c5890 2024-05-09T08:08:28.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:28.570+0000 7f8408c00640 1 --2- 172.21.15.1:0/3252021166 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f83e406be30 0x7f83e406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:08:28.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:28.571+0000 7f8408c00640 1 --2- 172.21.15.1:0/3252021166 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f83e406be30 0x7f83e406e2f0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f83f8000f60 tx=0x7f83f8007e10 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:08:28.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:28.575+0000 7f8400c00640 1 -- 172.21.15.1:0/3252021166 <== mon.2 v2:172.21.15.7:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f83fc022020 con 0x7f84040c5890 2024-05-09T08:08:28.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:28.826+0000 7f840a000640 1 -- 172.21.15.1:0/3252021166 --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f84040c7520 con 0x7f84040c5890 2024-05-09T08:08:29.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:29.347+0000 7f8400c00640 1 -- 172.21.15.1:0/3252021166 <== mon.2 v2:172.21.15.7:3301/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v12) ==== 104+0+0 (secure 0 0 0) 0x7f83fc010040 con 0x7f84040c5890 2024-05-09T08:08:29.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:29.350+0000 7f840a000640 1 -- 172.21.15.1:0/3252021166 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f83e406be30 msgr2=0x7f83e406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:08:29.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:29.350+0000 7f840a000640 1 --2- 172.21.15.1:0/3252021166 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f83e406be30 0x7f83e406e2f0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f83f8000f60 tx=0x7f83f8007e10 comp rx=0 tx=0).stop 2024-05-09T08:08:29.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:29.350+0000 7f840a000640 1 -- 172.21.15.1:0/3252021166 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f84040c5890 msgr2=0x7f84040c0d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:08:29.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:29.350+0000 7f840a000640 1 --2- 172.21.15.1:0/3252021166 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f84040c5890 0x7f84040c0d30 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f83fc00efc0 tx=0x7f83fc00c490 comp rx=0 tx=0).stop 2024-05-09T08:08:29.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:29.351+0000 7f840a000640 1 -- 172.21.15.1:0/3252021166 shutdown_connections 2024-05-09T08:08:29.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:29.351+0000 7f840a000640 1 --2- 172.21.15.1:0/3252021166 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f83e406be30 0x7f83e406e2f0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:29.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:29.351+0000 7f840a000640 1 --2- 172.21.15.1:0/3252021166 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8404072940 0x7f84040bfce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:29.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:29.351+0000 7f840a000640 1 --2- 172.21.15.1:0/3252021166 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f84040c5890 0x7f84040c0d30 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:29.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:29.351+0000 7f840a000640 1 --2- 172.21.15.1:0/3252021166 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f84040718e0 0x7f84041e4480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:29.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:29.351+0000 7f840a000640 1 -- 172.21.15.1:0/3252021166 >> 172.21.15.1:0/3252021166 conn(0x7f840406b250 msgr2=0x7f8404159550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:08:29.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:29.351+0000 7f840a000640 1 -- 172.21.15.1:0/3252021166 shutdown_connections 2024-05-09T08:08:29.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:29.351+0000 7f840a000640 1 -- 172.21.15.1:0/3252021166 wait complete. 2024-05-09T08:08:29.383 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-09T08:08:29.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:29.592+0000 7f8eddc00640 1 Processor -- start 2024-05-09T08:08:29.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:29.592+0000 7f8eddc00640 1 -- start start 2024-05-09T08:08:29.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:29.594+0000 7f8eddc00640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7f8ed8152550 0x7f8ed80734e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:08:29.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:29.594+0000 7f8eddc00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f8ed8072830 0x7f8ed815bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:08:29.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:29.594+0000 7f8eddc00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8ed8158f20 con 0x7f8ed80723d0 2024-05-09T08:08:29.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:29.594+0000 7f8eddc00640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7f8ed806ed90 con 0x7f8ed8152550 2024-05-09T08:08:29.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:29.594+0000 7f8eddc00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f8ed80c6b50 con 0x7f8ed8072830 2024-05-09T08:08:29.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:29.594+0000 7f8ed6000640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f8ed80723d0 0x7f8ed815b8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.1:38976/0 (socket says 172.21.15.1:38976) 2024-05-09T08:08:29.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:29.594+0000 7f8ed6000640 1 -- 172.21.15.1:0/2428381944 learned_addr learned my addr 172.21.15.1:0/2428381944 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T08:08:29.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:29.594+0000 7f8ed7400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f8ed8072830 0x7f8ed815bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:08:29.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:29.594+0000 7f8ed6a00640 1 --2- 172.21.15.1:0/2428381944 >> v2:172.21.15.1:3300/0 conn(0x7f8ed8152550 0x7f8ed80734e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:08:29.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:29.594+0000 7f8ed5600640 1 -- 172.21.15.1:0/2428381944 <== mon.1 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2804180221 0 0) 0x7f8ed8158f20 con 0x7f8ed80723d0 2024-05-09T08:08:29.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:29.595+0000 7f8ed5600640 1 -- 172.21.15.1:0/2428381944 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8ebc003520 con 0x7f8ed80723d0 2024-05-09T08:08:29.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:29.595+0000 7f8ed7400640 1 -- 172.21.15.1:0/2428381944 >> v2:172.21.15.1:3300/0 conn(0x7f8ed8152550 msgr2=0x7f8ed80734e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:08:29.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:29.595+0000 7f8ed7400640 1 --2- 172.21.15.1:0/2428381944 >> v2:172.21.15.1:3300/0 conn(0x7f8ed8152550 0x7f8ed80734e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:29.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:29.595+0000 7f8ed7400640 1 -- 172.21.15.1:0/2428381944 >> v1:172.21.15.7:6789/0 conn(0x7f8ed80723d0 legacy=0x7f8ed815b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:08:29.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:29.595+0000 7f8ed7400640 1 -- 172.21.15.1:0/2428381944 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ed806eab0 con 0x7f8ed8072830 2024-05-09T08:08:29.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:29.595+0000 7f8ed6a00640 1 --2- 172.21.15.1:0/2428381944 >> v2:172.21.15.1:3300/0 conn(0x7f8ed8152550 0x7f8ed80734e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:08:29.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:29.596+0000 7f8ed7400640 1 --2- 172.21.15.1:0/2428381944 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8ed8072830 0x7f8ed815bfe0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f8ed806c120 tx=0x7f8ed002a110 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b3fa3a7a12441b11 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:08:29.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:29.596+0000 7f8ed5600640 1 -- 172.21.15.1:0/2428381944 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ed0037070 con 0x7f8ed8072830 2024-05-09T08:08:29.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:29.596+0000 7f8ed5600640 1 -- 172.21.15.1:0/2428381944 <== mon.1 v2:172.21.15.7:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f8ed0032440 con 0x7f8ed8072830 2024-05-09T08:08:29.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:29.596+0000 7f8ed5600640 1 -- 172.21.15.1:0/2428381944 <== mon.1 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ed002ae60 con 0x7f8ed8072830 2024-05-09T08:08:29.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:29.597+0000 7f8eddc00640 1 -- 172.21.15.1:0/2428381944 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8ed8072830 msgr2=0x7f8ed815bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:08:29.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:29.597+0000 7f8eddc00640 1 --2- 172.21.15.1:0/2428381944 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8ed8072830 0x7f8ed815bfe0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f8ed806c120 tx=0x7f8ed002a110 comp rx=0 tx=0).stop 2024-05-09T08:08:29.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:29.597+0000 7f8eddc00640 1 -- 172.21.15.1:0/2428381944 shutdown_connections 2024-05-09T08:08:29.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:29.597+0000 7f8eddc00640 1 --2- 172.21.15.1:0/2428381944 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8ed8072830 0x7f8ed815bfe0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:29.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:29.597+0000 7f8eddc00640 1 --2- 172.21.15.1:0/2428381944 >> v2:172.21.15.1:3300/0 conn(0x7f8ed8152550 0x7f8ed80734e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:29.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:29.597+0000 7f8eddc00640 1 -- 172.21.15.1:0/2428381944 >> 172.21.15.1:0/2428381944 conn(0x7f8ed80c89b0 msgr2=0x7f8ed80c8dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:08:29.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:29.597+0000 7f8eddc00640 1 -- 172.21.15.1:0/2428381944 shutdown_connections 2024-05-09T08:08:29.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:29.597+0000 7f8eddc00640 1 -- 172.21.15.1:0/2428381944 wait complete. 2024-05-09T08:08:29.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:29.599+0000 7f8eddc00640 1 Processor -- start 2024-05-09T08:08:29.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:29.599+0000 7f8eddc00640 1 -- start start 2024-05-09T08:08:29.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:29.600+0000 7f8eddc00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8ed80723d0 0x7f8ed8147840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:08:29.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:29.600+0000 7f8eddc00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8ed8072830 0x7f8ed814c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:08:29.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:29.600+0000 7f8eddc00640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f8ed8152550 0x7f8ed813e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:08:29.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:29.600+0000 7f8eddc00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f8ed8071bd0 con 0x7f8ed8072830 2024-05-09T08:08:29.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:29.600+0000 7f8eddc00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f8ed806e860 con 0x7f8ed80723d0 2024-05-09T08:08:29.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:29.600+0000 7f8eddc00640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7f8ed8071eb0 con 0x7f8ed8152550 2024-05-09T08:08:29.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:29.600+0000 7f8ed6000640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8ed8072830 0x7f8ed814c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:08:29.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:29.600+0000 7f8ed6000640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8ed8072830 0x7f8ed814c7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49280/0 (socket says 172.21.15.1:49280) 2024-05-09T08:08:29.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:29.600+0000 7f8ed6000640 1 -- 172.21.15.1:0/4073910908 learned_addr learned my addr 172.21.15.1:0/4073910908 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:08:29.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:29.600+0000 7f8ed6a00640 1 --2- 172.21.15.1:0/4073910908 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8ed80723d0 0x7f8ed8147840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:08:29.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:29.600+0000 7f8ed7400640 1 --2- 172.21.15.1:0/4073910908 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f8ed8152550 0x7f8ed813e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:08:29.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:29.601+0000 7f8ed6000640 1 -- 172.21.15.1:0/4073910908 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f8ed8152550 msgr2=0x7f8ed813e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:08:29.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:29.601+0000 7f8ed6000640 1 --2- 172.21.15.1:0/4073910908 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f8ed8152550 0x7f8ed813e3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:29.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:29.601+0000 7f8ed6000640 1 -- 172.21.15.1:0/4073910908 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8ed80723d0 msgr2=0x7f8ed8147840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:08:29.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:29.601+0000 7f8ed6000640 1 --2- 172.21.15.1:0/4073910908 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8ed80723d0 0x7f8ed8147840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:29.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:29.601+0000 7f8ed6000640 1 -- 172.21.15.1:0/4073910908 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ed80c5ca0 con 0x7f8ed8072830 2024-05-09T08:08:29.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:29.601+0000 7f8ed6000640 1 --2- 172.21.15.1:0/4073910908 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8ed8072830 0x7f8ed814c7a0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f8ec000c400 tx=0x7f8ec000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:08:29.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:29.601+0000 7f8ecea00640 1 -- 172.21.15.1:0/4073910908 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec0017070 con 0x7f8ed8072830 2024-05-09T08:08:29.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:29.601+0000 7f8ecea00640 1 -- 172.21.15.1:0/4073910908 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f8ec0005d00 con 0x7f8ed8072830 2024-05-09T08:08:29.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:29.601+0000 7f8eddc00640 1 -- 172.21.15.1:0/4073910908 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ed80c5f10 con 0x7f8ed8072830 2024-05-09T08:08:29.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:29.601+0000 7f8ecea00640 1 -- 172.21.15.1:0/4073910908 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec0007330 con 0x7f8ed8072830 2024-05-09T08:08:29.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:29.602+0000 7f8eddc00640 1 -- 172.21.15.1:0/4073910908 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ed8156d80 con 0x7f8ed8072830 2024-05-09T08:08:29.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:29.603+0000 7f8eddc00640 1 -- 172.21.15.1:0/4073910908 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ed8072e10 con 0x7f8ed8072830 2024-05-09T08:08:29.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:29.604+0000 7f8ecea00640 1 -- 172.21.15.1:0/4073910908 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8ec001a070 con 0x7f8ed8072830 2024-05-09T08:08:29.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:29.605+0000 7f8ecea00640 1 --2- 172.21.15.1:0/4073910908 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f8eb406be30 0x7f8eb406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:08:29.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:29.605+0000 7f8ecea00640 1 -- 172.21.15.1:0/4073910908 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6800+0+0 (secure 0 0 0) 0x7f8ec008a7b0 con 0x7f8ed8072830 2024-05-09T08:08:29.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:29.605+0000 7f8ed6a00640 1 --2- 172.21.15.1:0/4073910908 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f8eb406be30 0x7f8eb406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:08:29.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:29.606+0000 7f8ed6a00640 1 --2- 172.21.15.1:0/4073910908 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f8eb406be30 0x7f8eb406e2f0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f8ec8009bd0 tx=0x7f8ec800b040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:08:29.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:29.610+0000 7f8ecea00640 1 -- 172.21.15.1:0/4073910908 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f8ed8072e10 con 0x7f8ed8072830 2024-05-09T08:08:29.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:29.813+0000 7f8eddc00640 1 -- 172.21.15.1:0/4073910908 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f8ed8068520 con 0x7f8ed8072830 2024-05-09T08:08:30.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:30.629+0000 7f8ecea00640 1 -- 172.21.15.1:0/4073910908 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v29) ==== 181+0+0 (secure 0 0 0) 0x7f8ec00585c0 con 0x7f8ed8072830 2024-05-09T08:08:30.639 INFO:teuthology.orchestra.run.smithi001.stderr:pool 'cephfs_metadata' removed 2024-05-09T08:08:30.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:30.634+0000 7f8eddc00640 1 -- 172.21.15.1:0/4073910908 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f8eb406be30 msgr2=0x7f8eb406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:08:30.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:30.634+0000 7f8eddc00640 1 --2- 172.21.15.1:0/4073910908 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f8eb406be30 0x7f8eb406e2f0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f8ec8009bd0 tx=0x7f8ec800b040 comp rx=0 tx=0).stop 2024-05-09T08:08:30.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:30.634+0000 7f8eddc00640 1 -- 172.21.15.1:0/4073910908 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8ed8072830 msgr2=0x7f8ed814c7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:08:30.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:30.634+0000 7f8eddc00640 1 --2- 172.21.15.1:0/4073910908 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8ed8072830 0x7f8ed814c7a0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f8ec000c400 tx=0x7f8ec000c8c0 comp rx=0 tx=0).stop 2024-05-09T08:08:30.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:30.635+0000 7f8eddc00640 1 -- 172.21.15.1:0/4073910908 shutdown_connections 2024-05-09T08:08:30.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:30.635+0000 7f8eddc00640 1 --2- 172.21.15.1:0/4073910908 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7f8eb406be30 0x7f8eb406e2f0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:30.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:30.635+0000 7f8eddc00640 1 --2- 172.21.15.1:0/4073910908 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7f8ed8152550 0x7f8ed813e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:30.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:30.635+0000 7f8eddc00640 1 --2- 172.21.15.1:0/4073910908 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8ed8072830 0x7f8ed814c7a0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:30.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:30.635+0000 7f8eddc00640 1 --2- 172.21.15.1:0/4073910908 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8ed80723d0 0x7f8ed8147840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:30.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:30.635+0000 7f8eddc00640 1 -- 172.21.15.1:0/4073910908 >> 172.21.15.1:0/4073910908 conn(0x7f8ed80c89b0 msgr2=0x7f8ed8151290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:08:30.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:30.636+0000 7f8eddc00640 1 -- 172.21.15.1:0/4073910908 shutdown_connections 2024-05-09T08:08:30.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:30.636+0000 7f8eddc00640 1 -- 172.21.15.1:0/4073910908 wait complete. 2024-05-09T08:08:30.671 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-09T08:08:30.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:30.863+0000 7fed14200640 1 Processor -- start 2024-05-09T08:08:30.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:30.864+0000 7fed14200640 1 -- start start 2024-05-09T08:08:30.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:30.865+0000 7fed14200640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fed0c1512f0 0x7fed0c0c0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:08:30.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:30.865+0000 7fed14200640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fed0c0c94d0 0x7fed0c0c7650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:08:30.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:30.865+0000 7fed14200640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fed0c0c1530 con 0x7fed0c072ce0 2024-05-09T08:08:30.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:30.865+0000 7fed14200640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7fed0c0c17f0 con 0x7fed0c1512f0 2024-05-09T08:08:30.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:30.865+0000 7fed14200640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7fed0c157240 con 0x7fed0c0c94d0 2024-05-09T08:08:30.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:30.866+0000 7fed11800640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fed0c1512f0 0x7fed0c0c0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:08:30.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:30.866+0000 7fed11800640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fed0c1512f0 0x7fed0c0c0ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49290/0 (socket says 172.21.15.1:49290) 2024-05-09T08:08:30.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:30.866+0000 7fed11800640 1 -- 172.21.15.1:0/2031321929 learned_addr learned my addr 172.21.15.1:0/2031321929 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:08:30.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:30.866+0000 7fed10e00640 1 --2- 172.21.15.1:0/2031321929 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fed0c0c94d0 0x7fed0c0c7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:08:30.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:30.866+0000 7fed11800640 1 -- 172.21.15.1:0/2031321929 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fed0c0c94d0 msgr2=0x7fed0c0c7650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:08:30.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:30.866+0000 7fed11800640 1 --2- 172.21.15.1:0/2031321929 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fed0c0c94d0 0x7fed0c0c7650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:30.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:30.866+0000 7fed11800640 1 -- 172.21.15.1:0/2031321929 >> v1:172.21.15.1:6789/0 conn(0x7fed0c072ce0 legacy=0x7fed0c15a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:08:30.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:30.866+0000 7fed11800640 1 -- 172.21.15.1:0/2031321929 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed0c157520 con 0x7fed0c1512f0 2024-05-09T08:08:30.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:30.867+0000 7fed11800640 1 --2- 172.21.15.1:0/2031321929 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fed0c1512f0 0x7fed0c0c0ed0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fecfc009e50 tx=0x7fecfc02a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2dd2ab760cd0434 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:08:30.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:30.868+0000 7fed0be00640 1 -- 172.21.15.1:0/2031321929 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecfc036070 con 0x7fed0c1512f0 2024-05-09T08:08:30.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:30.868+0000 7fed0be00640 1 -- 172.21.15.1:0/2031321929 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fecfc031600 con 0x7fed0c1512f0 2024-05-09T08:08:30.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:30.868+0000 7fed0be00640 1 -- 172.21.15.1:0/2031321929 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecfc0319f0 con 0x7fed0c1512f0 2024-05-09T08:08:30.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:30.868+0000 7fed14200640 1 -- 172.21.15.1:0/2031321929 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fed0c1512f0 msgr2=0x7fed0c0c0ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:08:30.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:30.868+0000 7fed14200640 1 --2- 172.21.15.1:0/2031321929 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fed0c1512f0 0x7fed0c0c0ed0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fecfc009e50 tx=0x7fecfc02a7a0 comp rx=0 tx=0).stop 2024-05-09T08:08:30.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:30.868+0000 7fed14200640 1 -- 172.21.15.1:0/2031321929 shutdown_connections 2024-05-09T08:08:30.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:30.868+0000 7fed14200640 1 --2- 172.21.15.1:0/2031321929 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fed0c0c94d0 0x7fed0c0c7650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:30.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:30.868+0000 7fed14200640 1 --2- 172.21.15.1:0/2031321929 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fed0c1512f0 0x7fed0c0c0ed0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:30.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:30.868+0000 7fed14200640 1 -- 172.21.15.1:0/2031321929 >> 172.21.15.1:0/2031321929 conn(0x7fed0c0c89b0 msgr2=0x7fed0c0c8dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:08:30.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:30.868+0000 7fed14200640 1 -- 172.21.15.1:0/2031321929 shutdown_connections 2024-05-09T08:08:30.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:30.868+0000 7fed14200640 1 -- 172.21.15.1:0/2031321929 wait complete. 2024-05-09T08:08:30.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:30.870+0000 7fed14200640 1 Processor -- start 2024-05-09T08:08:30.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:30.870+0000 7fed14200640 1 -- start start 2024-05-09T08:08:30.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:30.871+0000 7fed14200640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fed0c072ce0 0x7fed0c14b630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:08:30.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:30.871+0000 7fed14200640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fed0c0c94d0 0x7fed0c1f0490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:08:30.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:30.871+0000 7fed14200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed0c1512f0 0x7fed0c1f09d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:08:30.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:30.871+0000 7fed14200640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fed0c0c6210 con 0x7fed0c0c94d0 2024-05-09T08:08:30.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:30.871+0000 7fed14200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fed0c157ae0 con 0x7fed0c1512f0 2024-05-09T08:08:30.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:30.871+0000 7fed14200640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7fed0c158000 con 0x7fed0c072ce0 2024-05-09T08:08:30.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:30.871+0000 7fed10e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fed0c0c94d0 0x7fed0c1f0490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:08:30.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:30.871+0000 7fed10e00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fed0c0c94d0 0x7fed0c1f0490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49292/0 (socket says 172.21.15.1:49292) 2024-05-09T08:08:30.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:30.871+0000 7fed10e00640 1 -- 172.21.15.1:0/665293832 learned_addr learned my addr 172.21.15.1:0/665293832 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:08:30.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:30.872+0000 7fed12200640 1 --2- 172.21.15.1:0/665293832 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed0c1512f0 0x7fed0c1f09d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:08:30.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:30.872+0000 7fed11800640 1 --2- 172.21.15.1:0/665293832 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fed0c072ce0 0x7fed0c14b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:08:30.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:30.872+0000 7fed10e00640 1 -- 172.21.15.1:0/665293832 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fed0c072ce0 msgr2=0x7fed0c14b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:08:30.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:30.872+0000 7fed10e00640 1 --2- 172.21.15.1:0/665293832 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fed0c072ce0 0x7fed0c14b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:30.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:30.872+0000 7fed10e00640 1 -- 172.21.15.1:0/665293832 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed0c1512f0 msgr2=0x7fed0c1f09d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:08:30.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:30.872+0000 7fed10e00640 1 --2- 172.21.15.1:0/665293832 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed0c1512f0 0x7fed0c1f09d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:30.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:30.872+0000 7fed10e00640 1 -- 172.21.15.1:0/665293832 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed0c0cb7a0 con 0x7fed0c0c94d0 2024-05-09T08:08:30.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:30.872+0000 7fed10e00640 1 --2- 172.21.15.1:0/665293832 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fed0c0c94d0 0x7fed0c1f0490 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fecf800e8a0 tx=0x7fecf800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:08:30.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:30.872+0000 7fed09600640 1 -- 172.21.15.1:0/665293832 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecf8017070 con 0x7fed0c0c94d0 2024-05-09T08:08:30.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:30.873+0000 7fed09600640 1 -- 172.21.15.1:0/665293832 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fecf800c940 con 0x7fed0c0c94d0 2024-05-09T08:08:30.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:30.873+0000 7fed14200640 1 -- 172.21.15.1:0/665293832 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed0c1f36e0 con 0x7fed0c0c94d0 2024-05-09T08:08:30.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:30.873+0000 7fed09600640 1 -- 172.21.15.1:0/665293832 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecf800cd50 con 0x7fed0c0c94d0 2024-05-09T08:08:30.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:30.873+0000 7fed14200640 1 -- 172.21.15.1:0/665293832 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed0c1f3ca0 con 0x7fed0c0c94d0 2024-05-09T08:08:30.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:30.874+0000 7fed14200640 1 -- 172.21.15.1:0/665293832 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed0c068520 con 0x7fed0c0c94d0 2024-05-09T08:08:30.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:30.875+0000 7fed09600640 1 -- 172.21.15.1:0/665293832 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fecf801a070 con 0x7fed0c0c94d0 2024-05-09T08:08:30.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:30.877+0000 7fed09600640 1 --2- 172.21.15.1:0/665293832 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fecec06be60 0x7fecec06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:08:30.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:30.877+0000 7fed09600640 1 -- 172.21.15.1:0/665293832 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6362+0+0 (secure 0 0 0) 0x7fecf808afe0 con 0x7fed0c0c94d0 2024-05-09T08:08:30.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:30.877+0000 7fed11800640 1 --2- 172.21.15.1:0/665293832 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fecec06be60 0x7fecec06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:08:30.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:30.878+0000 7fed11800640 1 --2- 172.21.15.1:0/665293832 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fecec06be60 0x7fecec06e320 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fecfc033000 tx=0x7fecfc009000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:08:30.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:30.880+0000 7fed09600640 1 -- 172.21.15.1:0/665293832 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fecf8058fc0 con 0x7fed0c0c94d0 2024-05-09T08:08:31.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:31.091+0000 7fed14200640 1 -- 172.21.15.1:0/665293832 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fed0c06d820 con 0x7fed0c0c94d0 2024-05-09T08:08:31.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:31.632+0000 7fed09600640 1 -- 172.21.15.1:0/665293832 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v30) ==== 169+0+0 (secure 0 0 0) 0x7fecf805d2c0 con 0x7fed0c0c94d0 2024-05-09T08:08:31.642 INFO:teuthology.orchestra.run.smithi001.stderr:pool 'cephfs_data' removed 2024-05-09T08:08:31.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:31.636+0000 7fed14200640 1 -- 172.21.15.1:0/665293832 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fecec06be60 msgr2=0x7fecec06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:08:31.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:31.636+0000 7fed14200640 1 --2- 172.21.15.1:0/665293832 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fecec06be60 0x7fecec06e320 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fecfc033000 tx=0x7fecfc009000 comp rx=0 tx=0).stop 2024-05-09T08:08:31.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:31.636+0000 7fed14200640 1 -- 172.21.15.1:0/665293832 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fed0c0c94d0 msgr2=0x7fed0c1f0490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:08:31.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:31.636+0000 7fed14200640 1 --2- 172.21.15.1:0/665293832 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fed0c0c94d0 0x7fed0c1f0490 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fecf800e8a0 tx=0x7fecf800ed60 comp rx=0 tx=0).stop 2024-05-09T08:08:31.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:31.636+0000 7fed14200640 1 -- 172.21.15.1:0/665293832 shutdown_connections 2024-05-09T08:08:31.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:31.636+0000 7fed14200640 1 --2- 172.21.15.1:0/665293832 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fecec06be60 0x7fecec06e320 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:31.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:31.636+0000 7fed14200640 1 --2- 172.21.15.1:0/665293832 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed0c1512f0 0x7fed0c1f09d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:31.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:31.636+0000 7fed14200640 1 --2- 172.21.15.1:0/665293832 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fed0c0c94d0 0x7fed0c1f0490 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:31.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:31.636+0000 7fed14200640 1 --2- 172.21.15.1:0/665293832 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fed0c072ce0 0x7fed0c14b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:31.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:31.636+0000 7fed14200640 1 -- 172.21.15.1:0/665293832 >> 172.21.15.1:0/665293832 conn(0x7fed0c0c89b0 msgr2=0x7fed0c06fba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:08:31.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:31.637+0000 7fed14200640 1 -- 172.21.15.1:0/665293832 shutdown_connections 2024-05-09T08:08:31.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:31.639+0000 7fed14200640 1 -- 172.21.15.1:0/665293832 wait complete. 2024-05-09T08:08:31.677 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-05-09T08:08:31.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:31.863+0000 7fd125400640 1 Processor -- start 2024-05-09T08:08:31.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:31.864+0000 7fd125400640 1 -- start start 2024-05-09T08:08:31.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:31.865+0000 7fd125400640 1 --2- >> v2:172.21.15.1:3300/0 conn(0x7fd1200c4d70 0x7fd1200cb7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:08:31.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:31.865+0000 7fd125400640 1 -- --> v1:172.21.15.1:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd1200c5320 con 0x7fd1201549d0 2024-05-09T08:08:31.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:31.865+0000 7fd125400640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd1200caa00 con 0x7fd120067690 2024-05-09T08:08:31.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:31.865+0000 7fd125400640 1 -- --> v2:172.21.15.1:3300/0 -- mon_getmap magic: 0 -- 0x7fd1200ca7b0 con 0x7fd1200c4d70 2024-05-09T08:08:31.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:31.866+0000 7fd11e800640 1 --1- >> v1:172.21.15.1:6789/0 conn(0x7fd1201549d0 0x7fd12015b060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.1:6789/0 says I am v1:172.21.15.1:38848/0 (socket says 172.21.15.1:38848) 2024-05-09T08:08:31.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:31.866+0000 7fd11e800640 1 -- 172.21.15.1:0/754792656 learned_addr learned my addr 172.21.15.1:0/754792656 (peer_addr_for_me v1:172.21.15.1:0/0) 2024-05-09T08:08:31.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:31.866+0000 7fd11f200640 1 --2- 172.21.15.1:0/754792656 >> v2:172.21.15.1:3300/0 conn(0x7fd1200c4d70 0x7fd1200cb7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:08:31.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:31.866+0000 7fd11d400640 1 -- 172.21.15.1:0/754792656 <== mon.1 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2840938113 0 0) 0x7fd1200caa00 con 0x7fd120067690 2024-05-09T08:08:31.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:31.866+0000 7fd11d400640 1 -- 172.21.15.1:0/754792656 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd104003520 con 0x7fd120067690 2024-05-09T08:08:31.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:31.866+0000 7fd11f200640 1 -- 172.21.15.1:0/754792656 >> v1:172.21.15.7:6789/0 conn(0x7fd120067690 legacy=0x7fd1200cc4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:08:31.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:31.866+0000 7fd11f200640 1 -- 172.21.15.1:0/754792656 >> v1:172.21.15.1:6789/0 conn(0x7fd1201549d0 legacy=0x7fd12015b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:08:31.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:31.866+0000 7fd11f200640 1 -- 172.21.15.1:0/754792656 --> v2:172.21.15.1:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1200cace0 con 0x7fd1200c4d70 2024-05-09T08:08:31.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:31.867+0000 7fd11f200640 1 --2- 172.21.15.1:0/754792656 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd1200c4d70 0x7fd1200cb7c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fd118009d90 tx=0x7fd11802a780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72fde0c8a76b51a7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:08:31.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:31.868+0000 7fd11d400640 1 -- 172.21.15.1:0/754792656 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd118037070 con 0x7fd1200c4d70 2024-05-09T08:08:31.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:31.868+0000 7fd11d400640 1 -- 172.21.15.1:0/754792656 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fd118007740 con 0x7fd1200c4d70 2024-05-09T08:08:31.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:31.868+0000 7fd11d400640 1 -- 172.21.15.1:0/754792656 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd118007af0 con 0x7fd1200c4d70 2024-05-09T08:08:31.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:31.868+0000 7fd125400640 1 -- 172.21.15.1:0/754792656 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd1200c4d70 msgr2=0x7fd1200cb7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:08:31.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:31.868+0000 7fd125400640 1 --2- 172.21.15.1:0/754792656 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd1200c4d70 0x7fd1200cb7c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fd118009d90 tx=0x7fd11802a780 comp rx=0 tx=0).stop 2024-05-09T08:08:31.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:31.868+0000 7fd125400640 1 -- 172.21.15.1:0/754792656 shutdown_connections 2024-05-09T08:08:31.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:31.868+0000 7fd125400640 1 --2- 172.21.15.1:0/754792656 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd1200c4d70 0x7fd1200cb7c0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:31.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:31.868+0000 7fd125400640 1 -- 172.21.15.1:0/754792656 >> 172.21.15.1:0/754792656 conn(0x7fd12006ce10 msgr2=0x7fd120066ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:08:31.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:31.868+0000 7fd125400640 1 -- 172.21.15.1:0/754792656 shutdown_connections 2024-05-09T08:08:31.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:31.868+0000 7fd125400640 1 -- 172.21.15.1:0/754792656 wait complete. 2024-05-09T08:08:31.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:31.870+0000 7fd125400640 1 Processor -- start 2024-05-09T08:08:31.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:31.870+0000 7fd125400640 1 -- start start 2024-05-09T08:08:31.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:31.871+0000 7fd125400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd120067690 0x7fd1200c5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:08:31.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:31.871+0000 7fd125400640 1 --2- >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fd1200c4d70 0x7fd1201f0750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:08:31.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:31.871+0000 7fd125400640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd1201549d0 0x7fd1201f0c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:08:31.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:31.871+0000 7fd125400640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fd1200ccbd0 con 0x7fd1201549d0 2024-05-09T08:08:31.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:31.871+0000 7fd125400640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd1200cb2a0 con 0x7fd120067690 2024-05-09T08:08:31.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:31.871+0000 7fd125400640 1 -- --> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] -- mon_getmap magic: 0 -- 0x7fd1200ccd50 con 0x7fd1200c4d70 2024-05-09T08:08:31.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:31.871+0000 7fd11e800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd120067690 0x7fd1200c5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:08:31.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:31.871+0000 7fd11f200640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd1201549d0 0x7fd1201f0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:08:31.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:31.872+0000 7fd11e800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd120067690 0x7fd1200c5870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.1:57178/0 (socket says 172.21.15.1:57178) 2024-05-09T08:08:31.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:31.872+0000 7fd11e800640 1 -- 172.21.15.1:0/2586548902 learned_addr learned my addr 172.21.15.1:0/2586548902 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-09T08:08:31.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:31.872+0000 7fd11de00640 1 --2- 172.21.15.1:0/2586548902 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fd1200c4d70 0x7fd1201f0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:08:31.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:31.872+0000 7fd11e800640 1 -- 172.21.15.1:0/2586548902 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fd1200c4d70 msgr2=0x7fd1201f0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:08:31.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:31.872+0000 7fd11e800640 1 --2- 172.21.15.1:0/2586548902 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fd1200c4d70 0x7fd1201f0750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:31.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:31.872+0000 7fd11e800640 1 -- 172.21.15.1:0/2586548902 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd1201549d0 msgr2=0x7fd1201f0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:08:31.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:31.872+0000 7fd11e800640 1 --2- 172.21.15.1:0/2586548902 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd1201549d0 0x7fd1201f0c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:31.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:31.872+0000 7fd11e800640 1 -- 172.21.15.1:0/2586548902 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1201f4390 con 0x7fd120067690 2024-05-09T08:08:31.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:31.872+0000 7fd11f200640 1 --2- 172.21.15.1:0/2586548902 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd1201549d0 0x7fd1201f0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:08:31.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:31.872+0000 7fd11e800640 1 --2- 172.21.15.1:0/2586548902 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd120067690 0x7fd1200c5870 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fd11000c3c0 tx=0x7fd11000c880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:08:31.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:31.873+0000 7fd116a00640 1 -- 172.21.15.1:0/2586548902 <== mon.1 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd110017070 con 0x7fd120067690 2024-05-09T08:08:31.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:31.873+0000 7fd116a00640 1 -- 172.21.15.1:0/2586548902 <== mon.1 v2:172.21.15.7:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fd110005ce0 con 0x7fd120067690 2024-05-09T08:08:31.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:31.873+0000 7fd116a00640 1 -- 172.21.15.1:0/2586548902 <== mon.1 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd110007310 con 0x7fd120067690 2024-05-09T08:08:31.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:31.873+0000 7fd125400640 1 -- 172.21.15.1:0/2586548902 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1201f4670 con 0x7fd120067690 2024-05-09T08:08:31.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:31.873+0000 7fd125400640 1 -- 172.21.15.1:0/2586548902 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1200cf630 con 0x7fd120067690 2024-05-09T08:08:31.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:31.874+0000 7fd125400640 1 -- 172.21.15.1:0/2586548902 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd120068520 con 0x7fd120067690 2024-05-09T08:08:31.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:31.875+0000 7fd116a00640 1 -- 172.21.15.1:0/2586548902 <== mon.1 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd11001a070 con 0x7fd120067690 2024-05-09T08:08:31.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:31.876+0000 7fd116a00640 1 --2- 172.21.15.1:0/2586548902 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fd0fc06be30 0x7fd0fc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:08:31.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:31.876+0000 7fd116a00640 1 -- 172.21.15.1:0/2586548902 <== mon.1 v2:172.21.15.7:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 5853+0+0 (secure 0 0 0) 0x7fd11008a260 con 0x7fd120067690 2024-05-09T08:08:31.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:31.876+0000 7fd11de00640 1 --2- 172.21.15.1:0/2586548902 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fd0fc06be30 0x7fd0fc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:08:31.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:31.877+0000 7fd11de00640 1 --2- 172.21.15.1:0/2586548902 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fd0fc06be30 0x7fd0fc06e2f0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fd1200c61d0 tx=0x7fd108009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:08:31.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:31.881+0000 7fd116a00640 1 -- 172.21.15.1:0/2586548902 <== mon.1 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fd110058490 con 0x7fd120067690 2024-05-09T08:08:32.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:32.082+0000 7fd125400640 1 -- 172.21.15.1:0/2586548902 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) -- 0x7fd120158050 con 0x7fd120067690 2024-05-09T08:08:32.667 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:32.657+0000 7fd116a00640 1 -- 172.21.15.1:0/2586548902 <== mon.1 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v31) ==== 178+0+0 (secure 0 0 0) 0x7fd11005c790 con 0x7fd120067690 2024-05-09T08:08:32.667 INFO:teuthology.orchestra.run.smithi001.stderr:pool 'cephfs_data_ec' removed 2024-05-09T08:08:32.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:32.667+0000 7fd125400640 1 -- 172.21.15.1:0/2586548902 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fd0fc06be30 msgr2=0x7fd0fc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:08:32.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:32.667+0000 7fd125400640 1 --2- 172.21.15.1:0/2586548902 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fd0fc06be30 0x7fd0fc06e2f0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fd1200c61d0 tx=0x7fd108009040 comp rx=0 tx=0).stop 2024-05-09T08:08:32.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:32.667+0000 7fd125400640 1 -- 172.21.15.1:0/2586548902 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd120067690 msgr2=0x7fd1200c5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:08:32.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:32.667+0000 7fd125400640 1 --2- 172.21.15.1:0/2586548902 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd120067690 0x7fd1200c5870 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fd11000c3c0 tx=0x7fd11000c880 comp rx=0 tx=0).stop 2024-05-09T08:08:32.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:32.667+0000 7fd125400640 1 -- 172.21.15.1:0/2586548902 shutdown_connections 2024-05-09T08:08:32.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:32.667+0000 7fd125400640 1 --2- 172.21.15.1:0/2586548902 >> [v2:172.21.15.7:6832/3562612464,v1:172.21.15.7:6833/3562612464] conn(0x7fd0fc06be30 0x7fd0fc06e2f0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:32.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:32.667+0000 7fd125400640 1 --2- 172.21.15.1:0/2586548902 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd1201549d0 0x7fd1201f0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:32.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:32.667+0000 7fd125400640 1 --2- 172.21.15.1:0/2586548902 >> [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] conn(0x7fd1200c4d70 0x7fd1201f0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:08:32.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:32.667+0000 7fd125400640 1 --2- 172.21.15.1:0/2586548902 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd120067690 0x7fd1200c5870 secure :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fd11000c3c0 tx=0x7fd11000c880 comp rx=0 tx=0).stop 2024-05-09T08:08:32.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:32.667+0000 7fd125400640 1 -- 172.21.15.1:0/2586548902 >> 172.21.15.1:0/2586548902 conn(0x7fd12006ce10 msgr2=0x7fd1200cc2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:08:32.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:32.668+0000 7fd125400640 1 -- 172.21.15.1:0/2586548902 shutdown_connections 2024-05-09T08:08:32.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-09T08:08:32.668+0000 7fd125400640 1 -- 172.21.15.1:0/2586548902 wait complete. 2024-05-09T08:08:32.711 INFO:teuthology.misc:Shutting down mds daemons... 2024-05-09T08:08:32.711 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-05-09T08:08:32.711 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T08:08:32.736 INFO:tasks.ceph.mds.a:Stopped 2024-05-09T08:08:32.736 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-05-09T08:08:32.736 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T08:08:32.800 INFO:tasks.ceph.mds.c:Stopped 2024-05-09T08:08:32.801 DEBUG:tasks.ceph.mds.e:waiting for process to exit 2024-05-09T08:08:32.801 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T08:08:32.866 INFO:tasks.ceph.mds.e:Stopped 2024-05-09T08:08:32.866 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-05-09T08:08:32.866 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T08:08:32.878 INFO:tasks.ceph.mds.b:Stopped 2024-05-09T08:08:32.878 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-05-09T08:08:32.878 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T08:08:32.931 INFO:tasks.ceph.mds.d:Stopped 2024-05-09T08:08:32.931 DEBUG:tasks.ceph.mds.f:waiting for process to exit 2024-05-09T08:08:32.931 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T08:08:32.983 INFO:tasks.ceph.mds.f:Stopped 2024-05-09T08:08:32.983 INFO:teuthology.misc:Shutting down osd daemons... 2024-05-09T08:08:32.984 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-05-09T08:08:32.984 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T08:08:37.535 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~0s 2024-05-09T08:08:39.085 INFO:tasks.ceph.osd.0:Stopped 2024-05-09T08:08:39.086 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-05-09T08:08:39.086 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T08:08:39.163 INFO:tasks.ceph.osd.1:Stopped 2024-05-09T08:08:39.163 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-05-09T08:08:39.163 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T08:08:39.248 INFO:tasks.ceph.osd.2:Stopped 2024-05-09T08:08:39.248 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-05-09T08:08:39.248 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T08:08:39.326 INFO:tasks.ceph.osd.3:Stopped 2024-05-09T08:08:39.326 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-05-09T08:08:39.326 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T08:08:39.361 INFO:tasks.ceph.osd.4:Stopped 2024-05-09T08:08:39.361 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-05-09T08:08:39.361 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T08:08:39.436 INFO:tasks.ceph.osd.5:Stopped 2024-05-09T08:08:39.437 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-05-09T08:08:39.437 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T08:08:39.516 INFO:tasks.ceph.osd.6:Stopped 2024-05-09T08:08:39.516 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-05-09T08:08:39.517 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T08:08:39.597 INFO:tasks.ceph.osd.7:Stopped 2024-05-09T08:08:39.597 INFO:teuthology.misc:Shutting down mgr daemons... 2024-05-09T08:08:39.597 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-05-09T08:08:39.597 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T08:08:42.939 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~0s 2024-05-09T08:08:42.940 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 has been restored 2024-05-09T08:08:45.699 INFO:tasks.ceph.mgr.y:Stopped 2024-05-09T08:08:45.700 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-05-09T08:08:45.700 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T08:08:45.731 INFO:tasks.ceph.mgr.x:Stopped 2024-05-09T08:08:45.731 INFO:teuthology.misc:Shutting down mon daemons... 2024-05-09T08:08:45.731 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-05-09T08:08:45.731 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T08:08:45.767 INFO:tasks.ceph.mon.a:Stopped 2024-05-09T08:08:45.767 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-05-09T08:08:45.767 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T08:08:45.785 INFO:tasks.ceph.mon.b:Stopped 2024-05-09T08:08:45.786 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-05-09T08:08:45.786 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T08:08:45.838 INFO:tasks.ceph.mon.c:Stopped 2024-05-09T08:08:45.839 INFO:tasks.ceph:Checking cluster log for badness... 2024-05-09T08:08:45.839 DEBUG:teuthology.orchestra.run.smithi001:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v 'overall HEALTH_' | egrep -v '\(OSDMAP_FLAGS\)' | egrep -v '\(OSD_' | egrep -v '\(PG_' | egrep -v '\(POOL_' | egrep -v '\(CACHE_POOL_' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(OBJECT_' | egrep -v '\(SLOW_OPS\)' | egrep -v '\(REQUEST_SLOW\)' | egrep -v '\(TOO_FEW_PGS\)' | egrep -v 'slow request' | egrep -v 'timeout on replica' | egrep -v 'late reservation from' | egrep -v MON_DOWN | egrep -v OSDMAP_FLAGS | egrep -v OSD_DOWN | egrep -v PG_DEGRADED | egrep -v PG_AVAILABILITY | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v '\(MON_DOWN\)' | head -n 1 2024-05-09T08:08:45.963 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi001.front.sepia.ceph.com 2024-05-09T08:08:45.964 DEBUG:teuthology.orchestra.run.smithi001:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-05-09T08:08:46.362 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi001.front.sepia.ceph.com 2024-05-09T08:08:46.363 DEBUG:teuthology.orchestra.run.smithi001:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-05-09T08:08:46.514 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi001.front.sepia.ceph.com 2024-05-09T08:08:46.515 DEBUG:teuthology.orchestra.run.smithi001:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-05-09T08:08:46.661 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi001.front.sepia.ceph.com 2024-05-09T08:08:46.661 DEBUG:teuthology.orchestra.run.smithi001:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-05-09T08:08:46.842 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi007.front.sepia.ceph.com 2024-05-09T08:08:46.842 DEBUG:teuthology.orchestra.run.smithi007:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-05-09T08:08:47.124 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi007.front.sepia.ceph.com 2024-05-09T08:08:47.124 DEBUG:teuthology.orchestra.run.smithi007:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-05-09T08:08:47.204 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi007.front.sepia.ceph.com 2024-05-09T08:08:47.204 DEBUG:teuthology.orchestra.run.smithi007:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-05-09T08:08:47.272 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi007.front.sepia.ceph.com 2024-05-09T08:08:47.272 DEBUG:teuthology.orchestra.run.smithi007:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-05-09T08:08:47.329 INFO:tasks.ceph:Archiving mon data... 2024-05-09T08:08:47.331 DEBUG:teuthology.misc:Transferring archived files from smithi001:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/teuthology-2024-05-08_21:24:04-fs-squid-distro-default-smithi/7698643/data/mon.a.tgz 2024-05-09T08:08:47.331 DEBUG:teuthology.orchestra.run.smithi001:> mktemp 2024-05-09T08:08:47.392 INFO:teuthology.orchestra.run.smithi001.stdout:/tmp/tmp.CIVvwdUa9s 2024-05-09T08:08:47.393 DEBUG:teuthology.orchestra.run.smithi001:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.CIVvwdUa9s 2024-05-09T08:08:47.578 DEBUG:teuthology.orchestra.run.smithi001:> sudo chmod 0666 /tmp/tmp.CIVvwdUa9s 2024-05-09T08:08:47.725 DEBUG:teuthology.orchestra.remote:smithi001:/tmp/tmp.CIVvwdUa9s is 399KB 2024-05-09T08:08:47.840 DEBUG:teuthology.orchestra.run.smithi001:> rm -fr /tmp/tmp.CIVvwdUa9s 2024-05-09T08:08:47.900 DEBUG:teuthology.misc:Transferring archived files from smithi007:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/teuthology-2024-05-08_21:24:04-fs-squid-distro-default-smithi/7698643/data/mon.b.tgz 2024-05-09T08:08:47.900 DEBUG:teuthology.orchestra.run.smithi007:> mktemp 2024-05-09T08:08:47.919 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/tmp.JZOfZiFwJt 2024-05-09T08:08:47.919 DEBUG:teuthology.orchestra.run.smithi007:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.JZOfZiFwJt 2024-05-09T08:08:47.941 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y has been restored 2024-05-09T08:08:48.080 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 0666 /tmp/tmp.JZOfZiFwJt 2024-05-09T08:08:48.165 DEBUG:teuthology.orchestra.remote:smithi007:/tmp/tmp.JZOfZiFwJt is 412KB 2024-05-09T08:08:48.262 DEBUG:teuthology.orchestra.run.smithi007:> rm -fr /tmp/tmp.JZOfZiFwJt 2024-05-09T08:08:48.278 DEBUG:teuthology.misc:Transferring archived files from smithi007:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/teuthology-2024-05-08_21:24:04-fs-squid-distro-default-smithi/7698643/data/mon.c.tgz 2024-05-09T08:08:48.278 DEBUG:teuthology.orchestra.run.smithi007:> mktemp 2024-05-09T08:08:48.333 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/tmp.Yk46FyY7wK 2024-05-09T08:08:48.333 DEBUG:teuthology.orchestra.run.smithi007:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.Yk46FyY7wK 2024-05-09T08:08:48.476 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 0666 /tmp/tmp.Yk46FyY7wK 2024-05-09T08:08:48.573 DEBUG:teuthology.orchestra.remote:smithi007:/tmp/tmp.Yk46FyY7wK is 419KB 2024-05-09T08:08:48.683 DEBUG:teuthology.orchestra.run.smithi007:> rm -fr /tmp/tmp.Yk46FyY7wK 2024-05-09T08:08:48.706 INFO:tasks.ceph:Cleaning ceph cluster... 2024-05-09T08:08:48.707 DEBUG:teuthology.orchestra.run.smithi001:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-05-09T08:08:48.721 DEBUG:teuthology.orchestra.run.smithi007:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-05-09T08:08:48.874 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-05-09T08:08:48.902 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-05-09T08:08:48.902 INFO:tasks.ceph:Archiving crash dumps... 2024-05-09T08:08:48.925 DEBUG:teuthology.misc:Transferring archived files from smithi001:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-05-08_21:24:04-fs-squid-distro-default-smithi/7698643/remote/smithi001/crash 2024-05-09T08:08:48.931 DEBUG:teuthology.orchestra.run.smithi001:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-05-09T08:08:49.057 DEBUG:teuthology.misc:Transferring archived files from smithi007:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-05-08_21:24:04-fs-squid-distro-default-smithi/7698643/remote/smithi007/crash 2024-05-09T08:08:49.136 DEBUG:teuthology.orchestra.run.smithi007:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-05-09T08:08:49.164 INFO:tasks.ceph:Shutting down logrotate 2024-05-09T08:08:49.164 DEBUG:teuthology.orchestra.run.smithi001:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T08:08:49.173 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T08:08:49.286 DEBUG:teuthology.orchestra.run.smithi001:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-05-09T08:08:49.380 DEBUG:teuthology.orchestra.run.smithi007:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-05-09T08:08:49.407 INFO:tasks.ceph:Compressing logs... 2024-05-09T08:08:49.407 DEBUG:teuthology.orchestra.run.smithi001:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-09T08:08:49.416 DEBUG:teuthology.orchestra.run.smithi007:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-09T08:08:49.475 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-05-09T08:08:49.475 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.f.log 2024-05-09T08:08:49.475 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-05-09T08:08:49.475 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-05-09T08:08:49.476 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-05-09T08:08:49.476 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36239.log 2024-05-09T08:08:49.476 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-osd.7.log: /var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-05-09T08:08:49.476 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.36239.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-05-09T08:08:49.481 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-osd.5.log: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.36239.log.gz 2024-05-09T08:08:49.481 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36329.log 2024-05-09T08:08:49.481 INFO:teuthology.orchestra.run.smithi007.stderr: 92.7% -- replaced with /var/log/ceph/ceph-mds.f.log.gz 2024-05-09T08:08:49.481 INFO:teuthology.orchestra.run.smithi007.stderr: 93.7% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-05-09T08:08:49.482 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-05-09T08:08:49.482 INFO:teuthology.orchestra.run.smithi007.stderr: 92.1% -- replaced with /var/log/ceph/ceph.log.gz 2024-05-09T08:08:49.482 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-05-09T08:08:49.482 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.36329.log: gzip -5/var/log/ceph/ceph-mgr.x.log: --verbose 92.7% -- /var/log/ceph/ceph-osd.6.log -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-05-09T08:08:49.482 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-09T08:08:49.482 INFO:teuthology.orchestra.run.smithi007.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.36329.log.gz 2024-05-09T08:08:49.482 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36289.log 2024-05-09T08:08:49.482 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36409.log 2024-05-09T08:08:49.482 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.36289.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36369.log 2024-05-09T08:08:49.482 INFO:teuthology.orchestra.run.smithi007.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.36289.log.gz 2024-05-09T08:08:49.482 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.36409.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.36409.log.gz 2024-05-09T08:08:49.482 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36449.log 2024-05-09T08:08:49.483 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.36369.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36158.log 2024-05-09T08:08:49.483 INFO:teuthology.orchestra.run.smithi007.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.36369.log.gz 2024-05-09T08:08:49.483 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.36449.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.36449.log.gz 2024-05-09T08:08:49.483 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-05-09T08:08:49.483 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-05-09T08:08:49.483 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.36158.log: /var/log/ceph/ceph-mds.d.log: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.36158.log.gz 2024-05-09T08:08:49.484 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36198.log 2024-05-09T08:08:49.484 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-mon.b.log: 92.7% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-05-09T08:08:49.495 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.36198.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.36198.log.gz 2024-05-09T08:08:49.496 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39616.log 2024-05-09T08:08:49.498 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48052.log 2024-05-09T08:08:49.499 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.39616.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44536.log 2024-05-09T08:08:49.500 INFO:teuthology.orchestra.run.smithi001.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.39616.log.gz 2024-05-09T08:08:49.500 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.48052.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40910.log 2024-05-09T08:08:49.501 INFO:teuthology.orchestra.run.smithi001.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.48052.log.gz 2024-05-09T08:08:49.502 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.44536.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.44536.log.gz 2024-05-09T08:08:49.502 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46512.log 2024-05-09T08:08:49.503 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.40910.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41125.log 2024-05-09T08:08:49.504 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.46512.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.46512.log.gz 2024-05-09T08:08:49.504 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43295.log 2024-05-09T08:08:49.505 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.41125.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45722.log 2024-05-09T08:08:49.505 INFO:teuthology.orchestra.run.smithi001.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.41125.log.gz 2024-05-09T08:08:49.506 INFO:teuthology.orchestra.run.smithi001.stderr: 92.5% -- replaced with /var/log/ceph/ceph-client.admin.40910.log.gz 2024-05-09T08:08:49.506 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.43295.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42119.log 2024-05-09T08:08:49.506 INFO:teuthology.orchestra.run.smithi001.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43295.log.gz 2024-05-09T08:08:49.507 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.45722.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48280.log 2024-05-09T08:08:49.507 INFO:teuthology.orchestra.run.smithi001.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.45722.log.gz 2024-05-09T08:08:49.508 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.42119.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.42119.log.gz 2024-05-09T08:08:49.509 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47052.log 2024-05-09T08:08:49.509 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.48280.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40319.log 2024-05-09T08:08:49.510 INFO:teuthology.orchestra.run.smithi001.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.48280.log.gz 2024-05-09T08:08:49.510 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.47052.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46020.log 2024-05-09T08:08:49.511 INFO:teuthology.orchestra.run.smithi001.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.47052.log.gz 2024-05-09T08:08:49.512 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.40319.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49470.log 2024-05-09T08:08:49.512 INFO:teuthology.orchestra.run.smithi001.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.40319.log.gz 2024-05-09T08:08:49.512 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.46020.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41209.log 2024-05-09T08:08:49.513 INFO:teuthology.orchestra.run.smithi001.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.46020.log.gz 2024-05-09T08:08:49.514 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.49470.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42300.log 2024-05-09T08:08:49.514 INFO:teuthology.orchestra.run.smithi001.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.49470.log.gz 2024-05-09T08:08:49.515 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.41209.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.41209.log.gzgzip 2024-05-09T08:08:49.515 INFO:teuthology.orchestra.run.smithi001.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.43169.log 2024-05-09T08:08:49.516 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.42300.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46122.log 2024-05-09T08:08:49.517 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.43169.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.42300.log.gz 2024-05-09T08:08:49.517 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47529.log 2024-05-09T08:08:49.517 INFO:teuthology.orchestra.run.smithi001.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.43169.log.gz 2024-05-09T08:08:49.518 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.46122.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46615.log 2024-05-09T08:08:49.519 INFO:teuthology.orchestra.run.smithi001.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46122.log.gz 2024-05-09T08:08:49.519 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.47529.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.47529.log.gz 2024-05-09T08:08:49.519 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48154.log 2024-05-09T08:08:49.520 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.46615.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39355.log 2024-05-09T08:08:49.521 INFO:teuthology.orchestra.run.smithi001.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.46615.log.gz 2024-05-09T08:08:49.521 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.48154.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35709.log 2024-05-09T08:08:49.521 INFO:teuthology.orchestra.run.smithi001.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.48154.log.gz 2024-05-09T08:08:49.523 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.39355.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41393.log 2024-05-09T08:08:49.523 INFO:teuthology.orchestra.run.smithi001.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.39355.log.gz 2024-05-09T08:08:49.523 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.35709.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50142.log 2024-05-09T08:08:49.524 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.41393.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.35709.log.gz 2024-05-09T08:08:49.524 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44662.log 2024-05-09T08:08:49.525 INFO:teuthology.orchestra.run.smithi001.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.41393.log.gz 2024-05-09T08:08:49.526 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.50142.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41435.log 2024-05-09T08:08:49.526 INFO:teuthology.orchestra.run.smithi001.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.50142.log.gz 2024-05-09T08:08:49.527 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.44662.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43379.log 2024-05-09T08:08:49.527 INFO:teuthology.orchestra.run.smithi001.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.44662.log.gz 2024-05-09T08:08:49.528 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.41435.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-05-09T08:08:49.528 INFO:teuthology.orchestra.run.smithi001.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.41435.log.gz 2024-05-09T08:08:49.529 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.43379.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45221.log 2024-05-09T08:08:49.529 INFO:teuthology.orchestra.run.smithi001.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.43379.log.gz 2024-05-09T08:08:49.530 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.44341.log 2024-05-09T08:08:49.531 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.45221.log: gzip -5 --verbose 89.7% -- -- replaced with /var/log/ceph/ceph-client.admin.45221.log.gz /var/log/ceph/ceph-client.admin.45367.log 2024-05-09T08:08:49.531 INFO:teuthology.orchestra.run.smithi001.stderr: 2024-05-09T08:08:49.532 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.0.44341.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.44341.log.gz 2024-05-09T08:08:49.532 INFO:teuthology.orchestra.run.smithi001.stderr: 86.1% -- replaced with /var/log/ceph/ceph.log.gz 2024-05-09T08:08:49.532 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39816.log 2024-05-09T08:08:49.533 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.45367.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47136.log 2024-05-09T08:08:49.534 INFO:teuthology.orchestra.run.smithi001.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.45367.log.gz 2024-05-09T08:08:49.535 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.39816.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.39816.log.gz 2024-05-09T08:08:49.535 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48450.log 2024-05-09T08:08:49.536 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.47136.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45095.log 2024-05-09T08:08:49.537 INFO:teuthology.orchestra.run.smithi001.stderr: 92.3% -- replaced with /var/log/ceph/ceph-client.admin.47136.log.gz 2024-05-09T08:08:49.537 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.48450.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50506.log 2024-05-09T08:08:49.537 INFO:teuthology.orchestra.run.smithi001.stderr: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.48450.log.gz 2024-05-09T08:08:49.538 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.45095.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39658.log 2024-05-09T08:08:49.539 INFO:teuthology.orchestra.run.smithi001.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.45095.log.gz 2024-05-09T08:08:49.539 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.50506.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.50506.log.gz 2024-05-09T08:08:49.539 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44494.log 2024-05-09T08:08:49.540 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.39658.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44914.log 2024-05-09T08:08:49.540 INFO:teuthology.orchestra.run.smithi001.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.39658.log.gz 2024-05-09T08:08:49.541 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.44494.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.44494.log.gz 2024-05-09T08:08:49.542 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49806.log 2024-05-09T08:08:49.543 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.44914.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-05-09T08:08:49.543 INFO:teuthology.orchestra.run.smithi001.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.44914.log.gz 2024-05-09T08:08:49.543 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.49806.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39700.log 2024-05-09T08:08:49.544 INFO:teuthology.orchestra.run.smithi001.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.49806.log.gz 2024-05-09T08:08:49.545 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35442.log 2024-05-09T08:08:49.546 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.39700.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.39700.log.gz 2024-05-09T08:08:49.546 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39868.log 2024-05-09T08:08:49.547 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.35442.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41561.log 2024-05-09T08:08:49.547 INFO:teuthology.orchestra.run.smithi001.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.35442.log.gz 2024-05-09T08:08:49.548 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.39868.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.39868.log.gz 2024-05-09T08:08:49.549 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49512.log 2024-05-09T08:08:49.550 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.41561.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46755.log 2024-05-09T08:08:49.550 INFO:teuthology.orchestra.run.smithi001.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.41561.log.gz 2024-05-09T08:08:49.551 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.49512.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49205.log 2024-05-09T08:08:49.551 INFO:teuthology.orchestra.run.smithi001.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.49512.log.gz 2024-05-09T08:08:49.552 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.46755.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41867.log 2024-05-09T08:08:49.552 INFO:teuthology.orchestra.run.smithi001.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.46755.log.gz 2024-05-09T08:08:49.554 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.49205.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46555.log 2024-05-09T08:08:49.554 INFO:teuthology.orchestra.run.smithi001.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.49205.log.gz 2024-05-09T08:08:49.560 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.41867.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48775.log 2024-05-09T08:08:49.561 INFO:teuthology.orchestra.run.smithi001.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.41867.log.gz 2024-05-09T08:08:49.561 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.46555.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.46555.log.gz 2024-05-09T08:08:49.561 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50380.log 2024-05-09T08:08:49.561 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.48775.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44872.log 2024-05-09T08:08:49.561 INFO:teuthology.orchestra.run.smithi001.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.48775.log.gz 2024-05-09T08:08:49.561 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.50380.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45768.log 2024-05-09T08:08:49.561 INFO:teuthology.orchestra.run.smithi001.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.50380.log.gz 2024-05-09T08:08:49.561 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.44872.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45852.log 2024-05-09T08:08:49.561 INFO:teuthology.orchestra.run.smithi001.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.44872.log.gz 2024-05-09T08:08:49.562 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.45768.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45768.log.gz 2024-05-09T08:08:49.562 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47403.log 2024-05-09T08:08:49.562 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.45852.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47968.log 2024-05-09T08:08:49.563 INFO:teuthology.orchestra.run.smithi007.stderr: 91.5% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-05-09T08:08:49.563 INFO:teuthology.orchestra.run.smithi001.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.45852.log.gz 2024-05-09T08:08:49.563 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.47403.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49848.log 2024-05-09T08:08:49.563 INFO:teuthology.orchestra.run.smithi001.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.47403.log.gz 2024-05-09T08:08:49.564 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.47968.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50735.log 2024-05-09T08:08:49.565 INFO:teuthology.orchestra.run.smithi001.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.47968.log.gz 2024-05-09T08:08:49.566 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.49848.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.49848.log.gz 2024-05-09T08:08:49.566 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48649.log 2024-05-09T08:08:49.566 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.50735.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45978.log 2024-05-09T08:08:49.567 INFO:teuthology.orchestra.run.smithi001.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.50735.log.gz 2024-05-09T08:08:49.568 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.48649.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44746.log 2024-05-09T08:08:49.568 INFO:teuthology.orchestra.run.smithi001.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.48649.log.gz 2024-05-09T08:08:49.569 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.45978.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.45978.log.gz 2024-05-09T08:08:49.569 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-05-09T08:08:49.571 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.44746.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42973.log 87.9% 2024-05-09T08:08:49.571 INFO:teuthology.orchestra.run.smithi001.stderr: -- replaced with /var/log/ceph/ceph-client.admin.44746.log.gz 2024-05-09T08:08:49.572 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47010.log 2024-05-09T08:08:49.573 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.42973.log: 89.1% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-05-09T08:08:49.573 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47884.log 2024-05-09T08:08:49.573 INFO:teuthology.orchestra.run.smithi001.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.42973.log.gz 2024-05-09T08:08:49.574 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.47010.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49890.log 2024-05-09T08:08:49.575 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.47884.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.47010.log.gz 2024-05-09T08:08:49.575 INFO:teuthology.orchestra.run.smithi001.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.47884.log.gz 2024-05-09T08:08:49.576 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49722.log 2024-05-09T08:08:49.577 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.49890.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50268.log 2024-05-09T08:08:49.578 INFO:teuthology.orchestra.run.smithi001.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.49890.log.gz 2024-05-09T08:08:49.578 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.49722.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43757.log 2024-05-09T08:08:49.579 INFO:teuthology.orchestra.run.smithi001.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.49722.log.gz 2024-05-09T08:08:49.579 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.50268.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42637.log 2024-05-09T08:08:49.580 INFO:teuthology.orchestra.run.smithi001.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.50268.log.gz 2024-05-09T08:08:49.581 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.43757.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42295.log 2024-05-09T08:08:49.581 INFO:teuthology.orchestra.run.smithi001.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.43757.log.gz 2024-05-09T08:08:49.582 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.42637.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43421.log 2024-05-09T08:08:49.583 INFO:teuthology.orchestra.run.smithi001.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.42637.log.gz 2024-05-09T08:08:49.584 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.42295.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47259.log 2024-05-09T08:08:49.584 INFO:teuthology.orchestra.run.smithi001.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.42295.log.gz 2024-05-09T08:08:49.584 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.43421.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47571.log 2024-05-09T08:08:49.584 INFO:teuthology.orchestra.run.smithi001.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.43421.log.gz 2024-05-09T08:08:49.586 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.47259.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43463.log 2024-05-09T08:08:49.586 INFO:teuthology.orchestra.run.smithi001.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47259.log.gz 2024-05-09T08:08:49.587 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.47571.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42969.log 2024-05-09T08:08:49.587 INFO:teuthology.orchestra.run.smithi001.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.47571.log.gz 2024-05-09T08:08:49.588 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.43463.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.43463.log.gz 2024-05-09T08:08:49.589 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44830.log 2024-05-09T08:08:49.590 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.42969.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39569.log 2024-05-09T08:08:49.590 INFO:teuthology.orchestra.run.smithi001.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.42969.log.gz 2024-05-09T08:08:49.591 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.44830.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39470.log 2024-05-09T08:08:49.591 INFO:teuthology.orchestra.run.smithi001.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.44830.log.gz 2024-05-09T08:08:49.592 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.39569.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-05-09T08:08:49.592 INFO:teuthology.orchestra.run.smithi001.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.39569.log.gz 2024-05-09T08:08:49.593 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.39470.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.39470.log.gz 2024-05-09T08:08:49.593 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42614.log 2024-05-09T08:08:49.594 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49680.log 2024-05-09T08:08:49.596 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.42614.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43211.log 2024-05-09T08:08:49.596 INFO:teuthology.orchestra.run.smithi001.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42614.log.gz 2024-05-09T08:08:49.597 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.49680.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44704.log 2024-05-09T08:08:49.597 INFO:teuthology.orchestra.run.smithi001.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.49680.log.gz 2024-05-09T08:08:49.598 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.43211.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.43211.log.gz 2024-05-09T08:08:49.598 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41351.log 2024-05-09T08:08:49.599 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.44704.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40277.log 2024-05-09T08:08:49.599 INFO:teuthology.orchestra.run.smithi001.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.44704.log.gz 2024-05-09T08:08:49.601 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.41351.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.41351.log.gz 2024-05-09T08:08:49.601 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47343.log 2024-05-09T08:08:49.602 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.40277.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45680.log 2024-05-09T08:08:49.603 INFO:teuthology.orchestra.run.smithi001.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.40277.log.gz 2024-05-09T08:08:49.604 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.47343.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45554.log 2024-05-09T08:08:49.604 INFO:teuthology.orchestra.run.smithi001.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.47343.log.gz 2024-05-09T08:08:49.605 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.45680.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49163.log 2024-05-09T08:08:49.605 INFO:teuthology.orchestra.run.smithi001.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.45680.log.gz 2024-05-09T08:08:49.606 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.45554.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2024-05-09T08:08:49.606 INFO:teuthology.orchestra.run.smithi001.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.45554.log.gz 2024-05-09T08:08:49.611 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.49163.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48817.log 2024-05-09T08:08:49.611 INFO:teuthology.orchestra.run.smithi001.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.49163.log.gz 2024-05-09T08:08:49.612 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49247.log 2024-05-09T08:08:49.613 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.48817.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.48817.log.gz 2024-05-09T08:08:49.613 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47655.log 2024-05-09T08:08:49.613 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48901.log 2024-05-09T08:08:49.613 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.49247.log: /var/log/ceph/ceph-client.admin.47655.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.49247.log.gz 2024-05-09T08:08:49.613 INFO:teuthology.orchestra.run.smithi001.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.47655.log.gz 2024-05-09T08:08:49.614 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49638.log 2024-05-09T08:08:49.615 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.48901.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50184.log 2024-05-09T08:08:49.616 INFO:teuthology.orchestra.run.smithi001.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.48901.log.gz 2024-05-09T08:08:49.616 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.49638.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.49638.log.gz 2024-05-09T08:08:49.617 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48963.log 2024-05-09T08:08:49.618 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.50184.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49079.log 2024-05-09T08:08:49.619 INFO:teuthology.orchestra.run.smithi001.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.50184.log.gz 2024-05-09T08:08:49.620 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.48963.log: gzip 88.0% -- replaced with /var/log/ceph/ceph-client.admin.48963.log.gz -5 2024-05-09T08:08:49.620 INFO:teuthology.orchestra.run.smithi001.stderr: --verbose -- /var/log/ceph/ceph-client.admin.44788.log 2024-05-09T08:08:49.622 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.49079.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40048.log 2024-05-09T08:08:49.622 INFO:teuthology.orchestra.run.smithi001.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.49079.log.gz 2024-05-09T08:08:49.623 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.44788.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.44788.log.gz 2024-05-09T08:08:49.623 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-05-09T08:08:49.623 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.40048.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40361.log 2024-05-09T08:08:49.624 INFO:teuthology.orchestra.run.smithi001.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.40048.log.gz 2024-05-09T08:08:49.625 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-05-09T08:08:49.626 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.40361.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41765.log 2024-05-09T08:08:49.626 INFO:teuthology.orchestra.run.smithi001.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.40361.log.gz 2024-05-09T08:08:49.627 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48408.log 2024-05-09T08:08:49.629 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.41765.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45493.log 2024-05-09T08:08:49.629 INFO:teuthology.orchestra.run.smithi001.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.41765.log.gz 2024-05-09T08:08:49.631 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.48408.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49121.log 2024-05-09T08:08:49.631 INFO:teuthology.orchestra.run.smithi001.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.48408.log.gz 2024-05-09T08:08:49.632 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.45493.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45638.log 2024-05-09T08:08:49.632 INFO:teuthology.orchestra.run.smithi001.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.45493.log.gz 2024-05-09T08:08:49.634 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.49121.log: gzip -5 --verbose 88.0% -- replaced with /var/log/ceph/ceph-client.admin.49121.log.gz 2024-05-09T08:08:49.634 INFO:teuthology.orchestra.run.smithi001.stderr: -- /var/log/ceph/ceph-client.admin.42971.log 2024-05-09T08:08:49.635 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.45638.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50651.log 2024-05-09T08:08:49.635 INFO:teuthology.orchestra.run.smithi001.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.45638.log.gz 2024-05-09T08:08:49.635 INFO:teuthology.orchestra.run.smithi001.stderr: 93.5% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-05-09T08:08:49.636 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.42971.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45451.log 2024-05-09T08:08:49.637 INFO:teuthology.orchestra.run.smithi001.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.42971.log.gz 2024-05-09T08:08:49.637 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.50651.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49596.log 2024-05-09T08:08:49.638 INFO:teuthology.orchestra.run.smithi001.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.50651.log.gz 2024-05-09T08:08:49.639 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.45451.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-05-09T08:08:49.639 INFO:teuthology.orchestra.run.smithi001.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.45451.log.gz 2024-05-09T08:08:49.640 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.49596.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.29409.log 2024-05-09T08:08:49.640 INFO:teuthology.orchestra.run.smithi001.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.49596.log.gz 2024-05-09T08:08:49.641 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46713.log 2024-05-09T08:08:49.643 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph.tmp-client.admin.29409.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48094.log 2024-05-09T08:08:49.643 INFO:teuthology.orchestra.run.smithi001.stderr: -- replaced with /var/log/ceph/ceph.tmp-client.admin.29409.log.gz 2024-05-09T08:08:49.644 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.46713.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42299.log 2024-05-09T08:08:49.644 INFO:teuthology.orchestra.run.smithi001.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.46713.log.gz 2024-05-09T08:08:49.646 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.48094.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42077.log 2024-05-09T08:08:49.646 INFO:teuthology.orchestra.run.smithi001.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.48094.log.gz 2024-05-09T08:08:49.647 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.42299.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42963.log 2024-05-09T08:08:49.647 INFO:teuthology.orchestra.run.smithi001.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.42299.log.gz 2024-05-09T08:08:49.648 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.42077.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48691.log 2024-05-09T08:08:49.648 INFO:teuthology.orchestra.run.smithi001.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42077.log.gz 2024-05-09T08:08:49.650 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.42963.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43715.log 2024-05-09T08:08:49.650 INFO:teuthology.orchestra.run.smithi001.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.42963.log.gz 2024-05-09T08:08:49.651 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.48691.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43253.log 2024-05-09T08:08:49.651 INFO:teuthology.orchestra.run.smithi001.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.48691.log.gz 2024-05-09T08:08:49.652 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.43715.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43715.log.gz 2024-05-09T08:08:49.653 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.43253.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43253.log.gz 2024-05-09T08:08:49.654 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41681.log 2024-05-09T08:08:49.655 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42035.log 2024-05-09T08:08:49.657 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.41681.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42293.log 2024-05-09T08:08:49.657 INFO:teuthology.orchestra.run.smithi001.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.41681.log.gz 2024-05-09T08:08:49.658 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.42035.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.42035.log.gz 2024-05-09T08:08:49.659 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.42293.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.42293.log.gz 2024-05-09T08:08:49.660 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42301.log 2024-05-09T08:08:49.661 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45137.log 2024-05-09T08:08:49.663 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.42301.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40235.log 2024-05-09T08:08:49.663 INFO:teuthology.orchestra.run.smithi001.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.42301.log.gz 2024-05-09T08:08:49.664 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.45137.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.45137.log.gz 2024-05-09T08:08:49.665 INFO:teuthology.orchestra.run.smithi007.stderr: 91.8% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-05-09T08:08:49.666 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.40235.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.40235.log.gz 2024-05-09T08:08:49.666 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45263.log 2024-05-09T08:08:49.667 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49932.log 2024-05-09T08:08:49.669 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.45263.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42296.log 2024-05-09T08:08:49.669 INFO:teuthology.orchestra.run.smithi001.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.45263.log.gz 2024-05-09T08:08:49.670 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.49932.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42624.log 2024-05-09T08:08:49.670 INFO:teuthology.orchestra.run.smithi001.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.49932.log.gz 2024-05-09T08:08:49.671 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.42296.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41251.log 2024-05-09T08:08:49.671 INFO:teuthology.orchestra.run.smithi001.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.42296.log.gz 2024-05-09T08:08:49.673 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.42624.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48322.log 2024-05-09T08:08:49.673 INFO:teuthology.orchestra.run.smithi001.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.42624.log.gz 2024-05-09T08:08:49.674 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.41251.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39962.log 2024-05-09T08:08:49.674 INFO:teuthology.orchestra.run.smithi001.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.41251.log.gz 2024-05-09T08:08:49.676 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.48322.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41909.log 2024-05-09T08:08:49.676 INFO:teuthology.orchestra.run.smithi001.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.48322.log.gz 2024-05-09T08:08:49.676 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.39962.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.39962.log.gz 2024-05-09T08:08:49.677 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-05-09T08:08:49.678 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.41909.log: gzip 87.4% -- replaced with /var/log/ceph/ceph-client.admin.41909.log.gz 2024-05-09T08:08:49.678 INFO:teuthology.orchestra.run.smithi001.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.46346.log 2024-05-09T08:08:49.679 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47217.log 2024-05-09T08:08:49.681 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.46346.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.46346.log.gz 2024-05-09T08:08:49.682 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.47217.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.47217.log.gz 2024-05-09T08:08:49.683 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50100.log 2024-05-09T08:08:49.685 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40004.log 2024-05-09T08:08:49.686 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.50100.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.50100.log.gz 2024-05-09T08:08:49.687 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.40004.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.40004.log.gz 2024-05-09T08:08:49.688 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49428.log 2024-05-09T08:08:49.689 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42634.log 2024-05-09T08:08:49.691 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.49428.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.49428.log.gz 2024-05-09T08:08:49.691 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43589.log/var/log/ceph/ceph-client.admin.42634.log: 2024-05-09T08:08:49.692 INFO:teuthology.orchestra.run.smithi001.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.42634.log.gz 2024-05-09T08:08:49.693 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46164.log 2024-05-09T08:08:49.694 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.43589.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43589.log.gz 2024-05-09T08:08:49.695 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.46164.log: /var/log/ceph/ceph-client.admin.49974.log 2024-05-09T08:08:49.696 INFO:teuthology.orchestra.run.smithi001.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.46164.log.gz 2024-05-09T08:08:49.697 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42631.log 2024-05-09T08:08:49.699 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.49974.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.49974.log.gz 2024-05-09T08:08:49.699 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.42631.log: -- /var/log/ceph/ceph-client.admin.43337.log 2024-05-09T08:08:49.700 INFO:teuthology.orchestra.run.smithi001.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.42631.log.gz 2024-05-09T08:08:49.701 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47613.log 2024-05-09T08:08:49.702 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.43337.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43337.log.gz 2024-05-09T08:08:49.703 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42926.log 2024-05-09T08:08:49.703 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.47613.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.47613.log.gz 2024-05-09T08:08:49.705 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40108.log 2024-05-09T08:08:49.706 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.42926.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.42926.log.gz 2024-05-09T08:08:49.709 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5/var/log/ceph/ceph-client.admin.40108.log: --verbose -- /var/log/ceph/ceph-client.admin.41519.log 2024-05-09T08:08:49.709 INFO:teuthology.orchestra.run.smithi001.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.40108.log.gz 2024-05-09T08:08:49.709 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41807.log 2024-05-09T08:08:49.710 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.41519.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.41519.log.gz 2024-05-09T08:08:49.711 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.41807.log: /var/log/ceph/ceph-client.admin.46470.log 2024-05-09T08:08:49.712 INFO:teuthology.orchestra.run.smithi001.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.41807.log.gz 2024-05-09T08:08:49.712 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44578.log 2024-05-09T08:08:49.714 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.46470.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.46470.log.gz 2024-05-09T08:08:49.715 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.44578.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43631.log 2024-05-09T08:08:49.715 INFO:teuthology.orchestra.run.smithi001.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.44578.log.gz 2024-05-09T08:08:49.717 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48010.log 2024-05-09T08:08:49.718 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.43631.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.43631.log.gz 2024-05-09T08:08:49.719 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.48010.log: -- /var/log/ceph/ceph-client.admin.40473.log 2024-05-09T08:08:49.719 INFO:teuthology.orchestra.run.smithi001.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.48010.log.gz 2024-05-09T08:08:49.721 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39755.log 2024-05-09T08:08:49.722 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.40473.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.40473.log.gz 2024-05-09T08:08:49.723 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41083.log/var/log/ceph/ceph-client.admin.39755.log: 2024-05-09T08:08:49.724 INFO:teuthology.orchestra.run.smithi001.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.39755.log.gz 2024-05-09T08:08:49.725 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43547.log 2024-05-09T08:08:49.726 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.41083.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.41083.log.gz 2024-05-09T08:08:49.727 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.43547.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46883.log 2024-05-09T08:08:49.727 INFO:teuthology.orchestra.run.smithi001.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43547.log.gz 2024-05-09T08:08:49.728 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46968.log 2024-05-09T08:08:49.730 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.46883.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46883.log.gz 2024-05-09T08:08:49.730 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47301.log 2024-05-09T08:08:49.733 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.46968.log: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.46968.log.gz 2024-05-09T08:08:49.733 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41477.log 2024-05-09T08:08:49.734 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.47301.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.47301.log.gz 2024-05-09T08:08:49.735 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50548.log 2024-05-09T08:08:49.735 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.41477.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.41477.log.gz 2024-05-09T08:08:49.736 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42632.log 2024-05-09T08:08:49.737 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.50548.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.50548.log.gz 2024-05-09T08:08:49.739 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.42632.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48196.log 2024-05-09T08:08:49.739 INFO:teuthology.orchestra.run.smithi001.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.42632.log.gz 2024-05-09T08:08:49.740 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41639.log 2024-05-09T08:08:49.742 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.48196.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.48196.log.gz 2024-05-09T08:08:49.742 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42967.log 2024-05-09T08:08:49.748 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.41639.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.41639.log.gz 2024-05-09T08:08:49.749 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45324.log 2024-05-09T08:08:49.749 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.42967.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.42967.log.gz 2024-05-09T08:08:49.749 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.e.log/var/log/ceph/ceph-client.admin.45324.log: 2024-05-09T08:08:49.749 INFO:teuthology.orchestra.run.smithi001.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.45324.log.gz 2024-05-09T08:08:49.749 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47697.log 2024-05-09T08:08:49.750 INFO:teuthology.orchestra.run.smithi007.stderr: 91.3% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-05-09T08:08:49.751 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-mds.e.log: /var/log/ceph/ceph-client.admin.47697.log: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.47697.log.gz 2024-05-09T08:08:49.752 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- 92.6% -- replaced with /var/log/ceph/ceph-mds.e.log.gz 2024-05-09T08:08:49.752 INFO:teuthology.orchestra.run.smithi001.stderr: /var/log/ceph/ceph-client.admin.41951.log 2024-05-09T08:08:49.753 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39912.log 2024-05-09T08:08:49.754 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.41951.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.41951.log.gz 2024-05-09T08:08:49.755 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50693.log/var/log/ceph/ceph-client.admin.39912.log: 2024-05-09T08:08:49.757 INFO:teuthology.orchestra.run.smithi001.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.39912.log.gz 2024-05-09T08:08:49.757 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50608.log 2024-05-09T08:08:49.758 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.50693.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.50693.log.gz 2024-05-09T08:08:49.759 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5/var/log/ceph/ceph-client.admin.50608.log: --verbose -- /var/log/ceph/ceph-client.admin.50226.log 2024-05-09T08:08:49.760 INFO:teuthology.orchestra.run.smithi001.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.50608.log.gz 2024-05-09T08:08:49.761 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40151.log 2024-05-09T08:08:49.766 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.50226.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.50226.log.gz 2024-05-09T08:08:49.769 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42294.log 2024-05-09T08:08:49.769 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.40151.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.40151.log.gz 2024-05-09T08:08:49.769 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41723.log 2024-05-09T08:08:49.769 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.42294.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42294.log.gz 2024-05-09T08:08:49.769 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5/var/log/ceph/ceph-client.admin.41723.log: --verbose -- /var/log/ceph/ceph-client.admin.40403.log 2024-05-09T08:08:49.769 INFO:teuthology.orchestra.run.smithi001.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.41723.log.gz 2024-05-09T08:08:49.770 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45936.log 2024-05-09T08:08:49.771 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.40403.log: /var/log/ceph/ceph-client.admin.45936.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.40403.log.gz 2024-05-09T08:08:49.771 INFO:teuthology.orchestra.run.smithi001.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.45936.log.gz 2024-05-09T08:08:49.772 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41293.log 2024-05-09T08:08:49.773 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45409.log 2024-05-09T08:08:49.775 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.41293.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.41293.log.gz 2024-05-09T08:08:49.776 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.45409.log: 91.7% -- replaced with /var/log/ceph/ceph-client.admin.45409.log.gz 2024-05-09T08:08:49.776 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45810.log 2024-05-09T08:08:49.778 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50422.log 2024-05-09T08:08:49.779 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.45810.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.45810.log.gz 2024-05-09T08:08:49.780 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45894.log 2024-05-09T08:08:49.781 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.50422.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.50422.log.gz 2024-05-09T08:08:49.782 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42622.log 2024-05-09T08:08:49.783 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.45894.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.45894.log.gz 2024-05-09T08:08:49.784 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48238.log 2024-05-09T08:08:49.785 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.42622.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42622.log.gz 2024-05-09T08:08:49.786 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44956.log 2024-05-09T08:08:49.787 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.48238.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.48238.log.gz 2024-05-09T08:08:49.788 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43799.log 2024-05-09T08:08:49.790 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.44956.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.44956.log.gz 2024-05-09T08:08:49.790 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.43799.log: /var/log/ceph/ceph-client.admin.47740.log 2024-05-09T08:08:49.791 INFO:teuthology.orchestra.run.smithi001.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43799.log.gz 2024-05-09T08:08:49.791 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48859.log 2024-05-09T08:08:49.793 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.47740.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47740.log.gz 2024-05-09T08:08:49.794 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.48859.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35811.log 2024-05-09T08:08:49.794 INFO:teuthology.orchestra.run.smithi001.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.48859.log.gz 2024-05-09T08:08:49.795 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47926.log 2024-05-09T08:08:49.797 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.35811.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.35811.log.gz 2024-05-09T08:08:49.798 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5/var/log/ceph/ceph-client.admin.47926.log: --verbose -- /var/log/ceph/ceph-client.admin.48365.log 2024-05-09T08:08:49.799 INFO:teuthology.orchestra.run.smithi001.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.47926.log.gz 2024-05-09T08:08:49.799 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47445.log 2024-05-09T08:08:49.802 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.48365.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.48365.log.gz 2024-05-09T08:08:49.802 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50058.log 2024-05-09T08:08:49.803 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.47445.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.47445.log.gz 2024-05-09T08:08:49.803 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.50058.log: /var/log/ceph/ceph-client.admin.50340.log 2024-05-09T08:08:49.805 INFO:teuthology.orchestra.run.smithi001.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.50058.log.gz 2024-05-09T08:08:49.805 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46062.log 2024-05-09T08:08:49.807 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.50340.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.50340.log.gz 2024-05-09T08:08:49.807 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.46062.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46671.log 2024-05-09T08:08:49.808 INFO:teuthology.orchestra.run.smithi001.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.46062.log.gz 2024-05-09T08:08:49.810 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.46671.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.46671.log.gz 2024-05-09T08:08:49.811 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50464.log 2024-05-09T08:08:49.812 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47782.log 2024-05-09T08:08:49.814 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.50464.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.50464.log.gz 2024-05-09T08:08:49.814 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43673.log 2024-05-09T08:08:49.815 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.47782.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.47782.log.gz 2024-05-09T08:08:49.816 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43841.log 2024-05-09T08:08:49.817 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.43673.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.43673.log.gz 2024-05-09T08:08:49.819 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.43841.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.43841.log.gz 2024-05-09T08:08:49.819 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46841.log 2024-05-09T08:08:49.821 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-05-09T08:08:49.822 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.46841.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46841.log.gz 2024-05-09T08:08:49.823 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48733.log 2024-05-09T08:08:49.826 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-mds.a.log: /var/log/ceph/ceph-client.admin.48733.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.48733.log.gz 2024-05-09T08:08:49.826 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42289.log 2024-05-09T08:08:49.826 INFO:teuthology.orchestra.run.smithi001.stderr: 92.6% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-05-09T08:08:49.829 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.42289.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48493.log 2024-05-09T08:08:49.829 INFO:teuthology.orchestra.run.smithi001.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42289.log.gz 2024-05-09T08:08:49.830 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45014.log 2024-05-09T08:08:49.832 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.48493.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.48493.log.gz 2024-05-09T08:08:49.833 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.45014.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46925.log 88.0% -- replaced with /var/log/ceph/ceph-client.admin.45014.log.gz 2024-05-09T08:08:49.833 INFO:teuthology.orchestra.run.smithi001.stderr: 2024-05-09T08:08:49.834 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41167.log 2024-05-09T08:08:49.836 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.46925.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.46925.log.gz 2024-05-09T08:08:49.837 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43883.log 2024-05-09T08:08:49.838 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.41167.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.41167.log.gz 2024-05-09T08:08:49.838 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48552.log 2024-05-09T08:08:49.841 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.43883.log: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.43883.log.gz 2024-05-09T08:08:49.841 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.48552.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45596.log 2024-05-09T08:08:49.841 INFO:teuthology.orchestra.run.smithi001.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.48552.log.gz 2024-05-09T08:08:49.843 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44620.log 2024-05-09T08:08:49.844 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.45596.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.45596.log.gz 2024-05-09T08:08:49.845 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45179.log 2024-05-09T08:08:49.846 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.44620.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.44620.log.gz 2024-05-09T08:08:49.847 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49289.log 2024-05-09T08:08:49.848 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.45179.log: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.45179.log.gz 2024-05-09T08:08:49.850 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42961.log 2024-05-09T08:08:49.850 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.49289.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.49289.log.gz 2024-05-09T08:08:49.851 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42633.log 2024-05-09T08:08:49.851 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.42961.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.42961.log.gz 2024-05-09T08:08:49.852 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41993.log 2024-05-09T08:08:49.854 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.42633.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.42633.log.gz 2024-05-09T08:08:49.854 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43505.log 2024-05-09T08:08:49.855 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.41993.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.41993.log.gz 2024-05-09T08:08:49.857 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46797.log 2024-05-09T08:08:49.857 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.43505.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.43505.log.gz 2024-05-09T08:08:49.858 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49005.log 2024-05-09T08:08:49.859 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.46797.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.46797.log.gz 2024-05-09T08:08:49.861 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.49005.log: 92.4% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-05-09T08:08:49.862 INFO:teuthology.orchestra.run.smithi001.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.49005.log.gz 2024-05-09T08:08:49.862 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47487.log 2024-05-09T08:08:49.863 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49764.log 2024-05-09T08:08:49.864 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.47487.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49554.log 2024-05-09T08:08:49.864 INFO:teuthology.orchestra.run.smithi001.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.47487.log.gz 2024-05-09T08:08:49.865 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.49764.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47842.log 2024-05-09T08:08:49.865 INFO:teuthology.orchestra.run.smithi001.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.49764.log.gz 2024-05-09T08:08:49.866 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.49554.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.49554.log.gz 2024-05-09T08:08:49.867 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42972.log 2024-05-09T08:08:49.868 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.47842.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.47842.log.gz 2024-05-09T08:08:49.868 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40193.log 2024-05-09T08:08:49.869 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50016.log 2024-05-09T08:08:49.870 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.42972.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.42972.log.gz 2024-05-09T08:08:49.871 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.40193.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44011.log 2024-05-09T08:08:49.871 INFO:teuthology.orchestra.run.smithi001.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.40193.log.gz 2024-05-09T08:08:49.872 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.50016.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.50016.log.gz 2024-05-09T08:08:49.872 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46428.log 2024-05-09T08:08:49.873 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.44011.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.44361.log 2024-05-09T08:08:49.873 INFO:teuthology.orchestra.run.smithi001.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.44011.log.gz 2024-05-09T08:08:49.875 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.admin.46428.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.46428.log.gz 2024-05-09T08:08:49.875 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/ceph-client.0.44361.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.44361.log.gz 2024-05-09T08:08:50.080 INFO:teuthology.orchestra.run.smithi001.stderr: 90.9% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-05-09T08:08:55.602 INFO:teuthology.orchestra.run.smithi007.stderr: 93.1% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-05-09T08:08:55.623 INFO:teuthology.orchestra.run.smithi007.stderr: 93.2% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-05-09T08:08:55.759 INFO:teuthology.orchestra.run.smithi001.stderr: 93.1% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-05-09T08:08:56.843 INFO:teuthology.orchestra.run.smithi001.stderr: 93.1% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-05-09T08:08:57.359 INFO:teuthology.orchestra.run.smithi007.stderr: 93.1% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-05-09T08:08:57.684 INFO:teuthology.orchestra.run.smithi007.stderr: 93.0% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-05-09T08:08:57.685 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-09T08:08:57.685 INFO:teuthology.orchestra.run.smithi007.stderr:real 0m8.223s 2024-05-09T08:08:57.686 INFO:teuthology.orchestra.run.smithi007.stderr:user 0m27.988s 2024-05-09T08:08:57.686 INFO:teuthology.orchestra.run.smithi007.stderr:sys 0m0.913s 2024-05-09T08:08:59.210 INFO:teuthology.orchestra.run.smithi001.stderr: 93.1% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-05-09T08:09:00.130 INFO:teuthology.orchestra.run.smithi001.stderr: 93.0% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-05-09T08:09:00.134 INFO:teuthology.orchestra.run.smithi001.stderr: 2024-05-09T08:09:00.135 INFO:teuthology.orchestra.run.smithi001.stderr:real 0m10.670s 2024-05-09T08:09:00.135 INFO:teuthology.orchestra.run.smithi001.stderr:user 0m31.794s 2024-05-09T08:09:00.135 INFO:teuthology.orchestra.run.smithi001.stderr:sys 0m3.227s 2024-05-09T08:09:00.136 INFO:tasks.ceph:Archiving logs... 2024-05-09T08:09:00.136 DEBUG:teuthology.misc:Transferring archived files from smithi001:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-05-08_21:24:04-fs-squid-distro-default-smithi/7698643/remote/smithi001/log 2024-05-09T08:09:00.137 DEBUG:teuthology.orchestra.run.smithi001:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-09T08:09:05.412 DEBUG:teuthology.misc:Transferring archived files from smithi007:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-05-08_21:24:04-fs-squid-distro-default-smithi/7698643/remote/smithi007/log 2024-05-09T08:09:05.413 DEBUG:teuthology.orchestra.run.smithi007:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-09T08:09:09.774 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-05-09T08:09:09.785 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-05-09T08:09:09.785 DEBUG:teuthology.orchestra.run.smithi001:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-09T08:09:09.793 DEBUG:teuthology.orchestra.run.smithi007:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-09T08:09:09.920 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-05-09T08:09:09.931 INFO:teuthology.task.clock:Checking final clock skew... 2024-05-09T08:09:09.932 DEBUG:teuthology.orchestra.run.smithi001:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-09T08:09:09.941 DEBUG:teuthology.orchestra.run.smithi007:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-09T08:09:09.959 INFO:teuthology.orchestra.run.smithi007.stderr:bash: line 1: ntpq: command not found 2024-05-09T08:09:09.963 INFO:teuthology.orchestra.run.smithi007.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-09T08:09:09.963 INFO:teuthology.orchestra.run.smithi007.stdout:=============================================================================== 2024-05-09T08:09:09.963 INFO:teuthology.orchestra.run.smithi007.stdout:^+ hv01.front.sepia.ceph.com 3 6 377 10 +826us[ +827us] +/- 66ms 2024-05-09T08:09:09.964 INFO:teuthology.orchestra.run.smithi007.stdout:^* hv02.front.sepia.ceph.com 3 6 377 10 -561us[ -560us] +/- 58ms 2024-05-09T08:09:09.964 INFO:teuthology.orchestra.run.smithi007.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 9 -290us[ -290us] +/- 60ms 2024-05-09T08:09:09.964 INFO:teuthology.orchestra.run.smithi007.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T08:09:09.989 INFO:teuthology.orchestra.run.smithi001.stderr:bash: line 1: ntpq: command not found 2024-05-09T08:09:10.004 INFO:teuthology.orchestra.run.smithi001.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-09T08:09:10.004 INFO:teuthology.orchestra.run.smithi001.stdout:=============================================================================== 2024-05-09T08:09:10.004 INFO:teuthology.orchestra.run.smithi001.stdout:^+ hv01.front.sepia.ceph.com 3 6 377 10 +820us[ +820us] +/- 66ms 2024-05-09T08:09:10.004 INFO:teuthology.orchestra.run.smithi001.stdout:^* hv02.front.sepia.ceph.com 3 6 377 10 -570us[ -608us] +/- 58ms 2024-05-09T08:09:10.004 INFO:teuthology.orchestra.run.smithi001.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 10 -285us[ -322us] +/- 60ms 2024-05-09T08:09:10.004 INFO:teuthology.orchestra.run.smithi001.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T08:09:10.005 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-05-09T08:09:10.021 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-05-09T08:09:10.022 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-05-09T08:09:10.036 DEBUG:teuthology.orchestra.run.smithi001:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-09T08:09:10.238 DEBUG:teuthology.orchestra.run.smithi007:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-09T08:09:10.305 DEBUG:teuthology.orchestra.run.smithi001:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-09T08:09:10.405 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T08:09:10.406 DEBUG:teuthology.orchestra.run.smithi007:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-09T08:09:10.436 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T08:09:10.437 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-05-09T08:09:10.450 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-05-09T08:09:10.460 INFO:teuthology.task.internal:Duration was 950.527936 seconds 2024-05-09T08:09:10.461 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-05-09T08:09:10.471 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-05-09T08:09:10.471 DEBUG:teuthology.orchestra.run.smithi001:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-09T08:09:10.479 DEBUG:teuthology.orchestra.run.smithi007:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-09T08:09:10.521 INFO:teuthology.orchestra.run.smithi007.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-09T08:09:10.625 INFO:teuthology.orchestra.run.smithi001.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-09T08:09:10.747 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-05-09T08:09:10.747 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi001.front.sepia.ceph.com 2024-05-09T08:09:10.747 DEBUG:teuthology.orchestra.run.smithi001:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-09T08:09:10.811 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi007.front.sepia.ceph.com 2024-05-09T08:09:10.811 DEBUG:teuthology.orchestra.run.smithi007:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-09T08:09:10.839 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-05-09T08:09:10.839 DEBUG:teuthology.orchestra.run.smithi001:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-09T08:09:10.844 DEBUG:teuthology.orchestra.run.smithi007:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-09T08:09:10.921 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-05-09T08:09:10.921 DEBUG:teuthology.orchestra.run.smithi001:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-09T08:09:10.927 DEBUG:teuthology.orchestra.run.smithi007:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-09T08:09:11.105 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-05-09T08:09:11.126 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-05-09T08:09:11.126 DEBUG:teuthology.orchestra.run.smithi001:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-09T08:09:11.223 DEBUG:teuthology.orchestra.run.smithi007:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-09T08:09:11.250 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-05-09T08:09:11.263 DEBUG:teuthology.orchestra.run.smithi001:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-09T08:09:11.272 DEBUG:teuthology.orchestra.run.smithi007:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-09T08:09:11.314 INFO:teuthology.orchestra.run.smithi007.stdout:kernel.core_pattern = core 2024-05-09T08:09:11.371 INFO:teuthology.orchestra.run.smithi001.stdout:kernel.core_pattern = core 2024-05-09T08:09:11.415 DEBUG:teuthology.orchestra.run.smithi001:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-09T08:09:11.466 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T08:09:11.467 DEBUG:teuthology.orchestra.run.smithi007:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-09T08:09:11.482 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T08:09:11.482 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-05-09T08:09:11.494 INFO:teuthology.task.internal:Transferring archived files... 2024-05-09T08:09:11.495 DEBUG:teuthology.misc:Transferring archived files from smithi001:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-05-08_21:24:04-fs-squid-distro-default-smithi/7698643/remote/smithi001 2024-05-09T08:09:11.495 DEBUG:teuthology.orchestra.run.smithi001:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-09T08:09:11.594 DEBUG:teuthology.misc:Transferring archived files from smithi007:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-05-08_21:24:04-fs-squid-distro-default-smithi/7698643/remote/smithi007 2024-05-09T08:09:11.594 DEBUG:teuthology.orchestra.run.smithi007:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-09T08:09:11.630 INFO:teuthology.task.internal:Removing archive directory... 2024-05-09T08:09:11.630 DEBUG:teuthology.orchestra.run.smithi001:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-09T08:09:11.638 DEBUG:teuthology.orchestra.run.smithi007:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-09T08:09:11.690 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-05-09T08:09:11.703 INFO:teuthology.task.internal:Not uploading archives. 2024-05-09T08:09:11.703 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-05-09T08:09:11.714 INFO:teuthology.task.internal:Tidying up after the test... 2024-05-09T08:09:11.715 DEBUG:teuthology.orchestra.run.smithi001:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-09T08:09:11.721 DEBUG:teuthology.orchestra.run.smithi007:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-09T08:09:11.736 INFO:teuthology.orchestra.run.smithi007.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 9 08:09 /home/ubuntu/cephtest 2024-05-09T08:09:11.771 INFO:teuthology.orchestra.run.smithi001.stdout: 393619 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 9 08:09 /home/ubuntu/cephtest 2024-05-09T08:09:11.775 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-05-09T08:09:11.786 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-05-09T08:09:11.814 INFO:teuthology.run:Summary data: client.0-kernel-sha1: 806ec36b21129f113ce356c007dd59694d340b37 description: fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a5s-mds-1c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-ec-root overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/no prefetch_entire_dirfrags/no races session_timeout thrashosds-health} ranks/1 tasks/{1-thrash/mon 2-workunit/suites/iozone}} duration: 950.5279357433319 flavor: default owner: scheduled_teuthology@teuthology success: true 2024-05-09T08:09:11.814 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-09T08:09:11.901 INFO:teuthology.run:pass